Systems and methods for monitoring bacterial growth of colonies and predicting colony biomass

Abstract

Imaging method for earliest microbial growth detection. This method uses images to determine colony biomass, and the colony biomass determines when colonies can be picked for identification or analysis of antibiotic susceptibility testing. If the sample source is not a pure sample source, additional incubation may be required to allow an increase in colony biomass before picking.

Description

[0001] Related Applications

[0002] This application claims the benefit of priority to U.S. Provisional Application No. 62 / 782,513, filed on Dec. 20, 2018, the entire content of which is incorporated herein by reference. Background Art

[0003] There is increasing interest in digital images of culture plates for detecting microbial growth. Techniques for imaging culture plates to detect microbial growth thereon are described in PCT Publication No. WO / 2015 / 114121, published on Aug. 6, 2016, entitled “A System and Method for Image Acquisition Using Supervised High Quality Imaging,” WO / 2016 / 172527, published on Oct. 27, 2016, entitled “Colony Contrast Gathering,” and WO / 2016 / 172532, published on Oct. 27, 2017, entitled “A Method and System for Automatic Microbial Colony count from Streaked Sample on Plated media,” the entire content of each of which is incorporated herein by reference. All of the above references are commonly assigned with this application.

[0004] WO / 2015 / 114121 describes techniques for identifying colony targets by controlling the signal-to-noise ratio and distinguishing these targets from non-colony artifacts and background. WO / 2016 / 172527 describes systems and methods for interpreting culture plate images based on automated imaging techniques. Colony targets are identified early in the process of promoting microbial growth on a nutrient medium, and the changes in these identified targets are observed over time and under conditions that support colony growth. WO / 2016 / 172532 describes systems and methods by which identified colonies are counted to determine whether the colonies are from the same microorganism and to further determine whether the colony count has reached or exceeded a predetermined number.

[0005] The detection of colonies, colony counting, colony population differentiation, and colony identification define the goals for modern microbiology imaging systems. Achieving these goals as early as possible meets the goals of delivering results to patients quickly and providing these results and analyses economically. Automated laboratory workflows and decision-making can improve the speed and cost at which these goals can be achieved.

[0006] Although significant progress has been made in imaging techniques for detecting signs of microbial growth, there is still a search to extend such imaging techniques to support automated workflows. Instruments and methods for examining culture plates to indicate microbial growth are difficult to automate, in part due to the highly visual nature of plate examination. In this regard, there is a desire to develop techniques that can automatically interpret culture plate images and, based on the automated interpretation, determine subsequent steps to be performed (e.g., colony identification, susceptibility testing, etc.).

[0007] Identifying and differentiating colonies on a culture plate can be difficult, especially when the colonies have different sizes and shapes and are in contact with each other. Colonies that "grow together" make it more difficult to pick colonies of interest because there is a risk of picking microorganisms from adjacent colonies, where the adjacent colonies are different microorganisms. Picking a colony sample for downstream processing requires a sufficient number of colonies and the purity of the target colony to avoid picking multiple microorganisms, which can contaminate downstream test results. These problems are more severe when growth has reached confluence in some areas of the plate. For these reasons, if possible, it is preferred to identify colonies and determine growth early in the process. However, there is still a need for incubation time to allow at least some growth of the colonies. Thus, on the one hand, the longer the colonies are allowed to grow, the stronger the contrast they form with their background and with each other, and the easier it becomes to identify them. However, on the other hand, if the colonies are allowed to grow too long and they start to fill the plate and / or contact each other, it becomes more difficult to pick pure colonies. The problem can be minimized or even solved if the incubation time can be detected for colonies when they are still small enough to be separated from each other - although the contrast is relatively poor - but still large enough to provide a sufficient amount of sample for testing. SUMMARY OF THE INVENTION

[0008] An automated method for evaluating microbial growth on a plated medium is described herein. According to the method, the provided medium is inoculated with a biological sample placed in a substantially optically transparent container. The inoculated medium is incubated in an incubator. The inoculated medium is placed in an optically transparent container carrying the inoculated medium in a digital imaging device. The automated method further includes, at a first time (t 0)Obtain a first digital image of the inoculated culture medium, the first digital image having a plurality of pixels. The automated method further includes determining the coordinates of the pixels in the first digital image relative to the transparent container carrying the inoculated culture medium. The automated method further includes removing the transparent container carrying the inoculated culture medium from the digital imaging device and placing the inoculated culture medium in an incubator for further incubation. The automated method further includes, after further incubation, placing the transparent container carrying the inoculated culture medium in the digital imaging device. The automated method further includes at a second time (t x )Obtain a second digital image of the inoculated culture medium, the second digital image having a plurality of pixels. The automated method further includes aligning the first digital image with the second digital image such that the pixel coordinates in the second digital image correspond to the coordinates of the corresponding pixels in the first digital image. The automated method further includes comparing the pixels of the second digital image with the corresponding pixels of the first digital image. The automated method further includes identifying the pixels that have changed between the first digital image and the second digital image, wherein the pixels that have not changed between the first digital image and the second digital image indicate the background. The automated method further includes determining which of the identified pixels in the second digital image have a predetermined level of threshold contrast with the pixels indicating the background. The automated method further includes identifying one or more targets in the second digital image, each target including pixels that have the level of threshold contrast with the pixels indicating the background and are not separated from each other by background pixels. The automated method further includes associating the identified targets with biomass. The automated method further includes determining whether the biological sample is identified as a pure sample, and if so, further determining whether the biomass is higher than a first threshold, the first threshold being a predetermined area of the culture medium covered by the identified target, and if the area of the identified target is higher than the first threshold, picking at least a portion of the growth material for further analysis. The automated method further includes if the biological sample is not a pure sample, further incubating the inoculated culture medium in an optically transparent container. BRIEF DESCRIPTION OF THE DRAWINGS

[0009] Figure 1 is a schematic diagram for imaging analysis and testing cultures according to one aspect of the present disclosure.

[0010] Figure 2 is a flowchart illustrating an automated laboratory workflow procedure for imaging analysis and testing cultures according to one aspect of the present disclosure.

[0011] Figure 3A 、 3B and 3C are images showing the time contrast of colony morphology change over time by visualizing colony morphology according to one aspect of the present disclosure.

[0012] Figure 3D and 3Eis an image showing the spatial contrast under different illumination conditions.

[0013] Figure 4 is a flowchart of an example program for obtaining and analyzing image information according to one aspect of the present disclosure.

[0014] Figure 5 is a flowchart of an example program for obtaining spatial contrast according to one aspect of the present disclosure.

[0015] Figure 6 is a flowchart of an example program for obtaining temporal contrast according to one aspect of the present disclosure.

[0016] Figure 7 is a flowchart of an example program for filtering artifacts from an image according to one aspect of the present disclosure.

[0017] Figure 8 is a flowchart of an example program for labeling pixels of an image according to one aspect of the present disclosure.

[0018] Figure 9 is a flowchart of an example program for splitting colonies into separate objects according to one aspect of the present disclosure.

[0019] Figure 10 is a flowchart of an example object segmentation program according to one aspect of the present disclosure.

[0020] Figure 11 is a schematic diagram showing the measurement of confluent colonies, which is Figure 10 part of the segmentation program in

[0021] Figure 12 is according to one aspect of the present disclosure Figure.

[0022] Figure 13A 、 13B and 13C are diagrams illustrating the measurement of the isolation factor according to one aspect of the present disclosure.

[0023] Figure 14 is a diagram illustrating the affected area according to one aspect of the present disclosure.

[0024] Figure 15A 、 15B and 15C are images illustrating the characterization of colony growth according to one aspect of the present disclosure.

[0025] Figure 16A and 16B show a part of the imaging plate, with magnified and redirected images of sample colonies in the image.

[0026] Figure 16C shows the Figure 16B vector diagrams of the respective images in

[0027] Figure 17 depicts the SHQI, spatial contrast, and temporal contrast images of a sample according to one aspect of the present disclosure.

[0028] Figure 18 is a flowchart comparing Figure 2 the timeline of the program in

[0029] Figure 19 is a diagram of a growth timeline that will identify colony candidates of sufficient size to support colony picking. DETAILED DESCRIPTION

[0030] The present disclosure provides apparatuses, systems, and methods for identifying and analyzing microbial growth on a plated medium based at least in part on contrast detected in one or more digital images of the plated medium. The various methods described herein may be fully or partially automated, for example, integrated as part of a fully or partially automated laboratory workflow.

[0031] The systems described herein can be implemented in an optical system for imaging microbial samples for identifying microorganisms and detecting microbial growth of such microorganisms. There are many such commercially available systems that are not described in detail herein. One example is the BD Kiestra TM ReadA Compact Smart Incubation and Imaging System. Other example systems include those described in PCT Publication No. WO2015 / 114121 and U.S. Patent Publication No. 2015 / 0299639, the entireties of which are incorporated herein by reference. Such optical imaging platforms are well known to those skilled in the art and are not described in detail herein.

[0032] Figure 1 is a schematic diagram of a system 100 having a processing module 110 and an image acquisition device 120 (e.g., a camera) for providing high-quality imaging of a plated medium. The processing module and the image acquisition device may be further connected to other system components and thereby further interact with other system components, such as an incubation module (not shown) for incubating the plated medium to allow the cultures inoculated on the plated medium to grow. Such connections can be fully or partially automated using a track system that receives samples for incubation and transports them to an incubator and then transports them between the incubator and the image acquisition device.

[0033] The processing module 110 can direct other components of the system 100 to perform tasks based on the processing of various types of information. The processor 110 can be hardware that performs one or more operations. The processor 110 can be any standard processor, such as a central processing unit (CPU), or can be a specialized processor, such as an application specific integrated circuit (ASIC) or a field programmable gate array (FPGA). Although one processor block is shown, the system 100 can also include multiple processors - which may or may not operate in parallel, or other specialized logic and memory for storing and tracking information related to the sample containers in the incubator and / or the image acquisition device 120. In this regard, the processing unit can track and / or store several types of information related to the samples in the system 100, including but not limited to the location of the sample in the system (incubator or image acquisition device, location and / or orientation therein, etc.), incubation time, pixel information of the captured images, type of sample, type of culture medium, preventive treatment information (e.g., harmful samples), and so on. In this regard, the processor can be capable of automating all or part of the various procedures described herein. In one embodiment, the instructions for implementing the procedures described herein can be stored on a non-transitory computer-readable medium (e.g., a software program).

[0034] Figure 2 is a flowchart that illustrates an example automated laboratory procedure 200 for imaging, analyzing, and optionally testing a culture. The procedure 200 can be implemented by an automated microbiology laboratory system, such as BD Kiestra TM Total Lab Automation or BD Kiestra TM Work CellAutomation. Example systems include interconnected modules, each configured to perform one or more steps of the procedure 200.

[0035] At 202, a culture medium is provided and inoculated with a biological sample. The culture medium can be an optically transparent container so that it can be observed within the container when the biological sample is illuminated from different angles. The inoculation can follow a predetermined pattern. The streaking patterns and automated methods for streaking samples on plates are well known to those skilled in the art and will not be discussed in detail here. One automated method uses magnetic beads to streak the sample on the plate. At 204, the culture medium is incubated to allow the biological sample to grow.

[0036] At 206, one or more digital images of the capture medium and the biological sample are captured. As will be described in more detail below, during the incubation process, digital imaging of the medium can be performed multiple times (e.g., at the start of incubation, at an intermediate time during incubation, at the end of incubation), so that changes in the medium can be observed and analyzed. Imaging of the medium can include removing the medium from the incubator. When multiple images of the medium are taken at different times, between imaging sessions, the medium can be returned to the incubator for further incubation.

[0037] At 208, the biological sample is analyzed based on information from the captured digital images. Analysis of the digital images can include analysis of the pixel information contained in the images. In some examples, the pixel information can be analyzed on a pixel-by-pixel basis. In other examples, the pixel information can be analyzed on a block-by-block basis. In further examples, the pixels can be analyzed based on the entire region of the pixels, whereby the pixel information of the individual pixels in the region can be obtained by combining the information of the individual pixels, selecting sample pixels, or by using other statistical methods such as statistical histogram operations described in more detail below. In the present disclosure, operations described as applied to "pixels" can equally be applied to blocks or other pixel groupings, and the term "pixel" is thus intended to include such applications.

[0038] The analysis can include determining whether growth in the medium is detected. From the perspective of image analysis, growth can be detected in the image by identifying the object of imaging (based on the difference between the object and its adjacent environment) and then identifying the change of the object over time. As described in more detail herein, these differences and changes are two forms of "contrast". In addition to detecting growth, the image analysis at 208 can further include quantitatively detecting the amount of growth detected, identifying unique colonies, identifying sister colonies, etc.

[0039] At 210, it is determined whether the biological sample (in particular, the identified sister colonies) exhibits a quantitatively significant growth. If no growth is found or only a negligible amount of growth is found, the program 200 can proceed to 220, where a final report is output. In the case of proceeding from 210 to 220, the final report will likely indicate a lack of significant growth, or report the growth of normal flora.

[0040] If it is determined that the biological sample exhibits a quantitatively significant increase, then at 212, based on the existing analysis, one or more colonies can be picked from the image. Picking colonies can be a fully automated process where each picked colony is sampled and tested. Optionally, picking colonies can be a partially automated process where multiple candidate colonies are automatically identified and visually presented to the operator on a digital image so that the operator can input a selection of one or more candidates for sampling and further testing. Sampling of the selected or picked colonies themselves can be performed automatically by the system.

[0041] At 214, the sampled colonies are prepared for further testing, for example by plating the sample in a suspension of organisms. At 216, the sample is tested using matrix-assisted laser desorption ionization (MALDI) imaging to identify the type of sample sampled from the initial medium. At 218, the sample is also, or optionally, subjected to antibiotic susceptibility testing (AST) to identify possible treatments for the identified sample.

[0042] At 220, the test results are output in a final report. The report can include MALDI as well as AST results. As described above, the report can also indicate the quantification of sample growth. Thus, the automated system is able to start from the inoculated medium and generate a final report on the samples found in the culture with little or no additional input.

[0043] In an example procedure such as Figure 2 the detected and identified colonies are typically referred to as colony-forming units (CFUs). A CFU is a microscopic object that starts as one or several bacteria. Over time, the bacteria grow to form colonies. The earlier the time from when the bacteria are placed on the plate, the fewer bacteria are detected, and thus the smaller the colonies and the lower the contrast with the background. In other words, a smaller colony size produces a smaller signal, and a smaller signal on a constant background results in a lower contrast. This is reflected by the following equation:

[0044] (1)

[0045] Contrast plays an important role in identifying objects in an image such as CFUs or other artifacts. An object can be detected if it has a significant difference in brightness, color, and / or texture from its surroundings. Once an object is detected, the analysis can also include identifying the type of the detected object. Such identification can also rely on contrast measurements, such as the smoothness of the edges of the identified object, or the uniformity (or lack thereof) of the color and / or brightness of the object. For an object to be detected by an image sensor, the contrast must be large enough to overcome the image noise (background signal).

[0046] Human perception of contrast (governed by Weber's law) is limited. Under optimal conditions, the human eye can detect a 1% difference in light level. The quality and confidence of an image measurement (e.g., brightness, color, contrast) can be characterized by the signal-to-noise ratio (SNR) of the measurement, where, independent of pixel intensity, an SNR value of 100 (or 40 decibels using 20 log 10 will match human detection capabilities. Digital imaging techniques that utilize high SNR imaging information and known SNR information for each pixel can allow detection of colonies even when those colonies remain invisible to the human eye.

[0047] In the present disclosure, contrast can be collected in at least two ways: spatially and temporally. Spatial contrast or local contrast quantifies the difference in color or brightness between a given region (e.g., a pixel, a group of adjacent pixels) in a single image and its surroundings. Temporal contrast or time contrast quantifies the difference in color or brightness between a given region in one image and the same region in another image taken at a different time. The equation for controlling temporal contrast is similar to the equation for spatial contrast:

[0048] (2)

[0049] where t 1 is the time after t 0 . Both the spatial and temporal contrast of a given image can be used to identify targets. The identified targets can then be further tested to determine their significance (e.g., whether they are CFUs, normal flora, dust, etc.).

[0050] Figure 3A 、 3B and 3C provide a visual demonstration of the effect that temporal contrast can have on imaging a sample. Figure 3A The images shown in Figure 3A were captured at different time points (from left to right, from the top row to the bottom row), showing the total growth in the sample. Although the growth in Figure 3B is significant, from the corresponding contrast time images in Figure 3C , the growth is even more significant and can even be noticed earlier in the sequence. For clarity, Figure 3B shows an enlarged portion of Figure 3C . It can be seen from

[0051] that the longer a portion of a colony is imaged over time, the brighter the spot becomes in the contrast image. In this way, the centroid of each colony can be indicated by the bright center or peak of the colony. Thus, image data obtained over time can reveal important information about changes in colony morphology.

[0051] To maximize the spatial or temporal contrast of a target relative to its background, the system can capture images using different incident light on different backgrounds. For example, arbitrary top illumination, bottom illumination, or side illumination can be used on a black or white background.

[0052] Figure 3D and 3E provides a visual demonstration of the effect that lighting conditions can have on an imaging sample. Figure 3D The image in Figure 3E was captured using top illumination, while the image in Figure 3D and 3E was captured using bottom illumination at a time that was nearly the same (e.g., close enough in time that no noticeable or significant growth occurred). As can be seen, Figure 3D each of the images of the samples in Figure 3E contains several colonies, but additional information about the colonies that is visible (in this case, hemolysis) requires

[0053] back illumination or bottom illumination of the image in

[0054] Figure 4 is a flowchart that shows an example procedure for analyzing an imaging plate based at least in part on contrast. Figure 4 The procedure in Figure 2 can be considered a subroutine example of the procedure 200 in Figure 2 such that Figure 4 206 and 208 in

[0055] are performed at least in part using the procedure in0 Capture a first digital image. Time t 0 can be the time immediately after the start of the incubation process, such that the bacteria in the imaging plate have not yet begun to form visible colonies.

[0056] At 404, assign coordinates to one or more pixels in the first digital image. In some cases, the coordinates can be polar coordinates, which have a radial coordinate extending from the center point of the imaging plate and an angular coordinate around the center point. The coordinates can be used in subsequent steps to assist in aligning the first digital image with other digital images of the plate taken at different angles and / or at different times. In some cases, the imaging plate can have specific landmarks or fiducial marks. Advantageously, such fiducial marks can be detected by the sensor (i.e., optically detectable). Orienting the target in the coordinate space using fiducial marks is well known to those skilled in the art or persons of ordinary skill. Examples of suitable optically detectable fiducial marks include eccentric marks, such as a point or line on the bottom of an optically transparent petri dish. Such marks can be detected by "aiming" the sensor at the bottom of the petri dish. If the support of the petri dish is optically transparent, the sensor can be located below the petri dish and detect the fiducial mark. If the culture medium placed in the petri dish is optically transparent, the sensor can detect such a fiducial mark if mounted above the petri dish. To avoid difficulties in detecting the fiducial mark on the bottom of the culture plate from the top or bottom of the plate; the fiducial mark can be a label attached to the side of the petri dish, such as a barcode label. The barcode label can be detected by the sensor. In these examples, the side or center of the barcode label can be the fiducial mark. Pixel coordinates can be assigned with respect to the fiducial mark. In this way, the orientation of the plate in the coordinate space of the imaging device can be replicated between imaging events. The coordinates of the pixel(s) covering the landmark in the first image can be assigned to the pixel(s) covering the same landmark in other images. Thus, the pixels in the earlier image can be compared with the same pixel in the later image because they share the same coordinates. In this way, any changes in the pixels between the images can be easily observed.

[0057] At 406, at time t x Capture a second digital image. Time t x is a time after t 0 at which the bacteria in the imaging plate have had the opportunity to form visible colonies.

[0058] At 408, align the second digital image with the first digital image based on the previously assigned coordinates. Aligning the images can further include normalization and standardization of the images, for example, using the methods and systems described in PCT Publication No. WO2015 / 114121.

[0059] At 410, the contrast information of the second digital image is determined. The contrast information can be collected on a pixel-by-pixel basis. For example, the pixels of the second digital image can be compared with the corresponding pixels (at the same coordinates) of the first digital image to determine the presence of temporal contrast. Additionally, adjacent pixels of the second digital image can be compared with each other, or with other pixels known as background pixels, to determine the presence of spatial contrast. Changes in pixel color and / or brightness indicate contrast, and the magnitude of such changes from one image to the next or from one pixel (or region of pixels) to the next pixel (or region of pixels) can be measured, calculated, estimated, or otherwise determined. In the case where both the temporal and spatial contrast of a given image are determined, the total contrast of a given pixel of the image can be determined based on the combination (e.g., average, weighted average) of the spatial and temporal contrast of the given pixel.

[0060] At 412, based on the contrast information calculated at 410, the objects in the second digital image are identified. Adjacent pixels of the second digital image with similar contrast information can be considered to belong to the same object. For example, if the difference in brightness between an adjacent pixel and its background, or between pixels, and their brightness in the first digital image is substantially the same (e.g., within a predetermined threshold amount), then the pixels can be considered to belong to the same object. As an example, the system can assign a "1" to any pixel with significant contrast (e.g., exceeding the threshold amount), and then identify the group of all adjacent pixels assigned a "1" as an object. The object can be given a specific label or mask, such that pixels with the same label share certain characteristics. The label can assist in distinguishing the object from other objects and / or the background during subsequent processes of subroutine 400. Identifying the objects in a digital image can include segmenting or partitioning the digital image into multiple regions (e.g., foreground and background). The purpose of segmentation is to transform the image into a representation of multiple components, so as to more easily analyze the components. Image segmentation is used to locate the objects of interest in the image.

[0061] At 414, the characteristics of a given object (identified at 412) can be characterized. The characterization of object characteristics can include deriving descriptive statistics of the object (e.g., area, reflectance, size, optical density, color, plate position, etc.). Descriptive statistics can ultimately quantitatively describe certain characteristics of the set of information collected about the object (e.g., from the SHQI image, from the contrast image). Such information can be evaluated as a function of species, concentration, mixture, time, and medium. However, at least in some cases, characterizing the object can start from a set of qualitative information related to the object characteristics, whereby such qualitative information is subsequently quantitatively represented. Table 1 below provides a list of example characteristics that can be qualitatively evaluated and subsequently transformed into a quantitative representation:

[0062] Table 1: Qualitative attributes of the object and criteria for quantitatively transforming the attribute

[0063]

[0064] Some features of the object, such as shape or the time until it can be visually observed, can be measured once for the object as a whole. Other features can be measured multiple times (e.g., for each pixel, for each row of pixels with a common Y coordinate, for each column of pixels with a common X coordinate, for each ray of pixels with a common angular coordinate, for each circle of pixels with a common radial coordinate) and then combined into a single measurement, for example, using a histogram. For example, color can be measured for each pixel, and growth rate or size can be measured for each row, column, ray, or circle of pixels, etc.

[0065] At 416, based on the characterized features, it is determined whether the object is a colony candidate. The determination of colony candidates can include inputting quantitative features (such as the scores shown in Table 1 above) or a subset thereof into a classifier. The classifier can include a confusion matrix for implementing a supervised machine learning algorithm or a matching matrix for implementing an unsupervised machine learning algorithm to evaluate the object. Supervised learning may be preferred in cases where the object is to be distinguished from a finite set (e.g., two or three) of possible organisms (in which case the algorithm can be trained on a relatively finite set of training data). In contrast, unsupervised learning may be preferred in cases where the object is to be distinguished from an entire database of possible organisms, in which case it is difficult to provide comprehensive - or even sufficient - training data. In the case of a confusion or matching matrix, the difference can be numerically measured within a certain range. For example, for a given pair of objects, "0" may mean that the two objects should be distinguished from each other, while "1" may mean that the objects are difficult to distinguish from each other.

[0066] Colony candidates can be stored in the memory of an automated system for further applications (e.g., testing, the segmentation procedure described below, etc.).

[0067] Application of multiple culture media

[0068] In the above example, the evaluation of the culture is described for a single culture medium. However, the example is equally applicable to examples where the culture is evaluated in multiple culture media.

[0069] Since the characteristics of bacteria (e.g., color, growth rate, etc.) may vary depending on the type of culture medium (“medium”) used, different confusion matrices may be used for each medium during classification (e.g., 416 of subroutine 400). Thus, for two targets, for one medium, the classifier outputs “0”, while for the same two targets, for a different medium, the classifier outputs “1”, which is entirely reasonable. The collective results of the classifiers can then be evaluated together (manually or based on further machine-driven relationships) to achieve an overall or final discrimination or classification of the targets.

[0070] The evaluation of multiple media can be implemented using a single container. The single container can be configured to hold multiple media (e.g., dual plates, triple plates, quadruple plates, etc.) so that multiple media can be imaged simultaneously together. Optionally, multiple media can be evaluated by streaking the culture sample in several containers, each container holding one or more media. Each of the multiple containers can then undergo the imaging procedure described above. The information obtained from each medium (e.g., characterized features) can then be collectively input into the classifier to give a more informed identification of the growth spotted in different media.

[0071] Contrast information

[0072] Figure 5 is a flowchart that shows an example subroutine 500 for obtaining spatial contrast as part of Figure 4 410 in. Subroutine 500 receives the following as input: one or more backgrounds of a group and lighting condition 551 and filter 554. At 502, a digital image is obtained under the specified lighting and background conditions from input group 551. At 504, then, the image is cloned. At 506, one of the cloned images is filtered using filter 554. In Figure 5 the example, a low-pass kernel is used as the filter, but other filters known to those skilled in the art can be used. At 508, the ratio of the unfiltered image minus the filtered image to the filtered image plus the unfiltered image is calculated. At 510, a spatial contrast image is obtained based on the ratio calculated at 508. This procedure 500 can be repeated for each background and lighting condition 551. Each repetition of procedure 500 produces another spatial contrast image, which can be stored at 510 for iteratively updating a previously stored spatial contrast image. Thus, a comprehensive contrast image (including the contrast from each illumination condition) can be built iteratively. In one embodiment, in each iteration, a clear contrast image - where the contrast setting is still set to 0 (compared to the iteratively built contrast image) - can be provided as the input for each illumination setting. If it is determined at 512 that the final image has been processed, then procedure 500 ends.

[0073] Figure 6 is a flow chart that shows the subroutine 600 for obtaining temporal contrast, which is also part of 410 in Figure 4 . The subroutine 600 receives the following as inputs: one or more backgrounds and illumination conditions 651 of a set; and a filter 655. At 602, each of the first and second digital images taken under the specified illumination and background conditions is obtained. At 604, the 0 images are filtered. In the example of Figure 6 , a low-pass kernel is used as the filter, but those skilled in the art know other filters that can be used. At 606, the unfiltered x image minus the filtered 0 image is calculated with the ratio of the filtered 0 image plus the unfiltered x image. At 608, a temporal contrast image is obtained based on the ratio calculated at 606. This program 600 can be repeated under different illumination conditions and / or different background conditions. Each repetition of the program 600 produces another temporal contrast image, which can be used at 608 to iteratively update the previously stored temporal contrast image. Similar to the establishment of the spatial contrast image, the temporal contrast image can be established iteratively, where a clear contrast image is provided as the input for each illumination condition. If it is determined at 610 that the final image has been processed, the program 600 ends.

[0074] To make a comprehensive or overall determination regarding contrast, the spatial and temporal contrast results can be further combined. The combination of spatial and temporal contrast is referred to herein as "mixed contrast" (MC). In one embodiment, the mixed contrast can be obtained from the spatial contrast (SC) image at time t 0 , the spatial contrast image at time t x , and the temporal contrast (TC) image obtained from the comparison of the 0 and x images according to the following equation:

[0075] (3)

[0076] Filter

[0077] To enhance image analysis, additional processes can be included in the subroutine 400 of Figure 4 . For example, the first digital image can be analyzed to find the targets that appear in the image at time t 0 . Since it is known that no bacteria have started to grow significantly at t 0 , it can be assumed that at time t 0Any objects found are merely dust, bubbles, artifacts, condensation, etc. that do not constitute colony candidates.

[0078] A filtering process can be applied to the captured images to subtract dust and other artifacts adhering to the imaging plate or lens. When considering transparent culture media (e.g., MacConkey agar, CLED agar, CHROMagar, etc.), some level of dust can be expected to be present in the captured images. The impact of dust on a given image can be determined at least in part by the specific lighting and background conditions under which the image is taken. For example, when using a white culture medium, reflection artifacts and dust are most easily observed when the medium is illuminated from above with a black background below. As another further example, when using a colored or black culture medium, artifacts and dust will be most easily observed when the medium is illuminated from above with a white background below. As a further example, in most any culture medium, regardless of the background, absorption light artifacts and dust are observed when the medium is illuminated from below. In any case, the management of dust and artifacts is a complex image processing challenge that can significantly affect the detection of microbial growth.

[0079] Dust and artifacts can be classified into two types: (A) those that can change position; and (B) those that cannot change position. Dust and artifacts can accumulate over time, meaning that the quantities of both type A and type B can change over time. Nevertheless, observations show that type A is more likely to change over time in terms of quantity. Of course, type A is also more prone to change, for example due to the plate being moved into or out of the imaging chamber.

[0080] Typically, type B is caused by artifacts associated with the plate itself such as ink dots (brand, lot number, and information printed under the plate), defects or frosted regions associated with the plastic mold injection points. Type B can also be caused by dust or bubbles that adhere to the top of the culture medium, are trapped within the culture medium, or are electrostatically adhered to the underside of the plate.

[0081] From the perspective of the observed imaging points, even the dust and artifacts of type A themselves hardly change position. However, due to the plastic of the plate and the culture medium acting as filters and lenses, the characteristics and positions of the observed type A artifacts may change slightly, depending on the culture medium color, culture medium level, and plastic. The dust and artifacts of type B also do not change position. However, in so far as the type B dust and artifacts are attached to the culture medium and the culture medium undergoes slight movement and drift over time (mainly due to slight drying over time in the incubator), the type B dust and artifacts can move with the culture medium. Therefore, the positions of the type B dust and artifacts are also at least somewhat prone to subtle changes.

[0082] In terms of contrast, the speck of dust of type A can be said to be at t 0 at the position "p 0 " in the spatial contrast image and at t x at the position "p x " in the spatial contrast image. Assuming that p 0 and p x are different positions, the dust or artifacts will also be present at two positions in the temporal contrast image (e.g., showing positive contrast at the p x position and negative contrast at the p 0 position). By comparison, the speck of dust of type B will be present in the common position of the two spatial contrast images at times t 0 and t x and will not be present in the temporal contrast image.

[0083] As explained above, in order to obtain a mixed contrast result, the spatial and temporal contrast images can be combined. The effects of dust and artifacts for both type A and B can be further eliminated from the mixed contrast result. In one embodiment, if a target (e.g., a CFU candidate) is identified in the mixed contrast result, it can be compared with the dust and artifacts in the target neighborhood N(x, y) detected in the spatial contrast result at time t 0 . Then, if a similar target is found in the spatial contrast result at time t 0 , the target identified in the mixed contrast result is labeled as a false positive of type A or type B. Even if the initial target is not labeled as a false positive of type A or type B, if it is found that the target does not change significantly in size over time, it may still be determined later as a false positive of type B. The false positives can be stored and later applied to subsequent images, for example, through a filter mask (e.g., a binary mask) further described below.

[0084] Another filtering process can be used to subtract the condensation formed on the plate (e.g., during the transfer from the refrigerator to the incubator at the start of the incubation run). In one example condensation filter, the plate is illuminated with bottom light so that less light passes through the positions with condensation compared to the positions without condensation. The optical density of the image can then be evaluated, and the regions of low optical density can be subtracted from the image.

[0085] Additionally or alternatively, an image mask can be constructed to discount the target from any analysis of the t 0 image and / or subsequent digital images. Figure 7 is a flowchart that shows the use of t 0The spatial contrast of an image generates an example program 700 for an image mask. In an example of program 700, the only input provided is the SHQI image 751 taken at time t 0 At 702, the spatial contrast of the t 0 image is determined. At 704, the spatial contrast information is used to collect statistical information about the pixels in the region of interest of the t 0 image, such as the mean and standard deviation (e.g., regarding brightness). At 706, the contrast threshold is adjusted to ensure that an appropriate number of pixels exceed the threshold. For example, if a high percentage of the pixels are not considered as the background of the image, the threshold can be increased. At step 708, the threshold is further adjusted based on the statistical information related to those pixels below the threshold. Finally, at 710, a binary mask is generated. The binary mask differentiates between multiple artifacts of non-effective pixels and other pixels that are considered effective. The binary mask can then be used at a subsequent time when there are potential colonies to be detected, and targets that occupy non-effective pixels are excluded from the candidate colonies.

[0086] The above filtering process can improve subroutine 400 by avoiding accidentally including dust, condensation, or other artifacts as targets and accelerating property characterization at 414 - since such characterization will only be performed on effective pixels.

[0087] Defining targets and labels

[0088] Another process that can be added Figure 4 to subroutine 400 is to assign labels to the targets identified at 412. The targets can be given specific labels such that pixels with the same label share certain characteristics. The labels can help differentiate the targets from other targets and / or the background during subsequent processes of subroutine 400. Figure 8 is a flowchart that shows an example program 800 for labeling the pixels of an image taken at time t x (or "t x image"). In an Figure 8 example, a binary mask 851 (e.g., the output of program 700), an uninitialized candidate mask 852 for the t x image, and a temporal contrast image 853 (e.g., the output of subroutine 600) are received as inputs. At 802, the candidate mask 852 is initialized. The initialization can include identifying the region of interest on the imaging plate and using the binary mask 851 to identify the "effective pixels" in the image taken at time t x . Effective pixels are the imaging pixels that are not ignored as candidate colonies and will be considered for labeling. At 804, the temporal contrast image 853 is used to collect information about the t xStatistics of valid pixels in the region of interest of the image, such as the mean and standard deviation (e.g., with respect to luminance). Then, at 806, the statistics of each of the temporal contrast image 853 and the binary mask 851 (which are preferably generated under similar illumination and background conditions) are combined to form a threshold contrast image. Using the threshold defined by the threshold contrast image, t x The "connex components" of the image are labeled at 808. The connex components effectively serve as labels indicating connections between adjacent pixels (or groupings within adjacent pixels), which in turn indicates that the pixel is part of the same object.

[0089] Once for t x the image has defined the connex components, each connex component can be analyzed individually (at 810) to verify its status as a single object. In Figure 8 an example, the statistical calculation of the pixels associated with the label is performed at 812. The calculation can utilize a histogram to determine the mean and / or standard deviation of the luminance or color of the pixels. At 814, it is determined whether the pixels satisfy a threshold area. If the threshold area is not satisfied, the operation proceeds to 830, where the label is updated. Updating the label can include keeping the analyzed component as one label, or splitting the component into two labels. In the case where the threshold area is not satisfied, the component remains as a single label. If the threshold area is satisfied, then at 816, the histogram is smoothed, and at 818, the peaks of the distributed labeled pixels are identified. The peaks can be further defined by having a minimum area - since peaks smaller than this minimum area can be ignored. At 820, the number of identified peaks is counted. If there is only one peak, the operation proceeds to 830, and the label is updated such that the component remains as one object. If there are more than one peak, then at 824, the threshold contrast image is used to further evaluate whether the contrast between the peaks is significant. Then the operation proceeds to 830, and the label is updated based on the multiple identified peaks such that the significant contrast results in the component are split into two, and otherwise remains as one.

[0090] Segmentation

[0091] Another process that can be included as part of the subroutine 400 is the segmentation process, which is used to separate the confluent colonies at time t x into separate objects. If at time t x the colonies have grown to the point where they overlap or touch each other, in order to evaluate the individual colonies within the region, it may be necessary to draw boundaries through the confluent area.

[0092] In some cases, when two adjacent colonies have different characteristics (e.g., different colors, different textures), the segmentation may simply involve an analysis of the characteristics of the confluent region. However, spatial and temporal contrast is not always sufficient to identify the boundary between colonies alone. Figure 9 is a flow chart that shows an example procedure 900 for separating such colonies into separate objects (e.g., using separate labels), or in other words, segmenting the colonies. Figure 9 The example procedure 900 uses a first digital image 951 taken at time t 0 and a second digital image taken at time t x as well as a t 0 image binary mask 953 (e.g., a mask generated by procedure 700) as inputs. At 902, a temporal contrast image is generated based on the t 0 and t x images 951 and 952. At 904, the temporal contrast image is segmented using the binary mask 953. At 906, labels are applied to the segmented parts of the image. At 908, the peaks or maxima of each label are identified. The maximum value of a given segmented part is typically the center point or centroid of the segmented part. At 910, for each label, the maximum value (e.g., of the analyzed label, of adjacent labels) is used to make further determinations regarding whether a given label is unique with respect to its neighbors or whether it should be combined with one or more adjacent labels. Once the labels are reduced to their unique components, the characterization of each label (e.g., steps 414 and 416 of procedure 400) can be performed at 912, and a global list of candidate colonies can be generated at 914.

[0093] A variety of factors, such as an inclusion factor, can be applied to determine whether the local maximum of a given label belongs to one colony or a different colony. The inclusion factor is a factor that indicates whether adjacent pixels are associated with adjacent objects. Such a factor can be used in the segmentation strategy to determine whether to split two local maxima in a given label into two separate objects or to merge them into a single object.

[0094] Figure 10 is a flow chart that shows such an example segmentation strategy. Procedure 1000 can be used as a Figure 9 subroutine for step 910 in Figure 10 As shown in Figure 10In an example, region "A" surrounds the maximum value 1051, and region "B" surrounds the maximum value 1052. For the purposes of the example equations below, it is assumed that the size of region A is greater than or equal to that of region B. In some cases, each region can be given an elliptical shape, which has a horizontal distance (xA, xB) along the horizontal axis of the region and a vertical distance (yA, yB) along the vertical axis of the region. To clarify Figure 10 the procedure Figure 11 provides an example diagram of regions A and B and their respective maximum values.

[0095] At 1004, for each local maximum 1051 and 1052, the distance from the maximum to the target edge is determined. In some cases, the determined distance is the average or median distance of the region assigned at 1002, hereinafter referred to as the distance map. The distance map of region A is hereinafter referred to as rA, and the distance map of region B is referred to as rB.

[0096] At 1006, based on the distance "d" between the two local maxima and the distances determined at 1004, the inclusion factor is calculated. In one embodiment, the inclusion factor is calculated using the following equation:

[0097] (4)

[0098] At 1008, it is determined whether the inclusion factor is less than a predetermined range, greater than a predetermined range, or within a predetermined range (e.g., between 0.5 and 1). If the inclusion factor is less than the predetermined range, it is determined that the maxima are associated with the same target. If it is greater than the predetermined range, it is determined that the maxima are associated with different targets.

[0099] For inclusion factors that fall within the range, it is not immediately clear whether the maxima belong to the same target or different targets, and more processing is required. The procedure 1000 then continues at 1010, where the convexity of the respective surrounding regions of the two maxima is calculated using the coordinates of a third region "C" at the position between the two maxima. In some cases, this region can be the weighted center of the two regions, such that the center point of region C is closer to the smaller region B than to the larger region A. For region C, the horizontal and vertical distances xC, yC, and the distance map H can also be calculated. For example, the convexity can be calculated according to the following equation using the above values and the distance d(A, C) between the center point of region C and the maximum value A:

[0100] (5)

[0101] (6)

[0102] (7)

[0103] (8) ΔH = H - (0.9R)

[0104] At 1012, determine whether the convexity value is greater than (more convex) a given threshold. For example, ΔH can be compared with a threshold of 0. If the convexity value is greater than the threshold, the maximum value is determined to be associated with a different target. Otherwise, at 1014, one or more parameters of region C are updated so that the size of region C increases. For example, the offset distance (distOffset) is updated based on ΔH. For example, the value of ΔH is limited between 0 and 1 (if ΔH is greater than 1, it is rounded to 1) and then added to the offset distance.

[0105] At 1016, determine whether the size of region C meets or exceeds a threshold. If the threshold is met or exceeded, the maximum value is determined to be associated with the same target. In other words, if the difference between regions A and B is so uncertain that region C increases until it starts to cover regions A and B, this is a good indication that the maximum values 1051 and 1052 should belong to the same target. In the above example, this can be indicated by the offset distance meeting or exceeding the distance d between the maximum values. Otherwise, the operation returns to 1010, and the convexity of regions A and B is recalculated based on the updated parameter(s) of region C.

[0106] Once the association of each maximum value is determined, the determined association can be stored, for example, in a matrix (also called an association matrix). The stored information can be used to reduce the full list of maximum values to a final list of candidate targets. For example, in the case of an association matrix, a master list can be generated from the full list of maximum values, and then each maximum value can be iteratively checked, and if the associated maximum value is still on the list, it is removed from the master list.

[0107] At Figure 9 and 10 In the example of x (and, therefore, the earliest time at which program 900 can be executed), it can be only a few hours after entering the incubation process. Such a time is considered too early to identify fully formed colonies, but may be sufficient to generate a segmented image. The segmented image can optionally be applied to future images taken at a subsequent time. For example, the boundaries between colonies can be drawn to predict the expected growth of the colonies. Then, in the case of confluence between colonies, the boundaries can be used to separate the confluent colonies.

[0108] Using the time t 0 Analysis of the last two or more images

[0109] Although the above processes and programs only need to be performed at the time t 0A subsequent image is taken (e.g., a first digital image at time t 0 and a second digital image at time t x ), and other procedures require taking at least a second image after time t 0 . For example, if the image at time t x is found to include confluent colonies, another image taken at time t n (where 0 < n < x) may be used to identify and separate individual colonies.

[0110] For example, if t 0 = 0 hours of incubation (when no growth occurs), and t x = 24 hours of incubation (when so much growth has occurred that the colonies are now confluent), an image at time t n = 12 hours (when the colonies will start to grow but are not yet confluent) will reveal the presence of individual colonies. Colony growth is then predicted based on the image at time t n to evaluate the boundaries between the confluent colonies at time t x . In this regard, the image at time t n can help distinguish between rapidly growing colonies and slowly growing colonies. Those skilled in the art should recognize that as the number of images taken between time t 0 and time t x increases, the growth rate of the colonies will be predicted more accurately.

[0111] In one application of the foregoing concept, the image taken at time t n (or more generally, the images taken between time t 0 and t x ) can be used to identify colony seeds, which are targets of colonies suspected to grow over time, and associate the seeds with corresponding masks and labels. Each seed receives a unique label and the label will be stored together with other features (e.g., location, morphology, and histogram based on an image generated from the SHQI image: red channel, green channel, blue channel, illuminance, chromaticity, hue, or composite image) as well as properties (e.g., isolated / non-isolated status, other information for predicting time-dependent proliferation). To extract plate global metrics, some stored features (e.g., histograms) can also be calculated at the plate level rather than being attributed to specific seeds. The features stored for the seeds can then be used to perform colony extraction at time t x and be provided as input to a classifier for training and / or testing.

[0112] Using at t 0Tracking the growth rate of multiple images taken later can also be used to detect dust, artifacts, or other foreign objects that appear on the plate or in the imaging lens during the middle of the workflow process. For example, if a dust particle lands on the imaging lens after t 0 but before t n , the spot produced by the particle may initially be interpreted as a growing colony because it is not visible at time t 0 . However, subsequent imaging reveals that the size of the spot has not changed, then it can be determined that the spot is not growing and thus is not a colony.

[0113] In addition to tracking growth rate and segmentation, other aspects of the colony can be tracked with the help of additional images between t 0 and t x . In the case of subtle morphological changes that develop slowly over time in the colony, those subtle changes can be identified more quickly by capturing more images. In some cases, in addition to or instead of measuring growth along the normal x and y axes, growth can also be measured along the z axis. For example, it is known that Streptococcus pneumoniae slowly forms a sunken center when growing in blood agar, but the sunken center is usually not visible until the second day of analysis. By looking at the time course of bacterial growth, the initially sunken center can be detected, and the bacteria can be identified much earlier than if the technician had to wait for the center to sink completely.

[0114] In other cases, it is known that the colony changes color over time. Therefore, imaging the colony with a first color (e.g., red) at a time after t 0 and then imaging the colony with a second color (e.g., green) at a subsequent time can be used to determine the identity of the bacteria growing in the colony. The color change can be measured as a vector or path through a color space (e.g., RGB, CMYK, etc.). Changes in other chromaticity characteristics of the colony can also be measured similarly.

[0115] Target features

[0116] As discussed above in conjunction with Figure 4 , the features of the target on the imaging plate can be characterized as part of the image analysis performed on the imaging plate. The characterized features can include both static features (related to a single image) and dynamic images (related to multiple images).

[0117] Static features are intended to reflect the target attributes and / or the surrounding background at a given time. Static features include the following:

[0118] (i) Center of gravity: This is a static feature that provides the center of gravity of the imaging target in a coordinate space (e.g., x-y, polar). The center of gravity of the target, like the polar coordinates of the target, provides invariance of the feature set under given lighting and background conditions. The center of gravity can be obtained by first determining the weighted centroid for all colonies in the image (where M is the binary mask of all detected colonies). The weighted centroid can be determined based on the assumption that each pixel in the image has an equal value. The center of gravity of a given colony can then be described in the x-y coordinate system by the following equation (where E = {p | p ∈ M} (E is the binary mask of the current colony), the x coordinate ranges from [0, image width], the y coordinate ranges from [0, image height], and each pixel is a unit):

[0119] (9)

[0120] (ii) Polar coordinates: This is also a static feature and can be used to further characterize the position on the imaging plate, such as the center of gravity. Generally speaking, the polar coordinates are measured along the radial axis (d) and the angular axis (θ), where the coordinates of the center of the plate are [0,0]. The coordinates d and θ of Igv (x,y) are given by the following equations (where the unit of d is millimeters and the unit of θ is degrees) (where k is the pixel density of the pixels corresponding to millimeters, and "barcode" is a landmark feature of the imaging plate used to ensure alignment of the plate with previous and / or future images):

[0121] (10) d = k × distance(igv (x,y) ,0 (x,y) )

[0122] (11) θ = angle(barcode, O (x,y) ,igv (x,y) )

[0123] (iii) Image vector: The two-dimensional polar coordinates can be transformed into a one-dimensional image vector. This image vector can characterize the intensity of the pixels in the image as a function of the radial axis (usually, the center of the colony has the highest intensity) and / or as a function of the angular axis. In many cases, the image vector can be more precise in classifying similarities / differences in the imaging target.

[0124] (iv) Morphological features: These describe the shape and size of a given target.

[0125] (a) Area: This is a morphological feature and can be determined based on the number of pixels in the imaging target (also known as "blob"), without counting the holes in the target. When the pixel density is available, the area can be measured in physical dimensions (e.g., mm 2)。Conversely, when the pixel density is not available, the total number of pixels can indicate the size, and the pixel density (k) can be set equal to 1. In one embodiment, the area is calculated using the following equation:

[0126] (12)A = k 2 ×∑ p∈E 1

[0127] (b) Perimeter: The perimeter of the target is also a morphological feature and can be determined by measuring the edges of the target and summing up the total length of the edges (e.g., a single pixel with an area of 1 square unit has a perimeter of 4 units). Like the area, the length can be measured in pixel units (e.g., when k is not available) or physical length (e.g., when k is available). In some cases, the perimeter may also include the perimeter of any holes in the target. Additionally, the stair-step effect (resulting from digitizing diagonal edges as stepped boxes) can be compensated for by counting interior angles as rather than 2. In one embodiment, the perimeter is determined using the following equation:

[0128] (13)P = k×∑ p∈E q(n p )

[0129] (14)

[0130] (15)If:

[0131] {∑(t∈M, l∈M, r∈M, b∈M) = 2, (l∈M≠r∈M), (t∈M≠b∈M)}

[0132] (p is internal and p is a corner)

[0133] Then: q(n p ) = √2

[0134] Otherwise: q(n p ) = 4 - ∑(t∈M, l∈M, r∈M, b∈M)

[0135] (c) Circularity: The circularity of the target is also a morphological feature and can be determined based on the combination of the area and the perimeter. In one embodiment, the circularity is calculated using the following equation:

[0136] (16)

[0137] (d) Coefficient of Variation of Radius (RCV): This is also a morphological feature and is used to obtain the average radius of the target in all N directions or angles θ extending from the center of gravity The ratio to the standard deviation σR of the radius indicates the variation of the target radius. In one embodiment, this value can be calculated using the following equation:

[0138] (17)

[0139] (18)

[0140] (19)

[0141] (v) Contextual feature, which describes the topographical relationship of the neighborhood of the target under the surveillance of other detected targets and the edge of the plate wall. For example, in the case of an imaged colony, a contextual feature of the colony can be whether the colony is free, has limited free space, or competes with other surrounding colonies for access to resources. Such features tend to help classify colonies growing in the same perceptual environment and / or distinguish colonies growing in different environments.

[0142] (a) Region of influence: This is a contextual feature that considers the space between the target and the surrounding targets and predicts the region that the target under analysis can expand to occupy (without other, different targets first expanding to occupy the same region). The region of influence can be represented in the form of a figure, such as the figure shown in Figure 12 , which shows the region of influence (shaded part) based on the distance d between colony 1201 and its surrounding colonies such as 1205. In one embodiment, the distance (D NC ) from the edge of the target to the edge of the region of influence can be characterized using the following equation:

[0143] (20)

[0144] (b) Distance to the plate wall: This is a contextual feature that calculates the distance (D PW ) from the edge of the target to the nearest plate wall.

[0145] In one embodiment, this distance can be characterized using the following equation:

[0146] (21)

[0147] (c) Isolation factor: This is a contextual feature that characterizes the relative isolation of a given target based on the size of the target and the distance to the nearest edge (e.g., the nearest edge of another target, the plate wall). Figure 13A -Figure C illustrates aspects of the isolation factor. Figure 13A Figure illustrates an example where the nearest edge is the distance d from the colony to the plate wall. Figure 13B and13C Illustrates an example where the nearest edge belongs to another colony. In such a case, a circle is drawn around the colony to be analyzed with the colony as the center, and then expanded (first small, such as Figure 13B , and then larger, such as Figure 13C ) until the circle touches the adjacent colony. In the Figure 13A -C embodiment, the isolation factor (IF) can be characterized using the following equation:

[0148] (22)

[0149] (d) Peripheral occupancy ratio: This is a situation characteristic that characterizes the area portion of the boundary influence region (V) of the plate within a given distance d for a given target. In one embodiment, the peripheral occupancy ratio (OR) can be characterized using the following equation (where for this equation, E = {p|p ∈ V, dist(p, igv (x,y) ) < d}):

[0150] (23)

[0151] (e) Relative peripheral occupancy ratio: In some examples, the average radius of the target can be multiplied by a predetermined factor to obtain a given distance d. The result is the relative peripheral occupancy ratio (RNOR), and for a given factor x, it can be derived using the following equation:

[0152] (24) RNOR(x) = NOR(d)

[0153] (vi) Spectral characteristics, which describe the optical properties of a given target. Color (red, green, and blue channels; hue, illuminance, and chromaticity, or any other color space transformation), texture, and contrast (over time and / or across space) are examples of such characteristics. Spectral characteristics can be derived from images captured at different time points and / or under different lighting conditions during the incubation process using the colony mask, and can be further associated with the influence region of a given colony.

[0154] (a) Channel image: This is a spectral characteristic where a specific color channel (e.g., red (R), green (G), and blue (B)) is used for spectral resolution of the image.

[0155] (b) Brightness: This is also a spectral characteristic, which is used to characterize the brightness of an image using the RGB channels as input.

[0156] (c) Hue: This is a spectral characteristic where the area of the image is characterized as exhibiting similarity to a perceived color (e.g., red, yellow, green, blue) or a combination thereof. Hue (H2) is typically characterized using the following equation:

[0157] (25)H 2 = atan2(β,α)

[0158] (26)

[0159] (27)

[0160] (d) Chroma: This is a spectral characteristic used to characterize the colorfulness of an image region relative to its brightness - if the region were similarly illuminated as white. Chroma (C 2 ) is typically characterized using the following equation:

[0161] (28)

[0162] (e) Radial dispersion: Analyzing the radial dispersion of hue and chroma contrast enables the differentiation of α, β, and γ hemolysis.

[0163] (f) Maximum contrast: This feature characterizes the resolution of an image by calculating the maximum value of the measured average contrast of pixels at a given radius r from the center point of the image (e.g., the center of the imaged colony). This feature can be used to describe the perceptible differences between images taken at times t 0 and t x based on the growth induced by the analysis target. The maximum contrast can be characterized as follows:

[0164] (29) Maximum contrast r = MAX(average contrast r ) 目标

[0165] (vii) Background features, which describe changes in the culture medium in the neighborhood of the analysis target. For example, in the case of imaging a colony, this change can be caused by the growth of microorganisms around the colony (e.g., signs of hemolysis, changes in pH, or specific enzymatic reactions).

[0166] Dynamic features aim to reflect changes in the target properties and / or the surrounding environment over time. Time series processing allows static features to be correlated over time. The discrete first and second derivatives of these features provide the instantaneous "speed" and "acceleration" (or level of stability or deceleration) of the features to be characterized as they change over time. Examples of dynamic features include the following:

[0167] (i) Time series processing for tracking the changes of the above static features over time. Each feature measured at a given incubation time can be referenced according to its relative incubation time to allow these features to be associated with the features measured at subsequent incubation times. The time series of images can be used to detect the emergence and growth of targets such as CFU over time, as described above. Based on the ongoing analysis of the previously captured target images, the imaging time points can be preset or defined through an automated process. At each time point, the image can be in a given acquisition configuration, either a single acquisition configuration for the entire series or an image of the entire series captured from multiple acquisition configurations.

[0168] (ii) Discrete first and second derivatives of the above features for providing the instantaneous velocity and acceleration (or steady level or deceleration) of these features over time (e.g., tracking the growth rate, as described above):

[0169] (a) Velocity: The first derivative of the feature with respect to time. Based on the following equation, the velocity (V) of feature x can be characterized in (x units) / hour, where Δt is the time span in hours:

[0170] (30)

[0171] (31)

[0172] (32)

[0173] (b) Acceleration: The second derivative of the feature with respect to time, which is also the first derivative of velocity. The acceleration (A) can be characterized based on the following equation:

[0174] (33)

[0175] The above image features can be measured from the target or the situation of the target, and are aimed at capturing the specificities of organisms growing under various culture media and incubation conditions. The listed features are not meant to be exhaustive, and those skilled in the art can improve, expand, or limit this feature set based on various image processing-based features known in the art.

[0176] Image features can be collected for each pixel, group of pixels, target, or group of targets in the image. To more generally characterize the region or even the entire image, the distribution of the collected features can be constructed as a histogram. For analyzing or otherwise processing the incoming image feature data, the histogram itself can rely on several statistical features.

[0177] Statistical histogram features can include the following:

[0178] (i) Minimum value: The smallest numerical value of the distribution captured within the histogram. This can be characterized by the following relationship:

[0179] (34)

[0180] (ii) Maximum value: The largest numerical value of the distribution captured within the histogram. This can be characterized by the following relationship:

[0181] (35)

[0182] (iii) Sum: The sum of all individual values captured within the histogram. The sum can be defined by the following relationship:

[0183] (36)

[0184] (iv) Mean value: The arithmetic mean, or average. This is the sum of all scores divided by the number of scores (N), according to the following relationship:

[0185] (37)

[0186] (v) First quartile (Q1): The score at the 25th percentile of the distribution. 25% of the scores are below Q1, and 75% of the scores are above Q1. This can be described by the following relationship:

[0187] (38)

[0188] (vi) Median (Q2): The score at the 50th percentile of the distribution. 50% of the scores are below the median, and 50% of the scores are above the median. The median is less sensitive to extreme scores compared to the mean, and this generally makes it a better measure than the mean for highly skewed distributions. This can be described by the following relationship:

[0189] (39)

[0190] (vii) Third quartile (Q3): The score at the 75th percentile of the distribution. 75% of the scores are below Q3 and 25% of the scores are above Q3. This is described by the following relationship:

[0191] (40)

[0192] (viii) Mode: The score that occurs most frequently in a distribution. This is used as a measure of central tendency. The advantage of the mode as a measure of central tendency is that its meaning is obvious. Also, it is the only measure of central tendency that can be used with nominal data. The mode is highly subject to sample fluctuations and is therefore not usually used as the sole measure of central tendency. Also, many distributions have more than one mode. These distributions are called "multimodal". The mode can be described by the following relationship:

[0193] (41)

[0194] (ix) Trimean: The score calculated by adding the 25th percentile, twice the 50th percentile (median), and the 75th percentile, and dividing by 4. The trimean is almost as resistant to extreme scores as the median, and in skewed distributions, is less subject to sampling fluctuations compared to the arithmetic mean. However, for a normal distribution, it is usually less efficient than the mean. The trimean can be described according to the following relationship:

[0195] (42)

[0196] (x) Trimmed mean: The score calculated by discarding a certain percentage of the lowest and highest scores and then calculating the mean of the remaining scores. For example, the trimmed 50% mean is calculated by discarding the lower and higher 25% of the scores and calculating the mean of the remaining scores. A further example is that the median is the trimmed 100% mean and the arithmetic mean is the trimmed 0% mean. Compared to the arithmetic mean, the trimmed mean is usually less affected by extreme scores. Therefore, for skewed distributions, it is less affected by sampling fluctuations compared to the mean. For a normal distribution, it is usually less efficient. As an example, the trimmed 50% mean is described by the following relationship:

[0197] (43)

[0198] (xi) Range: The difference between the smallest and largest numerical values. The range can be a measure of the spread available. However, since it is based on only two values, it is sensitive to extreme scores. Due to this sensitivity, the range is not usually used as the sole measure of spread, but it can still be meaningful if used as a complement to other measures of spread such as the standard deviation or the semi-interquartile range.

[0199] (xii) Semi - interquartile range: An extended measure calculated as half the difference between the 75th percentile (Q3) and the 25th percentile (Q1). Since half of the scores in a distribution lie between Q3 and Q1, the semi - interquartile range is half the distance required to cover that half of the scores. In a symmetric distribution, the spacing from one semi - interquartile range below the median to one semi - interquartile range above the median will contain half of the scores. However, this is not the case for skewed distributions. Unlike the range, the semi - interquartile range is generally substantially unaffected by extreme scores. However, in a normal distribution, it is more vulnerable to sampling fluctuations than the standard deviation and is therefore not often used for data that is approximately normally distributed. The semi - interquartile range is defined according to the following relationship:

[0200] (44)

[0201] (xiii) Variance: A measure of the spread of a distribution. The variance is calculated by taking the mean square deviation of each number from its mean according to the following relationship:

[0202] (45)

[0203] (xiv) Standard deviation: A function of the variance that measures how widely the values in a distribution are dispersed from the mean. The standard deviation is the square root of the variance. Although it is generally less sensitive to extreme scores compared to the range, the standard deviation is usually more sensitive than the semi - interquartile range. Therefore, when there is a possibility of extreme scores, the semi - interquartile range can be used to supplement the standard deviation.

[0204] (xv) Skewness: A measure of the asymmetry of a distribution about its mean. If one of the tails of the distribution is longer than the other, the distribution is skewed. Positive skewness indicates a distribution with an asymmetric tail extending towards more positive values (greater than the mean). Negative skewness indicates a distribution with an asymmetric tail extending towards more negative values (less than the mean). Skewness can be calculated according to the following relationship:

[0205] (46)

[0206] (xvi) Kurtosis: A measure of the steepness or flatness of a distribution (or relative peak width) compared to a normal distribution. Positive kurtosis indicates a relatively peaked distribution. Negative kurtosis indicates a relatively flat distribution. Kurtosis is based on the size of the tails of the distribution and can be determined by the following relationship:

[0207] (47)

[0208] The above statistical methods can be used to analyze the spatial distribution of gray values, which is carried out by calculating local features at each point in the image and obtaining a set of statistical results from the distribution of local features. Using these statistical methods, the texture of the analyzed region can be described and statically defined.

[0209] Texture can be characterized using texture descriptors. Texture descriptors can be calculated over a given region of the image (discussed in more detail below). A commonly used texture method is the co-occurrence method, proposed by Haralick, R. et al., "Textural Features for Image Classification", IEEE Transactions of System, Man and Cybernetics, Vol. 3, pp. 610-621 (1973), which is incorporated herein by reference. In this method, the relative frequencies of pairs of gray levels of pixels separated by a distance d in the θ direction are combined to form a relative displacement vector (d, θ). The relative displacement vector is calculated and stored in a matrix called the gray-level co-occurrence matrix (GLCM). This matrix is used to extract texture features of second-order statistics. Haralick proposed 14 different features to describe the two-dimensional probability density function p ij , four of which are more commonly used than the others:

[0210] Texture can be characterized using texture descriptors. Texture descriptors can be calculated over a given region of the image (discussed in more detail below). A commonly used texture method is the co-occurrence method, proposed by Haralick, R. et al., "Textural Features for Image Classification", IEEE Transactions of System, Man and Cybernetics, Vol. 3, pp. 610-621 (1973), which is incorporated herein by reference. In this method, the relative frequencies of pairs of gray levels of pixels separated by a distance d in the θ direction are combined to form a relative displacement vector (d, θ). The relative displacement vector is calculated and stored in a matrix called the gray-level co-occurrence matrix (GLCM). This matrix is used to extract texture features of second-order statistics. Haralick proposed 14 different features to describe the two-dimensional probability density function p ij , four of which are more commonly used than the others:

[0211] (i) Angular second moment (ASM) is calculated by the following formula:

[0212] (48)

[0213] (ii) Contrast (Con) is calculated by the following formula:

[0214] (49)

[0215] (iii) The correlation (Cor) is calculated by the following formula (where σ x and σ y are the standard deviations of the corresponding distributions):

[0216] (50)

[0217] (iv) The entropy (Ent) is calculated by the following formula:

[0218] (51)

[0219] These four features are also listed in Strand, J., et al., "Local Frequency Features for Texture Classification", Pattern Recognition, Vol. 27, No. 10, pp 1397 - 1406 (1994) [Strand94], which is hereby incorporated by reference.

[0220] For a given image, the image regions where the above features are evaluated can be defined by a mask (e.g., a colony mask) or by the influence region extending beyond the mask. The influence region is defined.

[0221] Figure 14 Several possible regions are shown. Region 1410 is a colony mask that extends only as far as the colony itself. Region 1420 is the influence region (defined by the edges of the image or the plate). For further illustration, pixel 1430 is a pixel within region 1420 but outside region 1410. In other words, the colony represented by region 1410 is expected to expand into pixel 1630, but has not yet. Pixel 1440 is a pixel outside both regions 1410 and 1420. In other words, the colony not only did not occupy pixel 1410 at the time of imaging, but is also not expected to be occupied by the colony at any time in the future (in this case, it has been occupied by a different colony).

[0222] As described above, using the colony masks at different time points during the incubation process and their associated influence regions, multiple histograms can be generated that depict different aspects of the colonies and their influence on the local surrounding growth medium. The colony masks and influence regions themselves can be adjusted over time, for example as the colonies grow. For example, Figure 15A -C illustrates how the growth of the colonies can be used to adjust the colony masks over time. Figure 15A is part of a blood culture grown in agar for 24 hours. Figure 15B is the same as the previous one at time t 0The contrast image of the same culture, as compared to the captured image. Figure 15C Is a grayscale image depicting growth at 9 hours (lightest), 12 hours (medium), and 24 hours (darkest). Figure 15C Each of the shades in Figure 15C can be used to design different masks for the colonies. Optionally or additionally, as growth occurs, the masks can be separated according to their respective Affected areas.

[0223] Any one feature or combination of features from the above list of features can be used as a feature set for capturing the specificity of organisms growing on various media of an imaging plate under different incubation conditions. This list is not meant to be exhaustive, and any person skilled in the art can modify, expand, or limit these feature sets based on the intended objectives to be imaged and the various image processing-based features known in the art. Thus, the above example features are provided by way of illustration and not limitation.

[0224] Those skilled in the art can be aware of other measurements and methods to determine the target shape and features, and the above examples are provided by way of illustration and not limitation.

[0225] Contrast establishment

[0226] It is often difficult to initially predict which image in a series of images will provide value for growth detection, counting, or identification. This is partly because image contrast varies for different colony-forming units (CFUs) and in different media. In a given image of several colonies, one colony may have a highly desirable contrast with the background, while another colony may not have sufficient contrast with the background for growth detection. This also makes it difficult to use a single method to identify colonies in a medium.

[0227] Therefore, it is desirable to establish contrast from all available materials, spatially (spatial differences) and temporally (temporal differences under the same imaging conditions), and by using different imaging conditions (e.g., red, green, and blue channels, light and dark backgrounds, spectral images, or any other color space transformation). It is also desirable to collect contrast from multiple available sources to provide standardized images as input to algorithms for detecting colonies.

[0228] Image data can be defined based on any number of factors. For example, image data can be defined as specific time points and / or specific information sought (e.g., spatial image information may not require as many time points as temporal image information). Illumination configurations and color spaces can also be selected to achieve specific contrast objectives. To detect targets of a desired size (or within a target range of sizes), the spatial frequency can also be varied.

[0229] To detect discrete objects, the contrast can be set to the absolute value on [0,1], or [-1,1] with signs. The scale and offset of the contrast output can also be specified (for example, for an 8-bit image with signed contrast, the offset can be 127.5 and the scale can be 127.5). In instances where the contrast is set to an extreme value, the absolute offset can be set to 0 and the scale to 256.

[0230] Spatial contrast can be used to detect discrete objects on a uniform background. The spatial contrast at position (x,y) within a distance r on image I can be provided using an equation for automated evaluation. In one embodiment, where the distance r is defined as greater than or equal to a distance, and a contrast operator K is used to control the contrast setting, the following equation is applied:

[0231] (52)

[0232] (53)

[0233] Temporal contrast can be used to detect moving objects or objects that change over time (such as CFUs that appear and / or expand on an imaging plate). The temporal contrast at position (x,y) on image I between times t 0 and t x can be provided using a formula for automated evaluation. In one embodiment, the following equation is applied:

[0234] (54)(54)

[0235] Spatial contrast collection can be implemented in an automated manner according to a pre-programmed sequence by generating multiple SHQI images of the plate. Multiple images can be generated at a given incubation time for further colony detection studies. In one embodiment, where, according to the following equation, at a given time, at several different radii (R min to R max ) from the detected colony, for several configurations (from CFG 1 to CFG N ), image data (specifically, the vector "vect" for providing contrast input to the contrast collection operator) is collected:

[0236] (55)

[0237] If for a given image I (x,y), if the SNR is known (e.g., when the SHQI imaging is the source), then when the following equation is satisfied, the configuration in which the SNR-weighted contrast is maximized can be identified as the best configuration (Best CFG):

[0238] (56) is the maximum value, exceeding:

[0239] (57)

[0240] The contrast operator K further benefits from this known SNR information, and the above equation becomes:

[0241] (58)

[0242] Temporal contrast collection can also be implemented in an automated manner according to a pre-programmed sequence by generating multiple SHQI images of the plate. Multiple images can be generated at multiple incubation times, at least one of which is t 0 , for further colony detection studies. In one embodiment, according to the following equation, at time t 0 and one or more subsequent incubation times up to t x , image data is collected for several configurations:

[0243] (59)

[0244] In the above example, the vector can be the vector between two time points (e.g., t 0 and t x ), which is based on the difference in the images at those two times. However, in other applications, including additional times between t 0 and t x , vectors can be drawn at as many points as the times at which the images are taken. Mathematically, there is no limit to the number of points that can be included in the vector.

[0245] Similar to spatial contrast, if for a given image I (x,y) , the SNR is known, then when the following equation is satisfied, the configuration in which the SNR-weighted contrast is maximized can be identified as the best configuration (Best CFG):

[0246] (60) is the maximum value, exceeding

[0247] (61)

[0248] The contrast operator K further benefits from this known SNR information, and the above equation becomes:

[0249] (62)

[0250] In the above example, the maximum operator can be replaced by any other statistical operator, such as a percentile (e.g., Q1, median, Q3, or any other percentile) or a weighted sum. The weighted values can be from previous work extracted from the training database, thereby opening the field of supervised contrast extraction to neural networks. Additionally, multiple algorithms can be used, and the results of the multiple algorithms are further combined using another operator such as the maximum operator.

[0251] Image alignment

[0252] When multiple images are taken over time, in order to obtain an effective time estimate from them, the images need to be aligned very precisely. Such alignment can be achieved by means of a mechanical alignment device and / or algorithms (e.g., image tracking, image matching). The knowledge in this field of these schemes and algorithms for achieving this goal is already known.

[0253] For example, in the case where multiple images of a target on a plate are collected, the coordinates of the target position can be determined. Based on these coordinates, the image data of the target collected at a subsequent time can then be associated with the previous image data and subsequently used to determine the change of the target over time.

[0254] In order to utilize images quickly and valuably (e.g., when used as input to a classifier), it is important to store the images in a spatial reference frame to maximize their invariance. Since the basic shape descriptor of a colony is generally circular, a polar coordinate system can be used to store colony images. When a colony is first detected, the centroid of the colony can be identified as the center of the colony position. This center point can later be used as the initial center for the polar coordinate transformation of each subsequent image of the colony. Figure 16A An enlarged portion of an imaging plate with a center point "O" is shown. Two rays "A" and "B" extending from point "O" are shown overlaid on the image (for clarity purposes). Each ray intersects with a respective colony (circled). Figure 16A The circled colonies in Figure 16B are shown in more detail in images 1611 and 1612 of Figure 16B In Figure 16A , image 1611 (the colony intersecting ray "A") is reoriented within image 1613 ("A'") such that the radial axis of image 1613 is aligned with the radial axis of image 1612, so that the leftmost part of the reoriented image is closest to the point "O" of Figure 16A and the rightmost part of the reoriented image is farthest from point "O". This polar coordinate reorientation allows for easier analysis of colonies in different orientations of the imaging plate (taking into account factors such as illumination).

[0255] InFigure 16C In, yes Figure 16B Each of the images 1611, 1612 and 1613 is polar transformed. In the polar transformed images 1621, 1622 and 1623, the radial axis (extending from the center of each respective imaged colony) of the respective reoriented images 1611, 1612 and 1613 is at Figure 16C The angular axes (of the respective colony) are drawn from left to right in the image and the angular axes (of the respective colony) are drawn from top to bottom.

[0256] For each polar coordinate image, a summary set of one-dimensional vectors may be generated along the radial and / or angular axes using, for example, shape features and / or histogram features (e.g., the mean and / or standard deviation of the color or intensity of the objects). Even when shape and histogram features are generally invariant when rotation is considered, it is possible that some texture features show significant changes when rotated; thus, invariance cannot be guaranteed. Therefore, it is significantly beneficial to present each colony image from the same viewpoint or angle of illumination, because object texture differences can then be used to distinguish one from another. Since illumination conditions generally show changes with angular position about the imaging center of the plate, rays passing through the colony and the center of the plate (at Figure 16B ) shown as a line in each of the images 1611, 1612 and 1613 may serve as the origin (θ) of each image polar coordinate transformation.

[0257] Further alignment challenges arise from the fact that the plate culture medium is not absolutely frozen and rigid, and can therefore shift slightly from one shot to the next. Therefore, it cannot be absolutely assumed that the area of ​​the plate at certain coordinates of an image taken at one time will necessarily align perfectly with the area of ​​the plate at the same coordinates taken at a later time. In other words, slight deformations of the culture medium can lead to small uncertainties with respect to precisely matching a given pixel with a corresponding pixel captured at a different time point during the incubation process.

[0258] To take this uncertainty into account, the time t a The intensity value of a given pixel Can be compared with different time points t b The closest intensity value within the local neighborhood N(x,y) of the pixel Selection of the local neighborhood may include determining one or more errors or inaccuracies in the repositioning of the imaged plate from one time to the next (e.g., positional accuracy errors due to imperfect repositioning of the imaged medium, parallax errors due to unknown height of the imaged medium from one time to the next). In one example, it has been observed that setting "x" and "y" within the range of about 3 to about 7 pixels is suitable for images with a resolution of about 50 microns per pixel.

[0259] Any person skilled in the art will recognize an effective solution for generating two t b images from a source image: the first image corresponds to the t b gray-scale dilation (referred to as b ) of t and the second image corresponds to the t gray-scale erosion (referred to as b ) of t, both having a kernel size that matches the distance uncertainty d of the repositioning. )

[0260] If then the contrast is 0, otherwise the contrast is estimated from the closest values of and in the following equation:

[0261] (63)

[0262] (64) If

[0263] (65) If

[0264] Improvement of SNR

[0265] Under typical illumination conditions, photon shot noise (statistical variation in the arrival rate of incident photons on the sensor) limits the SNR of the detection system. Modern sensors have sufficient capacity, which is approximately 1,700 to approximately 1,900 electrons per effective square micron. Therefore, when imaging a target on the imaging plate, the main concern is not the number of pixels used to image the target, but the area covered by the target within the sensor space. Increasing the area of the sensor improves the SNR for imaging the target.

[0266] The image quality can be improved by capturing images using such illumination conditions where photon noise controls the SNR without saturating the sensor (the maximum number of photons that can be recorded per pixel per frame). To maximize the SNR, image averaging techniques are typically used. Since the SNR in the dark regions is much lower than that in the bright regions, these techniques are used to process images with significant brightness (or color) differences, as shown in the following formula:

[0267] (66)

[0268] Where I is the average current generated by the electron flow at the sensor. Color is perceived due to the differences in absorption / reflection of matter and light across the electromagnetic spectrum, and the confidence in the captured color will depend on the system's ability to record intensities with a high SNR. Image sensors (e.g., CCD sensors, CMOS sensors, etc.) are well known to those skilled in the art and will not be described in detail herein.

[0269] To overcome classical SNR imaging limitations, the imaging system can analyze the imaging plate during image acquisition and adjust the illumination conditions and exposure time in real time based on that analysis. This process is described in PCT Publication No. WO2015 / 114121 - which is incorporated by reference and is generally referred to as supervised high-quality imaging (SHQI). The system can also customize the imaging conditions for different brightness regions of the plate within different color channels.

[0270] For a given pixel x, y of an image, the SNR information of the pixel acquired during the current frame N can be combined with the SNR information of the same pixel acquired during a previous or subsequent frame (e.g., N - 1, N + 1). As an example, the combined SNR is determined by the following formula:

[0271] (67)

[0272] After updating the image data with the new acquisition, the acquisition system is able to predict the optimal next acquisition time, which will maximize the SNR according to environmental constraints (e.g., the minimum required SNR for each pixel within the region of interest). For example, 5 images captured under average non-saturated conditions will promote an increase in the SNR of the dark regions (10% of the maximum intensity). When combining the information of two images captured under bright and dark conditions, only in two acquisitions, the optimal illumination will promote an increase in the SNR of the dark regions.

[0273] Image Modeling

[0274] In some cases, when calculating the spatial or temporal contrast between the pixels of one or more images, the pixel information of a given image may be unavailable or may be degraded. For example, if an image of the plate was not captured within the time before bacterial growth (e.g., the plate was not imaged at time t 0 or within a short time thereafter), then the unavailability may occur. For example, when at time t 0Capture of an image, but when the pixels of the captured image do not accurately reflect the imaging plate before bacterial growth, attenuation of signal information may occur. Such inaccuracies may be caused by temporal artifacts that do not reoccur in subsequent time series images of the plate (e.g., condensation that temporarily forms on the bottom of the plate due to thermal shock when the plate is first placed in the incubator).

[0275] In such cases, unavailable or attenuated images (or certain pixels in the image) can be replaced or enhanced using a model image of the plate. The model image can provide pixel information that reflects how the plate is expected to look at a particular time in the unavailable or attenuated image. In the case of the model image at time t 0 the model can be a plain or standard image of the plate and can be mathematically constructed using three-dimensional imaging / modeling techniques. The model can include each of physical design parameters (e.g., diameter, height, dividers for holding multiple media, plastic material, etc.), media parameters (e.g., type of media, media composition, media height or thickness, etc.), illumination parameters (e.g., angle of the light source, color or wavelength(s) of the light source, color of the background, etc.), and positioning parameters (e.g., position of the plate in the imaging chamber) to produce as realistic a model as possible.

[0276] In the case of an attenuated image, when the pixel information is less sharp than the remainder of the image (e.g., because underlying condensation blocks some light from passing through the plate and thus makes the portion of the plate with condensation slightly less opaque), signal recovery can be used to sharpen the attenuated pixel features in the image to complete enhancement of the attenuated image. Signal recovery can include determining the intensity information of the remainder of the image, identifying the median intensity of the intensity information, and then replacing the intensity information of the less sharp regions of the image with the median intensity of the remainder of the image.

[0277] Applications

[0278] The present disclosure is largely based on tests conducted in saline at different dilutions to simulate typical urine reporting volumes (CFU / ml bucket groups). The suspensions for each isolate were adjusted to 0.5 McFarland Standard and used for an estimated 1×10 6 、1×10 5 、5×10 4 、1×10 4 、1×10 3 and 1×10 2Prepare dilutions in a suspension of CFU / ml. Using Kiestra InoqulA (WCA1), process the sample tubes using the standard urine streaking pattern - 4# zigzag (dispensing 0.01 ml per plate).

[0279] Process the plates using a ReadA Compact (35 °C, non-CO 2 ) and over a total incubation time of 48 hours, image every two hours for the first 24 hours and every six hours for the second 24 hours. At 1 hour, the incubation time is entered as the first reading and the allowed margin is set at + / - 15 minutes. For the next readings from 2 - 24 hours, set at every two hours with an allowed margin of + / - 30 minutes. For readings from 24 - 48 hours, set at every six hours with an allowed margin of + / - 30 minutes. After a pure feasibility study, this was changed to eliminate the allowed margins. This was done to improve image acquisition in the desired 18 - 24 hour time frame.

[0280] In other cases, images can be obtained over a 48-hour span - every two hours for the first 24 hours and every six hours for the next 24 hours. In such a case, a total of 17 images will be obtained over the 48-hour span, including the image obtained at time t 0 (0 hours).

[0281] Correct the lens geometric aberration and chromatic aberration of all acquired images and spectrally balance all acquired images using a known target pixel size, normalized illumination conditions, and a high signal-to-noise ratio per pixel per band. Suitable cameras for the methods and systems described herein are known to those skilled in the art and are not described in detail herein. As an example, when the diameter of the colonies is in the range of 100 μm and has sufficient contrast, using a 4-megapixel camera to capture images of 90 mm plates should allow enumeration of up to 30 colonies / mm 2 of the local density (> 10 5 CFU / plate).

[0282] Evaluate the contrast of the colonies growing thereon using the following media:

[0283] TSAII 5% sheep blood (BAP) is a non-selective medium widely used in urine cultures.

[0284] BAP is used for colony counting and presumptive ID based on colony morphology and hemolysis.

[0285] MacConkey II Agar (MAC): A selective medium for the most common Gram-negative UTI pathogens. MAC is used for the differentiation of lactose-producing colonies. MAC also inhibits swarming of Proteus species. BAP and MAC are commonly used for urine cultures. Some media are not recommended for colony counts due to partial inhibition of some Gram-negatives.

[0286] Colistin Nalidixic Acid Agar (CNA): A selective medium for the most common Gram-positive UTI pathogens. CNA is not as commonly used for urine cultures as MAC, but it can help identify colonies if there is overgrowth of Gram-negative colonies.

[0287] CHROMagar Orientation (CHROMA): A non-selective medium widely used for urine cultures. CHROMA is used for colony counts and ID based on colony color and morphology. Escherichia coli and Enterococcus are identified by this medium and no confirmatory tests are required. Due to cost, CHROMA is used less than BAP. CLED medium is also used for mixed samples.

[0288] Cystine Lactose Electrolyte Deficient (CLED) Agar: Based on lactose fermentation, used for colony counts and presumptive ID of urinary pathogens.

[0289] Sample Processing BD Kiestra TM InoqulA TM For the automated processing of bacteriological samples to enable standardization and ensure consistent and high-quality streaking. BD Kiestra TM InoqulA TM The sample processor uses magnetic bead technology to streak the culture medium plates using a customizable pattern. The diameter of the magnetic beads is 5 mm.

[0290] Figure 17 Illustrates an example of the contrast collection algorithm performance in the growth of Morganella morganii on both CHROMagar Orientation (top panel) and blood agar (bottom panel) media. CHROMagar and BAP are both non-selective growth media widely used in microbiology laboratories. Figure 17 Each of the middle images presents an imaging plate illuminated by light from above the plate (top illumination). The left image shows the corresponding spatial contrast of the middle image. The spatial contrast is based on a 1-mm median kernel. Finally, the right image shows the temporal contrast of the middle image. The temporal contrast is determined by the difference in intensity between the image at 0 hours (t 0 ) and 12 hours (t x)Compiled from images captured between incubations, with different lighting conditions (different color channels, illumination settings, etc.) at each image capture time.

[0291] As Figure 17 shown, when dealing with translucent colonies, especially when the edge transition is small, local contrast has its limitations because, using only the spatial contrast algorithm alone, only the confluent regions with the strongest contrast can be selected from the image. The spatial contrast may not select the target. It is also evident from Figure 17 the effectiveness of temporal contrast for isolated colonies.

[0292] Figure 17 (Especially the blood agar image at the bottom) also emphasizes the problem of growth on a transparent medium detected using only spatial contrast. The ink printed on the transparent case has very strong edges and thus strong spatial contrast. This ultimately makes it very difficult to see any other spatial contrast because the colonies do not have such strongly defined edges. Thus, in the present disclosure, there are appreciable benefits to using temporal contrast in combination with spatial contrast.

[0293] Ultimately, the result of the above contrast determination is that a method for the rapid detection and identification of colonies in an imaged culture medium can be automated. This automated method provides significant advantages over comparable manual methods.

[0294] Figure 18 shows a pair of flowcharts that compare the timeline of an automated test process 1800 (e.g., Figure 2 program 200 in) with the timeline of a comparable manually implemented test process 1805. Each process begins with the laboratory receiving samples 1810, 1815 for testing. Each process then proceeds to incubation 1820, 1825, during which the samples can be imaged multiple times. In the automated process, after approximately 12 hours of incubation, an automated evaluation 1830 is performed, after which time it can be clearly determined whether there is no growth (or normal growth) 1840 in the sample. As Figure 17 the results show, the use of temporal contrast in the automated process greatly improves the ability to detect colonies - even after only 12 hours. In contrast, in the manual process, manual evaluation 1835 cannot be performed until nearly 24 hours into the incubation process. Only after 24 hours can it be clearly determined whether there is no growth (or normal growth) 1845 in the sample.

[0295] Figure 19Illustrated is a timeline by which colony semi - quantification can be determined. Plate 2005 was inoculated at 2000. A reference image of the newly inoculated plate 2005 was obtained at 2010. This is the background image described previously herein. Incubation was carried out for a predetermined time interval, after which another image of plate 2005 was obtained at 2015. This image was compared with the image obtained at 2010 to obtain evidence of target changes or artifacts that might indicate microbial growth. If there was no indication of microbial growth, the plate was incubated, and after a predetermined time, another image was obtained at 2020. At step 2020, changes or artifacts in the target were detected as evidence of microbial growth. From the perspective of image analysis, growth in the image can be detected by identifying the imaging target (based on the difference between the target and its adjacent environment) and then identifying the changes in the target over time. As described in more detail above, these differences and changes are two forms of "contrast". In addition to detecting growth, the image analysis at 2020 can further involve determining whether the target identified as a colony has sufficient biomass to be harvested for analysis. Figure 19 Indicates that insufficient biomass was identified at step 2020 and further incubation is required. At 2025, the image demonstrated colonies, but those colony targets needed to be further evaluated to not only quantify the colonies but also assess the nature of the colonies (i.e., whether it is a sister of other identified colony targets; whether it is a pure (i.e., single microbial species) colony or a mixed (i.e., multiple microbial species) colony. Colony confluence and how it is determined were described in the above section on target segmentation. If there was no growth, or a negligible amount of growth was found after a predetermined time, then at 2022 plate 2005 was released as sterile. The final report might indicate no significant growth, or report the growth of normal flora.

[0296] Biomass is determined by correlating the target. The size of the target is determined and compared with a predetermined target size indicating threshold biomass. At step 2025, if the target is formed from a pure sample, then when the area of the medium covered by the target reaches or exceeds a first threshold, the biomass can be picked for downstream testing. If the biological sample is not a pure sample, the inoculated sample dish is further incubated. If not all of the targets of interest have been picked based on the image obtained at step 2025, then at step 2030 the inoculated culture plate is further incubated and imaged again. The size of the target in the image related to the colony is compared with a second predetermined size threshold. If the size of the target and thus the biomass is above the second threshold, then at least part of the biomass is picked. The second predetermined threshold is one of the following: i) the threshold area covered by the biomass if the target is from a pure sample; or ii) the diameter of the biomass if the target is from an impure sample. The second predetermined threshold biomass is greater than the first threshold. This can be achieved byFigure 19 This is seen by comparing the target sizes at step 2025 with those at step 2030. The target in step 2030 is significantly larger than that in step 2025, but the target identified in step 2025 can be picked if there is confidence that the target / biomass is from a pure sample.

[0297] If it is determined that the biological sample exhibits a quantitatively significant growth, then one or more colonies can be identified as colony candidates to be picked for analysis at 2025 or 2030 (depending on when a target colony with sufficient biomass is detected). Picking colonies can be a fully automated process where each picked colony is sampled and tested. Optionally, picking colonies can be a partially automated process where multiple colony candidates are automatically identified and visually presented to the operator in a digital image, such that the operator can input a selection of one or more candidates for sampling and further testing. Sampling of the selected or picked colonies themselves can be automated by the system. With respect to 2025 and 2030, the time at which colonies can be picked depends on what is known about the observed colonies at the first observation at 2025. If the colonies have developed from a presumed pure sample (such as a deep wound or incision), then the colonies can be picked once there is sufficient quantity to observe the colonies and collect them. The timing of processing such samples is crucial, and avoiding one or more additional incubation cycles means that test results can be provided hours earlier. If the sample is not presumptively pure, then additional incubation cycles are required to allow the colonies to grow further to aid in the identification and quantification of these colonies.

[0298] Referring again to Figure 18 , the use of automated processes also allows for faster AST and MALDI testing. Such testing 1850 in an automated process can start soon after the initial assessment 1830, and the results can be obtained 1860 and reported 1875 by the 24-hour mark. In contrast, such testing 1855 in a manual process typically does not start until close to the 36-hour mark and takes an additional 8 to 12 hours to complete before the data can be examined 1865 and reported 1875.

[0299] In summary, the manual testing process 1805 shows that it takes up to 48 hours, with an incubation period of 18 - 24 hours, and the growth of the plates is only evaluated after that, and there is also no way to track how long the samples have been incubated. In contrast, because the automated testing process 1800 can detect even relatively poor contrast between colonies (compared to the background and to each other), and can perform imaging and incubation without the need for a microbiologist to keep track of the time, it only requires an incubation of 12 - 18 hours before the sample can be identified and ready for further testing (e.g., AST, MALDI), and the entire process can be completed in about 24 hours. Thus, the automated process of the present disclosure, with the help of the contrast processing described herein, provides faster testing of samples without adversely affecting the quality and accuracy of the test results.

[0300] While the present disclosure has been described with reference to specific embodiments, it should be understood that these embodiments are merely illustrative of the principles and applications of the present disclosure. Thus, it should be understood that various modifications can be made to the illustrative embodiments, and other arrangements can be designed without departing from the spirit and scope of the present disclosure as defined by the appended claims.

Claims

1. An automated method performed by an automated laboratory system directed by one or more processors for evaluating microbial growth on a plated medium, the directing comprising: providing a medium inoculated with a biological sample disposed within a substantially optically transparent container; prior to the formation of visible colonies, placing the substantially optically transparent container carrying the inoculated medium in a digital imaging device; Obtain a first digital image of the inoculated culture medium at a first time (t 0 ), the first digital image having a plurality of pixels; determining coordinates of the pixels in the first digital image relative to the substantially optically transparent container carrying the inoculated medium; removing the substantially optically transparent container carrying the inoculated medium from the digital imaging device and placing the inoculated medium in an incubator for incubation for a predetermined time sufficient for microbial growth to occur in the inoculated medium; after further incubation, placing the substantially optically transparent container carrying the inoculated medium in the digital imaging device; and Obtain a second digital image of the inoculated culture medium at a predetermined second time (t x ), the second digital image having a plurality of pixels; aligning the first digital image with the second digital image such that coordinates of the plurality of pixels in the second digital image correspond to the determined coordinates of each corresponding pixel among the plurality of pixels in the first digital image; comparing pixels of the second digital image with corresponding pixels of the first digital image; identifying pixels that change between the first digital image and the second digital image, wherein pixels that do not change between the first digital image and the second digital image indicate background; determining which of the identified pixels in the second digital image have a threshold contrast of a predetermined level with the pixels indicating background; identifying one or more targets in the second digital image, each target comprising pixels having the threshold contrast of the predetermined level with the pixels indicating background and not separated from each other by background pixels; determining whether the size of the one or more identified targets indicates a threshold biomass by comparing the targets with a predetermined target size indicating the threshold biomass; determining whether the biological sample is identified as a sample comprising a single microbial species, and if so, further determining whether the biomass is higher than a first threshold, the first threshold being a predetermined area of the medium covered by the identified target, and if the area of the identified target is higher than the first threshold, obtaining at least a portion of the biomass for further analysis; and if the biological sample is not a sample of a single microbial species, further incubating the inoculated medium in the substantially optically transparent container.

2. The method according to claim 1, wherein if no target is identified after a predetermined period of time, the substantially optically transparent container having the inoculated medium is labeled as a no - growth plate.

3. The method according to claim 1, wherein after the step of further incubation, a third digital image comprising pixels is obtained, and the method further comprises: identifying additional pixels that change from the second digital image to the third digital image; Determine which identified pixels and additional identified pixels in the third digital image have a threshold contrast of a predetermined level with the pixels indicating the background; Identify one or more targets in the third digital image, each target including pixels having the threshold contrast of the predetermined level with the pixels indicating the background and not separated from each other by background pixels; Determine whether the size of the one or more identified targets indicates a threshold biomass; and Determine whether the biomass is higher than a second threshold, where the second threshold is one of the following: i) the threshold area covered by the biomass if the target is from a single-species sample; or ii) the diameter of the biomass if the target is from a sample that is not a single species; and If the biomass is higher than the second threshold, obtain at least a portion of the biomass for analysis, where the second threshold of the biomass is greater than the first threshold.

4. The method according to claim 1, further comprising aligning the second digital image with the first digital image, where the alignment is based on the position of fiducial marks on the substantially optically transparent container.

5. The method according to claim 3, further comprising aligning the third digital image with the second digital image, where the alignment is based on the position of fiducial marks on the substantially optically transparent container.

6. The method according to any one of claims 4 and 5, where the fiducial marks are selected from: an eccentric optically detectable marked point on the bottom of the substantially optically transparent container, an end of an optically detectable label provided on the side of the substantially optically transparent container, and a center of an optically detectable label on the substantially optically transparent container.

7. The method according to claim 1, further comprising obtaining a plurality of first digital images at the first time according to a predetermined series of illumination conditions, where each of the first digital images is obtained under different illumination conditions, and each illumination condition includes a specified orientation of the substantially optically transparent container carrying the inoculated culture medium relative to the illumination source, and a specified background color on which the substantially optically transparent container is placed within the digital imaging device.

8. The method according to claim 1, where the digital imaging device comprises: a lighting source pointing downward at the top of the substantially optically transparent container carrying the inoculated culture medium; a lighting source pointing upward at the bottom of the substantially optically transparent container carrying the inoculated culture medium; and a lighting source pointing at the side of the substantially optically transparent container carrying the inoculated culture medium.

9. The method according to claim 8, where for the specified orientation of the substantially optically transparent container relative to the top lighting source and the specified orientation of the substantially optically transparent container relative to the side lighting source, the specified background color is black; and for the specified orientation of the substantially optically transparent container relative to the bottom lighting source, the specified background color is white.

10. The method according to claim 8, wherein the illumination source includes an illumination source emitting a red wavelength; an illumination source emitting a green wavelength; and an illumination source emitting a blue wavelength.

11. The method according to claim 1, wherein the target is a microbial colony.

Images