Systems and methods for monitoring bacterial growth of colonies and predicting colony biomass

Abstract

The invention provides systems and methods for monitoring bacterial growth of a colony and predicting colony biomass. Imaging methods for earliest microbial growth detection. The method uses images to determine colony biomass, and colony biomass determines when colonies can be picked for identification or analysis of antibiotic sensitivity testing. If the sample source is not a pure sample source, additional incubation may be required to allow for an increase in the biomass of the colony prior to picking.

Description

[0001] This application is a divisional application of a patent application with application date of December 19, 2019, application number 2019800915444, and titled “System and method for monitoring bacterial growth of colonies and predicting colony biomass”.

[0002] Related applications

[0003] This application claims priority to U.S. Provisional Application No. 62 / 782,513, filed on December 20, 2018, the entire contents of which are incorporated herein by reference. Background Art

[0004] There is growing interest in digital images of culture plates for detecting microbial growth. Techniques for imaging culture plates to detect microbial growth are described in PCT Publication Nos. WO / 2015 / 114121, entitled “A System and Method for Image Acquisition Using Supervised High Quality Imaging,” published on August 6, 2016, WO / 2016 / 172527, entitled “Colony Contrast Gathering,” published on October 27, 2016, and WO / 2016 / 172532, entitled “A Method and System for Automatic Microbial Colonycount from Streaked Sample on Plated media,” published on October 27, 2017, the entire contents of which are incorporated herein by reference. All of the above references are commonly assigned with the present application.

[0005] WO / 2015 / 114121 describes techniques for identifying colony targets and distinguishing them from non-colony artifacts and background by controlling the signal-to-noise ratio. WO / 2016 / 172527 describes systems and methods for interpreting culture plate images based on automated imaging techniques. Colony targets are identified early in the process of promoting the growth of microorganisms on a nutrient medium, and changes in these identified targets are observed over time and under conditions that support colony growth. WO / 2016 / 172532 describes systems and methods for counting identified colonies to determine whether the colonies are from the same microorganism and further determine whether the colony count reaches or exceeds a predetermined number.

[0006] Colony detection, colony enumeration, colony population differentiation, and colony identification define the goals of modern microbiology imaging systems. Achieving these goals as early as possible supports the goal of delivering results to patients quickly and providing these results and analyses economically. Automating laboratory workflows and decision-making can improve the speed and cost of achieving these goals.

[0007] While significant progress has been made in imaging technology for detecting signs of microbial growth, there is still a need to expand this imaging technology to support automated workflows. Instruments and methods for inspecting culture plates for indications of microbial growth are difficult to automate, in part due to the highly visual nature of plate inspection. In this regard, it is desirable to develop technology that can automatically interpret culture plate images and, based on the automated interpretation, determine the subsequent steps to be performed (e.g., identification of bacterial colonies, susceptibility testing, etc.).

[0008] Identifying and distinguishing bacterial colonies on a culture plate can be difficult, especially when the colonies are of different sizes and shapes and are in contact with one another. Colonies that "grow together" make picking colonies of interest even more difficult because there's a risk of picking microorganisms from adjacent colonies, where the adjacent colonies are different microorganisms. Picking colony samples for downstream processing requires a sufficient number of colonies and the purity of the target colonies to avoid picking multiple microorganisms, which can contaminate downstream test results. These problems are exacerbated when growth has reached confluence in some areas of the plate. For these reasons, it's preferable to identify colonies and determine growth early in the process, if possible. However, incubation time is still required to allow at least some growth of the colonies. Therefore, on the one hand, the longer the colonies are allowed to grow, the stronger their contrast with their background and with each other becomes, making it easier to identify them. On the other hand, if the colonies are allowed to grow too long and begin to fill the plate and / or come into contact with one another, picking pure colonies becomes even more difficult. This problem could be minimized or even solved if colonies could be detected at an incubation time when they are still small enough to be separated from each other - despite the relatively poor contrast - but still large enough to provide a sufficient amount of sample for testing. Summary of the Invention

[0009] This article describes an automated method for evaluating microbial growth on a plating medium. According to the method, a provided culture medium is inoculated with a biological sample placed in a substantially optically transparent container. The inoculated culture medium is incubated in an incubator. The inoculated culture medium is placed in an optically transparent container carrying the inoculated culture medium in a digital imaging device. The automated method also includes obtaining a first digital image of the inoculated culture medium at a first time (t0), the first digital image having a plurality of pixels. The automated method also includes determining the coordinates of the pixels in the first digital image relative to the transparent container carrying the inoculated culture medium. The automated method also includes removing the transparent container carrying the inoculated culture medium from the digital imaging device and placing the inoculated culture medium in an incubator for further incubation. The automated method also includes, after further incubation, placing the ... second time (t0). x ) obtain a second digital image of the inoculated culture medium, the second digital image having a plurality of pixels. The automated method further includes aligning the first digital image with the second digital image such that pixel coordinates in the second digital image correspond to coordinates of corresponding pixels in the first digital image. The automated method further includes comparing pixels of the second digital image with corresponding pixels of the first digital image. The automated method further includes identifying pixels that changed between the first digital image and the second digital image, wherein pixels that did not change between the first digital image and the second digital image indicate background. The automated method further includes determining which of the identified pixels in the second digital image have a predetermined threshold contrast level with pixels indicating background. The automated method further includes identifying one or more objects in the second digital image, each object including pixels that have the threshold contrast level with pixels indicating background and are not separated from each other by background pixels. The automated method further includes associating the identified objects with biomass. The automated method further includes determining whether the biological sample is identified as a pure sample and, if so, further determining whether the biomass is above a first threshold, the first threshold being a predetermined area of the culture medium covered by the identified object, and, if the area of the identified object is above the first threshold, picking at least a portion of the growth material for further analysis. The automated method further includes incubating the inoculated culture medium in an optically transparent container if the biological sample is not a pure sample. BRIEF DESCRIPTION OF THE DRAWINGS

[0010] Figure 1 is a schematic diagram for imaging analysis and testing cultures according to one aspect of the present disclosure.

[0011] Figure 2 is a flow chart illustrating an automated laboratory workflow procedure for imaging analysis and testing cultures according to one aspect of the present disclosure.

[0012] Figure 3A 、 3B 3C and 3C are time contrast images illustrating changes in colony morphology over time by visually representing colony morphology according to one aspect of the present disclosure.

[0013] Figure 3D and 3E are images showing the spatial contrast under different lighting conditions.

[0014] Figure 4 is a flow chart of an example procedure for obtaining and analyzing image information according to one aspect of the present disclosure.

[0015] Figure 5 is a flow chart of an example procedure for obtaining spatial contrast according to one aspect of the present disclosure.

[0016] Figure 6 is a flow chart of an example procedure for obtaining temporal contrast according to one aspect of the present disclosure.

[0017] Figure 7 is a flow chart of an example procedure for filtering artifacts from an image according to one aspect of the present disclosure.

[0018] Figure 8 is a flow chart of an example procedure for labeling pixels of an image according to one aspect of the present disclosure.

[0019] Figure 9 is a flow chart of an example procedure for splitting a colony into individual targets according to one aspect of the present disclosure.

[0020] Figure 10 is a flowchart of an example object segmentation procedure according to one aspect of the present disclosure.

[0021] Figure 11 is a schematic diagram showing the measurement of confluent colonies, which is Figure 10 Part of the segmentation process.

[0022] Figure 12 According to one aspect of the present disclosure picture.

[0023] Figure 13A 、 13B 13A and 13C are graphs illustrating isolation factor measurements according to one aspect of the present disclosure.

[0024] Figure 14 is a diagram illustrating one aspect of the present disclosure Map of the affected area.

[0025] Figure 15A 、 15Band 15C are images illustrating characterization of colony growth according to one aspect of the present disclosure.

[0026] Figure 16A and 16B A portion of an imaging plate is shown with a magnified and reoriented image of a sample colony in the image.

[0027] Figure 16C Shown Figure 16B Vector diagram of the respective images in .

[0028] Figure 17 Depicted are SHQI, spatial contrast, and temporal contrast images of a sample according to one aspect of the present disclosure.

[0029] Figure 18 It is a comparison Figure 2 A flowchart showing the timeline of the procedure in FIG. 1 and the timeline of a comparable manually performed procedure.

[0030] Figure 19 is a diagram of the growth timeline that will result in the identification of colony candidates of sufficient size to support colony picking. DETAILED DESCRIPTION

[0031] The present disclosure provides instruments, systems and methods for identifying and analyzing microbial growth on a plated culture medium based, at least in part, on contrast detected in one or more digital images of the plated culture medium. The multiple methods described herein can be fully or partially automated, for example, integrated as part of a fully or partially automated laboratory workflow.

[0032] The system described herein can be implemented in an optical system for imaging microbial samples for identification of microorganisms and detection of microbial growth of such microorganisms. There are many such commercially available systems, which are not described in detail herein. An example is the BD Kiestra TM ReadA compact intelligent incubation and imaging system. Other example systems include those described in PCT Publication No. WO2015 / 114121 and U.S. Patent Publication No. 2015 / 0299639, which are incorporated herein by reference in their entirety. Such optical imaging platforms are well known to those skilled in the art and are not described in detail here.

[0033] Figure 11 is a schematic diagram of a system 100 having a processing module 110 and an image acquisition device 120 (e.g., a camera) for providing high-quality imaging of a plating medium. The processing module and image acquisition device can be further connected to, and thereby further interact with, other system components, such as an incubation module (not shown) for incubating the plating medium to allow cultures seeded thereon to grow. Such connections can be fully or partially automated using a track system that receives samples for incubation and transports them to an incubator, and then between the incubator and the image acquisition device.

[0034] The processing module 110 can instruct other components of the system 100 to perform tasks based on processing various types of information. The processor 110 can be hardware that performs one or more operations. The processor 110 can be any standard processor, such as a central processing unit (CPU), or can be a specialized processor, such as an application-specific integrated circuit (ASIC) or a field-programmable gate array (FPGA). Although a single processor block is shown, the system 100 can also include multiple processors—which may or may not operate in parallel—or other specialized logic and memory for storing and tracking information related to sample containers in the incubator and / or image acquisition device 120. In this regard, the processing unit can track and / or store several types of information related to samples in the system 100, including, but not limited to, the location of the sample in the system (incubator or image acquisition device, position and / or orientation therein, etc.), incubation time, pixel information of captured images, sample type, type of culture medium, preventative treatment information (e.g., hazardous samples), etc. In this regard, the processor can be capable of fully or partially automating various procedures described herein. In one embodiment, instructions for implementing the procedures described herein may be stored on a non-transitory computer-readable medium (eg, a software program).

[0035] Figure 2 is a flow chart illustrating an example automated laboratory procedure 200 for imaging, analyzing, and optionally testing cultures. The procedure 200 may be performed by an automated microbiology laboratory system, such as the BD Kiestra TM Total Lab Automation or BD Kiestra TM Work Cell Automation: The example system includes interconnected modules, each of which is configured to perform one or more steps in the process 200 .

[0036] At 202, a culture medium is provided and inoculated with the biological sample. The culture medium can be an optically transparent container so that the biological sample can be observed within the container when illuminated from different angles. Inoculation can follow a predetermined pattern. The streaking patterns and automated methods for streaking the sample onto a plate are well known to those skilled in the art and will not be discussed in detail here. One automated method uses magnetic beads to streak the sample onto a plate. At 204, the culture medium is incubated to allow the biological sample to grow.

[0037] At 206, one or more digital images of the culture medium and biological sample are captured. As will be described in more detail below, digital imaging of the culture medium can be performed multiple times during the incubation process (e.g., at the beginning of incubation, at a time during incubation, at the end of incubation) so that changes in the culture medium can be observed and analyzed. Imaging of the culture medium can include removing the culture medium from the incubator. When multiple images of the culture medium are captured at different times, the culture medium can be returned to the incubator for further incubation between imaging sessions.

[0038] At 208, based on the information from the captured digital image, the biological sample is analyzed. The analysis of the digital image may include analysis of the pixel information contained in the image. In some examples, the pixel information can be analyzed on a pixel-by-pixel basis. In other examples, the pixel information can be analyzed on a block-by-block basis. In a further example, the pixel can be analyzed based on the entire region of the pixel, whereby pixel information for the individual pixels in the region can be derived by combining the information of the individual pixels, selecting sample pixels, or by using other statistical methods such as statistical histogram operations that will be described in more detail below. In the present disclosure, operations described as being applied to "pixels" may also be applied to blocks or other pixel groupings, and the term "pixel" is therefore intended to include such applications.

[0039] The analysis may include determining whether growth is detected in the culture medium. From an image analysis perspective, growth can be detected in the image by identifying the imaged target (based on the differences between the target and its adjacent environment) and then identifying changes in the target over time. As described in more detail herein, these differences and changes are two forms of "contrast." In addition to detecting growth, the image analysis at 208 may further include quantifying the amount of growth detected, identifying unique colonies, identifying sister colonies, and the like.

[0040] At 210, it is determined whether the biological sample (particularly, the identified sister colonies) exhibits significant growth in number. If no growth or insignificant growth is found, then process 200 may proceed to 220 where a final report is output. In the event of proceeding from 210 to 220, the final report will likely indicate a lack of significant growth or report growth of normal flora.

[0041] If it is determined that the biological sample exhibits a significant increase in number, then, based on the existing analysis, one or more colonies may be picked from the image at 212. Picking colonies may be a fully automated process, wherein each picked colony is sampled and tested. Alternatively, picking colonies may be a partially automated process, wherein multiple candidate colonies are automatically identified and visually presented to an operator on a digital image, so that the operator can input a selection of one or more candidates for sampling and further testing. Sampling of the selected or picked colonies may itself be performed automatically by the system.

[0042] At 214, the sampled colonies are prepared for further testing, for example, by plating the samples in an organism suspension. At 216, the samples are tested using matrix-assisted laser desorption ionization (MALDI) imaging to identify the type of sample sampled from the initial culture medium. At 218, the samples may also, or alternatively, undergo antibiotic susceptibility testing (AST) to identify possible treatments for the identified samples.

[0043] At 220, the test results are output in a final report. The report may include MALDI and AST results. As described above, the report may also indicate quantification of sample growth. Thus, the automated system can start with an inoculated culture medium and generate a final report on the samples found in the culture with little or no additional input.

[0044] In the program Figure 2 In the example procedures of , the colonies detected and identified are often referred to as colony forming units (CFUs). A CFU is a microscopic object that starts as one or several bacteria. Over time, bacteria grow to form colonies. The earlier the bacteria are placed on the plate, the fewer bacteria are detected, and therefore the smaller the colonies are and the lower the contrast with the background. In other words, smaller colony sizes produce smaller signals, and smaller signals on a constant background result in less contrast. This is reflected by the following equation:

[0045] (1)

[0046] Contrast plays an important role in identifying objects in an image, such as CFUs or other artifacts. An object can be detected if it has a significant difference in brightness, color, and / or texture from its surroundings. Once an object is detected, analysis can also include identifying the type of object detected. Such identification can also rely on contrast measurements, such as the smoothness of the edges of the identified object, or the uniformity (or lack thereof) of the color and / or brightness of the object. In order to be detected by the image sensor, the contrast must be large enough to overcome the image noise (background signal).

[0047] Human perception of contrast (as determined by Weber's law) is limited. Under optimal conditions, the human eye can detect a 1% difference in light levels. The quality and confidence of image measurements (e.g., brightness, color, contrast) can be characterized by the measured signal-to-noise ratio (SNR), where an SNR value of 100 (or 20log 100 using 20log 100) is independent of pixel intensity. 10 Digital imaging techniques that utilize high SNR imaging information and known SNR information for each pixel may allow detection of colonies even when those colonies remain invisible to the human eye.

[0048] In the present disclosure, contrast can be collected in at least two ways: spatially and temporally. Spatial contrast, or local contrast, quantifies the difference in color or brightness between a given region (e.g., a pixel, a group of adjacent pixels) in a single image and its surroundings. Temporal contrast, or temporal contrast, quantifies the difference in color or brightness between a given region in one image and the same region in another image taken at a different time. The equations governing temporal contrast are similar to those for spatial contrast:

[0049] (2)

[0050] Where t1 is the time after t0. Both the spatial and temporal contrast of a given image can be used to identify objects. The identified objects can then be further tested to determine their significance (e.g., whether they are CFU, normal flora, dust, etc.).

[0051] Figure 3A 、 3B and 3C provide a visual demonstration of the effect that temporal contrast can have on an imaged sample. Figure 3A The images shown in the figure were captured at different time points (left to right, top to bottom row) and show the total growth in the sample. Figure 3A The growth in Figure 3B In the corresponding contrast time image of , the growth is even more pronounced and can be noticed even earlier in the sequence. Figure 3CShown Figure 3B The enlarged part of the Figure 3C As can be seen in the figure, the longer a portion of the colony is imaged, the brighter the spot appears in the contrast image. In this way, the centroid of each colony is indicated by the bright center, or peak, of the colony. Therefore, image data acquired over time can reveal important information about changes in colony morphology.

[0052] To maximize the spatial or temporal contrast of an object relative to its background, the system can capture images using different incident lighting on different backgrounds. For example, arbitrary top lighting, bottom lighting, or side lighting can be used on a black or white background.

[0053] Figure 3D and 3E Provides a visual demonstration of the effect that lighting conditions can have on imaged samples. Figure 3D The images in were captured using top lighting, while Figure 3E The images in were captured at approximately the same time (e.g., close enough in time that no noticeable or significant growth occurred) using bottom illumination. As can be seen, Figure 3D and 3E Each of the images of the samples in contains several colonies, but additional information about the colonies (in this case, hemolysis) needs to be visible. Figure 3D The image in the backlight or bottom light, and in Figure 3E In images with different resolutions, the same information is difficult to grasp.

[0054] At a given point in time, multiple images can be captured under multiple lighting conditions. Different light sources can be used to capture images, and due to the level of the lighting light, the lighting angle and / or the filters (e.g., red, green and blue filters) deployed between the target and the sensor, the different light sources are spectrally different. In this way, the image acquisition conditions are variable in terms of the light source position (e.g., top, side, bottom), background (e.g., black, white, any color, any intensity) and spectrum (e.g., red channel, green channel, blue channel). For example, the first image can be captured using top lighting and a black background, the second image can be captured using side lighting and a black background, and the third image can be captured using bottom lighting and no background (i.e., a white background). Moreover, in order to maximize spatial contrast, specific algorithms can be used to generate a set of different image acquisition conditions. These or other algorithms are also useful for maximizing temporal contrast by changing the image acquisition conditions according to a given order and / or over a period of time. Some such algorithms are described in PCT Publication No. WO2015 / 114121.

[0055] Figure 4is a flow chart illustrating an example procedure for analyzing an imaged slab based at least in part on contrast. Figure 4 The program can be seen as Figure 2 The example subroutine of program 200, thereby Figure 2 206 and 208 are at least partially used Figure 4 procedure to be carried out.

[0056] At 402, a first digital image is captured at time t0. Time t0 may be a time immediately after the incubation process begins, such that bacteria in the imaging plate have not yet begun to form visible colonies.

[0057] At 404, coordinates are assigned to one or more pixels in the first digital image. In some cases, the coordinates may be polar coordinates, having radial coordinates extending from the center point of the imaging plate and angular coordinates about the center point. The coordinates may be used in subsequent steps to help align the first digital image with other digital images of the plate taken at different angles and / or at different times. In some cases, the imaging plate may have specific landmarks or fiducial markers. Advantageously, such fiducial markers are detectable by sensors (i.e., optically detectable). Using fiducial markers to orient an object in coordinate space is well known to those skilled in the art. Examples of suitable optically detectable fiducial markers include off-center markers, such as dots or lines on the bottom of an optically transparent culture dish. Such markers can be detected by a sensor "aimed" at the bottom of the culture dish. If the support of the culture dish is optically transparent, the sensor can be positioned below the culture dish and detect the fiducial marker. If the culture medium disposed in the culture dish is optically transparent, the sensor can detect the fiducial marker if mounted above the culture dish. To avoid the difficulty of detecting a fiducial marker on the bottom of the culture plate from either the top or bottom of the plate, the fiducial marker may be a label affixed to the side of the culture dish, such as a barcode label. The barcode label can be detected by the sensor. In these examples, the side or center of the barcode label can be a fiducial marker. Pixel coordinates can be assigned with respect to the fiducial marker. In this way, the orientation of the plate in the coordinate space of the imaging device can be replicated between imaging events. The coordinates of the pixel(s) covering the landmark in the first image can be assigned to the pixel(s) covering the same landmark in the other image. Therefore, a pixel in an earlier image can be compared to the same pixel in a later image because they share the same coordinates. In this way, any changes in pixels between images can be easily observed.

[0058] At 406, at time t x A second digital image is captured. Time t x is the time after t0 when bacteria in the imaging plate have a chance to form visible colonies.

[0059] At 408, the second digital image is aligned with the first digital image based on the previously assigned coordinates.Aligning the images may further include normalizing and standardizing the images, for example, using the methods and systems described in PCT Publication No. WO2015 / 114121.

[0060] At 410, contrast information for the second digital image is determined. The contrast information can be collected on a pixel-by-pixel basis. For example, a pixel of the second digital image can be compared to a corresponding pixel (at the same coordinates) of the first digital image to determine the presence of temporal contrast. Additionally, adjacent pixels of the second digital image can be compared to each other or to other pixels known to be background pixels to determine the presence of spatial contrast. Changes in pixel color and / or brightness indicate contrast, and the magnitude of such changes from one image to the next or from one pixel (or region of pixels) to the next pixel (or region of pixels) can be measured, calculated, estimated, or otherwise determined. Where both temporal and spatial contrast are determined for a given image, an overall contrast for a given pixel of the image can be determined based on a combination (e.g., an average, weighted average) of the spatial and temporal contrast for the given pixel.

[0061] At 412, based on the contrast information calculated at 410, an object is identified in the second digital image. Adjacent pixels in the second digital image with similar contrast information can be considered to belong to the same object. For example, if the difference in brightness between adjacent pixels and their background, or between pixels themselves, is substantially the same as their brightness in the first digital image (e.g., within a predetermined threshold), the pixels can be considered to belong to the same object. As an example, the system can assign a value of "1" to any pixel with significant contrast (e.g., exceeding a threshold value), and then identify the group of adjacent pixels assigned a value of "1" as an object. The object can be given a specific label or mask so that pixels with the same label share certain characteristics. This label can help distinguish the object from other objects and / or the background during subsequent processing in subroutine 400. Identifying an object in a digital image can include segmenting, or dividing, the digital image into multiple regions (e.g., foreground and background). The purpose of segmentation is to transform an image into a representation of its multiple components so that the components can be more easily analyzed. Image segmentation is used to locate objects of interest in an image.

[0062] At 414, the features of a given target (identified at 412) can be characterized. Characterization of target features can include deriving descriptive statistics of the target (e.g., area, reflectance, size, optical density, color, plate position, etc.). Descriptive statistics can ultimately quantitatively describe certain features of the information set collected about the target (e.g., from the SHQI image, from the contrast image). Such information can be evaluated as a function of species, concentration, mixture, time, and culture medium. However, at least in some cases, characterizing a target can start with a collection of qualitative information related to the target features, whereby this qualitative information is then quantitatively represented. Table 1 below provides a list of example features that can be qualitatively evaluated and then converted to a quantitative representation:

[0063] Table 1: Qualitative attributes of a target and criteria for quantitatively transforming them

[0064]

[0065] Some characteristics of an object, such as its shape or the time until it is visually observed, can be measured a single time for the object as a whole. Other characteristics can be measured multiple times (e.g., for each pixel, for each row of pixels with a common Y coordinate, for each column of pixels with a common X coordinate, for each ray of pixels with a common angular coordinate, for a circle of pixels with a common radial coordinate) and then combined into a single measurement, for example, using a histogram. For example, color can be measured for each pixel, growth rate or size can be measured for each row, column, ray, or circle of pixels, and so on.

[0066] At 416, it is determined whether the target is a colony candidate based on the features of the characterization. Colony candidate determination may include inputting quantitative features (e.g., the scores shown in Table 1 above) or a subset thereof to a classifier. The classifier may include a confusion matrix (confusion matrix) for implementing a supervised machine learning algorithm, or a matching matrix for implementing an unsupervised machine learning algorithm, to evaluate the target. Supervised learning may be preferred in such a case: wherein the target is distinguished from a finite set (e.g., two or three) of possible organisms (in this case, the algorithm can be trained on a relatively finite set of training data). On the contrary, unsupervised learning may be preferred in such a case: wherein the target is distinguished from the entire database of possible organisms, in which case providing comprehensive - or even sufficient - training data is difficult. In the case of a confusion or matching matrix, the difference can be measured numerically within a certain range. For example, for a given pair of targets, "0" may mean that the two targets should be distinguished from each other, while "1" may mean that the targets are difficult to distinguish from each other.

[0067] The colony candidates can be stored in the memory of the automated system for further use (eg, testing, the segmentation procedure described below, etc.).

[0068] Application of various culture media

[0069] In the examples above, the evaluation of cultures is described for a single culture medium. However, the examples are equally applicable to instances in which cultures are evaluated in multiple culture media.

[0070] Since the characteristics of bacteria (e.g., color, growth rate, etc.) may vary depending on the type of culture medium ("medium") used, a different confusion matrix may be used for each medium during classification (e.g., 416 of subroutine 400). Thus, it is perfectly reasonable for the classifier to output "0" for two targets with one medium, while outputting "1" for the same two targets with different mediums. The aggregate results of the classifiers can then be evaluated together (manually or based on further machine-driven relationships) to arrive at an overall or final distinction or classification of the targets.

[0071] The assessment of multiple culture media can be carried out using a single container. This single container can be configured to preserve multiple culture media (e.g., double plates, three plates, four plates, etc.) so that multiple culture media can be imaged together simultaneously. Alternatively, multiple culture media can be assessed by streaking the culture sample in several containers, with each container preserving one or more culture media. Each of the multiple containers can then be subjected to the above-mentioned imaging procedure. The information obtained from each culture media (e.g., the features of characterization) can then be collectively input into a classifier to provide a more informed identification of the (spotted) growth found in the different culture media.

[0072] Contrast information

[0073] Figure 5 is a flow chart showing the Figure 4 4. Example subroutine 500 for obtaining spatial contrast as part of 410 in FIG. Subroutine 500 receives as input a set of one or more background and lighting conditions 551 and filters 554. At 502, a digital image is obtained under the specified lighting and background conditions from input set 551. At 504, the image is then cloned. At 506, one of the cloned images is filtered using filter 554. Figure 5In the example of , a low-pass kernel is used as a filter, but the skilled person will be aware of other filters that can be used. At 508, the ratio of the unfiltered image minus the filtered image to the filtered image plus the unfiltered image is calculated. At 510, a spatial contrast image is obtained based on the ratio calculated at 508. This process 500 can be repeated for each background and lighting condition 551. Each repetition of process 500 produces another spatial contrast image, which can be used at 510 to iteratively update the previously stored spatial contrast image. Thus, a comprehensive contrast image (including contrast from each lighting condition) can be iteratively established. In one embodiment, in each iteration, a clear contrast image - in which the contrast setting is still set to 0 (compared to the contrast image established iteratively) - can be provided as input for each lighting setting. If it is determined at 512 that the final image has been processed, then process 500 ends.

[0074] Figure 6 is a flow chart showing the same as Figure 4 4. Example subroutine 600 for obtaining temporal contrast as part of 410. Subroutine 600 receives as input: a set of one or more background and lighting conditions 651; and filters 655. At 602, each of a first and a second digital image captured under specified lighting and background conditions is obtained. At 604, the t0 image is filtered. Figure 6 In the example of , a low pass kernel is used as the filter, but the skilled person will know that other filters can be used. At 606, the unfiltered t x Image minus the filtered t0 image and the filtered t0 image plus the unfiltered t x At 608, a temporal contrast image is obtained based on the ratio calculated at 606. This process 600 can be repeated under different lighting conditions and / or different background conditions. Each repetition of process 600 generates another temporal contrast image, which can be used at 608 to iteratively update the previously stored temporal contrast image. Similar to the creation of the spatial contrast image, the temporal contrast image can be iteratively created, with a clear contrast image provided as input for each lighting condition. If it is determined at 610 that the final image has been processed, process 600 ends.

[0075] In order to make a comprehensive or overall determination of contrast, the spatial and temporal contrast results can be further combined. The combination of spatial and temporal contrast is referred to herein as "mixed contrast" (MC). In one embodiment, the spatial contrast (SC) image at time t0, the temporal contrast image at time t x The spatial contrast image and the spatial contrast images from t0 and t x The temporal contrast (TC) image obtained by comparing the images is used to obtain the mixed contrast according to the following equation:

[0076] (3)

[0077] Filtering

[0078] To enhance image analysis, Figure 4 Additional processes may be included in the subroutine 400. For example, the first digital image may be analyzed for objects that appear in the image at time t0. Since it is known that no bacteria have begun to grow significantly at t0, it can be assumed that any objects found at time t0 are merely dust, bubbles, artifacts, condensation, etc. that do not constitute colony candidates.

[0079] A filtering process can be applied to the captured image to subtract dust and other artifacts that may be attached to the imaging plate or lens. When considering transparent culture media (e.g., MacConkey agar, CLED agar, chromogenic agar (CHROMagar) etc.), it can be expected that some level of dust will be present in the captured image. The effect of dust on a given image may be determined, at least in part, by the specific lighting and background conditions under which the image was taken. For example, when using a white culture medium, reflective artifacts and dust will be most easily observed when the culture medium is illuminated from above with a black background below. As another further example, when using a colored or black culture medium, artifacts and dust will be most easily observed when the culture medium is illuminated from above with a white background below. As a further example, in most arbitrary culture media, regardless of the background, light-absorbing artifacts and dust will be observed when the culture medium is illuminated from below. In any case, the management of dust and artifacts is a complex image processing challenge that can significantly affect the detection of microbial growth.

[0080] Dust and artifacts can be divided into two types: (A) those that can change position; and (B) those that cannot. Dust and artifacts can accumulate over time, meaning the amount of both Type A and Type B can vary over time. However, observations have shown that Type A is more likely to change in amount over time than Type B. Of course, Type A is also more likely to change, for example, due to the plate being moved in and out of the imaging chamber.

[0081] Typically, Type B is caused by artifacts associated with the plate itself, such as ink spots (brand, batch number, and information printed on the underside of the plate), imperfections associated with the injection point of the plastic mold, or frosted areas. Type B can also be caused by dust or air bubbles stuck to the top of the culture medium, trapped within the culture medium, or electrostatically attached to the underside of the plate.

[0082] Even the type A dust and artifacts themselves hardly change position from the point of view of the imaging being observed. However, because the plastic and culture medium of the plate act as filters and lenses, the characteristics and position of the observed type A artifacts may vary slightly, depending on the culture medium color, culture medium level, and plastic. Type B dust and artifacts also do not change position. However, to the extent that type B dust and artifacts are attached to the culture medium and the culture medium is subject to slight movement and drift over time (primarily due to slight drying over time in the incubator), type B dust and artifacts may move with the culture medium. Therefore, the position of type B dust and artifacts is also susceptible to slight changes, at least to a certain extent.

[0083] In terms of contrast, a dust particle (speck) of type A can be said to exist at position "p0" in the t0 spatial contrast image and at position "p0" in the t0 spatial contrast image. x The position "p" in the spatial contrast image x " exists. Assume that p0 and p x are different locations, then dust or artifacts will also exist at two locations in the time contrast image (for example, at p x By comparison, the dust particles of type B will be at positions t0 and t x exists in the common position of the two spatial contrast images but not in the temporal contrast image.

[0084] As explained above, in order to obtain a mixed contrast result, the spatial and temporal contrast images can be combined. The effects of dust and artifacts of both types A and B can be further eliminated from the mixed contrast result. In one embodiment, if a target (e.g., a CFU candidate) is identified in the mixed contrast result, it can be compared with the dust and artifacts of the target neighborhood N (x, y) detected in the spatial contrast result at time t0. Then, if a similar target is found in the spatial contrast result at time t0, the target identified in the mixed contrast result is marked as a type A or type B false positive. Even if the target is not initially marked as a type A or type B false positive, if it is found that the target does not change size significantly over time, it may still be determined later that the target is a type B false positive. The false positive can be stored and later applied to subsequent images, for example, by a filter mask (e.g., a binary mask) described further below.

[0085] Another filtering process can be used to subtract condensation that forms on the plate (e.g., during transfer from the refrigerator to the incubator at the beginning of the incubation session). In one example condensation filter, the plate is illuminated using bottom lighting, so that less light is transmitted to locations with condensation than to locations without condensation. The optical density of the image can then be evaluated, and areas of low optical density can be subtracted from the image.

[0086] Additionally or alternatively, an image mask may be constructed to discount the target from any analysis of the t0 image and / or subsequent digital images. Figure 7 is a flow chart illustrating an example process 700 for generating an image mask using the spatial contrast of a t0 image. In this example of process 700, the only input provided is a SHQI image 751 captured at time t0. At 702, the spatial contrast of the t0 image is determined. At 704, statistical information, such as mean and standard deviation (e.g., regarding brightness), is collected about the pixels in the region of interest of the t0 image using the spatial contrast information. At 706, a contrast threshold is adjusted to ensure that an appropriate number of pixels exceed the threshold. For example, if pixels above a given percentage are not considered background in the image, the threshold can be increased. At step 708, the threshold is further adjusted based on the statistical information associated with those pixels below the threshold. Finally, at 710, a binary mask is generated. The binary mask distinguishes between multiple artifacts of inactive pixels and other pixels that are considered active. The binary mask can then be used at a later time when there are potential colonies to detect, and objects occupying inactive pixels are excluded from candidate colonies.

[0087] The filtering process described above may improve subroutine 400 by avoiding the accidental inclusion of dust, condensation, or other artifacts as targets and by speeding up the property characterization at 414 since such characterization will only be performed on valid pixels.

[0088] Limited Targets and Markers

[0089] Can join Figure 4 Another process of the subroutine 400 is to assign a label to the target identified at 412. The target can be given a specific label so that pixels with the same label share certain characteristics. The label can help distinguish the target from other targets and / or background during subsequent processes of the subroutine 400. Figure 8 is a flow chart showing the time t for marking x The captured image (or "t x Example process 800 for pixels of an image). Figure 8 In the example of receiving a binary mask 851 (eg, the output of process 700), t xThe uninitialized candidate mask 852 of the image and the time contrast image 853 (e.g., the output of subroutine 600) are used as input. At 802, the candidate mask 852 is initialized. Initialization may include identifying the region of interest on the imaging plate and using the binary mask 851 to identify the region of interest at time t x "Valid pixels" in the captured image. Valid pixels are pixels that have not been disregarded as candidate colonies and will be considered for labeling. At 804, the time contrast image 853 is used to collect information about the t x Statistics of the valid pixels in the region of interest of the image, such as the mean and standard deviation (e.g., with respect to brightness). Then, at 806, the statistics of each of the temporal contrast image 853 and the binary mask 851 (which are preferably generated under similar lighting and background conditions) are combined to form a threshold contrast image. Using the threshold defined by the threshold contrast image, t x "Connected components" of the image are labeled at 808. Connected components effectively serve as labels that indicate connections between adjacent pixels (or groups within adjacent pixels), which in turn indicates that the pixels are part of the same object.

[0090] Once the x The image has defined connected components, each of which can be analyzed individually (at 810) to verify its status as a single object. Figure 8 In the example of , statistical calculations of pixels associated with the marker are performed at 812. The calculations may utilize a histogram to determine the mean and / or standard deviation of the brightness or color of the pixel. At 814, a determination is made as to whether the pixel meets a threshold area. If the threshold area is not met, the operation proceeds to 830, where the marker is updated. Updating the marker may include maintaining the analyzed component as a single marker or splitting the component into two markers. In the event that the threshold area is not met, the component remains as a single marker. If the threshold area is met, the histogram is smoothed at 816, and peaks of distributed marker pixels are identified at 818. Peaks can be further defined by having a minimum area—peaks smaller than this minimum area can be ignored. At 820, the number of identified peaks is counted. If there is only one peak, the operation proceeds to 830, and the marker is updated so that the component remains as a target. If there is more than one peak, the threshold contrast image is used to further assess whether the contrast between the peaks is significant at 824. Operations then proceed to 830 and the labels are updated based on the plurality of identified peaks so that significant contrast results in the component are split into two and otherwise remain as one.

[0091] segmentation

[0092] Another process that may be included as part of subroutine 400 is a segmentation process for dividing the x The confluent colonies are separated into individual targets. If at time t x , colonies have grown to the point where they overlap or touch each other, and it may be necessary to draw a border across the confluent area in order to assess the individual colonies within the area.

[0093] In some cases, when two bordering colonies have different characteristics (e.g., different colors, different textures), segmentation can simply involve feature analysis of the confluent area. However, spatial and temporal contrast is not always sufficient to identify the boundaries between colonies alone. Figure 9 is a flow chart illustrating an example process 900 for separating such colonies into individual targets (eg, using individual markers), or in other words, segmenting the colonies. Figure 9 The example process 900 uses a first digital image 951 captured at time t0, a first digital image 952 captured at time t x The captured second digital image and the t0 image binary mask 953 (eg, a mask generated by the process 700) are used as input. At 902, based on t0 and t x Images 951 and 952 generate a time contrast image. At 904, the time contrast image is segmented using a binary mask 953. At 906, markers are applied to the segmented portions of the image. At 908, the peak or maximum of each marker is identified. The maximum value for a given segmented portion is typically the center point or centroid of the segmented portion. At 910, for each marker, the maximum value (e.g., of the marker being analyzed, of neighboring markers) is used to make further determinations regarding whether the given marker is unique to its neighbors or whether it should be combined with one or more neighboring markers. Once the markers are reduced to their unique components, characterization of each marker (e.g., steps 414 and 416 of process 400) can be performed at 912, and a global list of candidate colonies can be generated at 914.

[0094] Various factors, such as the inclusion factor, can be used to determine whether the local maximum of a given marker belongs to the same colony or different colonies. The inclusion factor indicates whether adjacent pixels are associated with adjacent objects. Such a factor can be used in a segmentation strategy to determine whether to split two local maxima in a given marker into two separate objects or merge them into a single object.

[0095] Figure 10 is a flowchart illustrating such an instance segmentation strategy. Figure 9 The subroutine of step 910 in FIG. Figure 10As shown in , two local maxima 1051 and 1052 are identified. At 1002, for each maximum, the surrounding area is identified. Figure 10 In the example of , region "A" surrounds maximum value 1051, and region "B" surrounds maximum value 1052. For the purposes of the example equations below, it is assumed that region A is larger than or equal to region B in size. In some cases, each region may be given an elliptical shape having a horizontal distance (xA, xB) along the region's horizontal axis and a vertical distance (yA, yB) along the region's vertical axis. To illustrate Figure 10 The program, Figure 11 An example illustration of regions A and B and their respective maximum values is provided.

[0096] At 1004, for each local maximum 1051 and 1052, the distance from the maximum to the target edge is determined. In some cases, the distance determined is the average or median distance for the region assigned at 1002, hereinafter referred to as a distance map. The distance map for region A is hereinafter referred to as rA, and the distance map for region B is hereinafter referred to as rB.

[0097] At 1006, the inclusion factor is calculated based on the distance "d" between the two local maxima and the distance determined at 1004. In one embodiment, the inclusion factor is calculated using the following equation:

[0098] (4)

[0099] At 1008, a determination is made as to whether the inclusion factor is less than a predetermined range, greater than a predetermined range, or within a predetermined range (e.g., between 0.5 and 1). If the inclusion factor is less than the predetermined range, the maximum values are determined to be associated with the same target. If the inclusion factor is greater than the predetermined range, the maximum values are determined to be associated with different targets.

[0100] For inclusion factors that fall within the range, it is not immediately clear whether the maxima belong to the same target or different targets, and more processing is required. The program 1000 then continues at 1010, where the convexity of the area surrounding each of the two maxima is calculated using the coordinates of a third region "C" at a location between the two maxima. In some cases, the region can be a weighted center of the two regions, so that the center point of region C is closer to the smaller region B than to the larger region A. For region C, the horizontal and vertical distances xC, yC, and the distance map H can also be calculated. For example, the convexity can be calculated according to the following equation using the above values and the distance d(A, C) between the center point of region C and the maximum value A:

[0101] (5)

[0102] (6)

[0103] (7)

[0104] (8) ΔH=H-(0.9R)

[0105] At 1012, a determination is made as to whether the convexity value is greater than (more convex than) a given threshold. For example, ΔH may be compared to a threshold of 0. If the convexity value is greater than the threshold, the maximum value is determined to be associated with a different target. Otherwise, at 1014, one or more parameters of region C are updated, thereby increasing the size of region C. For example, an offset distance (distOffset) is updated based on ΔH; for example, the value of ΔH is bounded between 0 and 1 (if ΔH is greater than 1, it is rounded to 1) and then added to the offset distance.

[0106] At 1016, a determination is made as to whether the size of region C meets or exceeds a threshold. If the threshold is met or exceeded, the maxima are determined to be associated with the same target. In other words, if the difference between regions A and B is so uncertain that region C increases until it begins to eclipse both regions A and B, this is a good indication that maxima 1051 and 1052 should belong to the same target. In the example above, this can be indicated by the offset distance meeting or exceeding the distance d between the maxima. Otherwise, the operation returns to 1010, and the convexity of regions A and B is recalculated based on the updated parameter(s) for region C.

[0107] Once the relevance of each maximum is determined, the determined relevance can be stored, for example, in a matrix (also known as an incidence matrix). The stored information can be used to reduce the entire list of maximum values to a final list of candidate targets. For example, in the case of an incidence matrix, a master list can be generated from the complete list of maximum values, and then each maximum value can be iteratively checked and, if the associated maximum value is still on the list, removed from the master list.

[0108] exist Figure 9 and 10 In the example, the second image is captured at time t x The earliest time (and, therefore, the earliest time that process 900 can be executed) can be only a few hours into the incubation process. Such a time is generally considered too early to identify fully formed colonies, but it may be sufficient to generate a segmented image. The segmented image can optionally be applied to future images taken at a later time. For example, the boundaries between colonies can be drawn to predict the expected growth of the colonies. Then, in the event of confluence between colonies, the boundaries can be used to separate the confluent colonies.

[0109] Analyze two or more images after time t0

[0110] Although the above processes and procedures only require taking one image after time t0 (e.g., the first digital image at time t0 and the second digital image at time t x , other processes require taking at least a second image after time t0. For example, if it is found that the image at time t x includes confluent colonies, another image taken at time t n (where 0 < n < x) may be used to identify and separate individual colonies.

[0111] For example, if t0 = the start of incubation for 0 hours (when no growth occurs), and t x = the start of incubation for 24 hours (when so much growth has occurred that the colonies are now confluent), an image at time t n = 12 hours (when the colonies would have started growing but are not yet confluent) will reveal the presence of individual colonies. Colony growth is then predicted based on the image at time t n to evaluate the boundaries between the confluent colonies at time t x . In this regard, the image at time t n can help distinguish fast-growing colonies from slow-growing colonies. Those skilled in the art should recognize that as the number of images taken between time t0 and time t x increases, the growth rate of the colonies will be predicted more accurately.

[0112] In one application of the foregoing concept, the image taken at time t n (or more generally, the images taken between time t0 and t x ) can be used to identify colony seeds, which are the targets of colonies suspected of growing over time, and associate the seeds with corresponding masks and labels. Each seed receives a unique label and the label will be stored together with other features (e.g., position, morphology, and histogram based on the image generated from the SHQI image: red channel, green channel, blue channel, illuminance, chromaticity, hue, or composite image) and properties (e.g., isolated / non-isolated status, other information for predicting time-dependent proliferation). To extract global plate metrics, some stored features (e.g., histograms) can also be calculated at the plate level rather than being attributed to specific seeds. The features stored for the seeds can then be used for colony extraction at time t x and provided as input to a classifier for training and / or testing.

[0113] Growth rate tracking using multiple images taken after t0 can also be used to detect dust, artifacts, or other foreign objects that appear on the plate or in the imaging lens in the middle of a workflow procedure. For example, if a dust particle appears after t0 but before t n Before landing on the imaging lens, the spot created by the particle might initially be interpreted as a growing colony because it was not visible at time t 0. However, subsequent imaging reveals that the spot has not changed in size, and it can be determined that the spot is not growing and is therefore not a colony.

[0114] In addition to tracking growth rate and division, other aspects of the colony can be tracked at t0 and t x In some cases, growth can be measured along the z-axis in addition to or instead of measuring growth along the usual x- and y-axes. For example, Streptococcus pneumoniae is known to slowly develop a sunken center when grown in blood agar, but this sunken center is not typically visible until the second day of analysis. By viewing the time course of bacterial growth, an incipient sinking center can be detected, allowing the bacteria to be identified much earlier than if the technician had to wait for the center to fully sink.

[0115] In other cases, it is known that colonies change color over time. Thus, imaging a colony having a first color (e.g., red) at a time after t0, and then imaging a colony having a second color (e.g., green) at a later time, can be used to determine the identity of the bacteria growing in the colony. Color changes can be measured as vectors or paths through a color space (e.g., RGB, CMYK, etc.). Changes in other colorimetric characteristics of the colony can also be measured similarly.

[0116] Target Features

[0117] As above combined Figure 4 As discussed, features of an object on an imaging panel may be characterized as part of the image analysis performed on the imaging panel. The characterized features may include both static features (associated with a single image) and dynamic features (associated with multiple images).

[0118] Static features are intended to reflect the target attributes and / or surrounding context at a given time. Static features include the following:

[0119] (i) Center of gravity: This is a static feature that provides the center of gravity of the imaged object in coordinate space (e.g., xy, polar). The center of gravity of an object, such as its polar coordinates, provides the invariance of the feature set under given lighting and background conditions. The center of gravity can be obtained by first determining a weighted center of mass for all colonies in the image (M is a binary mask of all detected colonies). The weighted center of mass can be determined based on the assumption that each pixel of the image has equal value. The center of gravity of a given colony can then be described in the xy coordinate system by the following equation (where E = {p|p∈M} (E is the binary mask of the current colony), the x coordinate ranges from [0, image width], the y coordinate ranges from [0, image height], and each pixel is a unit):

[0120] (9)

[0121] (ii) Polar coordinates: These are also static features and can be used to further characterize the location on the imaging plate, such as the center of gravity. Typically, polar coordinates are measured along the radial axis (d) and the angular axis (θ), where the coordinates of the center of the plate are [0,0]. (x,y) The coordinates d and θ of are given by the following equations (d is in millimeters, θ is in degrees) (where k is the pixel density in pixels corresponding to millimeters, and the "barcode" is a landmark feature of the imaging plate used to ensure alignment of the plate with previous and / or future images):

[0122] (10)d=k×distance(igv (x,y) ,0 (x,y) )

[0123] (11)θ=angle(barcode,O (x,y) ,igv (x,y) )

[0124] (iii) Image Vectors: Two-dimensional polar coordinates can be transformed into one-dimensional image vectors. This image vector can represent the intensity of a pixel in the image as a function of the radial axis (typically, the center of the colony has the highest intensity) and / or the angular axis. In many cases, image vectors can be more accurate in classifying similarities / differences in imaged objects.

[0125] (iv) Morphological features: These describe the shape and size of a given object.

[0126] (a) Area: This is a morphological characteristic and can be determined based on the number of pixels in the imaged object (also called "blobs"), not counting holes in the object. When pixel density is available, area can be measured in physical dimensions (e.g., mm). 2). Conversely, when pixel density is not available, the total number of pixels may indicate the size, and the pixel density (k) may be set equal to 1. In one embodiment, the area is calculated using the following equation:

[0127] (12)A=k 2 ×∑ p∈E 1

[0128] (b) Perimeter: The perimeter of an object is also a morphological feature and can be determined by measuring the edges of the object and summing the total length of the edges (e.g., a single pixel with an area of 1 square unit has a perimeter of 4 units). As with area, length can be measured in pixel units (e.g., when k is not available) or physical length (e.g., when k is available). In some cases, perimeter can also include the perimeter of any holes in the object. Additionally, stair-stepping effects (caused when diagonal edges are digitized as stepped boxes) can be determined by counting the interior angles as Instead of 2 to compensate. In one embodiment, the perimeter is determined using the following equation:

[0129] (13)P=k×∑ p∈E q(n p )

[0130] (14)

[0131] (15) If:

[0132] {∑(t∈M,l∈M,r∈M,n∈M)=2,(l∈M≠r∈M),(t∈M≠b∈M)}

[0133] (p is interior and p is angle)

[0134] So:

[0135] Otherwise: q(n p )=4-∑(t∈M,l∈M,r∈M,b∈M)

[0136] (c) Roundness: The roundness of an object is also a morphological characteristic and can be determined based on a combination of area and perimeter. In one embodiment, the roundness is calculated using the following equation:

[0137] (16)

[0138] (d) Radius coefficient of variation (RCV): This is also a morphological feature and is used to calculate the average radius of the target in all N directions or angles θ extending from the center of gravity. The ratio between σ and the standard deviation of the radius σR indicates the variation of the target radius. In one embodiment, this value can be calculated using the following equation:

[0139] (17)

[0140] (18)

[0141] (19)

[0142] (v) Contextual features, which describe the relationship of the target's surrounding terrain to other detected targets and wall edges. For example, in the case of imaged bacterial colonies, a contextual feature of the colony could be whether the colony is free, has limited free space, or competes with other surrounding colonies for access to resources. Such features tend to help classify colonies growing in the same perceived environment and / or distinguish colonies growing in different environments.

[0143] (a) Influence area: This is a situational feature that considers the space between the target and its surrounding targets and predicts the area that the analyzed target can expand to occupy (without other, different targets first expanding to occupy the same area). The influence area can be expressed as In the form of a graph, Figure 12 , which shows the impact area (shaded area) based on the distance d between the colony 1201 and its surrounding colonies, such as 1205. In one embodiment, the distance (D) from the edge of the target to the edge of the impact area is NC ) can be represented using the following equation:

[0144] (20)

[0145] (b) Distance to wall: This is the distance from the edge of the target to the nearest wall (D PW ). In one embodiment, the distance can be represented by the following equation:

[0146] (twenty one)

[0147] (c) Isolation Factor: This is a situational characteristic that characterizes the relative isolation of a given object based on its size and distance to the nearest edge (e.g., another object, the nearest edge of a wall). Figure 13A -C illustrates aspects of the isolation factor. Figure 13A An example is illustrated where the closest edge is the distance d from the colony to the plate wall. Figure 13B and 13C The example in which the nearest edge belongs to another colony is illustrated. In such a case, a circle is drawn around the colony being analyzed, centered on it, and then expanded (small ones at first, e.g. Figure 13B , and then larger ones, such as Figure 13C ) until the circle touches the adjacent colony. Figure 13A In the embodiment of -C, the isolation factor (IF) can be characterized using the following equation:

[0148] (twenty two)

[0149] (d) Peripheral occupancy ratio: This is the ratio of the boundary of the plate to the given target within a given distance d. The situation characteristics of the area portion of the impact region (V). In one embodiment, the peripheral occupancy ratio (OR) can be characterized using the following equation (wherein for this equation, E = {p|p∈V,dist(p,igv (x,y) )<d}):

[0150] (twenty three)

[0151] (e) Relative perimeter occupancy ratio: In some examples, the average radius of the target multiplied by a predetermined factor ( ) for a given distance d. The result is the relative peripheral occupancy ratio (RNOR) and can be derived for a given factor x using the following equation:

[0152] (24)RNOR(x)=NOR(d)

[0153] (vi) Spectral features, which describe the light properties of a given target. Color (red, green, and blue channels; hue, luminance, and chromaticity, or any other color space transformation), texture, and contrast (over time and / or across space) are examples of such features. Spectral features can be derived from images captured at different time points and / or under different lighting conditions during the incubation process using the colony mask, and can be further compared to the light properties of a given colony. The affected area is associated.

[0154] (a) Channel image: This is a spectral feature where specific color channels (e.g., red (R), green (G), and blue (B)) are used to spectrally resolve the image.

[0155] (b) Brightness: This is also a spectral feature, which is used to characterize the brightness of an image using RGB channels as input.

[0156] (c) Hue: This is a spectral characteristic in which areas of an image are characterized as appearing similar to a perceived color (e.g., red, yellow, green, blue) or a combination thereof. Hue (H2) is typically characterized using the following equation:

[0157] (25)H2=atan2(β,α)

[0158] (26)

[0159] (27)

[0160] (d) Chromaticity: This is a spectral characteristic used to characterize the apparent colorfulness of an image area relative to its brightness if the area were illuminated similarly to white. Chromaticity (C2) is typically characterized using the following equation:

[0161] (28)

[0162] (e) Radial scatter: Analysis of radial scatter of hue and chromatic contrast enables differentiation of α, β, and γ hemolysis.

[0163] (f) Maximum contrast: This feature characterizes the resolution of an image by calculating the maximum value of the measured average contrast of pixels at a given radius r from the center point of the image (e.g., the center of the imaged colony). This feature can be used to describe the growth induced by the target at times t0 and t x The perceptible difference between captured images. Maximum contrast can be characterized as follows:

[0164] (29) Maximum contrast r =MAX(average contrast r ) 目标

[0165] (vii) Background characteristics, which describe changes in the culture medium in the vicinity of the analytical target. For example, in the case of imaging bacterial colonies, the changes may be caused by the growth of microorganisms surrounding the colonies (e.g., signs of hemolysis, changes in pH, or specific enzymatic reactions).

[0166] Dynamic features aim to reflect changes in target properties and / or surroundings over time. Time series processing allows static features to be correlated over time. The discrete first-order and second-order derivatives of these features provide the instantaneous "velocity" and "acceleration" (or plateau or deceleration) of these features over time. Examples of dynamic features include the following:

[0167] (i) Time series processing for tracking the above-mentioned static features over time. Each feature measured at a given incubation time can be referenced according to its relative incubation time to allow these features to be associated with features measured at subsequent incubation times. The time series of images can be used to detect targets such as CFUs appearing and growing over time, as described above. Based on the ongoing analysis of the target image captured previously, the time points of imaging can be preset or limited by an automated process. At each time point, the image can be a given acquisition configuration, whether a single acquisition configuration for the entire series, or as an entire series of images captured from multiple acquisition configurations.

[0168] (ii) Discrete first and second derivatives of the above characteristics to provide the instantaneous velocity and acceleration (or plateau or deceleration) of these characteristics over time (e.g., to track growth rate, as described above):

[0169] (a) Velocity: The first derivative of a characteristic with respect to time. The velocity (V) of a characteristic x can be expressed in (x units) / hour based on the following equation, where Δt is the time span in hours:

[0170] (30)

[0171] (31)

[0172] (32)

[0173] (b) Acceleration: The second derivative of the characteristic with respect to time, which is also the first derivative of velocity. Acceleration (A) can be characterized based on the following equation:

[0174] (33)

[0175] The above-mentioned image features can be measured from the target or the situation of the target and are intended to capture the specificity of the organisms grown in various culture media and incubation conditions. The listed features are not meant to be exhaustive, and those skilled in the art can improve, expand or limit the feature set based on various known image processing-based features known in the art.

[0176] Image features can be collected for each pixel, group of pixels, object, or group of objects in the image. To more generally characterize image regions or even entire images, the distribution of collected features can be constructed as a histogram. To analyze or otherwise process the incoming image feature data, the histogram itself can rely on several statistical features.

[0177] Statistical histogram features can include the following:

[0178] (i) Minimum: The smallest value of the distribution captured in the histogram. This can be characterized by the following relationship:

[0179] (34)

[0180] (ii) Maximum: The largest value of the distribution captured in the histogram. This can be characterized by the following relationship:

[0181] (35)

[0182] (iii) Sum: The sum of all individual values captured in the histogram. The sum can be defined by the following relationship:

[0183] (36)

[0184] (iv) Mean: The arithmetic mean, or average. This is the sum of all the scores divided by the number of scores (N), according to the following relationship:

[0185] (37)

[0186] (v) Quartile (Q1): The score at the 25th percentile of the distribution. 25% of the scores are below Q1, and 75% of the scores are above Q1. This can be described by the following relationship:

[0187] (38)

[0188] (vi) Median (Q2): The score at the 50th percentile of the distribution. 50% of the scores are below the median, and 50% of the scores are above the median. The median is less sensitive to extreme scores than the mean, and this often makes it a better measure than the mean for highly skewed distributions. This can be described by the following relationship:

[0189] (39)

[0190] (vii) Quartile (Q3): The score at the 75th percentile of the distribution. 75% of the scores are below Q3 and 25% of the scores are above Q3. This is described by the following relationship:

[0191] (40)

[0192] (viii) Mode: The score that occurs most frequently in a distribution. This serves as a measure of central tendency. The advantage of the mode as a measure of central tendency is that its meaning is obvious. Furthermore, it is the only measure of central tendency that can be used with nominal data. The mode is heavily subject to sample fluctuations and is therefore not usually used as the sole measure of central tendency. Also, many distributions have more than one mode. These distributions are said to be "multimodal". The mode can be described by the following relationship:

[0193] (41)

[0194] (ix) Trimean: The score calculated by adding the 25th percentile to twice the 50th percentile (the median) plus the 75th percentile, and dividing by 4. The trimean is almost as robust to extreme scores as the median and, in skewed distributions, is less subject to sampling fluctuations than the arithmetic mean. However, it is generally less effective than the mean for normal distributions. The trimean can be described by the following relationship:

[0195] (42)

[0196] (x) Truncated mean: A score calculated by discarding a certain percentage of the lowest and highest scores and then averaging the remaining scores. For example, a mean with a 50% cutoff is calculated by discarding the lower and upper 25% of the scores and averaging the remaining scores. As a further example, the median is the mean with a 100% cutoff and the arithmetic mean is the mean with a 0% cutoff. Compared to the arithmetic mean, the truncated mean is generally less susceptible to extreme scores. Therefore, for skew distributions, it is less susceptible to sampling fluctuations than the mean. It is generally less efficient for normal distributions. As an example, the mean with a 50% cutoff is described by the following relationship:

[0197] (43)

[0198] (xi) Range: The difference between the smallest and largest values. Range can be a useful measure of spread. However, because it is based on only two values, it is sensitive to extreme scores. Due to this sensitivity, range is not usually used as the sole measure of spread, but it can be useful if used to supplement other measures of spread, such as standard deviation or semi-interquartile range.

[0199] (xii) Half-interquartile range: A measure of the spread of half the difference between the 75th percentile (Q3) and the 25th percentile (Q1). Since half of the scores in a distribution lie between Q3 and Q1, the half-interquartile range is half the distance required to cover that half of the scores. In a symmetric distribution, the distance between one half-interquartile range below the median and one half-interquartile range above the median will contain half of the scores. However, this is not the case for skew distributions. Unlike the range, the half-interquartile range is generally not substantially affected by extreme scores. However, in a normal distribution it is more susceptible to sampling fluctuations than the standard deviation and is therefore not often used to approximate normally distributed data. The half-interquartile range is defined according to the following relationship:

[0200] (44)

[0201] (xiii) Variance: A measure of the spread of a distribution. Variance is calculated by taking the mean squared deviation of each number from its mean, according to the following relationship:

[0202] (45)

[0203] (xiv) Standard Deviation: A function of the variance that measures how widely the values of a distribution are spread out from the mean. The standard deviation is the square root of the variance. While generally less sensitive to extreme scores than the range, the standard deviation is generally more sensitive than the semi-interquartile range. Therefore, the semi-interquartile range can be used to complement the standard deviation when extreme scores are likely.

[0204] (xv) Skewness: A measure of the asymmetry of a distribution around its mean. A distribution is skewed if one of its tails is longer than the other. A positive skewness indicates a distribution with asymmetric tails that extend toward more positive values (greater than the mean). A negative skewness indicates a distribution with asymmetric tails that extend toward more negative values (less than the mean). Skewness can be calculated according to the following relationship:

[0205] (46)

[0206] (xvi) Kurtosis: A measure of the steepness or flatness of a distribution compared to a normal distribution (or the relative width of its peak). A positive kurtosis indicates a relatively pointed distribution. A negative kurtosis indicates a relatively flat distribution. Kurtosis is based on the size of the tails of the distribution and can be determined by the following relationship:

[0207] (47)

[0208] The above statistical methods can be used to analyze the spatial distribution of grayscale values by calculating local features at each point in the image and obtaining a set of statistical results from the distribution of local features. Using these statistical methods, the texture of the analysis area can be described and statically defined.

[0209] Texture can be characterized using texture descriptors. Texture descriptors can be calculated over a given area of an image (discussed in more detail below). A commonly used texture method is the co-occurrence method, proposed by Haralick, R. et al., "Texture Features for Image Classification", IEEE Transactions of System, Man and Cybernetics, Vol. 3, pp. 610-621 (1973), which is incorporated herein by reference. In this method, the relative frequencies of grayscale pairs of pixels separated by a distance d in the θ direction are combined to form a relative displacement vector (d, θ). The relative displacement vector is calculated and stored in a matrix called the gray level co-occurrence matrix (GLCM). This matrix is used to extract texture features of second-order statistics. Haralick proposed 14 different features to describe the two-dimensional probability density function p ij , four of these features are more commonly used than the others:

[0210] Texture can be characterized using texture descriptors. Texture descriptors can be calculated over a given area of an image (discussed in more detail below). A commonly used texture method is the co-occurrence method, proposed by Haralick, R. et al., "Texture Features for Image Classification", IEEE Transactions of System, Man and Cybernetics, Vol. 3, pp. 610-621 (1973), which is incorporated herein by reference. In this method, the relative frequencies of grayscale pairs of pixels separated by a distance d in the θ direction are combined to form a relative displacement vector (d, θ). The relative displacement vector is calculated and stored in a matrix called the gray level co-occurrence matrix (GLCM). This matrix is used to extract texture features of second-order statistics. Haralick proposed 14 different features to describe the two-dimensional probability density function p ij , four of these features are more commonly used than the others:

[0211] (i) Angular second moment (ASM) is calculated by the following formula:

[0212] (48)

[0213] (ii) Contrast ratio (Con) is calculated by the following formula:

[0214] (49)

[0215] (iii) Correlation (Cor) is calculated by the following formula (where σ x and σ y is the standard deviation of the corresponding distribution):

[0216] (50)

[0217] (iv) Entropy (Ent) is calculated by the following formula:

[0218] (51)

[0219] These four features are also listed in Strand, J. et al., "Local Frequency Features for Texture Classification," Pattern Recognition, Vol. 27, No. 10, pp 1397-1406 (1994) [Strand94], which is also incorporated herein by reference.

[0220] For a given image, the image region where the above features are evaluated can be represented by a mask (e.g., a colony mask) or by a region extending beyond the mask. The affected area is limited.

[0221] Figure 14 Several possible regions are shown. Region 1410 is a colony mask that extends only as far as the colony itself. Region 1420 is the colony's The affected area (defined by the edge of the image or plate). To further illustrate, pixel 1430 is a pixel within area 1420 but outside area 1410. In other words, the colony represented by area 1410 is expected to extend into pixel 1630, but has not yet. Pixel 1440 is a pixel outside both areas 1410 and 1420. In other words, not only was pixel 1410 not occupied by the colony at the time of imaging, but it is also not expected to be occupied by the colony at any time in the future (in which case it is already occupied by a different colony).

[0222] As described above, use the colony masks at different time points along the incubation process and their associated Region of influence, which generates multiple histograms depicting different aspects of the colony and their impact on the local surrounding growth medium. Colony mask and The area of influence itself can be adjusted over time, for example as the colony grows. Figure 15A -C illustrates how the growth of a colony can be used to adjust the colony's mask over time. Figure 15A A portion of a blood culture grown on agar for 24 hours. Figure 15Bis a contrast image of the same culture compared to the previous image captured at t0. Figure 15C Grayscale images illustrating growth at 9 hours (lightest), 12 hours (medium), and 24 hours (darkest). Figure 15C Each shadow in can be used to design a different mask for the colony. Alternatively or additionally, as growth occurs, the colonies can be Influence area separation mask.

[0223] Any one of the features or combinations of features in the above list of features can be used as a feature set for capturing the specificity of organisms grown on various media on imaging plates under different incubation conditions. This list is not meant to be exhaustive, and those skilled in the art can modify, expand, or limit these feature sets based on the desired target to be imaged and various image processing-based features known in the art. Therefore, the above example features are provided by way of illustration and not limitation.

[0224] Those skilled in the art will recognize other measurements and methods to determine target shapes and features, and the above examples are provided by way of illustration, not limitation.

[0225] Contrast buildup

[0226] It is often difficult to initially predict which image in a series of images will provide value for growth detection, counting, or identification. This is partly because image contrast varies for different colony-forming units (CFUs) and in different culture media. In a given image of several colonies, one colony may have a highly desirable contrast with the background, while another colony may not have sufficient contrast with the background for growth detection. This also makes it difficult to identify colonies in a culture using a single method.

[0227] Therefore, it is desirable to establish contrast from all available materials, both spatially (spatial differences) and temporally (temporal differences under the same imaging conditions), as well as by using different imaging conditions (e.g., red, green, and blue channels, light and dark backgrounds, spectral images, or any other color space transformations). It is also desirable to collect contrast from multiple available sources to provide standardized images as input to algorithms for detecting colonies.

[0228] Image data can be limited based on any number of factors. For example, image data can be limited to specific time points and / or specific information sought (e.g., spatial image information may not require as many time points as temporal image information). Illumination configurations and color spaces can also be selected to achieve specific contrast targets. Spatial frequency can also be varied to detect objects of a desired size (or within a target range).

[0229] To detect discrete objects, contrast can be set to an absolute value between [0, 1], or to a signed value between [-1, 1]. The scale and offset of the contrast output can also be specified (e.g., for an 8-bit image with signed contrast, the offset can be 127.5 and the scale can be 127.5). In instances where contrast is set to an extreme value, the absolute offset can be set to 0 and the scale to 256.

[0230] Spatial contrast can be used to detect discrete objects on a uniform background. The spatial contrast at a position (x, y) within a distance r on an image I can be given by the equation In one embodiment, the distance r is defined as being greater than or equal to The distance, and the contrast operator K is used to control the contrast setting, using the following equation:

[0231] (52)

[0232] (53)

[0233] Temporal contrast can be used to detect moving objects or objects that change over time (e.g., CFUs that appear and / or expand on an imaging plate). The time t0 and t x Time contrast between the two positions (x, y) In one embodiment, the following equation is used:

[0234] (54)

[0235] Spatial contrast collection can be performed in an automated manner by generating multiple SHQI images of the plate according to a pre-programmed sequence. Multiple images can be generated at a given incubation time to further investigate colony detection. In one embodiment, the spatial ... min to R max ), several configurations (from CFG1 to CFG N ) collects the image data (specifically, the vector "vect" used to provide the contrast input to the contrast collection operator):

[0236] (55)

[0237] If for a given image I (x,y), the SNR is known (e.g., when SHQI imaging is the source), then the configuration in which the SNR-weighted contrast is maximized can be identified as the best configuration (Best CFG) when the following equation is satisfied:

[0238] (56) is the maximum value, exceeding:

[0239] (57)

[0240] The contrast operator K further benefits from this known SNR information, and the above equation becomes:

[0241] (58)

[0242] Time contrast collection can also be performed in an automated manner by generating multiple SHQI images of the plate according to a pre-programmed sequence. Multiple images can be generated at multiple incubation times, at least one of which is t0, to further investigate colony detection. In one embodiment, the time at t0 and one or more subsequent times up to t0 is calculated according to the following equation: x Image data were collected for several configurations with an incubation time of:

[0243] (59)

[0244] In the above example, the vector can be two time points (eg, t0 and t x ), which is based on the difference in the images at those two times. However, in other applications, including x The extra time between the two points can be used to draw a vector with as many points as the time it took to capture the image. Mathematically, there is no limit to the number of points that can be included in a vector.

[0245] As with spatial contrast, if for a given image I (x,y) , the SNR is known, then the configuration in which the SNR-weighted contrast is maximized can be identified as the optimal configuration (optimal CFG) when the following equation is satisfied:

[0246] (60) is the maximum value, exceeding

[0247] (61)

[0248] The contrast operator K further benefits from this known SNR information, and the above equation becomes:

[0249] (62)

[0250] In the above example, the maximum operator can be replaced by any other statistical operator, such as a percentile (e.g., Q1, median, Q3, or any other percentile) or a weighted sum. The weighted values can come from previous work extracted from a training database, thereby opening up the field of supervised contrast extraction to neural networks. Alternatively, multiple algorithms can be used, and the results of the multiple algorithms can be further combined using another operator, such as the maximum operator.

[0251] Image alignment

[0252] When multiple images are captured over time, in order to obtain valid time estimates from them, the images need to be aligned very accurately. Such alignment can be achieved by means of mechanical alignment devices and / or algorithms (e.g., image tracking, image matching). Those skilled in the art are already aware of these schemes and algorithms for achieving this goal.

[0253] For example, where multiple images of an object on a flat panel are collected, the coordinates of the object's location can be determined. Based on these coordinates, image data of the object collected at a later time can then be correlated with the previous image data and subsequently used to determine changes in the object over time.

[0254] In order to quickly and effectively utilize images (e.g., when used as input to a classifier), it is important to store the images in a spatial reference system to maximize their invariance. Because the basic shape descriptor of a colony is generally circular, a polar coordinate system can be used to store colony images. When a colony is first detected, the colony centroid can be identified as the center of the colony's location. This center point can then be used as the initial center for the polar coordinate transformation of each subsequent image of the colony. Figure 16A An enlarged portion of an imaging plate is shown with a center point "O". Two rays "A" and "B" extending from point "O" are shown overlaid on the image (for clarity). Each ray intersects a respective colony (circled). Figure 16A The encircled colonies are Figure 16B This is shown in more detail in images 1611 and 1612. Figure 16B In FIG. 1 , image 1611 (the colony intersected by ray “A”) is reoriented within image 1613 (“A′”) such that the radial axis of image 1613 is aligned with the radial axis of image 1612, so that the leftmost portion of the reoriented image is aligned with the leftmost portion of the image 1613. Figure 16A The polar coordinates of the image are closest to point "O" and the rightmost portion of the reoriented image is farthest from point "O." This polar coordinate reorientation allows for easier analysis of colonies in different orientations of the imaged plate (taking into account factors such as lighting).

[0255] exist Figure 16C In, right Figure 16BPolar transformation is performed on each of the images 1611, 1612 and 1613. In the polar transformed images 1621, 1622 and 1623, the radial axis (extending from the center of each respective imaged colony) of the respective reoriented images 1611, 1612 and 1613 is at Figure 16C The angular axes (of the respective colonies) are drawn from left to right in the image and from top to bottom.

[0256] For each polar image, a summary set of one-dimensional vectors may be generated along the radial and / or angular axes using, for example, shape features and / or histogram features (e.g., the mean and / or standard deviation of the color or intensity of the objects). Even when considering rotation, shape and histogram features are generally invariant, it is possible that some texture features show significant changes upon rotation; thus, invariance cannot be guaranteed. Therefore, it is significantly beneficial to present each colony image from the same viewpoint or angle of illumination, since object texture differences can then be used to distinguish one from another. Since illumination conditions generally show changes with angular position about the imaging center of the plate, rays passing through the colony and the center of the plate (at Figure 16B ) shown as a line in each of the images 1611 , 1612 , and 1613 may serve as the origin (θ) of each image polar coordinate transformation.

[0257] A further alignment challenge arises from the fact that the plate culture medium is not absolutely frozen and rigid, and can therefore shift slightly from one shot to the next. Therefore, it cannot be absolutely assumed that a plate region at certain coordinates in an image captured at one time will necessarily align perfectly with a plate region at the same coordinates captured at a later time. In other words, slight deformations of the culture medium can lead to small uncertainties in precisely matching a given pixel with a corresponding pixel captured at a different time point during the incubation process.

[0258] To take this uncertainty into account, the time t a The intensity value of a given pixel Can be compared with different time points t b The closest intensity value within the local neighborhood N(x,y) of the pixel Selection of the local neighborhood may include determining one or more errors or inaccuracies in the repositioning of the imaging plate from one time to the next (e.g., positional accuracy errors due to imperfect repositioning of the imaged medium, parallax errors due to unknown heights of the imaged medium from one time to the next). In one example, it has been observed that setting "x" and "y" within a range of about 3 to about 7 pixels is suitable for images having a resolution of about 50 microns per pixel.

[0259] Anyone skilled in the art will recognize thatb The source image generates two t b Valid solutions for images: The first image corresponds to t b Grayscale dilation (called ) and the second image corresponds to t b Grayscale erosion (called ), both with kernel sizes matching the relocalization distance uncertainty d.

[0260] if Then the contrast is 0, otherwise the contrast is expressed as follows: and middle The closest estimate of :

[0261] (63)

[0262] (64) if

[0263] (65) if

[0264] Improvement in SNR

[0265] Under typical lighting conditions, photon shot noise (the statistical variation in the arrival rate of incident photons on the sensor) limits the SNR of the detection system. Modern sensors have sufficient capacity, which is approximately 1,700 to 1,900 electrons per active square micron. Therefore, when imaging a target on a flat panel, the primary concern is not the number of pixels used to image the target, but the area covered by the target within the sensor volume. Increasing the area of the sensor improves the SNR for imaging the target.

[0266] Image quality can be improved by capturing images using lighting conditions where the photon noise dominates the SNR. Without saturating the sensor (the maximum number of photons that can be recorded per pixel per frame). To maximize SNR, image averaging techniques are often used. Because the SNR in dark areas is much lower than that in bright areas, these techniques are used to process images with significant brightness (or color) differences, as shown in the following formula:

[0267] (66)

[0268] Where I is the average current generated by the electron flow at the sensor. Color is perceived due to differences in the absorption / reflection of matter and light across the electromagnetic spectrum, and confidence in the captured color depends on the system's ability to record intensity with a high SNR. Image sensors (e.g., CCD sensors, CMOS sensors, etc.) are well known to those skilled in the art and will not be described in detail here.

[0269] To overcome the classic SNR imaging limitation, the imaging system can analyze the imaged panel during image acquisition and adjust the lighting conditions and exposure time in real time based on this analysis. This process is described in PCT Publication No. WO2015 / 114121, which is incorporated by reference, and is generally referred to as supervised high-quality imaging (SHQI). The system can also customize imaging conditions for different brightness regions of the panel within different color channels.

[0270] For a given pixel x, y of an image, the SNR information of the pixel acquired during the current frame N can be combined with the SNR information of the same pixel acquired during the previous or subsequent frames (e.g., N-1, N+1). As an example, the combined SNR is determined by the following formula:

[0271] (67)

[0272] After updating the image data with new acquisitions, the acquisition system is able to predict the best next acquisition time that will maximize the SNR according to environmental constraints (e.g., the minimum required SNR per pixel in the region of interest). For example, averaging 5 images captured under non-saturated conditions will result in an improvement in the SNR of dark areas (10% of maximum intensity). When combining information from two images captured in bright and dark conditions, optimal illumination in only two acquisitions will contribute to improved SNR in the dark areas.

[0273] Image Modeling

[0274] In some cases, when calculating spatial or temporal contrast between pixels of one or more images, pixel information for a given image may be unavailable or may be degraded. For example, unavailability may occur if an image of a plate was not captured at a time prior to bacterial growth (e.g., the plate was not imaged at time t0 or shortly thereafter). For example, degradation of signal information may occur when an image is captured at time t0, but the pixels of the captured image do not accurately reflect the imaged plate prior to bacterial growth. Such inaccuracies may be caused by temporal artifacts that do not reappear in subsequent time series images of the plate (e.g., condensation that temporarily forms on the bottom of the plate due to thermal shock when the plate is first placed in the incubator).

[0275] In such cases, the unavailable or weakened image (or certain pixels in the image) can be replaced or enhanced with a model image of the plate. The model image can provide pixel information that reflects how the plate is expected to look at the specific time of the unavailable or weakened image. In the case of the model image at time t0, the model can be a clear (plain) or standard image of the plate and can be mathematically constructed using three-dimensional imaging / modeling techniques. The model can include each of the physical design parameters (e.g., diameter, height, dividers for accommodating multiple culture media, plastic materials, etc.), culture medium parameters (e.g., type of culture medium, culture medium composition, culture medium height or thickness, etc.), lighting parameters (e.g., angle of the light source, color or wavelength (one or more) of the light source, color of the background, etc.), and positioning parameters (e.g., the position of the plate in the imaging chamber) to produce a model that is as realistic as possible.

[0276] In the case of a weakened image, when pixel information is less sharp than the rest of the image (e.g., because underlying condensation blocks some light from passing through the plate and thus makes the portion of the plate with condensation slightly less transparent), signal restoration can be used to sharpen the weakened pixel features in the image to enhance the weakened image. Signal restoration can include determining intensity information for the rest of the image, identifying a median intensity of the intensity information, and then replacing the intensity information of the less sharp areas of the image with the median intensity of the rest of the image.

[0277] application

[0278] This disclosure is largely based on testing performed in saline at varying dilutions to simulate typical urine reporting volumes (CFU / ml bucket group). The suspension for each isolate was adjusted to a 0.5 McFarland Standard and used for an estimated 1×10 6 , 1×10 5 , 5×10 4 , 1×10 4 , 1×10 3 and 1×10 2 Dilutions were prepared at a suspension of 100 CFU / ml. Using Kiestra InoqulA (WCA1), sample tubes were treated using a standard urine streak pattern of 4# zigzag (0.01 ml per plate).

[0279] Use ReadA Compact to process flat board (35 ℃, non-CO2) and incubate for 48 hours in total, image every two hours for the first 24 hours, and every 6 hours for the second 24 hours. Incubation time is entered as first reading at 1 hour, and the margin that allows is set to + / -15 minutes. For the next reading in 2-24 hours, be set to every 2 hours, the margin that allows is + / -30 minutes. For reading 24-48 hours, be set to every 6 hours, the margin that allows is + / -30 minutes. After pure feasibility study, it is changed to eliminate the margin that allows. Do this to improve image acquisition in the desired 18-24 hour time range.

[0280] In other cases, images may be acquired over a 48-hour span—at 2-hour intervals for the first 24 hours and at 6-hour intervals for the next 24 hours. In such a case, a total of 17 images would be acquired over the 48-hour span, including the image acquired from time t0 (0 hours).

[0281] All acquired images are corrected for lens geometry and chromatic aberrations and spectrally balanced using a known target pixel size, normalized lighting conditions, and a high signal-to-noise ratio per pixel per band. Suitable cameras for use in the methods and systems described herein are known to those skilled in the art and will not be described in detail herein. As an example, using a 4 megapixel camera to capture images of a 90 mm plate should allow enumeration of up to 30 colonies / mm when the colonies are in the range of 100 μm in diameter and have sufficient contrast. 2 The local density (>10 5 CFU / plate).

[0282] The contrast of colonies grown on the following media was assessed:

[0283] TSAII 5% Blood of Sheep (BAP) is a non-selective medium widely used for urine culture.

[0284] BAP is used for colony counts and putative ID based on colony morphology and hemolysis.

[0285] MacConkey II Agar (MAC): A selective medium for the most common Gram-negative UTI pathogens. MAC is used to differentiate lactose-producing colonies. MAC also inhibits colonization of Proteus species. BAP and MAC are commonly used for urine cultures. Due to partial inhibition of some Gram-negative species, some media are not recommended for colony counts.

[0286] Colistin Nalidixic Acid Agar (CNA): A selective medium for the most common Gram-positive UTI pathogens. CNA is not as commonly used for urine cultures as MAC, but it can help identify colonies if overgrowth of Gram-negative colonies occurs.

[0287] Chromogenic Positioning Agar (CHROMA): A nonselective medium widely used for urine cultures. CHROMA is used for colony counts and identification based on colony color and morphology. Escherichia coli and enterococci are identified using this medium, and confirmatory testing is not required. Due to cost considerations, CHROMA is used less frequently than BAP. CLED medium is also used for mixed samples.

[0288] Cystine Lactose Electrolyte Deficiency (CLED) Agar: Based on lactose fermentation, used for colony counts and presumptive ID of urinary pathogens.

[0289] Sample processing BD Kiestra TM InoqulA TM For automated processing of bacteriological specimens to enable standardization and ensure consistent and high-quality streaking. BD Kiestra TM InoqulA TM The sample processor uses magnetic bead technology to streak the culture medium plate using a customizable pattern. The magnetic beads are 5mm in diameter.

[0290] Figure 17 An example of the performance of the contrast collection algorithm in searching for Morganella morganii growth on both chromogenic agar (top) and blood agar (bottom) media is illustrated. Both chromogenic agar and BAP are non-selective growth media widely used in microbiology laboratories. Figure 17 Each of the middle images shows the imaged plate illuminated by light from above the plate (top illumination). The left image shows the corresponding spatial contrast of the middle image. The spatial contrast is based on a 1 mm median kernel. Finally, the right image shows the temporal contrast of the middle image. The temporal contrast is calculated by comparing the spatial contrast at 0 hours (t0) and 12 hours (t x ) incubations, using different lighting conditions (different color channels, lighting settings, etc.) at each image capture time.

[0291] like Figure 17 As shown in Figure 2, local contrast has its limitations when dealing with semi-transparent colonies, especially when the edge transition is small. This is because using spatial contrast algorithms alone, only the confluence areas with the strongest contrast can be selected from the image. Spatial contrast may not be able to select the target. Figure 17Evident in Figure 3 is the effectiveness of temporal contrast for isolating colonies.

[0292] Figure 17 The blood agar image (particularly the bottom one) also highlights the problem of using spatial contrast alone to detect growth on transparent media. Ink printed on a transparent case has very strong edges and, therefore, strong spatial contrast. This ultimately makes any other spatial contrast very difficult to see, since the colonies do not have such strongly defined edges. Therefore, in the present disclosure, temporal contrast is used in conjunction with spatial contrast to considerable advantage.

[0293] Ultimately, as a result of the above contrast determination, a method for rapid detection and identification of bacterial colonies in imaged culture medium can be automated. This automated method offers significant advantages over comparable manual methods.

[0294] Figure 18 A pair of flow charts are shown that compare the timeline of the automated testing process 1800 (e.g., Figure 2 200) and a comparable timeline for a manually performed testing process 1805. Each process begins with the laboratory receiving the sample for testing 1810, 1815. Each process is followed by an incubation 1820, 1825 during which the sample can be imaged multiple times. In the automated process, after approximately 12 hours of incubation, an automated assessment 1830 is performed, after which time it can be determined definitively whether there is no growth (or normal growth) in the sample 1840. Figure 17 As shown in the results of the automated process, the use of temporal contrast significantly improved the ability to detect colonies—even after only 12 hours. In contrast, in the manual process, manual assessment was not possible until nearly 24 hours into the incubation process. 1835 Only after 24 hours could it be definitively determined whether no growth (or normal growth) was present in the sample. 1845

[0295] Figure 19A timeline is illustrated in which semi-quantification of colony counts can be determined. Plate 2005 is inoculated at 2000. A reference image of the newly inoculated plate 2005 is obtained at 2010. This is the background image described earlier herein. Incubation is performed for a predetermined time interval, after which another image of plate 2005 is obtained at 2015. This image is compared to the image obtained at 2010 for evidence of target changes or artifacts that may indicate microbial growth. If there is no indication of microbial growth, the plate is incubated and, after a predetermined time, another image is obtained at 2020. At step 220, changes or artifacts in the target are detected as evidence of microbial growth. From an image analysis perspective, growth in an image can be detected by identifying the imaged target (based on the differences between the target and its surroundings) and then identifying changes in the target over time. As described in more detail above, these differences and changes are two forms of "contrast." In addition to detecting growth, the image analysis at 2020 can further involve determining whether the target identified as a colony has sufficient biomass to be harvested for analysis. Figure 19 This indicates that insufficient biomass was identified at step 2020 and further incubation is required. At 2025, the image demonstrates colonies, but those colony targets require further evaluation to not only quantify the colony, but also to assess the nature of the colony (i.e., whether it is a sister of other identified colony targets; whether it is a pure (i.e., single microbial species) colony or a mixed (i.e., multiple microbial species) colony). Colony confluence and how it is determined are described in the above section describing target segmentation. If there is no growth, or an insignificant amount of growth is found after a predetermined time, then at 2022 the plate 2005 is released as sterile. The final report may indicate no significant growth, or report growth of a normal flora.

[0296] The biomass is determined by associating the target with the biomass. The size of the target is determined and compared to a predetermined target size that indicates a threshold biomass. In step 2025, if the target is formed from a pure sample, then the biomass can be picked for downstream testing when the area of the culture medium covered by the target reaches or exceeds a first threshold. If the biological sample is not a pure sample, the inoculated sample dish is further incubated. If the targets of interest have not all been picked based on the image obtained at step 2025, the inoculated culture plate is further incubated and imaged again at step 2030. The size of the target associated with the colony in the image is compared to a second predetermined size threshold. If the size of the target, and therefore the biomass, is above the second threshold, then at least part of the biomass is picked. The second predetermined threshold is one of: i) a threshold area covered by the biomass if the target is from a pure sample; or ii) a diameter of the biomass if the target is from an impure sample. The second predetermined threshold biomass is greater than the first threshold. This can be achieved by Figure 19 This can be seen by comparing the target size at step 2025 with the target size at step 2030. The target in step 2030 is significantly larger than the target in step 2025, but the target identified in step 2025 could be picked if there is confidence that the target / biomass is from a pure sample.

[0297] If it is determined that the biological sample shows significant growth in number, then at 2025 or 2030 (depending on when a target colony with sufficient biomass is detected), one or more colonies can be identified as candidate colonies to be picked for analysis. Picking colonies can be a fully automated process, in which each picked colony is sampled and tested. Alternatively, picking colonies can be a partially automated process, in which multiple colony candidates are automatically identified and visually presented to the operator in a digital image, allowing the operator to input a selection of one or more candidates for sampling and further testing. The sampling of the selected or picked colonies can itself be automated by the system. With respect to 2025 and 2030, the time at which colonies can be picked depends on what is known about the observed colonies when they are first observed at 2025. If the colonies develop from a presumed pure sample (e.g., a deep wound or incision), colonies can be picked once there is a sufficient number to observe and collect them. The time to process such samples is critical, and avoiding one or more additional incubation cycles means that test results can be provided hours earlier. If the sample is not putatively pure, additional incubation periods are required to allow further growth of the colonies to aid in identification and quantification of these colonies.

[0298] Reference again Figure 18 The use of automated processes also allows for faster AST and MALDI testing. Such testing 1850 in an automated process can begin shortly after the initial assessment 1830, and results can be obtained 1860 and reported 1875 by the 24-hour mark. In contrast, such testing 1855 in a manual process is typically not started until near the 36-hour mark and takes an additional 8 to 12 hours to complete before the data can be reviewed 1865 and reported 1875.

[0299] In summary, the manual testing process 1805 is shown to take up to 48 hours, requires an 18-24 hour incubation period, only after which the plate is evaluated for growth, and has no way to track how long the sample has been incubated. In contrast, because the automated testing process 1800 can detect even relatively poor contrast between colonies (compared to the background and to each other), and can perform imaging and incubation without the microbiologist having to keep track of the timing, only 12-18 hours of incubation are required before the sample can be identified and prepared for further testing (e.g., AST, MALDI), and the entire process can be completed in approximately 24 hours. Thus, the automated process of the present disclosure, with the aid of the contrast processing described herein, provides for faster testing of samples without adversely affecting the quality and accuracy of the test results.

[0300] Although the present disclosure has been described with reference to specific embodiments, it should be understood that these embodiments are merely illustrative of the principles and applications of the present disclosure. Therefore, it should be understood that various modifications may be made to the illustrative embodiments, and other arrangements may be designed without departing from the spirit and scope of the present disclosure as defined by the appended claims.

Claims

1. A non-transitory machine-readable storage medium storing instructions that, when executed, cause a processor of a digital imaging device to: obtaining a first digital image of the inoculated culture medium at a first time (t0), the first digital image having a plurality of pixels; determining coordinates of the pixel in the first digital image relative to a substantially optically transparent container carrying the inoculated culture medium; providing instructions for removing the substantially optically transparent container carrying the inoculated culture medium from the digital imaging device and placing the inoculated culture medium in an incubator for further incubation; At the second time (t x ) obtaining a second digital image of the inoculated culture medium, the second digital image having a plurality of pixels; aligning the first digital image with the second digital image so that coordinates of the plurality of pixels in the second digital image correspond to determined coordinates of each corresponding pixel in the plurality of pixels in the first digital image; comparing pixels of the second digital image with corresponding pixels of the first digital image; identifying pixels that changed between the first digital image and the second digital image, wherein pixels that did not change between the first digital image and the second digital image designate background; determining which identified pixels in the second digital image have a predetermined level of threshold contrast with pixels of a designated background; identifying one or more objects in the second digital image, each object comprising pixels having the predetermined level of threshold contrast with the pixels indicative of background and not separated from one another by background pixels; correlating the identified target with biomass; determining whether the biomass is from a pure biological sample; responsive to determining that the biological sample is identified as a pure sample, further determining whether the biomass is above a first threshold, the first threshold being a predetermined area of the culture medium covered by the identified target, and responsive to determining that the area of the identified target is above the first threshold, causing at least a portion of the biomass to be picked for further analysis; and In response to determining that the biological sample is not a pure sample, the inoculated culture medium is incubated in the substantially optically transparent container.

2. The non-transitory machine-readable storage medium storing instructions of claim 1 , wherein in response to not identifying a target after a predetermined period of time, the processor marks the substantially optically transparent container with the inoculated culture medium as a no-growth plate.

3. The non-transitory machine-readable storage medium storing instructions according to claim 2, further comprising causing the processor to: At the third time (t y ) obtaining a third digital image of the inoculated culture medium, the third digital image having a plurality of pixels; identifying additional pixels that change from the second digital image to the third digital image; determining which identified pixels and further identified pixels in the third digital image have a predetermined level of threshold contrast with the pixels indicative of background; identifying one or more objects in the third digital image, each object comprising pixels having the predetermined level of threshold contrast with the pixels indicative of background and not separated from one another by background pixels; correlating the identified target with biomass; and In response to determining that the biomass is above a second threshold, wherein the second threshold is one of: i) a threshold area covered by the biomass when the target is from a pure sample; or ii) a diameter of the biomass when the target is from an impure sample; and At least a portion of the biomass is picked for analysis, wherein the second threshold is greater than the first threshold.

4. The non-transitory machine-readable storage medium storing instructions of claim 1, wherein the alignment is based on a position of a fiducial marking on the substantially optically transparent container.

5. The non-transitory machine-readable storage medium storing instructions according to claim 3, further comprising causing the processor to: The third digital image is aligned with the second digital image, wherein the alignment is based on a position of a fiducial mark on the substantially optically transparent container.

6. A non-transitory machine-readable storage medium storing instructions according to claim 4, wherein the reference mark is selected from: an off-center optically detectable mark point on the bottom of the substantially optically transparent container, an end of an optically detectable label set on the side of the substantially optically transparent container, and a center of the optically detectable label on the substantially optically transparent container.

7. The non-transitory machine-readable storage medium storing instructions according to claim 1 , further comprising causing the processor to: A plurality of first digital images are acquired at the first time according to a predetermined series of lighting conditions, wherein each of the first digital images is acquired under a different lighting condition, each lighting condition comprising a specified orientation of the substantially optically transparent container carrying the inoculated culture medium relative to an illumination source, and a specified background color on which the substantially optically transparent container is placed within the digital imaging device.

8. The non-transitory machine-readable storage medium storing instructions of claim 1 , wherein the processor controls a device comprising: an illumination source directed downwardly toward a top of said substantially optically transparent container carrying said inoculated culture medium; an illumination source directed upwardly toward a bottom of the substantially optically transparent container carrying the inoculated culture medium; and An illumination source is directed toward the side of the substantially optically transparent container carrying the inoculated culture medium.

9. The non-transitory machine-readable storage medium storing instructions according to claim 8, the instructions comprising: specifying an orientation of the substantially optically transparent container relative to the overhead illumination source; specifying an orientation of the substantially optically transparent container relative to the side illumination source; An orientation of the substantially optically transparent container relative to the bottom illumination source is specified.

10. The non-transitory machine-readable storage medium storing instructions of claim 8, wherein the illumination source of the device comprises an illumination source that emits red wavelengths; an illumination source that emits green wavelengths; and an illumination source that emits blue wavelengths.

11. The non-transitory machine-readable storage medium storing instructions of claim 1, wherein the target is a colony of microorganisms.

12. A non-transitory machine-readable storage medium storing instructions according to claim 5, wherein the reference mark is selected from: an off-center optically detectable mark point on the bottom of the substantially optically transparent container, an end of an optically detectable label set on the side of the substantially optically transparent container, and a center of an optically detectable label on the substantially optically transparent container.

Images