Colony contrast collection

Abstract

The invention is entitled Colony Contrast Collection Imaging systems and methods for microbial growth detection that use differences that develop over time to assess whether microbial growth has occurred for a given sample.

Description

[0001] This application is a divisional application of a patent application with application date of April 22, 2016, application number 2016800229997, and title “Colony Contrast Collection”.

[0002] CROSS-REFERENCE TO RELATED APPLICATIONS

[0003] This application claims the benefit of the filing dates of U.S. Provisional Application No. 62 / 151,681, filed April 23, 2015, and U.S. Provisional Application No. 62 / 318,483, filed April 5, 2016, the disclosures of which are hereby incorporated herein by reference. Background Art

[0004] There is increasing concern about digital imaging of culture plates for detecting microbial growth. The technology for imaging plates for detecting microbial growth is described in PCT Publication No. WO2015 / 114121, which is incorporated herein by reference in its entirety. Using this technology, laboratory personnel no longer need to read plates by direct visual inspection, but can use high-quality digital images for inspection of plates. Changing laboratory workflow and decision-making for digital image inspection of culture plates can also improve efficiency. Images can be calibrated by the operator for further inspection (work-up) by the operator or another person with appropriate skills. Additional images can also be captured and used to guide secondary processing.

[0005] Detection of colonies, colony enumeration, differentiation of colony populations, and identification of colonies define the goals for modern microbiology imaging systems. Achieving these goals as early as possible achieves the goal of delivering results to patients quickly and providing these results and analyses economically. Automating laboratory workflow and decision making can improve the speed and cost at which these goals can be achieved.

[0006] Although significant advances have been made in imaging technology for detecting signs of microbial growth, there is still a need to expand such imaging technology to support automated workflows. Instruments and methods for inspecting culture plates for indications of microbial growth are difficult to automate, in part due to the highly visual nature of plate inspection. In this regard, it is desirable to develop technology that can automatically interpret culture plate images and determine subsequent steps to be performed (e.g., identification of bacterial colonies, sensitivity testing, etc.) based on the automated interpretation.

[0007] For example, identification and differentiation of bacterium colonies on culture plates may be difficult, especially when bacterium colonies have different sizes and shapes and contact each other. When growth has reached convergence in some areas of plates, these problems are more serious. For these reasons, if possible, preferably identify bacterium colonies in the early stages of the process and determine growth. However, the time for incubation is still needed to allow at least some growth of bacterium colonies. Therefore, on the one hand, the longer the bacterium colonies are allowed to grow, the stronger the contrast between them and their background and each other is formed, and the easier it becomes to identify them. However, on the other hand, if it is allowed that bacterium colonies grow too long, and they begin to fill plates and / or contact each other, it becomes more difficult to compare them with their background and compare them with each other. If it is possible to detect bacterium colonies at the incubation time when bacterium colonies are still small enough to separate from each other, although the contrast is relatively poor, then this problem can be minimized or even solved.

[0008] Summary of the invention

[0009] One aspect of the present disclosure relates to an automated method for evaluating microbial growth on a plated culture medium, the method comprising: providing a culture medium inoculated with a biological sample disposed in a substantially optically transparent container; incubating the inoculated culture medium in an incubator; placing the transparent container with the inoculated culture medium in a digital imaging instrument; obtaining a first digital image of the inoculated culture medium at a first time (t0), the first digital image having a plurality of pixels; determining coordinates of the pixels within the first image relative to the container carrying the inoculated culture medium; removing the container carrying the inoculated culture medium from the digital imaging instrument and placing the inoculated culture medium in an incubator for further incubation; after further incubation, placing the container carrying the inoculated culture medium in the digital imaging instrument; at a second time (t x ) obtain a second digital image of the inoculated culture medium, the second digital image having a plurality of pixels; align the first digital image with the second digital image so that the coordinates of the pixels in the second digital image correspond to the coordinates of the corresponding pixels in the first digital image; compare the pixels in the second digital image with the corresponding pixels in the first digital image; identify pixels that change between the first digital image and the second digital image, wherein pixels that do not change between the first digital image and the second digital image indicate background; determine which of the identified pixels in the second digital image have a predetermined level of threshold contrast with pixels indicating background; identify one or more targets in the second digital image, each target consisting of pixels that meet the level of threshold contrast and which are not separated from each other by background pixels; for at least one of the identified targets, determine the morphology of the target from the pixels of the target; determine whether the target is a colony candidate from the morphology of the target; and provide the coordinates of the pixels associated with the target to a memory.

[0010] In some examples, the method may further include obtaining a plurality of first digital images at a first time according to a predetermined series of lighting conditions, wherein each first digital image is obtained under a different lighting condition, each lighting condition including a specified orientation of the optically transparent container carrying the inoculated culture medium relative to an illumination source, and a specified background color on which the optically transparent container is placed in the digital imaging instrument. The specified orientations may include: the illumination source pointing downward at the top of the optically transparent container carrying the inoculated culture medium; the illumination source pointing upward at the bottom of the optically transparent container carrying the inoculated culture medium; and the illumination source pointing at the side of the optically transparent container carrying the inoculated culture medium. For the specified top and side orientations, the specified background color may be black. For the specified bottom orientation, the specified background color may be white. The lighting conditions may further include a specified illumination spectrum, which includes an illumination source emitting red wavelengths; an illumination source emitting green wavelengths; and an illumination source emitting blue wavelengths.

[0011] In some examples, the method may further include obtaining a target feature from pixel information associated with the target, the target feature including at least one of a target shape, a target size, a target edge, and a target color, wherein the target morphology is determined based on the target feature. The target color may be determined by a spectral feature of a pixel associated with the target. The spectral feature may be selected from pixel color, hue, illumination, and chromaticity. Background feature information may also be obtained. The background feature information may include a culture medium type and a culture medium color, and the target morphology may be further determined based on the background feature information. The target feature and the background feature information may be compared with other target features and other background feature information stored in a memory, and the type of microorganism may be determined based on the target feature and the background feature information.

[0012] In some examples, aligning the first digital image with the second digital image may include assigning polar coordinates to pixels of each of the first and second digital images such that the polar coordinates of the pixels in the second digital image are the same as the polar coordinates of the corresponding pixels in the first digital image. Likewise, in some examples, a plurality of culture media may be provided, inoculated with the biological sample, arranged in one or more optically transparent containers, incubated in an incubator, and placed in a digital imaging instrument while the bacteria are growing, thereby obtaining first and second digital images for each culture media. Additionally, in some examples, the method may further include identifying pixel information in the first digital image that is evidence of condensation on the optically transparent container or the plated culture medium, and subtracting the pixel information attributable to condensation from the image. Pixel information that may be identified as evidence of condensation in a digital image for which the illumination source is directed upward toward the bottom of the optically transparent container, and in which pixel information of those pixels having an optical density below a predetermined threshold is subtracted from the image.

[0013] In some examples, the method may further include: identifying pixel information in either of the first and second digital images that is evidence of dust, and subtracting the pixel information attributable to dust from the image. Pixel information that may be identified as evidence of dust in a digital image for which the optically transparent container contains white culture medium, an illumination source is directed downwardly toward the top of the optically transparent container, and the background color is black; the optically transparent container contains colored or black culture medium, an illumination source is directed downwardly toward the top of the optically transparent container, and the background color is white; or an illumination source is directed upwardly toward the bottom of the optically transparent container. The dust may be dust on any culture medium, on the optically transparent container, or on an optical component of a digital imaging instrument.

[0014] In some examples, the method may further include: obtaining a third digital image of the inoculated culture medium, wherein the third digital image is obtained at a time between the time the first digital image is obtained and the time the second digital image is obtained, wherein between the time the first and second digital images are obtained and between the time the second and third digital images are obtained, the inoculated culture medium is removed from the digital imaging instrument and placed in an incubator; aligning the third digital image with the first digital image so that the coordinates of the pixels in the third digital image are the same as the coordinates of the corresponding pixels in the first digital image; comparing the pixels in the third and second digital images to each other; and identifying pixels that change between the third and second digital images. The incubation time between the first and third digital images may be equal to the incubation time between the third and second digital images. At least one of the identified targets may be associated with the identified pixels that change between the third and second digital images.

[0015] In some examples, the method may further include: obtaining a third digital image of the inoculated culture medium, wherein the third digital image is obtained after the time when the second digital image is obtained, wherein between the time when the second digital image and the third digital image are obtained, the inoculated culture medium is removed from the digital imaging instrument and placed in an incubator; aligning the third digital image with the first and second digital images so that the coordinates of the pixels in the third digital image are the same as the coordinates of the corresponding pixels in the first and second digital images; comparing the pixels of the third and second digital images to each other; identifying the pixels that have changed in the second and third digital images; and determining that the target identified in the second digital image has changed in the third digital image based on the compared pixels in the second and third digital images identified as having changed. The target identified in the second digital image that has been determined to have changed in the third digital image may be determined as a seed target, from which the degree of the determined change is evaluated. In such a case, the method may further include updating the coordinates of the pixels associated with the target in the memory based on the pixels identified as having changed between the second and third digital images.

[0016] Another aspect of the present disclosure relates to a method for identifying microbial growth on a plated medium that has been inoculated with a culture and incubated, the method comprising: at the beginning of incubation of the medium (t0), obtaining a first digital image of the medium, the first digital image having a plurality of pixels; assigning coordinates to one or more pixels of the first digital image; after an incubation period of the medium (t x ), obtaining a second digital image of the culture medium, the second digital image having a plurality of pixels; aligning the second digital image with the first digital image, wherein the alignment is based on coordinates assigned to pixels of the first image and coordinates assigned to one or more pixels in the second image corresponding to pixels to which coordinates are assigned in the first image; generating spatial contrast data indicating changes between local adjacent pixels in the second digital image; generating temporal contrast data indicating changes between corresponding pixels of the first and second digital images; for each of the plurality of pixels of the second digital image, assigning a contrast value to the pixel based on a combination of the spatial contrast data and the temporal contrast data for the pixel; associating adjacent pixels having contrast values ​​greater than a predetermined threshold and within a predetermined margin of error from each other, the associated pixels constituting targets for identification; and storing each identified target in a memory as a candidate colony.

[0017] In some examples, combining the spatial and temporal contrast data includes averaging the spatial and temporal contrast data. The spatial and temporal contrast data may be combined according to a weighted average.

[0018] Generating spatial contrast data may include: at time t x Acquire multiple images, each of the multiple images is acquired under different lighting conditions; process spatial data in each of the multiple t0 images; and combine the processed spatial data. Processing the spatial data may include separately processing the spatial data results for each lighting condition and selecting the largest result from the separately processed spatial data results.

[0019] Generating temporal contrast data may include: acquiring a plurality of images at time t0, each of the plurality of images being acquired under different lighting conditions; x Get multiple images at time t x The illumination condition of each image corresponds to the illumination condition of the image acquired at time t0; processing the corresponding t0 and t x The processing of the temporal data may include processing the temporal data results for each lighting condition separately and selecting a maximum result from the separately processed temporal data results.

[0020] In some examples, the method may further include, for a given identified target: obtaining a plurality of target features from pixel information associated with the target, wherein the target features include at least one morphological feature, the morphological feature being at least one of a target shape, a target area, a target perimeter, or a target edge; combining the target features using a classification algorithm; comparing the combined target features with target feature information of a plurality of microorganisms stored in a memory; and classifying the identified target into a type of microorganism based on the comparison. The target features may further include at least one spectral feature, the spectral feature being at least one of target color, target brightness, target hue, and target chromaticity, or at least one temporal feature (the temporal feature being at least one of a target growth rate, a target color change, or a projected time when the target is first visually observed). At least one of the target features may be obtained for each pixel of the identified target, and then combined using one or more statistical histogram features. The classification algorithm may be a supervised machine learning algorithm, wherein the combined target features may be compared with target feature information of four or fewer microorganisms stored in a memory.

[0021] In some examples, the method may further include, for a given identified target: for each pixel in the identified target, assigning a temporal contrast value to the pixel based on the temporal contrast data of the pixel; identifying one or more maxima from the assigned temporal contrast values; if more than one maximum value is identified, determining whether the maxima are associated with a common colony forming unit or with different colony forming units; and for any two maxima determined to be associated with different colony forming units, segmenting the identified target into two targets based at least in part on the respective positions of the two maxima. Determining whether two maxima are associated with a common colony forming unit or with different colony forming units may further include: for each maximum value, determining a distance from the maximum value to an edge of the identified target; calculating a value of an inclusion factor based on each determined distance and a distance between two maxima; and comparing the value of the inclusion factor to a predetermined range, wherein if the value of the inclusion factor is less than the predetermined range, the maximum value is associated with a common colony forming unit, and if the value of the inclusion factor is greater than the predetermined range, the maximum value is associated with different colony forming units. If the value of the inclusion factor is within a predetermined range, the method may further include: for each maximum value, determining an area surrounding the maximum value; and calculating the convexity of the respective areas surrounding the maximum value, wherein if the convexity is greater than a threshold, the maximum value is associated with a different colony forming unit.

[0022] In some instances, the method may further include identifying one or more targets in the temporally first digital image based on the spatial contrast data; and, for a given identified target in the temporally second digital image, classifying the target in the second digital image as an artifact if the combined spatial and temporal contrast data of the target in the temporally second digital image matches the spatial contrast data of the target identified in the temporally first digital image.

[0023] Yet another aspect of the present disclosure relates to a method for evaluating microbial growth on a plating medium that has been inoculated with a culture and incubated, the method comprising: obtaining first and second digital images of the plating medium, each digital image being obtained after an incubation period of the inoculated medium and at different times; aligning the second digital image with the first digital image, wherein the alignment is based on coordinates assigned to pixels of the first image and coordinates assigned to one or more pixels in the second image that correspond to pixels in the first image to which the coordinates are assigned; generating temporal contrast data indicating changes between corresponding pixels of the first and second digital images; identifying a target in the second digital image from the temporal contrast data; obtaining one or more dynamic target features of the identified target from the temporal contrast data; classifying the identified target into a type of organism based on the one or more dynamic target features; and storing the identified target and its classification in a memory. In some instances, the one or more dynamic target features may include a growth rate of the identified target, a change in a chromatic feature of the identified target, or a change in growth along an axis substantially perpendicular to the plating medium.

[0024] In some examples, the method may further include: obtaining a third digital image of the plated medium after an incubation period of the inoculated medium and at a different time than the first and second digital images; aligning the third digital image with the first and second digital images; and generating temporal contrast data indicating changes between corresponding pixels of the second and third digital images, wherein the target is further identified based on the temporal contrast data, and wherein the one or more dynamic target features of the identified target include a second derivative of the temporal contrast data. The one or more dynamic target features may include an acceleration rate of growth of the target.

[0025] Yet further aspects of the present disclosure relate to a computer-readable storage medium having program instructions encoded thereon configured to cause a processor to implement a method. The method may be any of the above methods for evaluating microbial growth on a plating medium, identifying microbial growth on a plating medium that has been inoculated with a culture and incubated, or evaluating microbial growth on a plating medium that has been inoculated with a culture and incubated.

[0026] Even further aspects of the present disclosure relate to a system for evaluating growth in a culture medium inoculated with a biological sample. The system includes an image acquisition device for capturing a digital image of the culture medium, a memory, and one or more processors operable to execute instructions to implement a method. In some instances, the memory may store information about the captured digital image, and the method implemented by the executed instructions may be any of the above-mentioned methods for evaluating microbial growth on a plating medium. In other instances, the memory may store information about candidate colony targets identified in the captured digital image, and the method implemented by the executed instructions may be any of the above-mentioned methods for identifying microbial growth on a plating medium that has been inoculated and incubated with a culture. In yet further instances, the memory may store information about one or more targets identified in the captured digital image and one or more classifications of the identified targets, and the method implemented by the executed instructions may be any of the above-mentioned methods for evaluating microbial growth on a plating medium that has been inoculated and incubated with a culture. BRIEF DESCRIPTION OF THE DRAWINGS

[0027] Figure 1 is a schematic diagram for imaging analysis and testing cultures according to one aspect of the present disclosure.

[0028] Figure 2 is a flow chart illustrating an automated laboratory workflow procedure for imaging analysis and testing cultures according to one aspect of the present disclosure.

[0029] Figure 3A , 3B 3C is an image of time contrast showing changes in colony morphology over time by visually representing colony morphology according to one aspect of the present disclosure.

[0030] Figure 3D and 3E are images showing the spatial contrast under different lighting conditions.

[0031] Figure 4 is a flow chart of an example procedure for obtaining and analyzing image information according to one aspect of the present disclosure.

[0032] Figure 5 is a flow chart of an example procedure for obtaining spatial contrast according to one aspect of the present disclosure.

[0033] Figure 6 is a flow chart of an example procedure for obtaining temporal contrast according to one aspect of the present disclosure.

[0034] Figure 7is a flow chart of an example procedure for filtering artifacts from an image according to one aspect of the present disclosure.

[0035] Figure 8 is a flow chart of an example procedure for labeling pixels of an image according to one aspect of the present disclosure.

[0036] Fig. 9 is a flow chart of an example procedure for splitting a colony into individual targets according to one aspect of the present disclosure.

[0037] Fig.10 is a flow chart of an example object segmentation procedure according to one aspect of the present disclosure.

[0038] Fig.11 is a schematic diagram showing the measurement of confluent colonies, which is Fig.10 Part of the segmentation procedure.

[0039] Fig.12 According to one aspect of the present disclosure picture.

[0040] Fig.13A , 13B 13A and 13C are graphs illustrating isolation factor measurements according to one aspect of the present disclosure.

[0041] Fig.14 is a diagram illustrating the effect of one aspect of the present disclosure Illustration of the area.

[0042] Fig.15A , 15B and 15C are images illustrating characterization of colony growth according to one aspect of the present disclosure.

[0043] Fig.16A and 16B A portion of an imaging plate is shown with a magnified and redirected image of a sample colony in the image.

[0044] Fig. 16C Shows Fig. 16B Vector diagram of the respective images in .

[0045] Fig.17 Depicted are SHQI, spatial contrast, and temporal contrast images of a sample according to one aspect of the present disclosure.

[0046] Fig.18 It is a comparison Figure 2 A flowchart showing the timeline of the procedure in FIG. 1 and the timeline of a comparable manually performed procedure. DETAILED DESCRIPTION

[0047] The present disclosure provides instruments, systems and methods for identifying and analyzing microbial growth on a plated medium based at least in part on contrast detected in one or more digital images of the plated medium. The multiple methods described herein can be fully or partially automated, for example, integrated as part of a fully or partially automated laboratory workflow.

[0048] The system described herein can be implemented in an optical system for imaging microbial samples for identification of microorganisms and detection of microbial growth of such microorganisms. There are many such commercially available systems, which are not described in detail herein. An example is the BD Kiestra TM ReadA compact intelligent incubation and imaging system. Other example systems include those described in PCT Publication No. WO2015 / 114121 and U.S. Patent Publication No. 2015 / 0299639, which are incorporated herein by reference in their entirety. Such optical imaging platforms are well known to those skilled in the art and are not described in detail herein.

[0049] Figure 1 1 is a schematic diagram of a system 100 having a processing module 110 and an image acquisition device 120 (e.g., a camera) for providing high quality imaging of the plating medium. The processing module and the image acquisition device may be further connected to and thereby further interact with other system components, such as an incubator module (not shown) for incubating the plating medium so that the culture seeded on the plating medium grows. Such connections may be fully or partially automated using a track system that receives samples for incubation and transports them to the incubator, and then between the incubator and the image acquisition device.

[0050] The processing module 110 can instruct other components of the system 100 to perform tasks based on the processing of various types of information. The processor 110 can be hardware that performs one or more operations. The processor 110 can be any standard processor, such as a central processing unit (CPU), or can be a specialized processor, such as an application-specific integrated circuit (ASIC) or a field programmable gate array (FPGA). Although a processor block is shown, the system 100 can also include multiple processors, which may or may not operate in parallel, or other dedicated logic and memory for storing and tracking information related to sample containers in the incubator and / or image acquisition device 120. In this regard, the processing unit can track and / or store several types of information related to the sample in the system 100, including but not limited to the location of the sample in the system (incubator or image acquisition device, location and / or orientation therein, etc.), incubation time, pixel information of the captured image, type of sample, type of culture medium, preventive treatment information (e.g., harmful samples), etc. In this regard, the processor can be capable of automating all or part of the various procedures described herein. In one embodiment, instructions for implementing the procedures described herein may be stored on a non-transitory computer-readable medium (eg, a software program).

[0051] Figure 2 is a flow chart illustrating an example automated laboratory procedure 200 for imaging, analyzing, and optionally testing cultures. The procedure 200 may be performed by an automated microbiology laboratory system, such as the BD Kiestra TM Total Lab Automation or BD Kiestra TM Work Cell Automation: The example system includes interconnected modules, each of which is configured to perform one or more steps in the process 200.

[0052] At 202, a culture medium is provided and inoculated with the biological sample. The culture medium can be an optically transparent container so that the biological sample can be observed within the container when illuminated from different angles. The inoculation can follow a predetermined pattern. The streaking patterns and automated methods for streaking the sample on the plate are well known to those skilled in the art and will not be discussed in detail herein. An automated method employs magnetically controlled beads to streak the sample on the plate. At 204, the culture medium is incubated to allow the biological sample to grow.

[0053] At 206, one or more digital images of the culture medium and the biological sample are captured. As will be described in more detail below, during the incubation process, digital imaging of the culture medium can be performed multiple times (e.g., at the beginning of incubation, at a time in the middle of incubation, at the end of incubation), so that changes in the culture medium can be observed and analyzed. Imaging of the culture medium can include removing the culture medium from the incubator. When multiple images of the culture medium are taken at different times, between imaging sessions, the culture medium can be returned to the incubator for further incubation.

[0054] At 208, based on the information from the captured digital image, the biological sample is analyzed. The analysis of the digital image may include the analysis of the pixel information contained in the image. In some examples, the pixel information may be analyzed on a pixel-by-pixel basis. In other examples, the pixel information may be analyzed on a block-by-block basis. In a further example, the pixel may be analyzed based on the entire region of the pixel, whereby the pixel information of the individual pixels in the region may be derived by combining the information of the individual pixels, selecting the sample pixels, or by using other statistical methods such as the statistical histogram operation to be described in more detail below. In the present disclosure, the operation described as being applied to "pixels" may also be used for blocks or other pixel groups, and the term "pixel" is therefore intended to include such applications.

[0055] The analysis may include determining whether growth is detected in the culture medium. From an image analysis perspective, growth may be detected in an image by identifying the imaged target (based on differences between the target and its adjacent environment) and then identifying changes in the target over time. As described in more detail herein, these differences and changes are both forms of "contrast." In addition to detecting growth, the image analysis at 208 may further include quantifying the amount of growth detected, identifying unique colonies, identifying sister colonies, and the like.

[0056] At 210, it is determined whether the biological sample (particularly, the identified sister colonies) exhibits significant growth in number. If no growth is found or an insignificant amount of growth is found, then the process 200 may proceed to 220 where a final report is output. In the case of proceeding from 210 to 220, the final report will likely indicate a lack of significant growth, or report growth of normal flora.

[0057] If it is determined that the biological sample exhibits a significant increase in number, then at 212, based on the existing analysis, one or more colonies can be picked from the image. Picking colonies can be a fully automated process, where each picked colony is sampled and tested. Alternatively, picking colonies can be a partially automated process, where multiple candidate colonies are automatically identified and visually presented to an operator on a digital image, so that the operator can input a selection of one or more candidates for sampling and further testing. The sampling of the selected and picked colonies itself can be performed automatically by the system.

[0058] At 214, the sampled colonies are prepared for further testing, such as by plating the samples in an organism suspension. At 216, the samples are tested using matrix-assisted laser desorption ionization (MALDI) imaging to identify the type of sample sampled from the initial culture medium. At 218, the samples are also, or optionally, subjected to antibiotic susceptibility testing (AST) to identify possible treatments for the identified samples.

[0059] At 220, the test results are output in a final report. The report may include MALDI and AST results. As described above, the report may also indicate quantification of sample growth. Thus, the automated system is able to start with an inoculated culture medium and generate a final report on the samples found in the culture with little or no additional input.

[0060] In the program Figure 2 In the example procedures of , the colonies detected and identified are usually referred to as colony forming units (CFU). CFU is a microscopic object that starts as one or several bacteria. Over time, bacteria grow to form colonies. The earlier the time from when the bacteria are placed on the plate, the fewer bacteria are detected, so the colonies are smaller and the contrast with the background is lower. In other words, smaller colony sizes produce smaller signals, and smaller signals on a constant background result in less contrast. This is reflected by the following equation:

[0061] (1)

[0062] Contrast plays an important role in identifying targets in an image, such as CFUs or other artifacts. A target can be detected if it has a significant difference in brightness, color, and / or texture from its surroundings. Once a target is detected, analysis may also include identifying the type of target detected. Such identification may also rely on contrast measurements, such as the smoothness of the edges of the identified target, or the uniformity (or lack of uniformity) of the color and / or brightness of the target. In order to be detected by the image sensor, the contrast must be large enough to overcome the image noise (background signal).

[0063] Human perception of contrast (as governed by Weber's law) is limited. Under optimal conditions, the human eye can detect a 1% difference in light levels. The quality and confidence of image measurements (e.g., brightness, color, contrast) can be characterized by the signal-to-noise ratio (SNR) of the measurements, where an SNR value of 100 (or 40 dB) will match human detection capabilities, independent of pixel intensity. Digital imaging techniques that utilize high SNR imaging information and known SNR information for each pixel can allow colonies to be detected even when those colonies remain invisible to the human eye.

[0064] In the present disclosure, contrast can be collected in at least two ways: spatially and temporally. Spatial contrast, or local contrast, quantifies the difference in color or brightness between a given region (e.g., a pixel, group of adjacent pixels) in a single image and its surroundings. Temporal contrast, or temporal contrast, quantifies the difference in color or brightness between a given region in one image and the same region in another image taken at a different time. The equations governing temporal contrast are similar to those for spatial contrast:

[0065] (2)

[0066] Where t1 is the time after t0. Both the spatial and temporal contrast of a given image can be used to identify objects. The identified objects can then be further tested to determine their significance (e.g., whether they are CFU, normal flora, dust, etc.).

[0067] Figure 3A , 3B and 3C provide a visual demonstration of the effect that temporal contrast can have on an imaged sample. Figure 3A The images shown in were captured at different time points (left to right, top to bottom row) and show the total growth in the sample. Figure 3A The growth in Figure 3B In the corresponding contrast time image of , the growth is even more pronounced and can be noticed even earlier in the sequence. Figure 3C Shows Figure 3B The enlarged part of Figure 3C As can be seen in the figure, the longer the portion of the colony is imaged, the brighter the spot appears in the contrast image. In this way, the centroid of each colony can be indicated by the bright center or peak of the colony. Therefore, image data acquired over time can reveal important information about changes in colony morphology.

[0068] To maximize the spatial or temporal contrast of an object relative to its background, the system can capture images using different incident lighting on different backgrounds. For example, any top lighting, bottom lighting, or side lighting can be used on either a black or white background.

[0069] Figure 3D and 3E Provides a visual demonstration of the effect that lighting conditions can have on imaged samples. Figure 3D The images in were captured using top lighting, while Figure 3E The images in were captured at nearly the same time (e.g., close enough in time that no noticeable or significant growth occurred) using bottom illumination. As can be seen, Figure 3D and 3E Each of the images of the samples in the image contains several colonies, but additional information about the colonies (in this case, hemolysis) needs to be visible. Figure 3D The image in is backlit or bottom lit, while in Figure 3E In images with different resolutions, the same information is difficult to grasp.

[0070] At a given point in time, multiple images can be captured under multiple lighting conditions. Different light sources can be used to capture images, and different light sources are spectrally different due to the level of the illumination light, the illumination angle and / or the filters (e.g., red, green and blue filters) deployed between the target and the sensor. In this way, image acquisition conditions are variable in terms of light source position (e.g., top, side, bottom), background (e.g., black, white, any color, any intensity) and spectrum (e.g., red channel, green channel, blue channel). For example, the first image can be captured using top illumination and a black background, the second image can be captured using side illumination and a black background, and the third image can be captured using bottom illumination and no background (i.e., white background). Moreover, in order to maximize spatial contrast, a specific algorithm can be used to generate a set of different image acquisition conditions. By changing the image acquisition conditions according to a given sequence and / or over a period of time, these or other algorithms are also useful for maximizing temporal contrast. Some such algorithms are described in PCT Publication No. WO2015 / 114121.

[0071] Figure 4 is a flow chart illustrating an example procedure for analyzing an imaged slab based at least in part on contrast. Figure 4 The procedure can be regarded as Figure 2 The example subroutine of program 200, thereby Figure 2 206 and 208 are at least partially used Figure 4 procedure to be carried out.

[0072] At 402, a first digital image is captured at time t0. Time t0 may be a time shortly after the incubation process begins, so that the bacteria in the imaging plate have not yet begun to form visible colonies.

[0073] At 404, coordinates are assigned to one or more pixels in the first digital image. In some cases, the coordinates may be polar coordinates, which have radial coordinates extending from a center point of the imaging flat panel and angular coordinates about the center point. The coordinates are used in later steps to help align the first digital image with other digital images of the flat panel taken at different angles and / or at different times. In some cases, the imaging flat panel may have a particular landmark (e.g., a point or line off center), so that the coordinates of the pixel(s) covering the landmark in the first image may be assigned to the pixel(s) covering the same landmark in other images. In other cases, the image itself may be considered a feature for further alignment.

[0074] At 406, at time t x A second digital image is captured. Time t x is the time after t0 when the bacteria in the imaging plate have a chance to form visible colonies.

[0075] At 408, the second digital image is aligned with the first digital image based on the previously assigned coordinates.Aligning the images may further include normalization and standardization of the images, for example, using the methods and systems described in PCT Publication No. WO2015 / 114121.

[0076] At 410, contrast information for the second digital image is determined. The contrast information may be collected on a pixel-by-pixel basis. For example, a pixel of the second digital image may be compared to a corresponding pixel (at the same coordinates) of the first digital image to determine the presence of temporal contrast. Additionally, adjacent pixels of the second digital image may be compared to each other, or to other pixels known to be background pixels, to determine the presence of spatial contrast. Changes in pixel color and / or brightness indicate contrast, and the magnitude of such changes from one image to the next or from one pixel (or region of pixels) to the next pixel (or region of pixels) may be measured, calculated, estimated, or otherwise determined. Where both temporal contrast and spatial contrast are determined for a given image, an overall contrast for a given pixel of the image may be determined based on a combination (e.g., average, weighted average) of the spatial and temporal contrast of the given pixel.

[0077] At 412, based on the contrast information calculated at 410, an object in the second digital image is identified. Adjacent pixels of the second digital image with similar contrast information may be considered to belong to the same object. For example, if the difference in brightness between adjacent pixels and their background, or between pixels, and their brightness in the first digital image is substantially the same (e.g., within a predetermined threshold amount), the pixels may be considered to belong to the same object. As an example, the system may assign "1" to any pixel with significant contrast (e.g., exceeding a threshold amount), and then identify the group of all adjacent pixels assigned "1" as an object. The object may be given a specific label or mask so that pixels with the same label share certain characteristics. The label may help distinguish the object from other objects and / or the background during subsequent processing of subroutine 400. Identifying an object in a digital image may include segmenting or dividing the digital image into multiple regions (e.g., foreground and background). The purpose of segmentation is to change the image into a representation of multiple components so that the components can be more easily analyzed. Image segmentation is used to locate objects of interest in an image.

[0078] At 414, the characteristics of a given target (identified at 412) can be characterized. Characterization of target characteristics can include deriving descriptive statistics of the target (e.g., area, reflectance, size, optical density, color, plate position, etc.). Descriptive statistics can ultimately quantitatively describe certain characteristics of the information set collected about the target (e.g., from the SHQI image, from the contrast image). Such information can be evaluated as a function of species, concentration, mixture, time, and culture medium. However, at least in some cases, characterizing the target can start with a collection of qualitative information related to the target characteristics, whereby this qualitative information is then quantitatively represented. Table 1 below provides a list of example features that can be qualitatively evaluated and then converted to quantitative representations:

[0079] Table 1: Qualitative attributes of a target and criteria for quantitatively transforming that attribute

[0080]

[0081]

[0082] Some features of an object, such as shape or the time until it is visually observed, may be measured a single time for the object as a whole. Other features may be measured multiple times (e.g., for each pixel, for each row of pixels with a common Y coordinate, for each column of pixels with a common X coordinate, for each ray of pixels with a common angular coordinate, for a circle of pixels with a common radial coordinate) and then combined into a single measurement, such as using a histogram. For example, color may be measured for each pixel, growth rate or size may be measured for each row, column, ray, or circle of pixels, and so on.

[0083] At 416, it is determined whether the target is a colony candidate based on the features of the characterization. Colony candidate determination may include inputting quantitative features (e.g., scores shown in Table 1 above) or a subset thereof to a classifier. The classifier may include a confusion matrix (confusion matrix) for implementing a supervised machine learning algorithm, or a matching matrix for implementing an unsupervised machine learning algorithm, to evaluate the target. Supervised learning is preferred in such a case: wherein the target is distinguished from a finite set (e.g., two or three) of possible organisms (in this case, the algorithm can be trained on relatively finite sets of training data). On the contrary, unsupervised learning is preferred in such a case: wherein the target is distinguished from the entire database of possible organisms, in which case it is difficult to provide comprehensive - or even sufficient - training data. In the case of confusion or matching matrices, differences can be measured numerically within a certain range. For example, for a given pair of targets, "0" may mean that the two targets should be distinguished from each other, and "1" may mean that the targets are difficult to distinguish from each other.

[0084] The colony candidates can be stored in the memory of the automated system for further use (eg, testing, the segmentation procedure described below, etc.).

[0085] Application of various culture media

[0086] In the examples above, the evaluation of cultures is described for a single culture medium. However, the examples are equally applicable to instances where cultures are evaluated in multiple culture media.

[0087] Since the characteristics of bacteria (e.g., color, growth rate, etc.) may vary depending on the type of culture medium ("medium") used, a different confusion matrix may be used for each medium during classification (e.g., 416 of subroutine 400). Therefore, it is entirely reasonable that for two targets, the classifier may output "0" for one medium, while for the same two targets, the classifier may output "1" for a different medium. The ensemble results of the classifiers can then be evaluated together (manually or based on further machine-driven relationships) to arrive at an overall or final distinction or classification of the targets.

[0088] The assessment of multiple culture media can be implemented using a single container. The single container can be configured to store multiple culture media (e.g., double plates, three plates, four plates, etc.) so that multiple culture media can be imaged together at the same time. Alternatively, multiple culture media can be assessed by streaking the culture sample in several containers, each container storing one or more culture media. Each of the multiple containers can then perform the above-mentioned imaging procedure. The information obtained from each culture media (e.g., the features of characterization) can then be collectively input into a classifier to provide a more informed identification of the (spotted) growth found in different culture media.

[0089] Contrast information

[0090] Figure 5 is a flow chart showing the Figure 4 4. Example subroutine 500 for obtaining spatial contrast as a portion of 410 in FIG. Subroutine 500 receives as input: a set of one or more background and lighting conditions 551 and filters 554. At 502, a digital image is obtained under the specified lighting and background conditions from input set 551. At 504, the image is then cloned. At 506, one of the cloned images is filtered using filter 554. Figure 5In the example of , a low pass kernel is used as a filter, but the skilled person is aware of other filters that can be used. At 508, the ratio of the unfiltered image minus the filtered image to the filtered image plus the unfiltered image is calculated. At 510, a spatial contrast image is obtained based on the ratio calculated at 508. This procedure 500 can be repeated for each background and lighting condition 551. Each repetition of procedure 500 produces another spatial contrast image, which can be used at 510 to iteratively update the previously stored spatial contrast image. Therefore, a comprehensive contrast image (including the contrast from each lighting condition) can be iteratively established. In one embodiment, in each iteration, a clear contrast image - where the contrast setting is still set to 0 (compared to the contrast image established by the iteration) - can be provided as an input for each lighting setting. If it is determined at 512 that the last image has been processed, the procedure 500 ends.

[0091] Figure 6 is a flow chart showing the same as Figure 4 410 of 600 for obtaining temporal contrast. The subroutine 600 receives as input: a set of one or more background and lighting conditions 651; and filters 655. At 602, each of the first and second digital images captured under specified lighting and background conditions is obtained. At 604, the t0 image is filtered. Figure 6 In the example of , a low pass kernel is used as a filter, but the skilled person will know other filters that can be used. At 606, the unfiltered t x Image minus the filtered t0 image and the filtered t0 image plus the unfiltered t x At 608, a temporal contrast image is obtained based on the ratio calculated at 606. The process 600 can be repeated under different lighting conditions and / or different background conditions. Each repetition of the process 600 generates another temporal contrast image, which can be used at 608 to iteratively update the previously stored temporal contrast image. As with the creation of the spatial contrast image, the temporal contrast image can be iteratively created, wherein a clear contrast image is provided as input for each lighting condition. If it is determined at 610 that the last image has been processed, the process 600 ends.

[0092] In order to obtain a comprehensive or overall determination of contrast, the spatial and temporal contrast results may be further combined. The combination of spatial and temporal contrast is referred to herein as "mixed contrast" (MC). In one embodiment, the spatial contrast (SC) image at time t0, the temporal contrast image at time t1, and the temporal contrast image at time t2 may be combined to obtain a comprehensive or overall determination of contrast. x The spatial contrast image and the spatial contrast images from t0 and t x The time contrast (TC) image obtained by comparing the images is obtained by the following equation to obtain the mixed contrast:

[0093] (3)

[0094] Light Filter

[0095] To enhance image analysis, Figure 4 Additional processes may be included in the subroutine 400 of . For example, the first digital image may be analyzed for objects that appear in the image at time t0. Since it is known that no bacteria have begun to grow significantly at t0, it can be assumed that any objects found at time t0 are merely dust, bubbles, artifacts, condensation, etc. that do not constitute colony candidates.

[0096] A filtering process may be used on the captured image to subtract dust and other artifacts attached to the imaging plate or lens. When considering transparent culture media (e.g., MacConkey agar, CLED agar, chromogenic agar (CHROMagar) and the like), it is expected that some level of dust will be present on the captured image. The effect of dust on a given image may be determined at least in part by the specific lighting and background conditions of the captured image. For example, when a white culture medium is used, reflection artifacts and dust are most easily observed when a black background is used below and the culture medium is illuminated from above. As another further example, when a colored or black culture medium is used, artifacts and dust will be most easily observed when a white background is used below and the culture medium is illuminated from above. As a further example, in most arbitrary culture media, regardless of the background, light-absorbing artifacts and dust will be observed when the culture medium is illuminated from below. In any case, the management of dust and artifacts is a complex image processing challenge that can significantly affect the detection of microbial growth.

[0097] Dust and artifacts can be divided into two types: (A) those that can change position; and (B) those that cannot. Dust and artifacts can accumulate over time, which means that the amount of both types A and B can change over time. Nevertheless, observations show that type A is more likely to change over time than type B in amount. Of course, type A is also more likely to change, for example due to the plate being moved in or out of the imaging chamber.

[0098] Typically, Type B is caused by artifacts associated with the plate itself such as ink dots (brand, batch number, and information printed on the underside of the plate), imperfections associated with the injection point of the plastic mold, or frosted regions. Type B can also be caused by dust or air bubbles stuck to the top of the medium, trapped within the medium, or statically stuck to the underside of the plate.

[0099] Even the type A dust and artifacts themselves hardly change position from the point of view of the imaging being observed. However, due to the fact that the plastic and culture medium of the plate act as filters and lenses, the characteristics and position of the observed type A artifacts may vary slightly, depending on the culture medium color, culture medium level, and plastic. Type B dust and artifacts also do not change position. However, to the extent that type B dust and artifacts are connected to the culture medium and the culture medium is subject to slight movement and drift over time (primarily because of slight drying over time in the incubator), type B dust and artifacts may move with the culture medium. Therefore, the position of type B dust and artifacts is also susceptible to slight changes, at least to some extent.

[0100] In terms of contrast, a dust particle (speck) of type A can be said to exist at position "p0" in the t0 spatial contrast image and at position "p0" in the t0 spatial contrast image. x The position "p" in the spatial contrast image x " exists. Assume that p0 and p x are different locations, then dust or artifacts will also exist at two locations in the time contrast image (for example, at p x By comparison, the dust particles of type B will be at positions t0 and t x It exists in the common position of the two spatial contrast images but not in the temporal contrast image.

[0101] As explained above, in order to obtain a mixed contrast result, the spatial and temporal contrast images can be combined. The influence of dust and artifacts of both types A and B can be further eliminated from the mixed contrast result. In one embodiment, if a target (e.g., a CFU candidate) is identified in the mixed contrast result, it can be compared with the dust and artifacts of the target neighborhood N (x, y) detected in the spatial contrast result at time t0. Then, if a similar target is found in the spatial contrast result at time t0, the target identified in the mixed contrast result is marked as a type A or type B false positive. Even if the target is not marked as a type A or type B false positive initially, if it is found that the target does not change size significantly over time, it may still be determined later that the target is a type B false positive. False positives can be stored and applied to subsequent images later, such as by a filter mask (e.g., a binary mask) further described below.

[0102] Another filtering process can be used to subtract condensation that forms on the plate (e.g., during transfer from the refrigerator to the incubator at the beginning of the incubation session). In one example condensation filter, the plate is illuminated using bottom lighting so that less light is transmitted through locations with condensation than locations without condensation. The optical density of the image can then be evaluated, and areas of low optical density are subtracted from the image.

[0103] Additionally or alternatively, an image mask may be constructed to discount the target from any analysis of the t0 image and / or subsequent digital images. Figure 7 7 is a flow chart showing an example procedure 700 for generating an image mask using spatial contrast of a t0 image. In the example of procedure 700, the only input provided is a SHQI image 751 taken at time t0. At 702, the spatial contrast of the t0 image is determined. At 704, statistical information about the pixels in the region of interest of the t0 image, such as the mean and standard deviation (e.g., about brightness), is collected using the spatial contrast information. At 706, the contrast threshold is adjusted to ensure that an appropriate number of pixels exceed the threshold. For example, if pixels above a given percentage are not considered to be the background of the image, the threshold can be increased. At step 708, the threshold is further adjusted based on the statistical information associated with those pixels below the threshold. Finally, at 710, a binary mask is generated. The binary mask distinguishes multiple artifacts of non-valid pixels from other pixels that are considered valid. The binary mask can then be used at a subsequent time when there are potential colonies to detect and exclude targets that occupy non-valid pixels from candidate colonies.

[0104] The filtering process described above may improve subroutine 400 by avoiding the inadvertent inclusion of dust, condensation, or other artifacts as targets and by speeding up the characterization of properties at 414 since such characterization may be performed only on valid pixels.

[0105] Limiting Targets and Tags

[0106] Another one can be added Figure 4 The process of subroutine 400 is to assign a label to the target identified at 412. The target can be given a specific label so that pixels with the same label share certain characteristics. The label can help distinguish the target from other targets and / or background during the subsequent process of subroutine 400. Figure 8 is a flow chart showing the time t for marking x The captured image (or "t x 800 of an example process for determining pixels of an image. Figure 8 In the example of receiving a binary mask 851 (eg, the output of process 700), t xAn uninitialized candidate mask 852 of an image and a temporal contrast image 853 (e.g., the output of subroutine 600) are used as input. At 802, candidate mask 852 is initialized. Initialization may include identifying regions of interest on the imaging plate and using binary mask 851 to identify regions of interest at time t x The “valid pixels” in the captured image. Valid pixels are pixels that have not been disregarded as candidate colonies and will be considered for labeling. At 804, the time contrast image 853 is used to collect information about t x Statistics of valid pixels in the region of interest of the image, such as mean and standard deviation (e.g., with respect to brightness). Then, at 806, the statistics of each of the temporal contrast image 853 and the binary mask 851 (which are preferably generated under similar lighting and background conditions) are combined to form a threshold contrast image. Using the threshold defined by the threshold contrast image, t x "Connected components" of the image are labeled at 808. Connected components effectively serve as labels that indicate connections between neighboring pixels (or groups within neighboring pixels), which in turn indicates that the pixels are part of the same object.

[0107] Once the x The image has defined connected components, each of which can be analyzed individually (at 810) to verify its status as a single target. Figure 8 In the example of , statistical calculations of pixels associated with the tag are performed at 812. The calculation may utilize a histogram to determine the mean and / or standard deviation of the brightness or color of the pixel. At 814, it is determined whether the pixel meets a threshold area. If the threshold area is not met, the operation proceeds to 830, where the tag is updated. Updating the tag may include maintaining the analyzed component as one tag, or dividing the component into two tags. In the event that the threshold area is not met, the component remains as a single tag. If the threshold area is met, then at 816, the histogram is smoothed, and at 818, the peak of the distributed tag pixel is identified. The peak can be further defined by having a minimum area - since peaks smaller than the minimum area can be ignored.

[0108] At 820, the number of identified peaks is counted. If there is only one peak, the operation proceeds to 830 and the flag is updated so that the component remains as one target. If there is more than one peak, then at 824, the contrast between the peaks is further evaluated using the threshold contrast image. The operation then proceeds to 830 and the flag is updated based on the multiple identified peaks so that the significant contrast results in the component are divided into two, and otherwise remain as one.

[0109] segmentation

[0110] Another process that may be included as part of subroutine 400 is a segmentation process for dividing the x If at time t x , colonies have grown to the point where they overlap or touch each other, and it may be necessary to draw a border across the confluent area in order to assess the individual colonies within the area.

[0111] In some cases, when two bordering colonies have different characteristics (e.g., different colors, different textures), segmentation can simply involve feature analysis of the confluent area. However, spatial and temporal contrast is not always sufficient to identify boundaries between colonies alone. Fig. 9 is a flow chart illustrating an example process 900 for separating such colonies into individual objects (eg, with individual markers), or in other words, segmenting the colonies. Fig. 9 The example process 900 uses a first digital image 951 captured at time t0, a first digital image 952 captured at time t x The captured second digital image and the t0 image binary mask 953 (eg, a mask generated by the process 700) are used as input. At 902, based on t0 and t x Images 951 and 952 generate a time contrast image. At 904, the time contrast image is segmented using a binary mask 953. At 906, labels are applied to the segmented portions of the image. At 908, the peak or maximum of each label is identified. The maximum value of a given segmented portion is typically the center point or centroid of the segmented portion. At 910, for each label, the maximum value (e.g., of the label being analyzed, of adjacent labels) is used to make further determinations regarding whether a given label is unique to its neighbors, or whether it should be combined with one or more adjacent labels. Once the labels are reduced to their unique components, characterization of each label (e.g., steps 414 and 416 of procedure 400) can be performed at 912, and a global list of candidate colonies is generated at 914.

[0112] Various factors, such as inclusion factors, can be applied to determine whether the local maximum of a given marker belongs to one colony or different colonies. An inclusion factor is a factor that indicates whether neighboring pixels are associated with neighboring objects. Such a factor can be used in a segmentation strategy to determine whether to split two local maxima in a given marker into two separate objects or merge them into a single object.

[0113] Fig.10 is a flow chart illustrating such an instance segmentation strategy. Process 1000 can be used as Fig. 9The subroutine of step 910 in FIG. Fig.10 As shown in , two local maxima 1051 and 1052 are identified. At 1002, for each maximum, the surrounding area is identified. Fig.10 In the example of , region "A" surrounds maximum value 1051, and region "B" surrounds maximum value 1052. For purposes of the example equations below, it is assumed that region A is greater than or equal in size to region B. In some cases, each region may be given an elliptical shape having a horizontal distance (xA, xB) along the region's horizontal axis and a vertical distance (yA, yB) along the region's vertical axis. To clarify Fig.10 The program, Fig.11 An example illustration of regions A and B and their respective maximum values ​​is provided.

[0114] At 1004, for each local maximum 1051 and 1052, the distance from the maximum to the target edge is determined. In some cases, the distance determined is the average or median distance of the region assigned at 1002, hereinafter referred to as a distance map. The distance map of region A is hereinafter referred to as rA, and the distance map of region B is referred to as rB.

[0115] At 1006, a coverage factor is calculated based on the distance "d" between the two local maxima and the distance determined at 1004. In one embodiment, the coverage factor is calculated using the following equation:

[0116] (4)

[0117] At 1008, it is determined whether the inclusion factor is less than a predetermined range, greater than a predetermined range, or within a predetermined range (e.g., between 0.5 and 1). If the inclusion factor is less than the predetermined range, then the maximum values ​​are determined to be associated with the same target. If it is greater than the predetermined range, then the maximum values ​​are determined to be associated with different targets.

[0118] For inclusion factors that fall within the range, it is not immediately clear whether the maxima belong to the same target or different targets, and more processing is required. Procedure 1000 then continues at 1010, where the convexity of the respective surrounding areas of the two maxima is calculated using the coordinates of a third region "C" at a location between the two maxima. In some cases, the region can be a weighted center of the two regions, so that the center point of region C is closer to the smaller region B than to the larger region A. For region C, the horizontal and vertical distances xC, yC, and the distance map H can also be calculated. For example, the convexity can be calculated according to the following equation using the above values ​​and the distance d(A, C) between the center point of region C and the maximum value A:

[0119] (5)

[0120] (6)

[0121] (7)

[0122] (8) ΔH = H - (0.9R)

[0123] At 1012, it is determined whether the convexity value is greater than (more convex) a given threshold. For example, ΔH may be compared to a threshold of 0. If the convexity value is greater than the threshold, then the maximum value is determined to be associated with a different target. Otherwise, at 1014, one or more parameters of region C are updated so that the size of region C is increased. For example, an offset distance (distOffset) is updated based on ΔH, e.g., the value of ΔH is limited between 0 and 1 (if ΔH is greater than 1, it is rounded to 1) and then added to the offset distance.

[0124] At 1016, it is determined whether the size of region C meets or exceeds a threshold. If the threshold is met or exceeded, then the maximum values ​​are determined to be associated with the same target. In other words, if the difference between regions A and B is so uncertain that region C increases until it begins to overshadow regions A and B, this is a good indication that maxima 1051 and 1052 should belong to the same target. In the example above, this can be indicated by the offset distance meeting or exceeding the distance d between the maxima. Otherwise, the operation returns to 1010 and the convexity of regions A and B is recalculated based on the updated parameter(s) of region C.

[0125] Once the relevance of each maximum is determined, the determined relevance is stored, for example, in a matrix (also referred to as an incidence matrix). The stored information can be used to reduce the full list of maximum values ​​to a final list of candidate targets. For example, in the case of an incidence matrix, a master list can be generated from the full list of maximum values, and then each maximum value can be iteratively checked and removed from the master list if the associated maximum value is still on the list.

[0126] exist Fig. 9 and 10 In the example, the second image is captured at a time t x (and, therefore, the earliest time that the procedure 900 can be executed) can be only a few hours into the incubation process. Such a time is generally considered too early to identify fully formed colonies, but may be sufficient for generating a segmented image. The segmented image can optionally be applied to future images taken at a later time. For example, the boundaries between colonies can be drawn to predict the expected growth of the colonies. Then, in the event of confluence between colonies, the boundaries can be used to separate the confluent colonies.

[0127] Image analysis using two or more images after time t0

[0128] Although the above processes and procedures only require taking one image after time t0 (e.g., a first digital image at time t0 and a second digital image at time t x ), other processes require taking at least a second image after time t0. For example, if it is found that the image at time t x includes confluent colonies, another image taken at time t n (where 0 < n < x) may be used to identify and separate individual colonies.

[0129] For example, if t0 = the start of incubation for 0 hours (when no growth occurs), and t x = the start of incubation for 24 hours (when so much growth has occurred that the colonies are now confluent), an image at time t n = 12 hours (when the colonies will start to grow but are not yet confluent) will reveal the presence of individual colonies. Colony growth is then predicted based on the image at time t n to evaluate the boundaries between the confluent colonies at time t x . In this regard, the image at time t n can help distinguish between fast-growing colonies and slow-growing colonies. Those skilled in the art should recognize that as the number of images taken between time t0 and time t x increases, the growth rate of the colonies will be predicted more accurately.

[0130] In one application of the foregoing concept, the image taken at time t n (or more generally, the images taken between time t0 and t x ) can be used to identify colony seeds, which are the targets of colonies suspected to grow over time, and associate the seeds with corresponding masks and labels. Each seed receives a unique label and the label will be stored together with other features (e.g., based on the position, morphology, and histogram of the image generated from the SHQI image: red channel, green channel, blue channel, illuminance, chromaticity, hue, or composite image) and properties (e.g., isolated / non-isolated status, other information for predicting time-dependent proliferation). To extract global plate metrics, some stored features (e.g., histograms) can also be calculated at the plate level rather than being attributed to specific seeds. The features stored for the seeds can then be used for colony extraction at time t x and provided as input to a classifier for training and / or testing.

[0131] Growth rate tracking using multiple images taken after t0 can also be used to detect dust, artifacts, or other foreign objects that appear on the platen or in the imaging lens in the middle of a workflow procedure. For example, if a dust particle appears after t0 but before t n Before landing on the imaging lens, the spot created by the particle might initially be interpreted as a growing colony because it was not visible at time t 0. However, subsequent imaging reveals that the spot has not changed in size, and it can be determined that the spot is not growing and is therefore not a colony.

[0132] In addition to growth rate and division, other aspects of the colony can be determined between t0 and t x In some cases, growth can be measured along the z-axis in addition to or instead of measuring growth along the usual x and y axes. For example, it is known that Streptococcus pneumoniae will slowly develop a depressed center when grown in blood agar, but the depressed center is not usually visible until the second day of analysis. By looking at the time course of bacterial growth, an incipient sinking center can be detected, allowing the bacteria to be identified much earlier than if the technician had to wait for the center to fully sink.

[0133] In other cases, it is known that colonies change color over time. Thus, imaging a colony having a first color (e.g., red) at a time after t0, and then imaging a colony having a second color (e.g., green) at a subsequent time, can be used to determine the identity of the bacteria growing in the colony. The color change can be measured as a vector or path through a color space (e.g., RGB, CMYK, etc.). Changes in other chromaticity characteristics of the colony can also be measured similarly.

[0134] Target characteristics

[0135] As above combined Figure 4 As discussed, features of an object on an imaging panel may be characterized as part of an image analysis performed on the imaging panel. The characterized features may include both static features (associated with a single image) and dynamic features (associated with multiple images).

[0136] Static features are intended to reflect the properties of the target and / or the surrounding context at a given time. Static features include the following:

[0137] (i) Center of gravity: This is a static feature that provides the center of gravity of the imaged object in coordinate space (e.g., xy, polar). The center of gravity of an object, such as its polar coordinates, provides invariance of the feature set under given lighting and background conditions. The center of gravity can be obtained by first determining a weighted center of mass for all colonies in the image (M is a binary mask of all detected colonies). The weighted center of mass can be determined based on the assumption that each pixel of the image has equal value. The center of gravity of a given colony can then be described in the xy coordinate system by the following equation (where E = {p|p∈M} (E is the binary mask of the current colony), the x coordinate ranges from [0, image width], the y coordinate ranges from [0, image height], and each pixel is a unit):

[0138] (9)

[0139] (ii) Polar coordinates: These are also static features and can be used to further characterize the location on the imaging plate, such as the center of gravity. Typically, polar coordinates are measured along the radial axis (d) and the angular axis (θ), where the center of the plate has coordinates [0,0]. (x,y) The coordinates d and θ of are given by the following equations (d is in millimeters and θ is in degrees) (where k is the pixel density corresponding to pixels to millimeters and the "barcode" is a signature feature of the imaging plate used to ensure alignment of the plate with previous and / or future images):

[0140] (10)d = k × distance (igv (x，y) , 0 (x，y) )

[0141] (11)θ=angle(barcode, 0 (x，y) , ig v (x，y) )

[0142] (iii) Image vector: The two-dimensional polar coordinates can be transformed into a one-dimensional image vector. The image vector can represent the intensity of the pixels of the image as a function of the radial axis (usually, the center of the colony has the highest intensity) and / or as a function of the angular axis. In many cases, the image vector can be more accurate in classifying similarities / differences in the imaged objects.

[0143] (iv) Morphological features: These describe the shape and size of a given object.

[0144] (a) Area: This is a morphological feature and can be determined based on the number of pixels in the imaged object (also called "blobs"), not counting holes in the object. When pixel density is available, area can be measured in physical dimensions (e.g., mm 2). Conversely, when pixel density is not available, the total number of pixels may indicate the size, and the pixel density (k) may be set equal to 1. In one embodiment, the area is calculated using the following equation:

[0145] (12)A=k 2 ×∑ p∈E 1

[0146] (b) Perimeter: The perimeter of an object is also a morphological feature and can be determined by measuring the edges of the object and summing the total length of the edges (e.g., a single pixel with an area of ​​1 square unit has a perimeter of 4 units). As with area, length can be measured in pixel units (e.g., when k is not available) or physical length (e.g., when k is available). In some cases, perimeter can also include the perimeter of any holes in the object. Additionally, staircase effects (caused when diagonal edges are digitized as stair-shaped boxes) can be determined by counting the number of interior angles. Instead of 2 to compensate. In one embodiment, the circumference is determined using the following equation:

[0147] (13)P=k×∑ p∈E q(n p )

[0148] (14)

[0149] if:

[0150]

[0151] (p is interior and p is angle)

[0152] So:

[0153] Otherwise: q(n p )=4-∑(t∈M,l∈M,r∈M,b∈M)

[0154] (c) Roundness: The roundness of an object is also a morphological feature and can be determined based on a combination of area and perimeter. In one embodiment, the roundness is calculated using the following equation:

[0155] (16)

[0156] (d) Radius coefficient of variation (RCV): This is also a morphological feature and is used to calculate the radius of the target by taking the average radius of the target in all N directions or angles θ extending from the center of gravity. The ratio between σR and the standard deviation of the radius indicates the change in the target radius. In one embodiment, this value can be calculated using the following equation:

[0157] (17)

[0158] (18)

[0159] (19)

[0160] (v) contextual features, which describe the neighboring terrain relationship of the target in the context of other detected targets and wall edges. For example, in the case of imaged bacterial colonies, a contextual feature of the colony may be whether the colony is free, has limited free space, or competes with other surrounding colonies for access to resources. Such features tend to help classify colonies growing in the same perceived environment and / or distinguish colonies growing in different environments.

[0161] (a) Influence area: This is a situational feature that considers the space between the target and its surrounding targets and predicts the area that the analyzed target can expand to occupy (without other, different targets first expanding to occupy the same area). The influence area can be expressed as In the form of a graph, Fig.12 , which shows the impact area (shaded area) based on the distance d between the colony 1201 and its surrounding colonies, such as 1205. In one embodiment, the distance (D) from the edge of the target to the edge of the impact area NC ) can be represented using the following equation:

[0162] (20)

[0163] (b) Distance to the wall: This is the distance from the edge of the target to the nearest wall (D PW ) is a situation characteristic. In one embodiment, the distance can be characterized using the following equation:

[0164] (twenty one)

[0165] (c) Isolation Factor: This is a situational characteristic that characterizes the relative isolation of a given object based on its size and distance to the nearest edge (e.g., the nearest edge of another object, a wall). Fig.13A -C illustrates aspects of the isolation factor. Fig.13A An example is illustrated where the closest edge is the distance d from the colony to the plate wall. Fig. 13B and 13C An example is illustrated where the closest edge belongs to another colony. In such a case, a circle is drawn around the colony being analyzed, centered on it, and then expanded (small ones first, such as Fig. 13B , and then larger ones, such as Fig. 13C) until the circle touches the adjacent colony. Fig.13A In the embodiment of -C, the isolation factor can be characterized by the following equation:

[0166] (twenty two)

[0167] (d) Peripheral occupancy ratio: This is the ratio of the boundary of the affected plate to the given target at a given distance d. The situation characteristics of the area portion of the region (V). In one embodiment, the peripheral occupancy ratio (OR) can be characterized using the following equation (wherein for this equation, E = {p|p∈V,dist(p,igv (x,y) )<d}):

[0168] (twenty three)

[0169] (e) Relative perimeter occupancy ratio: In some examples, the average radius of the target may be multiplied by a predetermined factor. to get a given distance d. The result is the relative peripheral occupancy ratio (RNOR) and can be derived for a given factor x using the following equation:

[0170] (24)RNOR(x)=NOR(d)

[0171] (vi) Spectral features, which describe the light properties of a given target. Color (red, green, and blue channels; hue, luminance, and chromaticity, or any other color space transformation), texture, and contrast (over time and / or across space) are examples of such features. Spectral features can be derived from images captured at different time points and / or under different lighting conditions during the incubation process using the colony mask, and can be further correlated with the impact of a given colony. Region associated.

[0172] (a) Channel image: This is a spectral feature where specific color channels (e.g., red (R), green (G), and blue (B)) are used to spectrally resolve the image.

[0173] (b) Brightness: This is also a spectral feature, which is used to characterize the brightness of an image using RGB channels as input.

[0174] (c) Hue: This is a spectral feature where areas of an image are characterized as appearing similar to perceived colors (e.g., red, yellow, green, blue) or a combination thereof. Hue (H2) is typically characterized using the following equation:

[0175] (25)H2=atan2(β,α)

[0176] (26)

[0177] (27)

[0178] (d) Chroma: This is a spectral characteristic used to characterize the colorfulness of an image area relative to its brightness if the area were illuminated similarly to white. Chroma (C2) is usually characterized using the following equation:

[0179] (28)

[0180] (e) Radial scatter: Analysis of radial scatter of hue and chromatic contrast enables differentiation of α, β, and γ hemolysis.

[0181] (f) Maximum contrast: This feature characterizes the resolution of an image by calculating the maximum value of the measured average contrast of pixels at a given radius r from the center point of the image (e.g., the center of the imaged colony). This feature can be used to describe the growth induced by the target at times t0 and t x The perceptible difference between captured images. Maximum contrast can be characterized as follows:

[0182] (29) Maximum contrast r =MAX(average contrast r ) 目标

[0183] (vii) Background features, which describe changes in the culture medium in the vicinity of the analytical target. For example, in the case of imaging bacterial colonies, the changes may be caused by the growth of microorganisms surrounding the colonies (e.g., signs of hemolysis, changes in pH, or specific enzymatic reactions).

[0184] Dynamic features aim to reflect changes in target attributes and / or surroundings over time. Time series processing allows static features to be correlated over time. The discrete first-order and second-order derivatives of these features provide the instantaneous "speed" and "acceleration" (or stable level or deceleration) of these features over time. Examples of dynamic features include the following:

[0185] (i) Time series processing for tracking the above-mentioned static features over time. Each feature measured at a given incubation time can be referenced according to its relative incubation time to allow these features to be associated with features measured at subsequent incubation times. The time series of images can be used to detect the emergence and growth of targets such as CFU over time, as described above. Based on the ongoing analysis of the previously captured target image, the time point of imaging can be preset or limited by an automatic process. At each time point, the image can be a given acquisition configuration, whether a single acquisition configuration for the entire series, or as an image of the entire series captured from multiple acquisition configurations.

[0186] (ii) Discrete first and second derivatives of the above characteristics to provide instantaneous velocity and acceleration (or plateau or deceleration) of these characteristics over time (e.g., to track growth rate, as described above):

[0187] (a) Speed: The first derivative of a characteristic with respect to time. The speed (V) of a characteristic x can be expressed in (x units) / hour based on the following equation, where Δt is the time span expressed in hours:

[0188] (30)

[0189] (31)

[0190] (32)

[0191] (b) Acceleration: The second derivative of the characteristic with respect to time, which is also the first derivative of velocity. Acceleration (A) can be characterized based on the following equation:

[0192] (33)

[0193] The above image features can be measured from the target or the situation of the target, and are intended to capture the specificity of organisms grown in various culture media and incubation conditions. The listed features are not meant to be exhaustive, and those skilled in the art can improve, expand or limit the feature set based on various known image processing-based features known in the art.

[0194] Image features can be collected for each pixel, group of pixels, object, or group of objects in the image. In order to more generally characterize a region of an image or even an entire image, the distribution of collected features can be constructed as a histogram. In order to analyze or otherwise process the incoming image feature data, the histogram itself can rely on several statistical features.

[0195] Statistical histogram features can include the following:

[0196] (i) Minimum: The smallest value of the distribution captured in the histogram. This can be characterized by the following relationship:

[0197] (34)

[0198] (ii) Maximum: The largest value of the distribution captured in the histogram. This can be characterized according to the following relationship:

[0199] (35)

[0200] (iii) Sum: The sum of all individual values ​​captured in the histogram. The sum can be defined by the following relationship:

[0201] (36)

[0202] (iv) Mean: The arithmetic mean, or average. This is the sum of all the scores divided by the number of scores (N), according to the following relationship:

[0203] (37)

[0204] (v) Quarter (Q1): 25 percent of the scores in the distribution. 25% of the scores are below Q1, and 75% of the scores are above Q1. This can be described by the following relationship:

[0205] (38)

[0206] (vi) Median (Q2): The score at 50 percent of the distribution. 50% of the scores are below the median, and 50% of the scores are above the median. The median is less sensitive to extreme scores than the mean, and this generally makes it a better measure than the mean for highly skewed distributions. This can be described by the following relationship:

[0207] (39)

[0208] (vii) Three Quarters (Q3): 75 percent of the scores in the distribution. 75 percent of the scores are below Q3 and 25 percent of the scores are above Q3. This is described by the following relationship:

[0209] (40)

[0210] (viii) Mode: The score that occurs most often in a distribution. This is used as a measure of central tendency. The advantage of mode as a measure of central tendency is that its meaning is obvious. Moreover, it is the only measure of central tendency that can be used with nominal data. The mode is largely subject to sample fluctuations and is therefore not usually used as the sole measure of central tendency. Also, many distributions have more than one mode. These distributions are called "multimodal". The mode can be described by the following relationship:

[0211] (41)

[0212] (ix) Trimean: The score calculated by adding the 25th percentile plus twice the 50th percentile (the median) plus the 75th percentile, and dividing by 4. The trimean is almost as resistant to extreme scores as the median, and in skewed distributions, is less subject to sampling fluctuations than the arithmetic mean. However, it is generally less effective than the mean for normal distributions. The trimean can be described according to the following relationship:

[0213] (42)

[0214] (x) Truncated mean: A score calculated by discarding a certain percentage of the lowest and highest scores and then calculating the average of the remaining scores. For example, a mean with a 50% cutoff is calculated by discarding the lower and upper 25% of the scores and calculating the average of the remaining scores. As a further example, the median is the mean with a 100% cutoff and the arithmetic mean is the mean with a 0% cutoff. Compared to the arithmetic mean, the truncated mean is generally less susceptible to extreme scores. Therefore, for skewed distributions, it is less susceptible to sampling fluctuations than the mean. It is generally less efficient for normal distributions. As an example, the mean with a 50% cutoff is described by the following relationship:

[0215] (43)

[0216] (xi) Range: The difference between the smallest and largest values. Range can be a useful measure of spread. However, since it is based on only two values, it is sensitive to extreme scores. Due to this sensitivity, range is not usually used as the sole measure of spread, but it is still meaningful if used as a supplement to other measures of spread such as standard deviation or semi-interquartile range.

[0217] (xii) Half-interquartile range: A measure of the spread of half the difference between the 75th percentile (Q3) and the 25th percentile (Q1). Since half of the scores in a distribution lie between Q3 and Q1, the half-interquartile range is half the distance required to cover that half of the scores. In a symmetric distribution, the distance between one half-interquartile range below the median and one half-interquartile range above the median will contain half of the scores. However, this is not the case for skew distributions. Unlike the range, the half-interquartile range is usually not substantially affected by extreme scores. However, in a normal distribution it is more susceptible to sampling fluctuations than the standard deviation, and is therefore not often used to approximate normally distributed data. The half-interquartile range is defined according to the following relationship:

[0218] (44)

[0219] (xiii) Variance: A measure of the spread of a distribution. Variance is calculated by taking the mean square deviation of each number from its mean according to the following relationship:

[0220] (45)

[0221] (xiv) Standard Deviation: A function of the variance that measures how widely the values ​​of a distribution are spread out from the mean. The standard deviation is the square root of the variance. Although it is generally less sensitive to extreme scores than the range, the standard deviation is generally more sensitive than the semi-interquartile range. Therefore, the semi-interquartile range can be used to supplement the standard deviation when there is a possibility of extreme scores.

[0222] (xv) Skewness: A measure of the asymmetry of a distribution around its mean. A distribution is skewed if one of its tails is longer than the other. A positive skewness indicates a distribution with asymmetric tails extending toward more positive values ​​(greater than the mean). A negative skewness indicates a distribution with asymmetric tails extending toward more negative values ​​(less than the mean). Skewness can be calculated according to the following relationship:

[0223] (46)

[0224] (xvi) Kurtosis: A measure of the steepness or flatness of a distribution compared to a normal distribution (or the relative width of its peak). A positive kurtosis indicates a relatively pointed distribution. A negative kurtosis indicates a relatively flat distribution. Kurtosis is based on the tail of the distribution.

[0225] The size of the part can be determined by the following relationship:

[0226] (47)

[0227] The above statistical methods can be used to analyze the spatial distribution of grayscale values ​​by calculating local features at each point in the image and obtaining a set of statistical results from the distribution of local features. Using these statistical methods, the texture of the analysis area can be described and statically defined.

[0228] Texture can be characterized using texture descriptors. Texture descriptors can be calculated over a given area of ​​an image (discussed in more detail below). A commonly used texture method is the co-occurrence method, proposed by Haralick, R. et al., "Texture Features for Image Classification", IEEE Transactions of System, Man and Cybernetics, Vol. 3, pp. 610-621 (1973), which is incorporated herein by reference. In this method, the relative frequencies of grayscale pairs of pixels separated by a distance d in the θ direction are combined to form a relative displacement vector (d, θ). The relative displacement vector is calculated and stored in a matrix called the gray level co-occurrence matrix (GLCM). This matrix is ​​used to extract second-order statistical texture features. Haralick proposed 14 different features to describe the two-dimensional probability density function p ij , four of the features are more commonly used than the others:

[0229] Texture can be characterized using texture descriptors. Texture descriptors can be calculated over a given area of ​​an image (discussed in more detail below). A commonly used texture method is the co-occurrence method, proposed by Haralick, R. et al., "Texture Features for Image Classification", IEEE Transactions of System, Man and Cybernetics, Vol. 3, pp. 610-621 (1973), which is incorporated herein by reference. In this method, the relative frequencies of grayscale pairs of pixels separated by a distance d in the θ direction are combined to form a relative displacement vector (d, θ). The relative displacement vector is calculated and stored in a matrix called the gray level co-occurrence matrix (GLCM). This matrix is ​​used to extract second-order statistical texture features. Haralick proposed 14 different features to describe the two-dimensional probability density function p ij , four of the features are more commonly used than the others:

[0230] (i) The angular second moment (ASM) is calculated by the following formula:

[0231] (48)

[0232] (ii) Contrast (Con) is calculated by the following formula:

[0233] (49)

[0234] (iii) Correlation (Cor) is calculated by the following formula (where σ x and σ y is the standard deviation of the corresponding distribution):

[0235] (50)

[0236] (iv) Entropy (Ent) is calculated by the following formula:

[0237] (51)

[0238] These four features are also listed in Strand, J. et al., "Local Frequency Features for Texture Classification," Pattern Recognition, Vol. 27, No. 10, pp 1397-1406 (1994) [Strand94], which is also incorporated herein by reference.

[0239] For a given image, the image region where the above features are evaluated can be either covered by a mask (e.g., a colony mask) or by an area extending beyond the influence of the mask. Regional restrictions.

[0240] Fig.14 Several possible regions are shown. Region 1410 is a colony mask that extends only as far as the colony itself. Region 1420 is the colony's influence. region (defined by the edge of the image or plate). To further illustrate, pixel 1430 is a pixel within region 1420 but outside of region 1410. In other words, the colony represented by region 1410 is expected to extend into pixel 1630, but currently does not. Pixel 1440 is a pixel outside of both regions 1410 and 1420. In other words, not only is pixel 1410 not occupied by the colony at the time of imaging, but it is also not expected to be occupied by the colony at any time in the future (in which case it is already occupied by a different colony).

[0241] As described above, use of colony masks at different time points along the incubation process and their associated effects Regions, multiple histograms can be generated depicting different aspects of the colony and their impact on the local surrounding growth medium. The area itself may be adjusted over time, for example as the colony grows. Fig.15A -C illustrates how the growth of a colony can be used to adjust the colony's mask over time. Fig.15A is a portion of a blood culture grown on agar for 24 hours. Fig. 15B is a contrast image of the same culture compared to the previous image captured at t0. Fig. 15C are grayscale images illustrating growth at 9 hours (lightest), 12 hours (medium), and 24 hours (darkest). Fig. 15CEach shadow in can be used to design a different mask for the colony. Alternatively or additionally, as growth occurs, the shadows can be individually determined according to their respective effects. Region separation mask.

[0242] Any one of the features or combinations of features in the above feature list can be used as a feature set for capturing the specificity of organisms grown on various media on imaging plates under different incubation conditions. This list is not meant to be exhaustive, and any person skilled in the art can improve, expand or limit these feature sets based on the desired target to be imaged and the various image processing-based features known in the art. Therefore, the above example features are provided by way of illustration and not by way of limitation.

[0243] Those skilled in the art will recognize other measurements and methods to determine target shapes and features, and the above examples are provided by way of illustration, not limitation.

[0244] Contrast build-up

[0245] It is often difficult to initially predict which image in a sequence of images will provide value for growth detection, counting or identification. This is partly because image contrast varies for different colony forming units (CFU) and in different culture media. In a given image of several colonies, one colony may have a highly desirable contrast with the background, while another colony may not have sufficient contrast with the background for growth detection. This also makes it difficult to identify colonies in culture using a single method.

[0246] Therefore, it is desirable to establish contrast from all available materials, both spatially (spatial differences) and temporally (temporal differences under the same imaging conditions), as well as by using different imaging conditions (e.g., red, green, and blue channels, light and dark backgrounds, spectral images, or any other color space transformations). It is also desirable to collect contrast from multiple available sources to provide standardized images as input to algorithms for detecting colonies.

[0247] The image data may be limited based on any number of factors. For example, the image data may be limited to specific time points and / or specific information sought (e.g., spatial image information may not require as many time points as temporal image information). The lighting configuration and color space may also be selected to achieve a specific contrast target. The spatial frequency may also be varied in order to detect an object of a desired size (or a size within a target range).

[0248] To detect discrete objects, contrast can be set to an absolute value on [0,1], or to [-1,1] with a sign. The scale and offset of the contrast output can also be specified (e.g., for an 8-bit image with signed contrast, the offset can be 127.5 and the scale can be 127.5). In the example where contrast is set to an extreme value, the absolute offset can be set to 0 and the scale to 256.

[0249] Spatial contrast can be used to detect discrete objects on a uniform background. The spatial contrast at a position (x, y) within a distance r on an image I can be given by the equation In one embodiment, the distance r is defined as being greater than or equal to The contrast operator K is used to control the contrast setting using the following equation:

[0250] (52)

[0251] (53)

[0252] Temporal contrast can be used to detect moving objects or objects that change over time (e.g., CFUs that appear and / or expand on an imaging plate). The time t0 and t2 on an image I can be provided by the equation x The temporal contrast between the positions (x, y) In one embodiment, the following equation is used:

[0253] (54)

[0254] Spatial contrast collection can be performed in an automated manner by generating multiple SHQI images of the plate according to a pre-programmed sequence. Multiple images can be generated at a given incubation time to further investigate colony detection. In one embodiment, the spatial ... min To R max ), for several configurations (from CFG1 to CFG N ) collects the image data (specifically, the vector "vect" used to provide the contrast input to the contrast collection operator):

[0255] (55)

[0256] If for a given image I (x,y), the SNR is known (e.g., when SHQI imaging is the source), then the configuration in which the SNR-weighted contrast is maximized can be identified as the best configuration (Best CFG) when the following equation is satisfied:

[0257] (56) is the maximum value, exceeding:

[0258] (57)

[0259] The contrast operator K further benefits from this known SNR information, and the above equation becomes:

[0260] (58)

[0261] Time contrast collection can also be performed in an automated manner by generating multiple SHQI images of the plate according to a pre-programmed sequence. Multiple images can be generated at multiple incubation times, at least one of which is t0, to further investigate colony detection. In one embodiment, the time at t0 and one or more subsequent times up to t0 are calculated according to the following equation: x The image data is collected for several configurations with an incubation time of: [For the previous equation using "N" for both vector and configuration, the following is an acceptable modified equation, or T can be replaced by N Use it back in the equation]

[0262] (59)

[0263] In the above example, the vector can be two time points (for example, t0 and t x ), which is based on the difference in the images at those two times. However, in other applications, including x The extra time between the two images can be used to draw a vector with as many points as the time it took to capture the image. Mathematically, there is no limit to the number of points that can be included in a vector.

[0264] As with spatial contrast, if for a given image I (x,y) , the SNR is known, then the configuration in which the SNR-weighted contrast is maximized can be identified as the best configuration (Best CFG) when the following equation is satisfied:

[0265] (60) is the maximum value, exceeding:

[0266] (61)

[0267] The contrast operator K further benefits from this known SNR information, and the above equation becomes:

[0268] (62)

[0269] In the above examples, the maximum operator can be replaced by any other statistical operator, such as percentile (e.g., Q1, median, Q3 or any other percentile) or weighted sum. The weighted values ​​can come from previous work extracted from the training database, thereby opening the field of supervised contrast extraction to neural networks. Additionally, multiple algorithms can be used, and the results of the multiple algorithms are further combined using another operator such as the maximum operator.

[0270] Image Alignment

[0271] When multiple images are taken over time, in order to obtain valid time estimates from them, the images need to be aligned very accurately. Such alignment can be achieved by means of mechanical alignment devices and / or algorithms (e.g., image tracking, image matching). Those skilled in the art are already aware of these schemes and algorithms to achieve this goal.

[0272] For example, where multiple images of a target on a flat panel are collected, the coordinates of the target's location can be determined. Based on these coordinates, image data of the target collected at a later time can then be correlated with the previous image data and subsequently used to determine changes in the target over time.

[0273] In order to quickly and meaningfully utilize images (e.g., when used as input to a classifier), it is important to store the images in a spatial reference frame to maximize their invariance. Because the basic shape descriptor of a colony is generally circular, a polar coordinate system can be used to store colony images. When a colony is first detected, the colony centroid can be identified as the center of the colony's location. This center point can then be used as the initial center for the polar coordinate transformation of each subsequent image of the colony. Fig.16A An enlarged portion of an imaged plate is shown with a center point "O". Two rays "A" and "B" extending from point "O" are shown overlaid on the image (for clarity purposes). Each ray intersects a respective colony (circled). Fig.16A The encircled colonies are Fig. 16B This is shown in more detail in images 1611 and 1612. Fig. 16B , image 1611 (the colony intersected by ray "A") is reoriented within image 1613 ("A'") so that the radial axis of image 1613 is aligned with the radial axis of image 1612, so that the leftmost portion of the reoriented image is aligned with Fig.16A The rightmost portion of the reoriented image is closest to point "O" and the rightmost portion of the reoriented image is farthest from point "O". This polar coordinate reorientation allows for easier analysis of colonies in different orientations of the imaged plate (taking into account factors such as illumination).

[0274] exist Fig. 16C In, yes Fig. 16B Each of the images 1611, 1612 and 1613 is polar transformed. In the polar transformed images 1621, 1622 and 1623, the radial axis (extending from the center of each respective imaged colony) of the respective reoriented images 1611, 1612 and 1613 is at Fig. 16C The angular axes (of the respective colony) are drawn from left to right in the image and the angular axes (of the respective colony) are drawn from top to bottom.

[0275] For each polar coordinate image, a summary set of one-dimensional vectors may be generated along the radial and / or angular axes using, for example, shape features and / or histogram features (e.g., the mean and / or standard deviation of the color or intensity of the objects). Even when shape and histogram features are generally invariant when rotation is considered, it is possible that some texture features show significant changes when rotated; thus, invariance cannot be guaranteed. Therefore, it is significantly beneficial to present each colony image from the same viewpoint or angle of illumination, because object texture differences can then be used to distinguish one from another. Since illumination conditions generally show changes with angular position about the imaging center of the plate, rays passing through the colony and the center of the plate (at Fig. 16B ) shown as a line in each of the images 1611, 1612 and 1613 may serve as the origin (θ) of each image polar coordinate transformation.

[0276] Further alignment challenges arise from the fact that the plate culture medium is not absolutely frozen and rigid, and can therefore shift slightly from one shot to the next. Therefore, it cannot be absolutely assumed that the area of ​​the plate at certain coordinates of an image taken at one time will necessarily align perfectly with the area of ​​the plate at the same coordinates taken at a later time. In other words, slight deformations of the culture medium can lead to small uncertainties with respect to precisely matching a given pixel with a corresponding pixel captured at a different time point during the incubation process.

[0277] To take this uncertainty into account, the time t a The intensity value of a given pixel Can be compared with different time points t b The closest intensity value within the local neighborhood N(x,y) of the pixel Selection of the local neighborhood may include determining one or more errors or inaccuracies in the repositioning of the imaged plate from one time to the next (e.g., positional accuracy errors due to imperfect repositioning of the imaged medium, parallax errors due to unknown height of the imaged medium from one time to the next). In one example, it has been observed that setting "x" and "y" within the range of about 3 to about 7 pixels is suitable for images with a resolution of about 50 microns per pixel.

[0278] Any person skilled in the art will recognize that b The source image generates two t b Valid scheme for images: The first image corresponds to t b Grayscale dilation (called ) and the second image corresponds to t b Grayscale erosion (called ), both with kernel sizes matching the relocalized distance uncertainty d.

[0279] if Then the contrast is 0, otherwise the contrast is expressed as and The closest The estimated value of:

[0280] (63)

[0281] (64)

[0282] (65) Improvement in SNR

[0283] Under typical lighting conditions, photon shot noise (the statistical variation in the arrival rate of incident photons on the sensor) limits the SNR of the detection system. Modern sensors have sufficient capacity, which is about 1,700 to about 1,900 electrons per active square micron. Therefore, when imaging a target on a flat panel, the main concern is not the number of pixels used to image the target, but the area covered by the target within the sensor space. Increasing the area of ​​the sensor improves the SNR for the imaged target.

[0284] Image quality can be improved by capturing images using lighting conditions where photon noise dominates the SNR. Without saturating the sensor (the maximum number of photons that can be recorded per pixel in each frame). To maximize the SNR, image averaging techniques are often used. Because the SNR in dark areas is much lower than that in bright areas, these techniques are used to process images with significant brightness (or color) differences, as shown in the following equation:

[0285] (66)

[0286] Where I is the average current produced by the electron flow at the sensor. Color is perceived due to differences in absorption / reflection of matter and light across the electromagnetic spectrum, and the confidence in the captured color will depend on the system's ability to record intensity with a high SNR. Image sensors (e.g., CCD sensors, CMOS sensors, etc.) are well known to those skilled in the art and will not be described in detail here.

[0287] To overcome the classic SNR imaging limitation, the imaging system can perform an analysis of the imaged panel during image acquisition and adjust the lighting conditions and exposure time in real time based on the analysis. This process is described in PCT Publication No. WO2015 / 114121, which is incorporated by reference, and is generally referred to as supervised high quality imaging (SHQI). The system can also customize imaging conditions for different brightness regions of the panel in different color channels.

[0288] For a given pixel x, y of an image, the SNR information of the pixel acquired during the current frame N can be combined with the SNR information of the same pixel acquired during the previous or subsequent frames (e.g., N-1, N+1). As an example, the combined SNR can be determined by the following formula:

[0289] (67)

[0290] After updating the image data with the new acquisition, the acquisition system is able to predict the best next acquisition time that will maximize the SNR according to the environmental constraints (e.g., the minimum required SNR per pixel in the region of interest). For example, averaging 5 images captured in non-saturated conditions will result in an improvement in the SNR in dark areas (10% of maximum intensity). When combining information from two images captured in bright and dark conditions, optimal illumination will promote SNR improvement in dark areas in only two acquisitions.

[0291] Image Modeling

[0292] In some cases, when calculating spatial or temporal contrast between pixels of one or more images, pixel information for a given image may not be available, or may be degraded. For example, if an image of a plate was not captured at a time prior to bacterial growth (e.g., the plate was not imaged at time t0 or shortly thereafter), degraded signal information may occur. For example, degraded signal information may occur when an image is captured at time t0, but the pixels of the captured image do not accurately reflect the imaged plate prior to bacterial growth. Such inaccuracies may be caused by temporal artifacts that do not reappear in subsequent time series images of the plate (e.g., condensation that temporarily forms on the bottom of the plate due to thermal shock when the plate is first placed in the incubator).

[0293] In such a case, the unavailable or weakened image (or certain pixels in the image) can be replaced or enhanced with a model image of the flat panel. The model image can provide pixel information that reflects how the flat panel is expected to look at a specific time of the unavailable or weakened image. In the case of a model image at time t0, the model can be a clear (plain) or standard image of the flat panel and can be mathematically constructed using three-dimensional imaging / modeling techniques. The model can include each of the physical design parameters (e.g., diameter, height, dividers for accommodating multiple culture media, plastic materials, etc.), culture medium parameters (e.g., type of culture medium, culture medium composition, culture medium height and thickness, etc.), illumination parameters (e.g., angle of the light source, color or wavelength (one or more) of the light source, color of the background, etc.), and positioning parameters (e.g., the position of the flat panel in the imaging chamber) to produce a model that is as realistic as possible.

[0294] In the case of a weakened image, when the pixel information is not as sharp as the rest of the image (e.g., because underlying condensation blocks some light from passing through the plate and thus makes the portion of the plate with condensation slightly less transparent), signal restoration may be used to sharpen the weakened pixel features in the image to accomplish enhancement of the weakened image. Signal restoration may include determining intensity information for the rest of the image, identifying a median intensity of the intensity information, and then replacing the intensity information of the less sharp regions of the image with the median intensity of the rest of the image.

[0295] application

[0296] The present disclosure is primarily based on testing performed in saline at various dilutions to simulate typical urine reporting volumes (CFU / ml bucket group). The suspension for each isolate was adjusted to 0.5 McFarland Standard and used to estimate 1×10 in a BD Vacutainer tube (Cat. No. 364951). 6 , 1×10 5 , 5×10 4 , 1×10 4 , 1×10 3 and 1×10 2 Dilutions were prepared at a suspension of 100 CFU / ml. Using Kiestra Inoqul A (WCA1), the sample tubes were treated using a standard urine streak pattern - 4# zigzag (0.01 ml per plate).

[0297] ReadA Compact was used to process the plates (35°C, non-CO2) and imaged every two hours for the first 24 hours and every 6 hours for the second 24 hours during a total incubation time of 48 hours. At 1 hour, the incubation time was entered as the first read, with a margin of + / -15 minutes allowed. For the next read at 2-24 hours, it was set to every 2 hours with a margin of + / -30 minutes allowed. For readings 24-48 hours, it was set to every 6 hours with a margin of + / -30 minutes allowed. After a pure feasibility study, it was changed to eliminate the margin allowed. This was done to improve image acquisition in the desired range of 18-24 hours.

[0298] In other cases, images may be acquired over a 48 hour span—2 hour intervals for the first 24 hours and 6 hour intervals for the next 24 hours. In such a case, a total of 17 images will be acquired over a 48 hour span, including the image acquired from time t0 (0 hours).

[0299] All acquired images are corrected for lens geometry and chromatic aberrations and spectrally balanced using a known target pixel size, normalized illumination conditions, and a high signal-to-noise ratio per pixel per band. Suitable cameras for use in the methods and systems described herein are known to those skilled in the art and will not be described in detail herein. As an example, using a 4 megapixel camera to capture images of a 90 mm plate should allow enumeration of up to 30 colonies / mm when the diameter of the colonies is in the range of 100 μm and with sufficient contrast. 2 The local density (>10 5 CFU / plate).

[0300] The contrast of colonies grown on the following media was assessed:

[0301] TSAII 5% Sheep Blood (BAP): A non-selective medium widely used for urine cultures.

[0302] BAV: For colony counts and putative IDs based on colony morphology and hemolysis.

[0303] MacConkey II Agar (MAC): A selective medium for the most common Gram-negative UTI pathogens. MAC is used for differentiation of lactose-producing colonies. MAC also inhibits colonization of Proteus species. BAP and MAC are commonly used for urine cultures. Some media are not recommended for colony counts due to partial inhibition of some Gram-negatives.

[0304] Colistin Nalidixic Acid Agar (CNA): A selective medium for the most common Gram-positive UTI pathogens. CNA is not as commonly used for urine cultures as MAC, but it can help identify colonies if overgrowth of Gram-negative colonies occurs.

[0305] Chromogenic Agar (CHROMA): A non-selective medium widely used for urine cultures. CHROMA is used for colony counts and ID based on colony color and morphology. E. coli and enterococci are identified with this medium and no confirmatory testing is required. CHROMA is used less than BAP due to cost reasons. For mixed samples, CLED medium is also used.

[0306] Cystine Lactose Electrolyte Deficiency (CLED) Agar: Based on lactose fermentation, used for colony counts and presumptive ID of urinary pathogens.

[0307] Sample processing BD Kiestra TM InoqulA TM For automated processing of bacteriological samples to enable standardization and ensure consistent and high-quality streaking. BD Kiestra TM InoqulA TM The sample processor uses magnetic bead technology to streak the culture plate using a customizable pattern. The diameter of the magnetic bead is 5mm.

[0308] Fig.17 Illustrated is an example of the performance of the contrast collection algorithm in searching for Morganella morganii growth on both chromogenic agar (upper panel) and blood agar (lower panel) media. Both chromogenic agar and BAP are non-selective growth media widely used in microbiology laboratories. Fig.17 Each of the middle images shows an imaged plate illuminated by light from above the plate (top illumination). The image on the left shows the corresponding spatial contrast of the middle image. The spatial contrast is based on a 1 mm median kernel. Finally, the image on the right shows the temporal contrast of the middle image. The temporal contrast is calculated by comparing the spatial contrast at 0 hours (t0) and 12 hours (t x ) incubations, with different lighting conditions (different color channels, lighting settings, etc.) used at each image capture time.

[0309] like Fig.17 As shown in Figure 1, local contrast has its limitations when dealing with semi-transparent colonies, especially when the edge transition is small, because using the spatial contrast algorithm alone, only the confluence area with strong contrast can be selected from the image. Spatial contrast may not be able to select the target. Fig.17It is also evident in Figure 3 that time contrast is effective for isolated colonies.

[0310] Fig.17 (particularly the blood agar image at the bottom) also highlights the problem of using spatial contrast alone to detect growth on transparent media. Inks printed on transparent media have very strong edges, and therefore strong spatial contrast. This ultimately makes any other spatial contrast very difficult to see, since the colonies do not have such strongly defined edges. Therefore, in the present disclosure, temporal contrast is used in conjunction with spatial contrast to considerable benefit.

[0311] Finally, the result of the contrast determination described above is that the method for rapid detection and identification of bacterial colonies in imaged culture medium can be automated. This automated method provides significant advantages over comparable manual methods.

[0312] Fig.18 A pair of flow charts are shown that compare the timeline of the automated testing process 1800 (e.g., Figure 2 200) and a comparable timeline of a manually performed testing process 1805. Each process begins with the laboratory receiving a sample for testing 1810, 1815. Each process is followed by an incubation 1820, 1825, during which the sample can be imaged multiple times. In the automated process, after approximately 12 hours of incubation, an automated assessment 1830 is performed, after which time it can be determined definitively whether there is no growth (or normal growth) in the sample 1840. Fig.17 As shown in the results of the automated process, the use of time contrast greatly improves the ability to detect colonies - even after only 12 hours. In contrast, in the manual process, manual evaluation was not possible until nearly 24 hours into the incubation process1835. Only after 24 hours could it be clearly determined whether there was no growth (or normal growth) in the sample1845.

[0313] The use of automated processes also allows for faster AST and MALDI testing. Such testing 1850 in an automated process can begin shortly after the initial assessment 1830, and results can be obtained 1860 and reported 1875 by the 24 hour mark. In contrast, such testing 1855 in a manual process is typically not started until close to the 36 hour mark, and takes an additional 8 to 12 hours to complete before the data can be reviewed 1865 and reported 1875.

[0314] In summary, the manual test process 1805 is shown to take up to 48 hours, requires an 18-24 hour incubation period, only after which the growth of the plate can be evaluated, and there is no way to track how long the sample has been incubated. In contrast, because the automated test process 1800 can detect even relatively poor contrast between colonies (compared to the background and to each other), and can perform imaging and incubation without the need for a microbiologist to have to keep track of the timing, only 12-18 hours of incubation are required before the sample can be identified and prepared for further testing (e.g., AST, MALDI), and the entire process can be completed in about 24 hours. Therefore, the automated process of the present disclosure, with the help of the contrast processing described herein, provides faster testing of samples without adversely affecting the quality and accuracy of the test results.

[0315] Although the present disclosure has been described with reference to specific embodiments, it should be understood that these embodiments are merely illustrations of the principles and applications of the present disclosure. Therefore, it should be understood that various modifications may be made to the illustrative embodiments, and other arrangements may be designed without departing from the spirit and scope of the present disclosure as defined by the appended claims.

Claims

1. A system for identifying growth in a culture medium inoculated with a biological sample and arranged in an optically transparent container, the system comprising: an image acquisition device for capturing a digital image of the culture medium; a memory for storing information about the captured digital images; as well as One or more processors operable to execute instructions to implement: When the inoculated culture medium received from the incubator is received into the image acquisition device, a first digital image of the inoculated culture medium is obtained at a first time (t0), the first digital image having a plurality of pixels; determining coordinates of pixels in the first digital image relative to an optically transparent container carrying the inoculated culture medium; After further incubating the inoculated culture medium, the inoculated culture medium is again received into the image acquisition device, and the one or more processors perform the following steps: At the second time (t x ) obtaining a second digital image of the inoculated culture medium, the second digital image having a plurality of pixels; aligning the first digital image with the second digital image so that coordinates of pixels in the second digital image correspond to coordinates of corresponding pixels in the first digital image; comparing pixels of the second digital image with corresponding pixels of the first digital image; identifying pixels that change between the first digital image and the second digital image, wherein pixels that do not change between the first digital image and the second digital image indicate background; determining which identified pixels in the second digital image have a predetermined level of threshold contrast with pixels indicative of background; identifying one or more objects in the second digital image, each object consisting of pixels satisfying a predetermined level of threshold contrast and which are not separated from each other by background pixels; For at least one of the identified objects, determining a morphology of the object from pixels of the object; and Determining whether the target is a colony candidate based on the morphology of the target; and Coordinates of the pixel associated with the target are provided to a memory.

2. The system of claim 1 , wherein the one or more processors are operable to execute the following instructions: Target features are obtained from information about pixels associated with the identified target, wherein the target features include at least one of target shape, target size, target edge, and target color, wherein the morphology of the identified target is determined based on the target features.

3. The system of claim 2, wherein the target color is determined by spectral characteristics of pixels associated with the identified target.

4. The system of claim 3, wherein the spectral feature is selected from the group consisting of pixel color, hue, luminance, and chromaticity.

5. The system of claim 2, wherein the one or more processors are operable to execute the following instructions: obtaining background characteristic information, wherein the background characteristic information includes culture medium type and culture medium color; and The morphology of the identified target is determined based on the background feature information.

6. The system of claim 5, wherein the one or more processors are operable to execute the following instructions: comparing the target feature and the background feature information with the target feature and background feature information stored in the memory; and The type of microorganism is determined based on the target feature and the background feature information.

7. The system of claim 1, wherein the one or more processors are operable to execute the following instructions: The first digital image is aligned with the second digital image by assigning polar coordinates to pixels of each of the first and second digital images such that the polar coordinates of pixels in the second digital image are the same polar coordinates as the polar coordinates of corresponding pixels in the first digital image.

8. The system of claim 1, wherein the one or more processors are operable to execute the following instructions: identifying pixel information in the first digital image that is evidence of condensation on the optically transparent container or plating medium; and The pixel information attributable to condensation is subtracted from pixel information in the first digital image.

9. The system of claim 8, wherein the one or more processors are operable to execute the following instructions: identifying condensation in a first digital image for which an illumination source is directed upward toward a bottom of the optically transparent container, and Pixel information of pixels having an optical density below a predetermined threshold is subtracted from the first digital image.

10. The system of claim 1, wherein the one or more processors are operable to execute the following instructions: identifying pixel information in either of the first digital image and the second digital image that is evidence of dust; and The pixel information attributable to dust is subtracted from the pixel information.

11. The system of claim 1 , wherein the one or more processors are operable to execute the following instructions: obtaining a third digital image of the inoculated culture medium, wherein the third digital image is obtained at a time between the time the first digital image is obtained and the time the second digital image is obtained, wherein between the obtaining of the first digital image and the second digital image and between the obtaining of the second digital image and the third digital image, the inoculated culture medium is removed from the image obtaining device and placed in the incubator; aligning the third digital image with the first digital image so that polar coordinates of pixels in the third digital image are the same as polar coordinates of corresponding pixels in the first digital image; comparing pixels in the third digital image and the second digital image with each other; as well as Pixels that change between the third digital image and the second digital image are identified.

12. The system of claim 11, wherein the one or more processors are operable to execute the following instructions: At least one of the identified objects is associated with a pixel identified as varying between the third digital image and the second digital image.

13. The system of claim 1, wherein the one or more processors are operable to execute the following instructions: obtaining a third digital image of the inoculated culture medium, wherein the third digital image is obtained after the time when the second digital image is obtained, wherein between the time when the second digital image and the third digital image are obtained, the inoculated culture medium is removed from the imaging instrument and placed in the incubator; aligning the third digital image with the first digital image and the second digital image so that polar coordinates of pixels in the third digital image are the same polar coordinates as the polar coordinates of corresponding pixels in the first digital image and the second digital image; comparing pixels of the third digital image and the second digital image to each other; identifying pixels that change between the second digital image and the third digital image; as well as Based on compared pixels in the second and third digital images identified as having changed, it is determined that an object identified in the second digital image has changed in the third digital image.

14. The system of claim 13, wherein the one or more processors are operable to execute the following instructions: The target identified by the second digital image is used as a seed target, the degree of change determined by evaluating the seed target, and the target that has changed in the third digital image is identified by the second digital image.

15. The system of claim 14, wherein the one or more processors are operable to execute the following instructions: Coordinates of pixels associated with the target in the memory are updated based on pixels identified as having changed between the second digital image and the third digital image.

Images