Identifying microorganisms using three-dimensional quantitative phase imaging

Abstract

Methods, systems and apparatus for identifying a predicted type of one or more microorganisms, including a computer program encoded on a computer storage medium. In one aspect, the system includes a phase contrast microscope and a microorganism classification system. The phase contrast microscope is configured to generate a three-dimensional quantitative phase image of one or more microorganisms. The microorganism classification system is configured to process the three-dimensional quantitative phase image using a neural network to generate a neural network output that characterizes the microorganism, and then use the neural network output to identify the predicted type of the microorganism.

Description

[0001] CROSS-REFERENCE TO RELATED APPLICATIONS

[0002] This application claims priority under 35 USC §119 to U.S. Provisional Patent Application Serial No. 62 / 817,680, filed on March 13, 2019, entitled “RAPID IDENTIFICATION OF BIOLOGICAL ENTITIES” and U.S. Provisional Patent Application No. 62 / 856,290, filed on June 3, 2019, entitled “IDENTIFYING MICROORGANISMS USING THREE-DIMENSIONAL QUANTITATIVE PHASE IMAGING”. The entire contents of both provisional patent applications are incorporated herein by reference. Technical Field

[0003] This specification relates to the use of three-dimensional (3D) quantitative phase imaging to identify microorganisms. This specification is incorporated by reference in its entirety into U.S. patent application serial number 62 / 817,680, entitled "Rapid identification of biological entities" filed on March 13, 2019. Background Art

[0004] The term microorganism may refer to, for example, bacteria, viruses or fungi. The type of microorganism may refer to, for example, the genus, species, strain, metabolism, morphology, motility or any other suitable characteristic of the microorganism.

[0005] Quantitative phase imaging characterizes samples by quantifying the phase shift induced when light passes through the sample. Summary of the invention

[0006] This specification describes a method and apparatus for identifying microorganisms by processing 3D quantitative phase images (QPI) of the microorganisms using a neural network (e.g., a 3D convolutional neural network).

[0007] According to a first aspect, a method for identifying a predicted type of one or more microorganisms is provided. The method includes generating a three-dimensional quantitative phase image of one or more microorganisms using a phase-contrast microscope, wherein the three-dimensional quantitative phase image includes a three-dimensional representation of the microorganism. A convolutional neural network having one or more three-dimensional convolutional layers is used to process the three-dimensional quantitative phase image. The neural network is configured to process the three-dimensional quantitative phase image according to trained parameter values ​​of the neural network to generate a neural network output that characterizes the microorganism. The predicted type of the microorganism is identified using the neural network output.

[0008] In some implementations, the neural network output includes a corresponding probability value for each of a predetermined number of microorganism types. The probability value for a given microorganism type indicates the likelihood that the microorganism belongs to the given microorganism type.

[0009] In some implementations, the predicted type of microorganism is identified as the type of microorganism having the highest probability value in the neural network output.

[0010] In some implementations, the neural network output includes a probability value for a microorganism type, which indicates the likelihood that the microorganism belongs to the microorganism type.

[0011] In some implementations, the predicted type of the microorganism is selected from the group consisting of genus, species, strain, gram-stainability, metabolic properties, morphology, and motility.

[0012] In some implementations, the three-dimensional quantitative phase image of the microorganism is a three-dimensional refractive index tomogram.

[0013] In some implementations, the microorganism is selected from the group consisting of bacteria, viruses, fungi, parasites, and microalgae.

[0014] In some implementations, the microorganisms include bacteria present in a patient's blood sample.

[0015] In some implementations, the method includes administering antibiotic therapy to the patient based on the predicted type of bacteria.

[0016] In some implementations, phase contrast microscopy, quantitative phase microscopy, or digital holographic microscopy is used to generate phase and amplitude images of the microorganisms at each of a plurality of illumination angles, and a three-dimensional refractive index tomogram is reconstructed using these phase and amplitude images.

[0017] In some implementations, this method takes up to one hour.

[0018] According to a second aspect, a method for identifying a predicted type of one or more microorganisms by processing a three-dimensional quantitative phase image of one or more microorganisms using a neural network according to trained parameter values ​​of the neural network is provided.

[0019] In some implementations, the neural network includes one or more three-dimensional convolutional layers.

[0020] In some implementations, the neural network is configured to generate a neural network output that characterizes the microorganism, and the predicted type of the microorganism is identified based at least in part on the neural network output.

[0021] In some implementations, this method takes up to one hour.

[0022] According to a third aspect, a method for identifying a predicted type of one or more microorganisms is provided. The method includes providing a biological sample from a patient suffering from a bacterial infection. A three-dimensional quantitative phase image of one or more bacteria isolated from the biological sample is processed using a neural network having one or more three-dimensional convolutional layers according to trained parameter values ​​of the neural network to generate a neural network output characterizing the bacteria, thereby identifying a predicted type of bacteria in the patient within one hour of obtaining the biological sample from the patient. The predicted type of bacteria is identified using the neural network output.

[0023] In some implementations, the biological sample includes a blood sample.

[0024] In some implementations, the predicted type of bacteria in a patient is identified within 45 minutes of providing a biological sample from the patient.

[0025] In some implementations, the predicted type of bacteria in a patient is identified within 30 minutes of providing a biological sample from the patient.

[0026] In some implementations, the predicted type of bacteria in a patient is identified within 15 minutes of providing a biological sample from the patient.

[0027] According to a fourth aspect, a method for identifying a predicted type of one or more microorganisms is provided. The method includes providing a biological sample from a patient suspected of having a bacterial infection. A three-dimensional quantitative phase image of one or more bacteria isolated from the biological sample is processed using a neural network having one or more three-dimensional convolutional layers according to trained parameter values ​​of the neural network to generate a neural network output that characterizes the bacteria, thereby identifying a predicted type of bacteria in the patient within one hour of providing the biological sample from the patient. The predicted type of bacteria is identified using the neural network output.

[0028] According to a fifth aspect, a system including a phase contrast microscope and a microbial classification system is provided. The phase contrast microscope is configured to generate a three-dimensional quantitative phase image of one or more microorganisms, wherein the three-dimensional quantitative phase image includes a three-dimensional representation of the microorganism. The microbial classification system is configured to process the three-dimensional quantitative phase image using a neural network, wherein the neural network is a convolutional neural network having one or more three-dimensional convolutional layers, and the neural network is configured to process the three-dimensional quantitative phase image according to trained parameter values ​​of the neural network to generate a neural network output characterizing the microorganism. The predicted type of the microorganism is identified using the neural network output.

[0029] According to a sixth aspect, a non-transitory computer-readable storage medium storing one or more instructions is provided, which, when executed by one or more computers, causes the one or more computers to receive a three-dimensional quantitative phase image of one or more microorganisms generated using a phase contrast microscope, wherein the three-dimensional quantitative phase image includes a three-dimensional representation of the microorganism. The three-dimensional quantitative phase image is processed using a neural network, wherein the neural network is a convolutional neural network having one or more three-dimensional convolutional layers. The neural network is configured to process the three-dimensional quantitative phase image according to trained parameter values ​​of the neural network to generate a neural network output that characterizes the microorganism. The predicted type of the microorganism is identified using the neural network output.

[0030] Particular embodiments of the subject matter described in this specification can be implemented to realize one or more of the following advantages.

[0031] The methods described herein may enable microorganisms to be identified more rapidly and / or more consistently than some conventional methods.

[0032] For example, screening based on blood culture is a conventional method for identifying bacterial species. Although blood culture can reveal the presence of bacteria, it usually takes hours to days for bacteria to grow and form distinct colonies. In addition, mass spectrometry or sensitivity testing for various antibiotics is often used to determine the exact type of bacteria. Other biochemical methods including antiserum testing, deoxyribonucleic acid (DNA) microarrays, and real-time polymerase chain reaction (PCR) can identify the type of bacteria. However, due to their reliance on biochemical reactions or molecular specific signals, these conventional methods also involve processes that take up to an hour to a day and involve the maintenance and use of specialized biochemical reagents.

[0033] In contrast, the techniques described in this specification can achieve accurate bacterial identification in under an hour (in some cases). For example, advances in microfluidic engineering and antibody engineering have enabled the isolation of bacteria in under an hour from concentrations as low as those found in the blood of patients with bacterial infections. Obtaining 3D QPI of bacterial samples does not involve time-consuming processes such as culturing or staining, and the classification neural network uses the input data to make predictions in milliseconds or less.

[0034] Compared to some conventional methods, the technology described in this specification can enable microorganisms to be identified more consistently because it is less susceptible to human factors in the measurement and analysis process. For example, some conventional methods rely on physical experiments performed by people, but the technology described in this specification can utilize consistent measurements (i.e., measurements generated using quantitative phase imaging) made using computational techniques (i.e., classification neural networks).

[0035] By processing 3D QPIs, the techniques described in this specification can (in some cases) more accurately identify microorganisms than would otherwise be possible by processing 2D QPIs. Specifically, 2D QPIs corresponding to a particular illumination angle often include noise image artifacts resulting from interference caused by scattering or multiple reflections from dust particles. Images of microorganisms (e.g., bacteria) that are only a few hundred nanometers thick are particularly susceptible to such artifacts. The signal-to-noise ratio (SNR) of a 3D QPI reconstructed from multiple 2D QPIs can be higher than that of a 2D QPI because noise artifacts depend on the illumination angle, while the effect of a microorganism remains consistent.

[0036] By using a 3D convolutional neural network, the techniques described herein can (in some cases) more accurately identify microorganisms than would otherwise be possible by using, for example, a 2D convolutional neural network or other machine learning model. Specifically, a 3D convolutional neural network can be trained to exploit the complex 3D spatial relationships between different parts of a 3D QPI to accurately identify microorganisms depicted in a 3D QPI.

[0037] Certain details of one or more embodiments of the subject matter of this specification are set forth in the accompanying drawings and the description that follows. Other features, aspects, and advantages of the subject matter will become apparent from the description, drawings, and claims. BRIEF DESCRIPTION OF THE DRAWINGS

[0038] Figure 1 is a block diagram of an example environment for predicting the type of a set of one or more microorganisms from a 3D QPI of the microorganisms.

[0039] Figure 2 An example training system is shown.

[0040] Figure 3 is a graphical representation of an example set of training data.

[0041] Figure 4 is an illustration of an example process for using phase contrast microscopy and a microbial classification system to determine appropriate antibiotic therapy for treating a patient with a bacterial infection.

[0042] Figure 5 is an illustration of an example process for generating 3D QPI of microorganisms using phase contrast microscopy.

[0043] Figures 6A-6G Shown are experimental results where a classification neural network was trained to predict the species of 19 sepsis-causing bacteria from 3D refractive index tomograms of the bacteria.

[0044] Figure 7 is an illustration of an example architecture of a classification neural network configured to process a 3D QPI of a microorganism to generate a classification output characterizing a predicted type of the microorganism.

[0045] Figure 8 is a flow chart of an example process for identifying a predicted type of one or more microorganisms.

[0046] Fig. 9 is an illustration of a Mach-Zehnder phase contrast microscope setup used to generate a 3D refractive index tomogram.

[0047] Like reference numbers and designations in the various drawings indicate like elements. DETAILED DESCRIPTION

[0048] This specification describes a microbial classification system that can predict the type (e.g., species) of a microorganism (e.g., bacteria) by processing a three-dimensional (3D) quantitative phase image (QPI) of the microorganism using a 3D convolutional classification neural network. The microbial classification system can be deployed in a clinical setting for diagnostic applications. These and other features will be described in more detail below.

[0049] Figure 1 is a block diagram of an example environment 100 in which types 102 of a set of one or more microorganisms 104 are predicted from 3D QPIs 106 of the microorganisms 104 .

[0050] Microorganism 104 can be bacteria, viruses, fungi, parasites, microalgae, or any other suitable type of microorganism. Microorganism 104 can be pathogenic (eg, sepsis-inducing), but need not be pathogenic.

[0051] The microorganism 104 can be derived from any of a variety of sources. For example, the microorganism 104 can be a bacterium (or fungus) obtained by processing a blood sample of a patient suffering from a bacterial (or fungal) infection using a bacterial (or fungal) isolation technique, as described in reference Figure 4 More specifically described.

[0052] The predicted type 102 of the microorganism 104 can define, for example, a predicted species of the microorganism (e.g., Acinetobacter baumannii, Bacillus subtilis, or Enterobacter cloacae), a predicted genus, a predicted strain, a predicted Gram staining property (e.g., Gram-positive or Gram-negative), a predicted metabolic property (e.g., aerobic or anaerobic), a predicted morphology (e.g., cocci, bacilli, or coccobacilli), a predicted motility (e.g., motile or non-motile), or any other suitable characteristic.

[0053] Optionally, the predicted type 102 of the microorganism 104 can define a prediction of multiple characteristics of the microorganism 104. For example, the predicted type 102 of the microorganism 104 can define both a predicted species of the microorganism and a predicted motility of the microorganism.

[0054] Environment 100 includes phase contrast microscope 108 and microorganism classification system 110. Microorganism classification system 110 may be implemented as a computer program on one or more computers at one or more locations.

[0055] Phase contrast microscopy 108 is used to generate a 3D quantitative phase image 106 (ie, 3D QPI 106) of microorganism 104. Broadly speaking, 3D QPI 106 is a 3D representation of microorganism 104 that characterizes the morphology and biophysical properties of the microorganism by quantifying the phase shift induced when light passes through microorganism 104.

[0056] In general, the 3D QPI 106 of the microorganism 104 may be represented as a 3D matrix of numerical values ​​(e.g., floating point values), where each component of the 3D matrix corresponds to a respective 3D spatial location in or around the microorganism 104. In one example, the 3D QPI 106 may be a "refractive index (RI) tomogram," where each component of the 3D matrix defines the refractive index of a material (e.g., a portion of a microorganism) at a spatial location corresponding to the component.

[0057] Phase contrast microscopy 108 may use any of a variety of different techniques to generate 3D QPI 106 of microorganism 104. Figure 5 An example method for generating a 3D QPI 106 of a microorganism 104 using a phase contrast microscope 108 is described in greater detail.

[0058] The microorganism classification system 110 is configured to process the 3D QPI 106 of the microorganism 104 using the classification neural network 112 according to the trained values ​​of the model parameters 114 of the classification neural network 112 to generate a classification output 116. Thereafter, the microorganism classification system 110 uses the classification output 116 to identify the predicted type 102 of the microorganism 104. Below are a few examples.

[0059] In one example, the classification output 116 defines a probability distribution of a predetermined number of possible microorganism types; that is, the classification output 116 includes a corresponding numerical probability value corresponding to each possible microorganism type. In this example, the system 110 can identify the predicted type 102 of the microorganism 104 as the microorganism type associated with the highest probability value via the classification output 116.

[0060] In another example, the classification output 116 defines an index (e.g., a positive integer value) from a predetermined set of possible indexes, where each index corresponds to a corresponding possible microorganism type. In this example, the system 110 can identify the predicted type 102 of the microorganism 104 as the microorganism type corresponding to the index defined by the classification output 116.

[0061] In some cases, the 3D QPI 106 generated by the phase contrast microscope 108 may depict a large number (e.g., thousands) of microorganisms. For example, the phase contrast microscope 108 may generate a 3D QPI 106 that depicts thousands of microorganisms (e.g., bacteria) mounted on a microscope slide. The system 110 may detect individual microorganisms in the 3D QPI 106 (e.g., using an object detection neural network trained to detect microorganisms) and crop a plurality of regions from the 3D QPI 106, each region depicting one or more corresponding microorganisms. For each region cropped from the 3D QPI 106, the system 110 may process the region using the classification neural network 112 to generate a corresponding classification output 116 that characterizes the one or more microorganisms depicted in the cropped region. Thereafter, the system 110 may use the corresponding classification output generated for the cropped region to generate an "overall" prediction of the type of microorganism depicted in the original 3D QPI 106. The following are a few examples.

[0062] In one example, a probability distribution of possible microorganism types is defined by the classification neural network 112 for the corresponding classification output 116 generated for each region cropped from the 3D QPI 106 (as previously described). In this example, the system 110 can combine (e.g., average) the probability distributions of possible microorganism types generated for each region in the region cropped from the 3D QPI 106 to generate an "overall" probability distribution of possible microorganism types. The system 110 can identify the overall predicted type of the microorganism depicted in the 3D QPI 106 as the microorganism type associated with the highest probability value through the overall probability distribution of possible microorganism types.

[0063] In another example, an index associated with a possible microorganism type is defined by the classification neural network 112 for a corresponding classification output 116 generated for each region cropped from the 3D QPI 106 (as previously described). In this example, the system 110 may determine the most frequently occurring index (i.e., a “pattern index”) in the set of indexes generated by the classification neural network 112 for the regions cropped from the 3D QPI 106. The system 110 may identify the overall predicted type of the microorganism depicted in the 3D QPI 106 as the possible microorganism type associated with the pattern index.

[0064] The robustness of the system 110 may be improved by processing multiple regions cropped from the 3D QPI to generate an overall prediction of the type of microorganism depicted in the 3D QPI.

[0065] In some cases, the system 110 can use an ensemble of multiple classification neural networks 112 (rather than using a single classification neural network 112) to determine the predicted microorganism type 102. The classification neural networks in the ensemble can differ from each other in any of a variety of ways. For example, some or all of the classification neural networks in the ensemble may have different neural network architectures, such as different 3D convolutional neural network architectures. As another example, some or all of the classification neural networks in the ensemble can be trained differently, such as by having their model parameter values ​​initialized differently. Each classification neural network in the ensemble can process the 3D QPI 106 to generate a corresponding classification output 116, and the system 110 can use the corresponding classification output to generate an overall prediction of the type of microorganism depicted in the 3D QPI 106 (as previously described).

[0066] Optionally, one or more intermediate outputs generated by the classification neural network 112 by processing the 3D QPI 106 can be provided for characterizing the microorganism 104. The intermediate outputs (which can also be referred to as latent features) refer to the outputs generated by the intermediate layers (i.e., the layers between the input layer and the output layer) of the classification neural network 112. The intermediate outputs can be represented as an ordered set of values, such as a vector or matrix of values. The intermediate outputs generated by the classification neural network 112 can be used as compact descriptors of the 3D QPI 106, which can be processed by another machine learning model (e.g., a support vector machine (SVM) or a random forest (RF)) to generate a predicted output that characterizes the microorganism 104. Figure 6B is a diagram of examples of some intermediate outputs (ie, latent features) generated by the classification neural network 112.

[0067] The model parameters 114 of the classification neural network 112 refer to parameters that define the operations performed by the layers of the classification neural network, such as weight matrices and bias vectors of the layers of the classification neural network 112 .

[0068] The classification neural network 112 may have any suitable neural network architecture. For example, the classification neural network 112 may be a 3D convolutional neural network, i.e., a neural network including one or more 3D convolutional layers. A 3D convolutional layer refers to a convolutional layer configured to process one or more 3D inputs (e.g., 3D matrices represented as numerical values) using one or more 3D convolutional filters to generate one or more 3D outputs.

[0069] In addition to the 3D convolutional layers, the classification neural network 112 may include any suitable type of additional neural network layers (e.g., pooling layers or fully connected layers) connected in any suitable configuration. Figure 7 An example classification neural network architecture is described in more detail.

[0070] The training system may determine trained values ​​of the model parameters 114 of the classification neural network 112 by training the classification neural network 112 on a set of training examples. Figure 2 Examples of training systems are described in more detail.

[0071] Phase contrast microscope 108 and microbial classification system 110 can be deployed in a clinical setting for diagnostic applications. Figure 4 As described, phase contrast microscopy 108 and microbial classification system 110 may be used to identify appropriate antibiotic therapy for a patient suffering from a bacterial infection.

[0072] Figure 2An example training system 200 is shown. Training system 200 is an example of a system implemented as a computer program on one or more computers in one or more locations in which the systems, components, and techniques described below are implemented.

[0073] The training system 200 uses a set of training data 202 and a training engine 204 to determine trained values ​​for the model parameters 114 of the classification neural network 112 .

[0074] The training data 202 includes a plurality of training examples. Each training example includes: (i) a 3D QPI of one or more microorganisms (e.g., bacteria), and (ii) a "target microorganism type" that defines the type of microorganism depicted in the 3D QPI. The target microorganism type specifies the classification output that should be generated by the classification neural network 112 by processing the 3D QPI.

[0075] Typically, the training data 202 includes at least one training example corresponding to each possible microorganism type (ie, from a predetermined set of possible microorganism types that can be predicted by the classification neural network 112 ). Figure 3 An illustration of an example set of training data 202 is provided.

[0076] The training engine 204 is configured to iteratively adjust current values ​​of the model parameters 114 of the classification neural network 112 at each of a plurality of training iterations.

[0077] At each training iteration, training engine 204 selects (eg, randomly samples) a batch (ie, a set) of one or more training examples 206 from training data 202 , and processes the 3D QPI included in each training example 206 to generate a corresponding predicted microorganism type 208 .

[0078] The training engine 204 then determines parameter value updates 210 to the current values ​​of the model parameters 114 based on: (i) the predicted microorganism types 208 generated by the classification neural network 112, and (ii) the target microorganism types specified by the training examples 206. More specifically, to determine the parameter value updates 210 at the current training iteration, the training engine 204 determines the gradient of the objective function with respect to the model parameters 114 of the classification neural network 112. Typically, the objective function depends on the predicted microorganism types 208 generated by the classification neural network 112 and the target microorganism types specified by the training examples 206. The training engine 204 can determine the parameter value updates 210 based on the gradient of the objective function (e.g., according to an update rule of any suitable gradient descent optimization algorithm (e.g., Adam or RMSprop)).

[0079] The parameter value updates 210 can be represented in any suitable digital format, for example, as an ordered set of numerical values ​​(e.g., a vector of numerical values) 112 that includes a respective numerical value for each model parameter 114 of the classification neural network. The training system 200 can determine updated values ​​of the model parameters 114 at training iterations as a result of adding the parameter value updates 210 to the current values ​​of the model parameters 114.

[0080] The training engine 204 may determine the gradient of the objective function in any suitable manner, for example, using a back-propagation technique. The objective function may be any suitable classification objective function, for example, a cross entropy objective function.

[0081] The training system 200 may determine that the training of the classification neural network 112 is complete when a training termination criterion is met. For example, the training termination criterion may be that a predetermined number of training iterations have been performed. As another example, the training termination criterion may be that the accuracy of the classification neural network 112, evaluated on a held-out set of training examples, reaches at least a predetermined threshold.

[0082] After determining that the training termination criteria are met, the training system 200 can provide the trained values ​​of the model parameters 114 of the classification neural network 112 to the microbial classification system 110 (i.e., reference Figure 1 A “trained value” of a model parameter 114 refers to the value of the model parameter 114 at the end of the last training iteration performed by the training system 200.

[0083] The training system 200 may be remote from the microorganism classification system 110, in which case the training system 200 may provide the trained values ​​of the model parameters 114 to the microorganism classification system, for example, via a wired or wireless connection.

[0084] Figure 3 is a diagram 300 of an example set of training data 202. The training examples of the training data 202 each include: (i) a 3D QPI of a bacterium causing sepsis, and (ii) a target microorganism type defining the species of the bacterium.

[0085] Graph 300 depicts exemplary 3D QPIs of bacteria corresponding to each of 19 different sepsis-causing species. Specifically, the images are sorted by image labels (i) to (xix), Figure 3The 3D QPI in the image depicts the following examples: Acinetobacter baumannii (A.baumannii), Bacillus subtilis (B.subtilis), Enterobacter cloacae (E.cloacae), Enterococcus faecalis (E.faecalis), Escherichia coli (E.coli), H.influenza (H.influenza), Klebsiella pneumoniae (K.pneumonia), Listeria monocytogenes (L.monocytogenes), Micrococcus luteus (M.luteus), Proteus mirabilis (P.mir abilis, Pseudomonas aeruginosa, Serratia marcescens, Staphylococcus aureus, Staphylococcus epidermidis, S. maltophilia, Streptococcus agalactiae, Streptococcus anginosus, S. pneumoniae, and Streptococcus pyogenes.

[0086] Optionally, the classification neural network 112 is trained using the training data 202 (e.g., as described in reference Figure 2 ), the training data 202 may be “enhanced” using the enhancement engine 302. More specifically, the enhancement engine 302 may process existing training examples in the training data 202 to generate “new” training examples that may then be used to train the classification neural network 112.

[0087] To generate new training examples from existing training examples, the enhancement engine 302 can modify the 3D QPI image from the existing training example, for example, by adding random noise values, rotating, elastically deforming, or translating the 3D QPI image from the existing training example. The new training example will include: (i) the modified 3D QPI image, and (ii) the same target microorganism type as the existing training example.

[0088] Training the classification neural network 112 using the training data 202 augmented by new training examples generated by the augmentation engine 302 may improve the accuracy of the trained classification neural network, for example, by mitigating potential effects of overfitting.

[0089] Figure 4 is for use with a phase contrast microscope 108 and a microbial classification system 110 (e.g., as described in reference Figure 1 ) is an illustration of an example process 400 for determining an antibiotic therapy 402 suitable for treating a patient 404 having a bacterial infection.

[0090] A blood sample 406 from a patient 404 is provided, for example by drawing blood from the patient 404. For convenience, Figure 4 The description refers to a "blood sample", but other biological samples may be used, such as a tissue biopsy sample (e.g., from stomach tissue), a Pap smear sample (e.g., from the cervix), a urine sample, a stool sample, or a saliva sample.

[0091] The bacterial separation technique is applied to the blood sample 406 to separate bacteria 408 that cause the bacterial infection in the patient 404. That is, the bacterial separation technique is applied to remove a sample of bacteria 408 from the blood sample 406. Any suitable bacterial separation technique can be applied to separate the bacteria 408 to any suitable purity level. Examples of bacterial separation techniques include centrifugation, filtration with a filter with holes, and separation using a microfluidic or microchip device.

[0092] The phase contrast microscope 108 is used to generate a 3D QPI 412 of the bacteria 408. For example, the phase contrast microscope may generate a 3D refractive index tomogram of the bacteria 408 (as previously described).

[0093] The microbial classification system 110 processes the 3D QPI 412 of the bacteria 408 to generate an output that defines a predicted class 410 of the bacteria 408 causing the bacterial infection in the patient 404. The bacteria 408 may be, for example, sepsis-causing, malaria-causing, or tuberculosis-causing.

[0094] The predicted species 410 of bacteria 408 is used to select an antibiotic therapy 402 (or other treatment) suitable for treating the bacterial infection in the patient 404. In particular, an antibiotic therapy that is particularly effective at inhibiting or destroying the predicted species 410 of bacteria can be selected.

[0095] The selected antibiotic therapy 402 may thereafter be used to treat the bacterial infection in the patient 404 , for example, by administering the selected antibiotic intravenously to the patient 404 .

[0096] Process 400 can achieve rapid identification of bacterial species that cause infection in patients. For example, advances in microfluidics and antibody engineering have achieved the separation of bacteria from concentrations as low as those present in the blood of patients with bacterial infections within one hour. Obtaining 3D QPI of bacterial samples does not require time-consuming processes such as culturing or staining, and microbial classification systems (specifically, trained classification neural networks) use input data to make predictions that consume milliseconds or less. Therefore, in some cases, process 400 can achieve accurate bacterial identification within one hour. In some specific cases, process 400 can achieve accurate bacterial identification within 45 minutes, within 30 minutes, or within 15 minutes. After bacteria 408 are separated from blood sample 406 (which may be the most time-consuming step of process 400), generating 3D QPI 412 and predicting 410 of bacterial species can occur within 10 minutes, within 5 minutes, within 3 minutes, or within 1 minute, such as 1 to 45 seconds (e.g., 1 to 30 seconds, 1 to 20 seconds, 1 to 10 seconds).

[0097] In contrast, typical bacterial species identification in a hospital (ie, without the aid of 3D quantitative phase imaging and microbial classification system 110 ) may take 12 hours or more because of the blood culture and subsequent analysis involved.

[0098] Rapidly identifying the bacterial species causing or contributing to a patient's infection can enable the appropriate antibiotic (i.e., an antibiotic that effectively inhibits or destroys that species of bacteria) to be rapidly administered to the patient. Rapid administration of appropriate antibiotics to patients with bacterial infections can greatly improve the patient's clinical outcome.

[0099] For example, in some cases, saving the life of a sepsis patient depends on effectively treating the patient within the first few hours after admission. With delays in initial antibiotic therapy, the mortality rate of sepsis patients may surge. Therefore, rapid antibiotic therapy facilitated by the 3D phase contrast imaging and microbial classification system 110 can play a vital role in reducing the mortality rate of sepsis patients.

[0100] Process 400 is just one example of a setup in which phase contrast microscope 108 and microbial classification system 110 may be used. Here are some other examples. In one example, phase contrast microscope 108 and classification system 110 may be used to continuously monitor hospitalized patients with infectious diseases, for example, by predicting the type of bacteria currently causing the infectious disease at each of a plurality of time points. In another example, phase contrast microscope 108 and classification system 110 may be used to identify predicted types of bacteria present in an agricultural environment (e.g., a fish farm). In another example, phase contrast microscope 108 and classification system 110 may be used to identify predicted types of bacteria present in food, medicine, or both.

[0101] Figure 5 5 is an illustration of an example process for generating a 3D QPI of a microorganism using a phase contrast microscope. By using a phase contrast microscope, a corresponding phase image (e.g., phase image 502) and a corresponding amplitude image (e.g., amplitude image 504) of the microorganism are obtained from multiple illumination angles (e.g., illumination angles θ1 to θ 70 ) measurements. Then, using optical diffraction tomography (ODT) technology (which may also be referred to as refractive index tomography technology, tomographic phase microscopy technology, etc.), a 3D QPI (e.g., 3D refractive index tomogram 506) is reconstructed based on the set of multi-angle phase and amplitude images.

[0102] Figures 6A-6G The results of an experiment in which a classification neural network was trained to predict the types of 19 sepsis-causing bacteria from 3D QPIs of bacteria are shown. The 19 sepsis-causing bacteria are: Acinetobacter baumannii, Bacillus subtilis, Enterobacter cloacae, Enterococcus faecalis, Escherichia coli, Haemophilus influenzae, Klebsiella pneumoniae, Listeria monocytogenes, Micrococcus luteus, Proteus mirabilis, Pseudomonas aeruginosa, Serratia marcescens, Staphylococcus aureus, Staphylococcus epidermidis, Stenotrophomonas maltophilia, Streptococcus agalactiae, Streptococcus anginosus, Streptococcus pneumoniae, and Streptococcus pyogenes bacteria. In this experiment, the 3DQPI is a refractive index tomogram, each corresponding to 7.2×7.2×4.0 μm 3 A rectangular area with one or more labeled samples. The pixel sizes in the x, y, and z directions are 0.1, 0.1, and 0.2 μm, respectively. Figure 7 The architecture of the classification neural network used in this experiment is described in more detail in [ 14 ]. The experiments were performed on 5041 3D RI tomograms of sepsis-causing bacteria; for each species, an equal number of tomograms were randomly assigned to the test and validation sets.

[0103] The trained classification neural network has an accuracy of 94.6% in the blind test of species classification. The average sensitivity, specificity and precision of each species are 94.6%, 99.7% and 94.7% respectively. The sensitivity, specificity and precision of any species are not less than 85%. For 7.5% of Escherichia coli and Streptococcus pneumoniae, errors related to the misclassification of Escherichia coli as Klebsiella pneumoniae and the misclassification of Streptococcus pneumoniae as Streptococcus pyogenes occurred. The recognition of four species including influenza bacilli, Listeria monocytogenes, Pseudomonas aeruginosa and Streptococcus anginosus has a sensitivity of 100%. Proteus mirabilis has an identification sensitivity of 85%. Proteus mirabilis, Pseudomonas aeruginosa and Stenotrophomonas maltophilia have a recognition specificity of 100%. Escherichia coli and Klebsiella pneumoniae have a specificity of 99.17%. Each of Proteus mirabilis, Pseudomonas aeruginosa and Stenotrophomonas maltophilia has a recognition accuracy of 100%. E. coli was identified with an identification accuracy of 85.7%.

[0104] Fig. 6A is an illustration of a table 600 representing the accuracy of a trained classification neural network. The shading of the portion of table 600 corresponding to row i and column j represents how frequently the classification neural network predicts that the bacteria of the species corresponding to row i belong to the species corresponding to column j.

[0105] Figure 6B A scatter plot 602 is shown, which shows the "latent features" extracted by the trained classification neural network for 3D refractive index tomograms of different bacterial species. Specifically, for the corresponding 3D refractive index tomogram, each point in the scatter plot is provided to the last fully connected layer of the classification neural network (i.e., as shown in reference Figure 7 604 corresponds to the 3D refractive index tomogram of Acinetobacter baumannii bacteria (that is, most of the points in cluster 604 correspond to the 3D refractive index tomogram of Acinetobacter baumannii bacteria). It can also be shown that scatter plot 602 is based on other characteristics (for example, Gram staining and metabolic properties) to cluster 3D refractive index tomograms approximately.

[0106] Figure 6CA bar graph 606 is shown which illustrates the accuracy of the classification neural network after it has been trained to process different forms of input data. A total of six types of data are compared: 2D optical phase delay images, a set of 2D optical phase delay images from multiple illumination angles (i.e., which can be processed to generate a 3D RI tomogram), a 3D RI tomogram, and a threshold-based binary mask of these three types of images (as described in reference Fig.6D As shown). These three types of images and the corresponding classification methods are referred to as 2D, multi-view and 3D in this specification. The thresholds in the 2D phase and 3D RI images are set to 0.05π and 1.3425 to accommodate the wide variety of phase and RI values ​​in bacteria. In the multi-view method and the 2D method, the classification neural network is a 2D convolutional neural network. In the multi-view method, the predictions for the 2D image can be integrated and averaged to make predictions for the multi-view data. Bar graph 606 shows that in this experiment, processing the 3D RI tomogram enables the classification neural network to make the most accurate prediction of the bacterial species. The accuracy of the trained classification neural network is 74.2%, 92.2% and 94.6% for 2D, multi-view and 3D tomograms, respectively, and 49.6%, 75.9% and 92.4% for the corresponding binary mask input data, respectively.

[0107] Fig.6D is a diagram of a 2D phase image 608 thresholded at 0.05π to generate a corresponding binary mask image 610, a set of multi-view phase delay images 612 from multiple illumination angles, each thresholded at 0.05π to generate a corresponding binary mask image 614, and a 3D RI tomogram 616 thresholded at 1.3425 to generate a corresponding binary mask image 618. Scale bar represents 2 μm.

[0108] Fig. 6EA 3D RI tomogram 620 correctly classified by a trained classification neural network and a 3D RI tomogram 628 incorrectly classified by a trained classification neural network are shown. The 3D RI tomogram 620 depicts Bacillus subtilis bacteria, 624 depicts three cross-sections of the 3D RI tomogram 620, and 626 depicts three cross-sections of a saliency map generated for the 3D RI tomogram 620 using the Grad-CAM++ technique. The intensity of a point in the saliency map of the 3D RI tomogram can be understood as indicating the importance of the point to the prediction generated by the trained classification neural network for the 3D RI tomogram. The 3D RI tomogram 628 depicts Staphylococcus aureus bacteria, 630 depicts three cross-sections of the 3D RI tomogram 628, and 632 depicts three cross-sections of a saliency map generated for the 3D RI tomogram 628 using the Grad-CAM++ technique. The scale bar represents 2 μm. Legend 622 represents the visualization of 3D RI tomograms 620 and 628; the horizontal axis represents the refractive index value (which can be unitless), and the vertical axis represents the gradient of the refractive index (which can be expressed in arbitrary units). For example, the darker areas of visualizations 620 and 628 correspond to portions of the 3D RI tomograms having a refractive index in the range of 1.375 to 1.4 and a refractive index gradient greater than 10.

[0109] Fig. 6F Three cross-sections 634 of a 3D RI tomogram depicting influenza bacilli bacteria correctly classified by a trained classification neural network by processing the 3D RI tomograms, and three cross-sections 636 of saliency maps generated for the 3D RI tomogram using the Grad CAM++ technique are shown. 638 shows a set of multi-angle 2D phase images of the same influenza bacilli bacteria processed by a trained classification neural network (i.e., having a 2D convolutional structure) and misclassified as depicting Staphylococcus aureus bacteria, and 640 shows the corresponding saliency maps generated for each of the 2D phase images 638 using the Grad-CAM++ technique. The scale bar represents 2 μm.

[0110] Figure 6GAn example of how accurately various properties of bacteria (e.g., genus, motility, morphology, Gram staining, and aerobicity) are identified using a trained classification neural network is shown. A set of one or more bacterial properties can be predicted by processing a 3D RI tomogram of the bacteria using a trained classification neural network to predict the species of the bacteria, and then identifying the properties of the bacteria based on the predicted species of the bacteria. Table 642 characterizes the accuracy of identifying 15 genera of 19 bacterial species that cause sepsis using a trained classification neural network (overall: 95.5%). Table 644 characterizes the accuracy of identifying bacterial species belonging to the genus Staphylococcus using a trained classification neural network (overall: 98.7%). Table 646 characterizes the accuracy of identifying bacterial species belonging to the genus Streptococcus using a trained classification neural network (overall: 96.2%). Bar chart 648 characterizes the accuracy of identifying the motility of bacteria using a trained classification neural network (overall: 98.2%). Bar graph 650 represents the accuracy of identifying the morphology of three groups of bacteria (bacillus, cocci and cocci) using the trained classification neural network (overall: 98.0%). Bar graph 652 represents the accuracy of identifying the Gram stain of bacteria using the trained classification neural network (overall: 97.5%). The aerobicity of bacteria was identified with an overall accuracy of 98.2% using the trained classification neural network.

[0111] Figure 7 is an illustration of an example architecture of a classification neural network configured to process a 3D QPI 702 of a microorganism to generate a classification output 704 characterizing a predicted type of the microorganism. The architecture has a series of layers, including, in order: a 3D convolutional layer 706 with a convolutional filter of dimension 7x7x7, a sequence of 6 bottleneck layers 708 (wherein the skip connections are such that the input to each bottleneck layer after the first bottleneck layer includes the sum of the output of the previous bottleneck layer and the input to the first bottleneck layer), a 3D convolutional layer 710 with a 1x1x1 convolutional filter, a 3D 2x2x2 average pooling layer 712, a sequence of 12 bottleneck layers 714, a 3D convolutional layer 716 with a 1x1x1 convolutional filter, a 3D 2x2x2 average pooling layer 718, a sequence of 32 bottleneck layers 720, a 3D convolutional layer 722 with a 1x1x1 convolutional filter, a 3D 2x2x2 average pooling layer 724, a sequence of 32 bottleneck layers 726, a 3D 7x7x7 global average pooling layer 728, a fully connected layer 730, and a soft-max output layer 732.

[0112] Certain hyperparameters of the classification neural network may be selected based on the nature of the predictions made by the classification neural network. For example, the dimensions of the filters of one or more convolutional layers of the classification neural network may be selected based on the (known) size of bacteria of the predicted bacterial species.

[0113] Figure 8 is a flow chart of an example process 800 for identifying a predicted type of one or more microorganisms.

[0114] Phase contrast microscopy is used to generate a 3D QPI of one or more microorganisms (802). The 3D QPI of the microorganism includes a 3D representation of the microorganism. The 3D QPI can be, for example, a 3D refractive index tomogram. The microorganism can be, for example, a bacterium, a virus, or a fungus. In a specific example, the microorganism can be a bacterium isolated from a blood sample of a patient suffering from a bacterial infection.

[0115] The 3DQPI is processed using a classification neural network according to trained parameter values ​​of the classification neural network to generate a neural network output that characterizes the microorganism (804). The classification neural network can be a convolutional neural network including one or more 3D convolutional neural network layers. In one example, the neural network output defines a probability distribution of a predetermined number of possible microorganism types. In another example, the neural network output defines an index from a predetermined set of possible indexes, where each index corresponds to a corresponding possible microorganism type.

[0116] The predicted type of the microorganism is identified using the neural network output (806). In one example, the predicted type of the microorganism can be identified by the neural network output as the type of microorganism associated with the highest probability value. In another example, the predicted type of the microorganism can be identified as the type of microorganism corresponding to the index defined by the neural network output.

[0117] Fig. 9 is used in reference Figures 6A-6G Illustration of an exemplary apparatus (Mach-Zehnder phase contrast microscope apparatus) 900 for generating 3D RI tomograms of bacteria in the described experiments. A set of multi-directional 2D QPI measurements are combined in the spatial frequency domain to generate a 3D RI tomogram using an optical diffraction tomography (ODT) technique. Each 2D QPI measurement is obtained using off-axis holography. A digital micromirror device (DMD) is used to control the angle of incidence of light illuminating a sample (e.g., a bacterial sample), i.e., using a +1 order beam diffracted by a grating pattern on the DMD. Using a continuous wave laser with a wavelength of 532 nm and two water immersion objectives with a numerical aperture of 1.2, the optical resolution is 110 nm in the horizontal axis and 330 nm in the vertical axis according to the Nyquist theorem. The 3D RI tomogram is resampled with voxels of 100 nm width and 200 nm height. In the illustration of device 6900, "BC" refers to beam collimator, "BS" refers to beam splitter, "CL" refers to condenser, "FC" refers to fiber coupler, "LP" refers to linear polarizer, "OL" refers to objective lens, and "TL" refers to tube lens.

[0118] In general, any suitable device may be used to generate a 3D RI tomogram. Reference: K. Kim, H. Yoon, M. Diez-Silva, M. Dao, R.R. Dasari, and Y. Park: "High-resolution three-dimensional imaging of red blood cells parasitized by Plasmodium falciparum and in situ hemozoin crystals using optical diffraction tomography", Journal of Biomedical Optics 19(1), 011005 (June 25, 2013), describes another example of a device that can be used to generate a 3D RI tomogram, which is incorporated herein by reference for the purposes of the device disclosed in the article. Reference: F. Charriere, A. Marian, F. Montfort, J. Kuehn, T. Colomb, E. Cuche, P. Marquet and C. Depeursinge: "Cell refractive index tomography by digital holographic microscopy," Optics Express, 31(2), pp. 178-180 (2006), describes another example of an apparatus that can be used to generate 3DRI tomograms, which article is incorporated herein by reference for the purposes of the apparatus disclosed in the article.

[0119] This specification uses the term "configured" in connection with system and computer program components. For a system of one or more computers to be configured to perform a particular operation or action means that the system has installed thereon software, firmware, hardware, or a combination thereof that, when operated, causes the system to perform the operation or action. For one or more computer programs to be configured to perform a particular operation or action means that the one or more programs include instructions that, when executed by a data processing device, cause the device to perform the operation or action.

[0120] The embodiments of the subject matter and functional operations described in this specification can be implemented in digital electronic circuits, in tangibly embodied computer software or firmware, in computer hardware including the structures disclosed in this specification and their structural equivalents, or in a combination of one or more of them. The embodiments of the subject matter described in this specification can be implemented as one or more computer programs, that is, one or more modules of computer program instructions encoded on a tangible non-temporary storage medium for execution by a data processing device or control of the operation of a data processing device. The computer storage medium can be a machine-readable storage device, a machine-readable storage substrate, a random or serial access memory device, or a combination of one or more of them. Alternatively or additionally, the program instructions can be encoded on an artificially generated propagation signal (e.g., a machine-generated electrical, optical or electromagnetic signal), which is generated to encode information for transmission to a suitable receiver device for execution by a data processing device.

[0121] The term "data processing apparatus" refers to data processing hardware and encompasses various apparatuses, devices, and machines for processing data, including, for example, a programmable processor, a computer, or multiple processors or computers. The apparatus may also be or include special-purpose logic circuits, such as an FPGA (field programmable gate array) or an ASIC (application-specific integrated circuit). In addition to hardware, the apparatus may optionally include code that creates an execution environment for a computer program, such as code constituting processor firmware, a protocol stack, a database management system, an operating system, or a combination of one or more of them.

[0122] A computer program, which may also be referred to or described as a program, software, software application, application, module, software module, script, or code, may be written in any form of programming language, including compiled or interpreted languages, or declarative or procedural languages; and it may be deployed in any form, including as a stand-alone program or as a module, component, subroutine, or other unit suitable for use in a computing environment. A program may, but need not, correspond to a file in a file system. A program may be stored as part of a file holding other programs or data (e.g., one or more scripts stored in a markup language document), in a single file dedicated to the associated program, or in multiple coordinated files (e.g., files storing one or more modules, subroutines, or portions of code). A computer program may be deployed for execution on a single computer or on multiple computers located at one site or distributed across multiple sites and interconnected by a data communications network.

[0123] In this specification, the term "engine" is used broadly to refer to a software-based system, subsystem, or process that is programmed to perform one or more specific functions. Typically, an engine will be implemented as one or more software modules or components, installed on one or more computers in one or more locations. In some cases, one or more computers will be dedicated to a specific engine; in other cases, multiple engines can be installed and run on the same one or more computers.

[0124] The processes and logic flows described in this specification may be performed by one or more programmable computers executing one or more computer programs to perform functions by operating on input data and generating output. The processes and logic flows may also be performed by a special purpose logic circuit (e.g., an FPGA or ASIC) or by a combination of a special purpose logic circuit and one or more programmed computers.

[0125] Computers suitable for executing computer programs can be based on general or special microprocessors or both, or any other kind of central processing unit. Typically, the central processing unit will receive instructions and data from a read-only memory or a random access memory or both. The basic elements of a computer are a central processing unit for executing or performing run instructions and one or more memory devices for storing instructions and data. The central processing unit and the memory can be supplemented by or incorporated into a special logic circuit. Typically, the computer will also include one or more large-capacity storage devices for storing data, such as disks, magneto-optical disks, or optical disks, or be operationally coupled to receive data from or transmit data to it, or both. However, the computer does not need to have such a device. In addition, the computer can be embedded in another device, such as a mobile phone, a personal digital assistant (PDA), a mobile audio or video player, a game console, a global positioning system (GPS) receiver, or a portable storage device such as a universal serial bus (USB) flash drive, to name a few.

[0126] Computer-readable media suitable for storing computer program instructions and data include all forms of non-volatile memory, media, and memory devices, including, for example, semiconductor memory devices, such as EPROM, EEPROM, and flash memory devices; magnetic disks, such as internal hard disks or removable disks; magneto-optical disks; and CD ROM and DVD-ROM disks.

[0127] To provide interaction with a user, embodiments of the subject matter described in this specification may be implemented on a computer having a display device (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor) for displaying information to the user, and a keyboard and pointing device (e.g., a mouse or trackball) through which the user can provide input to the computer. Other kinds of devices may also be used to provide interaction with the user; for example, the feedback provided to the user may be any form of sensory feedback, such as visual feedback, auditory feedback, or tactile feedback; and input from the user may be received in any form, including sound, voice, and / or tactile input. In addition, the computer may interact with the user by sending documents to and receiving documents from a device used by the user; for example, by sending a web page to a web browser on a user's device in response to a request received from the web browser. In addition, the computer may interact with the user by sending text messages or other forms of messages to a personal device (e.g., a smartphone running a messaging application) and receiving response messages from the user in return.

[0128] The data processing apparatus for implementing a machine learning model may also include, for example, dedicated hardware accelerator units for processing the general and computationally intensive parts of machine learning training or production, i.e., inference, workloads.

[0129] The machine learning model can be implemented and deployed using a machine learning framework (e.g., the TensorFlow framework, the Microsoft Cognitive Toolkit framework, the Apache Singa framework, or the Apache MXNet framework).

[0130] Embodiments of the subject matter described in this specification may be implemented in a computing system that includes a back-end component (e.g., as a data server), or includes a middleware component (e.g., an application server), or includes a front-end component (e.g., a client computer with a graphical user interface, a web browser, or an application through which a user can interact with an implementation of the subject matter described in this specification), or any combination of one or more such back-end, middleware, or front-end components. The components of the system may be interconnected by any form or medium of digital data communication (e.g., a communication network). Examples of communication networks include local area networks (LANs) and wide area networks (WANs), such as the Internet.

[0131] A computing system may include a client and a server. The client and the server are usually remote from each other and usually interact through a communication network. The relationship between the client and the server is due to the computer programs running on the corresponding computers, and there is a client-server relationship between each other. In some embodiments, the server sends data (e.g., an HTML page) to a user device, for example, to display data to a user interacting with the device as a client and to receive user input from it. Data generated at the user device, for example, the result of the user interaction, can be received from the device at the server.

[0132] Although this specification contains many specific implementation details, these should not be interpreted as limitations on the scope of any invention or the scope that may be claimed, but rather as descriptions of features that may be specific to a particular embodiment of a particular invention. Certain features described in this specification in the context of independent embodiments may also be implemented in combination in a single embodiment. Conversely, the various features described in the context of a single embodiment may also be implemented independently or in any suitable sub-combination in multiple embodiments. In addition, although the above features may be described as working in certain combinations, and even initially claimed to be so, in some cases, one or more features from the claimed combination may be deleted from the combination, and the claimed combination may be directed to a variant of a sub-combination or a sub-combination.

[0133] Similarly, although operations are described in the drawings and recited in the claims in a particular order, this should not be construed as requiring that the operations be performed in the particular order shown or in a sequential order, or requiring that all of the operations shown be performed in order to achieve the desired effect. In some cases, multitasking and parallel processing may be advantageous. In addition, the separation of various system modules and components in the above-described embodiments should not be construed as requiring such separation in all embodiments, and it should be understood that the described program components and systems can generally be integrated together into a single software product or packaged into multiple software products.

[0134] Although certain embodiments have been described, the present disclosure is not limited to such embodiments.

[0135] For example, although certain bacteria are disclosed above, in some embodiments, the bacteria may be related to sexually transmitted infections, such as, for example, gonorrhea, syphilis or chlamydia. In such embodiments, the microbial classification system can identify bacterial species as, for example, Treponema pallidum, Mycoplasma genitalium, Neisseria gonorrhoeae, Ureaplasma parvum, Mycoplasma hominis, Ureaplasma urealyticum, Gardnerella vaginalis, Chlamydia trachomatis, Trichomonas vaginalis, Candida albicans or Haemophilus ducreyi. Therefore, the present disclosure provides a relatively rapid diagnosis of sexually transmitted infections, which can also produce a relatively rapid treatment of such infections.

[0136] Specific embodiments of the subject matter have been described. Other embodiments are within the scope of the following claims. For example, the actions recited in the claims can be performed in a different order and still achieve the desired results. As an example, the processes depicted in the accompanying drawings do not necessarily require the particular order shown or sequential order to achieve the desired results. In some cases, multitasking and parallel processing may be advantageous.

Claims

1. A method for classifying one or more microorganisms, comprising: Use phase contrast microscopy to generate three-dimensional 3D quantitative phase images of one or more microorganisms. wherein the 3D quantitative phase image comprises a corresponding 3D representation of the one or more microorganisms as a 3D refractive index tomogram represented as a 3D matrix of numerical values, wherein each component of the 3D matrix corresponds to a corresponding 3D spatial position and defines a refractive index at the corresponding 3D spatial position; and One or more 3D quantitative phase images are processed using a neural network, wherein: The neural network is a convolutional neural network comprising one or more 3D convolutional layers, wherein each 3D convolutional layer performs operations comprising: receiving a layer input comprising one or more 3D matrices of features derived from the one or more 3D quantitative phase images input to the neural network; and processing the layer input to generate a layer output, comprising convolving one or more 3D convolutional filters with the one or more 3D matrices included in the layer input; and The neural network is configured to process the one or more 3D quantitative phase images according to trained parameter values ​​of the neural network to generate a neural network output characterizing the microorganism; and using the neural network output to classify the one or more microorganisms; obtaining one or more intermediate outputs, each intermediate output generated by a corresponding intermediate layer of the neural network during processing of the one or more 3D quantitative phase images, wherein each intermediate output defines a corresponding latent feature representation of the one or more microorganisms; and The one or more intermediate outputs of the neural network are processed using a separate machine learning model that is distinct from the neural network to generate one or more predicted outputs characterizing the one or more microorganisms.

2. The method of claim 1, wherein: The neural network output includes a corresponding probability value for each of a predetermined number of microorganism types; and The probability value for a given microorganism type indicates the likelihood that the microorganism belongs to the given microorganism type.

3. The method of claim 2, wherein: Using the neural network output to classify the one or more microorganisms comprises: The predicted type of the one or more microorganisms is identified as the type of microorganism having a highest probability value in the neural network output.

4. The method of claim 1, wherein: Classifying the one or more microorganisms includes classifying one or more of the following: genus, species, strain, Gram staining, metabolic properties, morphology, or motility of the one or more microorganisms.

5. The method of claim 1, wherein: The one or more microorganisms are selected from the group consisting of bacteria, viruses, fungi, parasites and microalgae.

6. The method of claim 5, wherein: The microorganisms include bacteria.

7. The method of claim 6, wherein: The bacteria are present in patient blood samples.

8. The method of claim 1, wherein: Generating a three-dimensional quantitative phase image of the one or more microorganisms using phase contrast microscopy includes: generating a phase and amplitude image of the microorganism at each of a plurality of illumination angles using the phase contrast microscope; and A three-dimensional refractive index tomogram is reconstructed using the phase and amplitude images.

9. The method of claim 1, wherein: The method takes a maximum of one hour.

10. A method for identifying a predicted type of bacteria, the method comprising: provide biological specimens from patients with bacterial infections; as well as within one hour of obtaining the biological sample from the patient, identifying a predicted type of bacteria in the patient, comprising: generating one or more three-dimensional 3D quantitative phase images of bacteria isolated from the biological sample using a phase contrast microscope, wherein each of the 3D quantitative phase images comprises a respective 3D representation of the bacteria isolated from the biological sample as a 3D refractive index tomogram represented as a 3D matrix of numerical values, wherein each component of the 3D matrix corresponds to a respective 3D spatial position and defines a refractive index at the corresponding 3D spatial position; The one or more 3D quantitative phase images of the bacteria isolated from the biological sample are processed according to trained parameter values ​​of the neural network using a neural network including one or more 3D convolutional layers to generate a neural network output characterizing the bacteria isolated from the biological sample, wherein each 3D convolutional layer of the neural network performs operations including: receiving a layer input comprising one or more 3D matrices of features derived from the one or more 3D quantitative phase images input to the neural network; and processing the layer input to generate a layer output, comprising convolving one or more 3D convolutional filters with the one or more 3D matrices included in the layer input; and using the neural network output to identify the predicted type of the bacteria isolated from the biological sample; obtaining one or more intermediate outputs, each intermediate output generated by a corresponding intermediate layer of the neural network during processing of the one or more 3D quantitative phase images, wherein each intermediate output defines a corresponding latent feature representation of the bacteria isolated from the biological sample; and The one or more intermediate outputs of the neural network are processed using a separate machine learning model distinct from the neural network to generate one or more predicted outputs characterizing the bacteria isolated from the biological sample.

11. The method of claim 10, wherein: The biological sample includes a blood sample.

12. The method of claim 10, wherein: The predicted type of bacteria in the patient is identified within 45 minutes of providing the biological sample from the patient.

13. The method of claim 10, wherein: The predicted type of bacteria in the patient is identified within 30 minutes of providing the biological sample from the patient.

14. The method of claim 10, wherein: The predicted type of bacteria in the patient is identified within 15 minutes of providing the biological sample from the patient.

15. A method for identification, comprising: Provide biological specimens from patients suspected of having bacterial infections; as well as Identifying a predicted type of bacteria in the patient within one hour of obtaining the biological sample from the patient, comprising: generating one or more three-dimensional 3D quantitative phase images of bacteria isolated from the biological sample using a phase contrast microscope, wherein each of the 3D quantitative phase images comprises a respective 3D representation of the bacteria isolated from the biological sample as a 3D refractive index tomogram represented as a 3D matrix of numerical values, wherein each component of the 3D matrix corresponds to a respective 3D spatial position and defines a refractive index at the corresponding 3D spatial position; processing the one or more 3D quantitative phase images of the bacteria isolated from the biological sample using a neural network including one or more 3D convolutional layers according to trained parameter values ​​of the neural network to generate a neural network output characterizing the bacteria isolated from the biological sample, Each 3D convolutional layer of the neural network performs operations including: receiving a layer input comprising one or more 3D matrices of features derived from the one or more 3D quantitative phase images input to the neural network; and processing the layer input to generate a layer output, comprising convolving one or more 3D convolutional filters with the one or more 3D matrices included in the layer input; and using the neural network output to identify the predicted type of the bacteria isolated from the biological sample; obtaining one or more intermediate outputs, each intermediate output generated by a corresponding intermediate layer of the neural network during processing of the one or more 3D quantitative phase images, wherein each intermediate output defines a corresponding latent feature representation of the bacteria isolated from the biological sample; and The one or more intermediate outputs of the neural network are processed using a separate machine learning model distinct from the neural network to generate one or more predicted outputs characterizing the bacteria isolated from the biological sample.

16. A system for identification, comprising: one or more computers; as well as One or more storage devices communicatively coupled to the one or more computers, wherein the one or more storage devices store instructions that, when executed by the one or more computers, cause the one or more computers to perform the operations of the corresponding method as described in any one of claims 1-15.

17. One or more non-transitory computer-readable storage media storing instructions that, when executed by one or more computers, cause the one or more computers to perform the operations of the corresponding method of any one of claims 1-15.

Images