Identifying microorganisms using three-dimensional quantitative phase imaging

Abstract

To provide methods, systems, and apparatus for identifying the predicted type of one or more microorganisms.SOLUTION: A system comprises a phase-contrast microscope and a microorganism classification system. The phase-contrast microscope is configured to generate a three-dimensional quantitative phase image of one or more microorganisms. The microorganism classification system is configured to process the three-dimensional quantitative phase image using a neural network to generate a neural network output characterizing the microorganisms, and thereafter identify the predicted type of the microorganisms using the neural network output.SELECTED DRAWING: Figure 1

Description

[Technical field]

[0001] (CROSS REFERENCE TO RELATED APPLICATIONS) This application is a joint venture of a patent application filed on March 13, 2019 under 35 U.S.C. § 119 entitled "Express Method for Producing Biological Entities." The U.S. Provisional Patent, entitled "RAPID IDENTIFICATION OF BIOLOGICAL ENTITIES," has been issued. Application No. 62 / 817,680 and application No. 62 / 817,680 filed June 3, 2019, entitled "A Novel Method for Detecting and Treating Microorganisms Using 3D Quantitative Phase Imaging," IDENTIFYING MICROORGANISMS USING THREE-DIMENSIONAL QUANTITATIVE This application claims priority from U.S. Provisional Patent Application No. 62 / 856,290, entitled "PATENT PHASE IMAGING" The entire contents of these two provisional patent applications are incorporated herein by reference. (Field) This specification relates to the identification of microorganisms using three-dimensional (3D) quantitative phase imaging. This specification relates to the "Rapid Identification of Biological Entities" filed on March 13, 2019. U.S. Provisional Patent Application No. 62 / 817,680, entitled "Chemical Identification of Biological Entities," , the entire contents of which are incorporated by reference. [Background technology]

[0002] (background) The term microorganism may refer to, for example, a bacterium, a virus, or a fungus. Types of microorganisms are may refer to, for example, the genus, species, strain, metabolism, morphology, motility, or other suitable characteristics of the microorganism. .

[0003] Quantitative phase imaging involves quantifying the phase shift induced in light passing through a specimen. Characterize the specimen by: Summary of the Invention [Means for solving the problem]

[0004] (overview) This specification describes a neural network, e.g., a 3D convolutional neural network. To identify microorganisms by processing 3D quantitative phase images (QPIs) of microorganisms using A method and apparatus are described.

[0005] According to a first aspect, a method for identifying a predicted species of one or more microorganisms is provided. The method uses phase contrast microscopy to quantitatively determine the three-dimensional location of one or more microorganisms. generating a three-dimensional quantitative phase image, the three-dimensional quantitative phase image including a three-dimensional representation of the microorganism. The three-dimensional quantitative phase image is generated using a convolutional neural network with one or more three-dimensional convolution layers. The neural network is processed using the neural network. Process the 3D quantitative phase images according to the learned parameter values ​​to characterize the microorganisms. The neural network is configured to generate a neural network output that predicts the predicted The species is identified using the neural network output.

[0006] In some implementations, the neural network output is a set of microbial species for each of a predetermined number of microbial species. The probability value for a given type of microorganism is the probability that the microorganism It indicates the likelihood of being a given type of microorganism.

[0007] In some implementations, the predicted type of microorganism is determined by the neural network output. The species is identified as the type of microorganism having the highest probability value.

[0008] In some implementations, the neural network output is the type of microorganism that the microorganism is. The probability value for that type of microorganism indicates the likelihood that the microorganism is present.

[0009] In some implementations, the predicted type of microorganism is determined by genus, species, strain, Gram staining, metabolism, morphology, and motility.

[0010] In some implementations, the three-dimensional quantitative phase image of the microorganism is a three-dimensional refractive index tomogram.

[0011] In some implementations, the microorganisms include bacteria, viruses, fungi, parasites, and microalgae. The compound is selected from the group consisting of:

[0012] In some implementations, the microorganisms include bacteria present in a patient's blood sample.

[0013] In some implementations, the method includes administering antibiotic therapy to a patient based on the predicted type of bacteria. This includes enforcing the law.

[0014] In some implementations, phase contrast microscopy, quantitative phase microscopy, or digital holography Using holographic microscopy, they generated phase and amplitude images of the microorganism at each of several illumination angles. ,The 3D refractive index tomogram is reconstructed using the phase and amplitude images.

[0015] In some implementations, the method can take up to an hour.

[0016] According to the second embodiment, according to the learned parameter values ​​of the microorganisms of the neural network, and generating a three-dimensional quantitative phase image of one or more microorganisms using the neural network. The method includes the steps of: identifying a predicted species of one or more microorganisms by processing the microorganism; Provided.

[0017] In some implementations, the neural network includes one or more 3D convolutional layers. nothing.

[0018] In some implementations, the neural network comprises a neural network that characterizes the microorganism. and configured to generate a network output, the predicted type of microorganism being at least partially is identified based on the neural network output.

[0019] In some implementations, the method can take up to an hour.

[0020] According to a third aspect, a method for identifying a predicted species of one or more microorganisms is provided. The method includes providing a biological sample from a patient infected with the bacteria. The predicted types of bacteria in the patient are generated using a neural network with one or more 3D convolutional layers. According to the learned parameter values ​​of the neural network, the neural network is used to generate a biological The three-dimensional quantitative phase image of one or more bacteria isolated from the biological sample is processed to identify the bacteria. and generating neural network outputs that characterize the biological samples from the patient. The bacteria are identified within an hour of obtaining the sample. The predicted species of bacteria are then analyzed using a neural network. The signal is identified using the network output.

[0021] In some implementations, the biological sample includes a blood sample.

[0022] In some implementations, the predicted bacterial species of a patient are determined using a biological sample from the patient. The detection is performed within 45 minutes of the initial detection.

[0023] In some implementations, the predicted bacterial species of a patient are determined using a biological sample from the patient. The detection is performed within 30 minutes of the detection.

[0024] In some implementations, the predicted bacterial species of a patient are determined using a biological sample from the patient. The sample is identified within 15 minutes of being recorded.

[0025] According to a fourth aspect, a method for identifying a predicted species of one or more microorganisms is provided. The method includes providing a biological sample from a patient suspected of having a bacterial infection. The predicted type of bacteria in the patient is generated using a neural network with one or more 3D convolutional layers. A neural network is used to generate a trained parameter value of the network. The three-dimensional quantitative phase images of one or more bacteria isolated from the biological sample are processed to identify the bacteria. Characterizing biological samples from patients by generating neural network outputs The predicted species of bacteria are identified within an hour of preparation. The signal is identified using the signal output.

[0026] According to a fifth aspect, there is provided a system comprising a phase contrast microscope and a microbial classification system. The phase contrast microscope is configured to generate three-dimensional quantitative phase images of one or more microorganisms. and the three-dimensional quantitative phase image comprises a three-dimensional representation of the microorganism. The method is configured to process three-dimensional quantitative phase images using a neural network, The neural network is a convolutional neural network with one or more 3D convolution layers. The neural network is a trained neural network. A neural network that processes 3D quantitative phase images according to parameter values ​​to characterize microorganisms. The network is configured to generate a network output. The predicted type of microorganism is The model is identified using the neural network output.

[0027] According to a sixth aspect, a method for implementing a method for a computer-based system comprising: 3D quantification of one or more microorganisms generated using a phase contrast microscope, on a computer one or more non-transitory computer readable memories storing instructions for receiving a target phase image. A medium is provided, the three-dimensional quantitative phase image including a three-dimensional representation of a microorganism. The phase images are processed using a neural network, the neural network comprising: A convolutional neural network has one or more three-dimensional convolutional layers. The neural network is trained according to the learned parameter values ​​of the neural network. The raw quantitative phase images are processed to generate neural network outputs that characterize the microorganisms. The predicted species of microorganisms are calculated using the neural network output. It is identified by

[0028] Particular implementations of the subject matter described herein realize one or more of the following advantages: It can be implemented to do so.

[0029] The methods described herein are capable of detecting microorganisms more quickly and / or consistently than some conventional methods. It may be possible to identify an object.

[0030] For example, blood culture-based screening is the traditional method for identifying bacterial species. Blood cultures may reveal the presence of bacteria, but in many cases they do not grow bacteria clearly. It takes hours to days for visible colonies to form. In addition, they typically contain a variety of antibiotics. Mass spectrometry or susceptibility testing to substances is used to determine the exact species of bacteria. , deoxyribonucleic acid (DNA) microarrays, and real-time polymerase chain reaction (P Other biochemical methods, including PCR, can identify bacterial species. Because these conventional methods rely on chemical reactions or molecular specific signals, they require time of one hour to one day. It may also include processes such as those described above, as well as the maintenance and use of specialized biochemicals.

[0031] In contrast, the techniques described herein provide accurate bacterial identification within an hour (in some cases). For example, advances in microfluidics and antibody engineering can allow for the determination of Isolation of bacteria from concentrations as low as those in the blood of infected patients was achieved within one hour. To obtain 3D QPI of bacterial samples, time-consuming processes such as culturing or staining are not required. Instead, classification neural networks can use input data to make predictions in milliseconds or less. It only takes a little time.

[0032] The techniques described herein are immune to human factors during both measurement and analysis. It allows for more consistent identification of microorganisms than some traditional methods. For example, some traditional methods rely on human-run physical experiments. However, the techniques described herein use computational techniques (i.e., classification neural networks). Use consistent measurements (i.e., generated using quantitative phase imaging) across the can be done.

[0033] By processing 3D QPIs, the techniques described herein can (in some cases) This allows for more accurate identification of microorganisms than would be possible by processing. For a given illumination angle, 2D QPI often contains multiple reflections from dust particles or Contains noisy image artifacts due to interference caused by scattering Images of microorganisms (e.g., bacteria) that are only a few hundred nanometers thick are particularly difficult to image. The signal-to-noise ratio (SNR) of a 3D QPI reconstructed from multiple 2D QPIs is The SNR is the ratio of noisy artifacts to illumination angle, whereas the influence of microorganisms is Because it is consistent, it can be higher than 2D QPI.

[0034] By using 3D convolutional neural networks, the techniques described herein: (Possibly) e.g. 2D convolutional neural networks or other machine learning models This allows for more accurate identification of microorganisms than would be possible using Next, a 3D convolutional neural network is used to extract the complex 3D spatial relationships between different parts of the 3D QPI. can be used to train the system to accurately identify microorganisms depicted in 3D QPIs. .

[0035] Certain details of one or more embodiments of the subject matter of this specification are set forth in the accompanying drawings and the description below. Other features, aspects, and advantages of the subject matter will be set forth in the description, drawings, and claims. It will become clear from this. [Brief description of the drawings]

[0036] BRIEF DESCRIPTION OF THE DRAWINGS [Figure 1] FIG. 1 is a block diagram of an example environment in which one or more sets of microbial species are predicted from 3D QPIs of the microorganisms.

[0037] [Diagram 2] FIG. 2 shows an example of a learning system.

[0038] [Diagram 3] FIG. 3 is a diagram of an example set of training data.

[0039] [Figure 4] FIG. 4 is a diagram of an example process for determining appropriate antibiotic therapy to treat a patient with a bacterial infection using phase contrast microscopy and a microbial classification system.

[0040] [Diagram 5] Figure 5 is a diagram of an example of a process for generating 3D QPIs of microorganisms using phase contrast microscopy.

[0041] [Figure 6] 6A-6G illustrate the results of an experiment in which a classification neural network was trained to predict 19 sepsis-inducing bacterial species from 3D refractive index tomograms of the bacteria.

[0042] [Figure 7] FIG. 7 is a diagram of an example of a classification neural network architecture configured to process 3D QPIs of a microorganism and generate a classification output that characterizes the predicted type of the microorganism.

[0043] [Figure 8] FIG. 8 is a flow diagram of an example process for identifying the predicted species of one or more microorganisms.

[0044] [Figure 9] Figure 9 is a diagram of the Mach-Zehnder phase contrast microscopy setup used to generate the 3D refractive index tomograms.

[0045] Like reference numbers and designations in the various drawings indicate like elements. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0046] (Detailed Description) The present specification describes a method for classifying microorganisms (e.g., bacteria) using a 3D convolutional classification neural network. By processing the three-dimensional (3D) quantitative phase images (QPIs), the type of the microorganism (e.g. The present invention describes a microbial classification system that can predict the number of bacteria (species) that cause bacterial infection. It can be deployed in a clinical environment for diagnostic purposes. These and other features are described in , as described in more detail below.

[0047] FIG. 1 illustrates a method for predicting the types 102 of one or more sets of microorganisms 104 from a 3D QPI 106 of the microorganisms 104. FIG. 1 is a block diagram of an example 100 of a monitored environment.

[0048] The microorganisms 104 may be bacteria, viruses, fungi, parasites, microalgae, or any other suitable type. The microorganism 104 may be capable of causing disease (e.g., capable of inducing sepsis). ), but do not necessarily cause disease.

[0049] The microorganism 104 can be derived from any of a variety of sources. For example, the microorganism 104 can be any of the microorganisms shown in FIG. As described in more detail below, bacterial (or fungal) isolation techniques may be used to isolate bacteria (or fungi). ) may be a bacterial (or fungal) obtained by processing a blood sample from a patient with an infectious disease. .

[0050] The predicted type 102 of the microorganism 104 may be, for example, a predicted species (e.g., Acinobacter Acinobacter baumannii, Bacillus subtilis, or Enterobacter cloacae), predicted genus, predicted The strain to be tested, the predicted Gram staining property (e.g., Gram-positive or Gram-negative), the predicted substitute microbial activity (e.g., aerobic or anaerobic), expected morphology (e.g., cocci, bacilli, or coccobacilli) , predicted motility (e.g., motile or non-motile), or other suitable microbial characteristics. can be defined.

[0051] Optionally, the predicted type 102 of a microorganism 104 includes predictions for a plurality of characteristics of the microorganism 104. For example, a predicted type 102 of a microorganism 104 can be defined as a predicted Both the species to be detected and the predicted motility of the microorganism can be defined.

[0052] The environment 100 includes a phase contrast microscope 108 and a microbial classification system 110. 10, a computer program on one or more computers at one or more locations It can be implemented as:

[0053] A phase contrast microscope 108 is used to generate a 3D quantitative phase image 106 (i.e., 3D QPI 106) of the microorganism 104. In general, the 3D QPI 106 detects the phase shift induced by light passing through the microorganism 104. The microorganisms are characterized by their morphological and biophysical properties by quantifying This is a 3D representation of 104.

[0054] Generally, the 3D QPI 106 of a microorganism 104 is represented as a 3D matrix of numerical values ​​(e.g., floating point values). Each element of the 3D matrix may represent a respective 3D space within or around the microorganism 104. In one example, the 3D QPI 106 can be a "refractive index (RI) tomogram," where ,Each element of the 3D matrix represents the spatial location of a substance (e.g., a part of a microorganism) at the corresponding component. Define the refractive index.

[0055] The phase contrast microscope 108 may perform 3D QPI 106 of the microorganism 104 using any of a variety of different techniques. A phase contrast microscope 108 can be used to generate a 3D QPI 106 of a microorganism 104. An example method for this is described in more detail with reference to FIG.

[0056] The microbial classification system 110 comprises a classification neural network 112, a model parameter 114, and a According to the learned values, the classification neural network 112 is used to calculate the 3D QPI 106 of the microorganism 104. to generate a classification output 116. The system 10 uses the classification output 116 to identify the predicted species 102 of the microorganism 104. Shown below.

[0057] In one example, the classification output 116 defines a probability distribution for a given number of possible microbial types. That is, the classification output 116 is a set of classifications corresponding to each possible type of microorganism. In this example, the system 110 estimates the predicted species 102 of the microorganism 104. may be identified by the classification output 116 as the type of microorganism with the highest probability value.

[0058] In another example, the classification output 116 may be a quotient from a predetermined set of possible quotients (e.g., We define a set of positive integer values, each of which corresponds to a different possible type of microorganism. In this example, the system 110 defines a predicted type 102 of the microorganism 104 by a classification output 116. The type of microorganism corresponding to the index thus determined can be identified.

[0059] In some cases, the 3D QPIs 106 generated by the phase contrast microscope 108 may be multiple (e.g., For example, a phase contrast microscope 108 may show microorganisms (thousands) mounted on a microscope slide. The system 110 can generate 3D QPIs 106 that represent thousands of microorganisms (e.g., bacteria). is an object detection neural network trained to detect microorganisms (e.g., 106) to detect individual microorganisms in a 3D QPI system, each of which is one or more Multiple regions can be cut out from the 3D QPI 106 showing different microorganisms. For each region taken, the system 110 uses a classification neural network 112 to identify the corresponding The areas are processed to characterize one or more microorganisms represented in the excised areas. The system 110 can then generate a classification output 116 for each of the cropped regions. Using the classification output generated for each of the 106 original 3D QPIs, It is possible to generate "overall" predictions for species. A few examples are given below:

[0060] In one example, a classification neural network 11 is run on each region cropped from the 3D QPI 106. Each classification output 116 produced by 2 is a set of possible microbial In this example, the system 110 extracts a probability distribution for each type of object from the 3D QPI 106. The probability distribution for the types of microorganisms that could have been generated for each of the regions was calculated. Combined (e.g., on average), they provide an "overall" probability of success for a given type of microorganism. The system 110 can generate a rate distribution of all of the microorganisms shown on the 3D QPI 106. The predicted species of the organism are calculated based on the overall probability distribution for the possible species of microorganisms. , the type of microorganism with the highest probability value can be identified.

[0061] In another example, a classification neural network was run on each region cut from the 3D QPI 106. Each classification output 116 produced by 112 is a classification of possible microorganisms (as described above). In this example, the system 110 defines an index associated with the type of cut out image from the 3D QPI 106. The set of indices generated by the classification neural network 112 for the selected region is It is possible to determine the most frequently occurring index (i.e., the "modal index") in the system. System 110 was identified in the 3D QPI 106 as a type of microorganism that may be related to the mode index. The overall expected variety of microorganisms present can be identified.

[0062] The microscopic images shown on the 3D QPI are obtained by processing multiple regions cut from the 3D QPI. Improve the robustness of the system 110 by generating a global prediction for the species. It can be improved.

[0063] In some cases, rather than using a single classification neural network 112, the system The neural network 110 uses an ensemble of multiple classification neural networks 112 to generate predicted microarrays. The species of the organism 102 can be determined. The classes can differ from each other in any of a variety of ways. For example, the classification skills in the ensemble Some or all of the neural networks may be based on different neural network architectures. For example, the different neural networks may have different 3D convolutional neural network architectures. For example, some or all of the classification neural networks in the ensemble may be For example, we can train the model in different ways by initializing the model parameter values ​​in different ways. Each classification neural network in the ensemble processes 106 3D QPIs and The system 110 can generate classification outputs 116 for each of the classes (as described above). Using the class output to generate an overall prediction of the types of microorganisms shown on the 3D QPI 106. can be done.

[0064] Optionally, by processing the 3D QPI 106 by a classification neural network 112 Providing one or more of the intermediate outputs generated for use in characterizing the microorganism 104. The intermediate output (also called latent features) can be fed to a classification neural network 11. This refers to the output generated by the intermediate layer (i.e., the layer between the input layer and the output layer) of the A force can be represented as an ordered collection of numbers, e.g., a vector or matrix of numbers. The intermediate output generated by the classification neural network 112 can be used for other machine learning A model (e.g., Support Vector Machine (SVM) or Random Forest (RF)) The 3D QPI 106 components can be processed to generate predictive outputs that characterize the microorganism 104. FIG. 6B shows a classification neural network 112. FIG. 1 is a diagram of some example intermediate outputs (i.e., latent features) generated by

[0065] The model parameters 114 of the classification neural network 112 are parameters that define the operations performed by the layers of the neural network, e.g. These refer to the weight matrices and bias vectors of the layers of network 112.

[0066] The classification neural network 112 may be implemented using any suitable neural network architecture. For example, the classification neural network 112 may include a 3D convolutional neural network. The neural network may be a neural network that includes one or more 3D convolutional layers. A 3D convolutional layer uses one or more 3D convolution filters to process one or more 3D inputs ( For example, a 3D matrix of numbers may be processed to produce one or more 3D outputs. This refers to a convolutional layer configured in

[0067] In addition to the 3D convolutional layers, the classification neural network 112 may be connected in any suitable configuration. Any suitable type of additional neural network layer (e.g., pooling or completion) may be added to the An example of a classification neural network architecture is shown in Figure 1. This is explained in more detail with reference to 7.

[0068] The training system trains a classification neural network 112 with a set of training examples. Determine the learned values ​​of the model parameters 114 of the classification neural network 112 by An example of a learning system is described in more detail with reference to FIG.

[0069] The phase contrast microscope 108 and microbial classification system 110 can be deployed in a clinical environment for diagnostic applications. For example, as described with reference to FIG. The system 110 can be used to determine appropriate antibiotic therapy for a patient with a bacterial infection. .

[0070] FIG. 2 illustrates an example learning system 200. The learning system 200 may include any of the following systems: On one or more computers at one or more locations where the components and techniques are implemented 1 is an example of a system implemented as a computer program.

[0071] The learning system 200 uses a set of training data 202 and a learning engine 204 to generate a classification The trained values ​​of the model parameters 114 of the neural network 112 are determined.

[0072] The training data 202 includes a plurality of training examples. Each training example includes: (i) one or more microorganisms ( (ii) a “target” 3D QPI that defines the type of microorganism represented in the 3D QPI; The target microbial species are classified by 3D QPI processing. The classification output that should be generated by the neural network 112 is specified in the above formula.

[0073] In general, the training data 202 is a set of samples of each possible microbial species (i.e., (from a given set of possible microbial species that can be predicted by network 112) 3 shows a diagram of an example set of training data 202. .

[0074] The learning engine 204 trains a classification neural network in each of a number of learning iterations. The system is configured to iteratively adjust current values ​​of model parameters 114 of the class 112 .

[0075] In each training iteration, the training engine 204 selects one or more training examples 206 from the training data 202. , select (e.g., randomly sample) a batch (i.e., set) of , The 3D QPIs contained in are processed to generate corresponding predicted microbial species 208.

[0076] The learning engine 204 then: (i) extracts the vectors generated by the classification neural network 112; (ii) the predicted types of microorganisms to be tested 208; and (ii) the target microorganisms defined by the training examples 206. Determine parameter value updates 210 for the current values ​​of the model parameters 114 based on the type of More specifically, to determine the parameter value updates 210 in the current learning iteration, The learning engine 204 performs objective training on the model parameters 114 of the classification neural network 112. Determine the gradient of the function. In general, the objective function is calculated by the classification neural network 112 as The predicted microbial species 208 generated and the target microbial species defined by the training examples 206 The learning engine 204 may, for example, use any suitable gradient descent optimization algorithm. The parameters are adjusted based on the gradient of the objective function according to an update rule of the algorithm (e.g., Adam or RMSprop). It may then be possible to determine whether to update 210 the data values.

[0077] The parameter values ​​can be updated 210 in any suitable numerical format, for example, in a classification neural network. A numerical ordering including respective numerical values ​​corresponding to each model parameter 114 of the workpiece 112. The learning system 200 can be represented as a collection of data (e.g., a vector of numbers). The result of applying the parameter value updates 210 to the current values ​​of the model parameters 114 during the training iteration is Updated values ​​for the model parameters 114 may be determined.

[0078] The learning engine 204 may calculate the gradient of the objective function in any suitable manner, for example using backpropagation techniques. The objective function can be any suitable classification objective function, for example, a cross-sectional distribution. It can be the entropy objective function.

[0079] The training system 200 stops training the classification neural network 112 when a training termination criterion is met. For example, a training termination criterion may be that a certain number of training iterations are performed. As another example, the training termination criterion may be that the training results are evaluated on a set of provided training examples. The accuracy of the evaluated classification neural network 112 achieves at least a predetermined threshold. It could be.

[0080] After determining that the training termination criteria have been met, the training system 200 performs a classification neural network. The learned values ​​of the model parameters 114 of the network 112 are provided to the microbial classification system 110. (i.e., as described with reference to FIG. 1). The "trained values" are the model parameters at the end of the last training iteration performed by the training system 200. This refers to the value of data 114.

[0081] The learning system 200 can be located remotely from the microbial classification system 110, in which case The training system 200 may, for example, be connected via a wired or wireless connection to train the model parameters 114. The learned values ​​can be provided to a microbial classification system.

[0082] FIG. 3 is a diagram 300 of an example set of training data 202. Each training example of training data 202 is (i) 3D QPI of sepsis-inducing bacteria, and (ii) target microbial types to define bacterial species. include.

[0083] Figure 300 shows examples of 3D QPIs of bacteria corresponding to each of 19 different sepsis-inducing species. In,Figure 3, the 3D QPIs are ordered according to image labels,(i) to (xix), respectively.,A. A. baumannii, B. subtilis, E. cloacae cae, E. faecalis, E. coli, and H. influenzae. influenza, K. pneumoniae, L. monocytogenes, M. lutea M. luteus, P. mirabilis, P. aeruginosa, S. S. marcescens, S. aureus, S. epidermidis pidermidis, S. maltophilia, S. agalactiae ctiae, S. anginosus, S. pneumonia, and Streptococcus pyogenes. An example of S. pyogenes is shown below.

[0084] Optionally, training data 202 may be used to generate classification metrics (e.g., as described with reference to FIG. 2). Before training the neural network 112, the training data 202 is amplified using an augmentation engine 302. More specifically, the augmentation engine 302 can “augment” the training data 202 by The existing training examples are processed and later used to train the classification neural network 112. It is possible to generate "new" training examples that can be used.

[0085] To generate new training examples from existing training examples, the augmentation engine 302 may, for example, Add dumb noise values, rotate, elastically transform, or use existing training examples Correcting the 3D QPI image from existing training examples by converting the 3D QPI image from The new training examples include (i) modified 3D QPI images, and (ii) a combination of existing training examples. The same target microbial species will be included.

[0086] The training data 202 was augmented with new training examples generated using the augmentation engine 302. Training the classification neural network 112 can, for example, reduce the potential effects of overfitting. By doing so, the accuracy of the trained classification neural network can be improved.

[0087] FIG. 4 illustrates a phase contrast microscope 108 and a microbial classification system (e.g., as described with reference to FIG. 1). The system 110 is used to determine an appropriate antibiotic regimen 402 for treating a patient 404 with a bacterial infection. FIG. 4 illustrates an example process 400 for determining

[0088] A blood sample 406 of a patient 404 is prepared, for example, by drawing blood from the patient 404. For convenience, the description of FIG. 4 refers to a "blood sample," but other biological samples, e.g., tissue samples, may also be used. Biopsy samples, e.g., stomach biopsies (e.g., from stomach tissue), Pap smear samples (e.g., from fetal tissue), A sample from the cervix, urine sample, stool sample, or saliva sample can be used.

[0089] Bacterial isolation techniques are applied to blood samples 406 to isolate bacteria 408 that contribute to a bacterial infection in a patient 404. That is, a bacterial isolation technique is applied to remove the sample of bacteria 408 from the blood sample 406. Bacteria 408 can be isolated to any suitable level of purity using any suitable bacterial isolation technique. Examples of bacterial separation techniques include centrifugation, filtration through fine-pore filters, and Separations using microfluidic or microchip devices are included.

[0090] A phase contrast microscope 108 is used to generate a 3D QPI 412 of the bacteria 408. For example, a phase contrast microscope , a 3D refractive index tomogram of the bacteria 408 can be generated (as described above).

[0091] The microbial classification system 110 processes the 3D QPIs 412 of the bacteria 408 to identify bacterial infections in the patient 404. The method generates an output that defines a predicted species 410 of bacteria 408 that contribute to the infection. The bacteria 408 may be, for example, , sepsis-induced, malaria-induced, or tuberculosis-induced.

[0092] The predicted species 410 of bacteria 408 is used to identify appropriate antibiotics to treat the bacterial infection in patient 404. Select a therapeutic regimen 402 (or other treatment) that specifically inhibits or destroys the predicted bacterial species 410. A particularly effective antibiotic therapy can be selected to treat the infection.

[0093] A selected antibiotic therapy 402 is then used, e.g., to administer the selected antibiotic to the patient 404. The patient 404 may be treated for bacterial infection by intravenously administering the

[0094] Process 400 can allow for rapid identification of bacterial species contributing to a patient's infection. For example, advances in microfluidics and antibody engineering have enabled the detection of microbial infections in the blood of patients infected with bacteria. Isolation of bacteria was achieved within an hour from concentrations as low as those present. D QPIs do not require time-consuming processes such as culture or staining, and are a simple, easy-to-follow microbial taxonomic system. The system (specifically, a trained classification neural network) uses the input data to generate Thus, in some cases, the process 400 may predict This can allow for accurate bacterial identification. In some particular cases, process 400 can allow accurate bacterial identification in less than 45 minutes, less than 30 minutes, or less than 15 minutes. After the bacteria 408 are isolated from the blood sample 406 (this is the most time-consuming step of the process 400), (which may be about 10 minutes or less), generate 3D QPIs 412 and predict bacterial species 410 in less than 10 minutes, less than 5 minutes, less than 3 minutes, It can be done in 1 minute or less, for example, 1 to 45 seconds (for example, 1 to 30 seconds, 1 to 20 seconds, 1 to 10 seconds). can.

[0095] In contrast, identification of typical bacterial species in hospitals (i.e., 3D quantitative phase imaging and microbiome analysis) Identification without the benefit of a classification system 110) involves blood culture and subsequent assays. Therefore, 12 hours or more may be required.

[0096] By rapidly identifying the bacterial species causing or contributing to an infection in a patient, Promptly administering appropriate antibiotics (i.e., antibiotics that effectively inhibit or destroy the bacterial species) to patients This allows for the rapid administration of appropriate antibiotics to patients with bacterial infections. Administration can dramatically improve the clinical outcome for such patients.

[0097] For example, in some cases, saving the life of a septic patient depends on the first few hours of hospitalization. Mortality in patients with sepsis increases sharply if initial antibiotic therapy is delayed. Therefore, the 3D phase contrast imaging and microbial classification system 110 can be used to Prompt antibiotic therapy plays a vital role in reducing mortality in patients with sepsis. Possible.

[0098] The process 400 can use a phase contrast microscope 108 and a microbial classification system 110. This is just one example of a setup. A few other examples are given below. In one example, a phase contrast microscope 108 and a The classification system 110 may be used, for example, to identify cells currently contributing to an infection at each of a number of time points. By predicting the species of bacteria, hospitalized patients with infectious diseases can be serially monitored. In this example, a phase contrast microscope 108 and classification system 110 are used to classify fish in an agricultural environment (e.g., a fish farm). In another example, phase contrast microscopy can be used to identify the predicted species of bacteria present in the sample. and using the classification system 110 to predict the species of bacteria present in the food, drug, or both. can be identified.

[0099] Figure 5 is an illustration of an example of a process for generating 3D QPIs of microorganisms using phase contrast microscopy. Using a phase contrast microscope, a phase image (e.g., phase image 502) of each of the microorganisms and Each amplitude image (e.g., amplitude image 504) is captured at multiple illumination angles (e.g., illumination angle θ ~θ 70 ) is measured. A 3D QPI (e.g., a 3D refractive index tomogram 506) is then obtained by Optical Doppler Tomography (ODT) technology (refractive index tomography technology and tomography phase microscope technology, etc.) (which may also be referred to as a "multi-angle image") is used to perform reconstruction based on a set of multiple angles of the phase image and the amplitude image. It will be restored.

[0100] Figure 6A-6G show how the classification neural network identifies sepsis-inducing bacteria from 3D QPI of bacteria. The results of an experiment in which the virus was trained to predict 19 different species are shown. The bacteria are: A. baumannii, B. subtilis, E. cloacae E. cloacae, E. faecalis, E. coli, influenza H. influenza, K. pneumonia, L. monocytogenes ), M. luteus, P. mirabilis, P. aeruginosa sa), S. marcescens, S. aureus, S. epidermidis, S. maltophilia, S. agalactiae S. agalactiae, S. anginosus, S. pneumonia , and S. pyogenes. In this experiment, the 3D QPI was 7.2 × 7.2 × 4.0 μm 3 10A and 10B are refractive index tomograms corresponding to rectangular parallelepiped regions of the respective There are one or more specimens of a given species. The size of a pixel is 1 in the x, y, and z directions, respectively. The values ​​were 0.1, 0.1, and 0.2 μm. The classification neural network used in this experiment The architecture is described in more detail with reference to Figure 7. The experiment was carried out using the sepsis-inducing bacteria 5041 3D RI tomograms were used: for all species, the same number of tomograms were collected in the test and validation sets. The subjects were randomly assigned to one of the groups.

[0101] The trained classification neural network was 94.6% accurate in a blinded test of species classification. The mean sensitivity, specificity, and accuracy of the 100-fold increase were 94.6%, 99.7%, and 94.7%, respectively. The specificity and accuracy were never below 85% for any species. Misclassification of Streptococcus pneumoniae as K. pneumoniae and S. pneumoniae as S. Errors related to misclassification as Escherichia coli (E. coli) and S. pneumoniae (S. pneumoniae) were neumoniae) occurred in 7.5% of cases. L. monocytogenes, P. aeruginosa, and S. anginosus Four species, including P. inosus, were identified with 100% sensitivity. Identification of P. mirabilis The sensitivity was 85%. P. mirabilis, P. aeruginosa, and The specificity of identification of S. maltophilia was 100%. The specificity for P. mirabilis and K. pneumonia was 99.17%. ilis, P. aeruginosa, and S. maltophilia, respectively. The identification accuracy of Escherichia coli was 85.7%.

[0102] FIG. 6A is a diagram of a table 600 showing the accuracy of a trained classification neural network. The shaded portion of table 600 corresponding to the row and j-th column indicates that the classification neural network represents the frequency with which we predict that the corresponding species of bacteria is the species corresponding to row j.

[0103] Figure 6B shows the results of a classification neural network trained on 3D refractive index tomograms of different bacterial species. 6 shows a scatter plot 602 illustrating the “latent features” extracted by the The final fully connected neural network for classification of each 3D refractive index tomogram is A table of inputs provided to the connected layer (i.e., the fully connected layer 730 described with reference to FIG. 7). The high-dimensional latent features are extracted using t-SNE (t-SNE) Mapped to the original (2D) plane. Latent images corresponding to correctly classified 3D refractive index tomograms Features are indicated as circles, and potential features corresponding to incorrectly classified 3D refractive index tomograms are indicated as “ The trained classification neural network can then classify the 3D refractive index tomograms into Generating latent features that cluster into respective groups in the space of latent features. In general, each cluster in the scatter plot corresponds to a different bacterial species. For example, cluster 604 generally corresponds to the 3D Corresponding to the refractive index tomography (i.e., most of the points in cluster 604 are A. baumannii (A. baumannii bacterium). The scatter plot 602 also shows other properties, e.g. For example, 3D refractive index tomograms were shown to roughly cluster based on Gram staining and metabolism. It is possible.

[0104] Figure 6C shows the classification neural network after it has been trained to process input data in different formats. A bar graph 606 illustrating the accuracy of the network is shown. A total of six types of data were compared: 2D optical position Phase delay images, a set of 2D optical phase delay images from multiple illumination angles (i.e., 3D RI tomograms) (which can be processed to generate 3D RI tomograms, and threshold-based binary averaging of the three types of images) Remasking (illustrated with reference to FIG. 6D). Three types of images and corresponding classification approaches The techniques are referred to herein as 2D, multiview, and 3D. Thresholds for 2D phase and 3D RI images The values ​​were set to 0.05π and 1.3425 to accommodate the various phases and RI values ​​of bacteria. In the multi-view and 2D approach, the classification neural network is a 2D convolutional neural network. In the multi-view approach, the prediction of 2D images is performed by a multi-view The data was then ensemble averaged to make predictions about the data. In this study, the classification neural network was able to identify the most accurate bacterial species by processing 3D RI tomography images. It has been demonstrated that it is possible to make predictions. The accuracy rates for 2D, multiview, and 3D tomography were 74.2%, 92.2%, and 94.6%, respectively. and 49.6%, 75.9%, and 92.4% for the corresponding binary masked input data.

[0105] FIG. 6D shows the 2D position image that is thresholded by 0.05π to generate the corresponding binary mask image 610. 6, which are thresholded by 0.05π to generate the corresponding binary mask image 614. A set of multi-view phase-delayed images 612 from multiple illumination angles and corresponding binary FIG. 6 is a diagram of a 3D RI tomogram 616 thresholded at 1.3425 to generate a remasked image 618. The scale bar represents 2 μm.

[0106] Figure 6E shows 3D RI tomograms correctly classified by the trained classification neural network. True 620 and 3D RI tomography incorrectly classified by the trained classification neural network The 3D RI tomogram 620 illustrates the structure of Bacillus subtilis. 624 shows three slices of the 3D RI tomogram 620, and 626 shows the 3D RI tomogram 620 using Grad-CAM++ technology. 6 shows three cross sections of the saliency map generated for a 3D RI tomogram 620. The intensities of points in the saliency map of a tomogram are fed to a trained classification neural network for 3D RI tomograms. This can be understood as indicating the importance of the points to the predictions generated by the network. 3D RI tomography 628 shows Staphylococcus aureus, and 630 shows 3D RI tomography 628 shows three cross sections, and 632 shows 3D RI tomography 632 using Grad-CAM++ technology. Figure 2 shows three cross sections of the saliency map generated for the 28-well plate. Scale bars represent 2 μm. Legend 622 shows a visualization of the 3D RI tomograms 620 and 628; the horizontal axis represents the bending moment. The vertical axis represents the refractive index value (which can be unitless), and the vertical axis represents the gradient of the refractive index (in arbitrary units For example, the darker regions of visualizations 620 and 628 represent the refractive index gradient. 10, corresponding to the portion of the 3D RI tomogram with a refractive index in the range of 1.375–1.4.

[0107] Figure 6F shows the results of the training of the classification neural network by processing 3D RI tomograms. 3D RI tomography showing correctly classified Haemophilus influenzae Three cross sections of the photographs, and a micrograph generated for the 3D RI tomograms using Grad-CAM++ technology. Three cross sections of the saliency map 636 are shown. 638 shows the trained classification neural network (i.e. , 2D convolutional architecture) and Staphylococcus aureus The same Haemophilus influenzae that was misclassified as indicating 6 shows a set of 2D phase images of multiple angles of 640, which are captured using Grad-CAM++ technology. Figure 1 shows the respective saliency maps generated for each of the 638 images. Scale bar is 2 μm. Represents.

[0108] FIG. 6G shows the trained classification neural network used to classify various bacterial properties (e.g., We provide examples of how accurately the genus, motility, morphology, Gram staining, and aerobicity can be identified. We use a trained classification neural network to process 3D RI tomograms of bacteria. The species of the bacteria is predicted, and then the characteristics of the bacteria are revealed based on the predicted species of the bacteria. By doing so, one or more sets of bacteria can be characterized. , using a trained classification neural network to identify 15 genera of 19 sepsis-causing bacterial species. The accuracy of use (overall: 95.5%) is shown in Table 644. s) using a trained classification neural network to identify bacterial species belonging to the genus The accuracy of the genus Streptococcus is 98.7%. Accuracy of using trained classification neural networks in identifying bacterial species (all Bar 648 represents the number of trained classification models in identifying bacterial motility. The bar graph 650 shows the accuracy of the three groups when using the neural network (overall: 98.2%). : Bacillus, Coccobasillus, and Coccus Accuracy of using trained classification neural networks in identifying bacterial morphology in The percentage of bacteria that have been studied in the confirmation of Gram staining is shown in bar graph 652. This represents the accuracy (overall: 97.5%) when using a classification neural network. Using a trained classification neural network to confirm the identity, we achieved an overall accuracy of 98.2%. degree was achieved.

[0109] FIG. 7 shows a classification system in which a 3D QPI 702 of a microorganism is processed to characterize the predicted species of the microorganism. An example of a classification neural network architecture configured to generate output 704. This architecture has a series of layers including, in the following order: A 3D convolutional layer 706 with ×7 convolutional filters, a series of six bottleneck layers 708 (skin The bottleneck connections ensure that the input to each bottleneck layer after the first is connected to the The sum of the output of the bottleneck layer and the input to the first bottleneck layer), l × l × l convolution filters A 3D convolutional layer 710 with a filter, a 3D 2×2×2 average pooling layer 712, a series of 12 bottlenecks A 3D convolution layer 714 with l×l×l convolution filters, a 3D convolution layer 716 with 3D 2×2×2 average pooling filters, A 3D convolution layer 718, a series of 32 bottleneck layers 720, and a 3D convolution filter with l×l×l convolution filters. layer 722, a 3D 2x2x2 average pooling layer 724, a series of 32 bottleneck layers 726, a 3D 7x7x7 total an average pooling layer 728, a fully connected layer 730, and a softmax output layer 732.

[0110] A specific hyperparameter of a classification neural network is The selection can be based on the nature of the prediction made by the work. For example, We set the filter dimensions of one or more convolutional layers of the neural network to match the predicted bacterial species. The selection can be based on the (known) size of the bacteria.

[0111] FIG. 8 is a flow diagram of an example process 800 for identifying a predicted species of one or more microorganisms. This is a diagram.

[0112] A phase contrast microscope is used to generate 3D QPIs of one or more microorganisms (802). The QPI includes a 3D representation of the microorganism. The 3D QPI may be, for example, a 3D refractive index tomography. The microorganism may be, for example, a bacterium, a virus, or a fungus. The entity may be a bacterium isolated from a blood sample of a patient infected with the bacterium.

[0113] The 3D QPI is then assigned to the classification neural network according to the trained parameter values ​​of the classification neural network. The neural network output is processed using a neural network to characterize the microorganisms. The classification neural network is generated (804) by using one or more 3D convolutional neural networks. In one example, the neural network may be a convolutional neural network including a neural network layer. The neural network output defines a probability distribution for a given number of possible microbial species. In another example, the neural network output may be a set of possible exponents. We define indices, each corresponding to a different possible type of microorganism.

[0114] The predicted species of the microorganism is identified using the neural network output (806). In one example, the predicted species of a microorganism is determined by the neural network output based on the probability value. In another example, a predicted type of microorganism may be identified as a type of microorganism with a high is the type of microorganism corresponding to the index defined by the neural network output. It can be identified.

[0115] FIG. 9 shows the results of the experiment described with reference to FIGS. 6A to 6G to generate 3D RI tomograms of bacteria. FIG. 9 is a diagram showing an example of an apparatus (Mach-Zehnder phase contrast microscope apparatus) 900 used for the multi-directional A set of DQPI measurements is assembled in the spatial frequency domain to be analyzed using the optical diffraction tomography (ODT) technique. 3D RI tomograms were generated using the technique. Each 2D QPI measurement was obtained using off-axis holography. The angle of incidence of the light illuminating the specimen (e.g., a bacterial sample) was adjusted by a digital micromirror device. The first-order beam is diffracted by a grating pattern on the DMD. A continuous wave laser with a wavelength of 532 nm and two water immersion objectives with a numerical aperture of 1.2 were used. According to the Nyquist theorem, the optical resolution is 110 nm on the horizontal axis and 330 nm on the vertical axis. The 3D RI tomograms were resampled with voxels of 100 nm width and 200 nm height. In the diagram of the device 6900, "BC" refers to the beam collimator, "BS" refers to the beam splitter, "CL" stands for focusing lens, "FC" stands for fiber coupler, and "LP" stands for linear polarizer. where "OL" refers to the objective lens and "TL" refers to the tube lens.

[0116] In general, any suitable device can be used to generate 3D RI tomograms. Another example of an apparatus that can be used to generate layer photographs is cited for the apparatus disclosed in the article: K. Kim, H. Yoon, M. Diez-Silva, M. Dao, R.R. "Plasmodium falciparum-induced malaria in Japan" by Y. Dasari and Y. Park High-resolution three-dimensional imaging of parasitized erythrocytes and in situ hemozoin crystals using optical diffraction tomography Original Imaging (High-resolution three-dimensional imaging of red blood cells par asitized by Plasmodium falciparum and in situ hemozoin crystals using optical di Journal of Biomedical Optics 19(1), 011005 (25 June 20 Another example of a device that can be used to generate 3D RI tomograms is F. Charriere, which is incorporated herein by reference for the apparatus disclosed therein. , A. Marian, F. Montfort, J. Kuehn, T. Colomb, E. Cuche, P. Marquet, and C. Depe. Ursinge's paper "Cell refractive index tomography by digital holographic microscopy" l refractive index tomography by digital holographic microscopy)” (Optics Lett ers, 31(2), pp. 178-180 (2006) for further explanation.

[0117] This specification refers to "configured" with respect to system and computer program components. The term "programming language" is used to refer to one or more programs configured to perform a particular operation or action. In a computer system, software causes the system to perform operations or actions during operation. Any software, firmware, hardware, or combination thereof installed on the System It means that one or more devices are configured to perform a particular operation or action. In the case of a plurality of computer programs, one or more programs are executed on a data processing device. By executed, it is meant to include instructions that cause the device to perform an operation or action.

[0118] Implementations of the subject matter and functional operations described herein may be implemented in digital electronic circuitry, tangibly implemented, or in any suitable form. The computer software or firmware disclosed herein and the structures and Computer hardware, including structural equivalents of these, or any combination of one or more of them Implementations of the subject matter described herein may be implemented in combination with one or more components. computer programs, i.e. for execution by or on a data processing apparatus A computer program encoded on a tangible non-transitory storage medium to control the operation of The computer storage medium may be implemented as one or more modules of instructions. , machine-readable storage device, machine-readable storage substrate, random or serial access memory device , or a combination of one or more thereof. The program instructions are adapted to be transmitted to a suitable receiver device for execution by the data processing device. An artificially generated propagated signal, e.g., mechanically generated, that is produced to encode information. The signal may be encoded into an electrical, optical, or electromagnetic signal that is generated.

[0119] The term "data processing device" refers to data processing hardware, e.g. A programmable processor, computer, or multiple processors or computers , includes any kind of apparatus, device, or machine for processing data. Also, dedicated logic circuits, such as FPGAs (field programmable gate arrays) or The device may be or may further include an ASIC (Application Specific Integrated Circuit). In addition to the hardware, code that creates the execution environment for a computer program, e.g. Processor firmware, protocol stack, database management system, operating Optionally includes code that constitutes a software development system, obtain.

[0120] program, software, software application, app, module, software Software modules, scripts, or code that may also be referred to or described as such. The computer program may be written in a compiled or interpreted language, or in a declarative or It can be written in any style of programming language, including procedural languages; As a stand-alone program or as a module, component, subroutine, or command may be implemented in any form, including as other units suitable for use in a computer environment. A program can correspond to a file in a file system, but not necessarily to a A program does not need to correspond to other programs or data, e.g. A file that holds one or more scripts stored in an application language document. In part, in a single file dedicated to the program, or in multiple cooperating files, e.g. or a file that stores multiple modules, subprograms, or pieces of code. A computer program can be stored in one computer or in one sub-system. are located at one site or distributed across multiple sites and connected to a data communications network Thus, the implementation can be implemented to run on multiple interconnected computers. .

[0121] As used herein, the term "engine" refers to a device configured to perform one or more specific functions. Refers to a programmed software-based system, subsystem, or process. Generally, an engine consists of one or more software modules or Implemented as a component and installed on one or more computers at one or more locations In some cases, one or more computers are dedicated to a particular engine. In other cases, multiple engines may be installed on the same computer or computers. It can be executed.

[0122] The processes and logic flows described herein may be implemented using one or more computer programs. One or more programs that perform a function by manipulating input data and generating output. The process and logic flow can be implemented by any number of programmable computers. The row may also be implemented by dedicated logic circuitry, such as an FPGA or an ASIC, or in combination with a dedicated logic circuit. The method may be implemented in combination with one or more programmed computers. do.

[0123] A computer suitable for running a computer program may be a general-purpose or special-purpose microprocessor. It may be based on a central processor, a processor-based processor, or both, or other types of central processing units. The processing unit receives instructions from the read-only memory or the random access memory, or both. The essential elements of a computer are the internal memory and the operating system for carrying out or executing the instructions. a central processing unit and one or more memory devices for storing instructions and data. The central processing unit and memory are supplemented by or incorporated into special purpose logic circuitry. Typically, a computer also includes one or more a mass storage device, such as a magnetic, magneto-optical, or optical disk; or Operatively coupled to receive data or to transmit data, or both However, a computer need not have such devices. A computer may communicate with other devices, such as mobile phones, personal digital assistants, to name a few. Mobile phones (PDAs), portable audio or video players, game consoles, global satellite GPS (Global Positioning System) receivers, or portable storage devices, such as those using the Universal Serial Bus (U Serial Bus) SB) It can be built into a flash drive.

[0124] A computer readable medium suitable for storing computer program instructions and data. Examples include semiconductor memory devices such as EPROM, EEPROM, and flash memory. Devices; magnetic disks, such as internal hard disks or removable disks; optical magnetic and all forms of non-volatile memory, including CD-ROM and DVD-ROM disks , media, and memory devices.

[0125] To provide for interaction with a user, embodiments of the subject matter described herein may provide the user with Display devices for displaying information, such as CRTs (cathode ray tubes) or LCDs (liquid crystal displays) A display (monitor), and a keyboard and pointer that allow the user to input information into the computer. Implemented on a computer with an pointing device, e.g., a mouse or trackball Other types of devices can also be used to provide for interaction with the user. For example, the feedback provided to the user may be any form of sensory feedback. feedback, e.g. visual feedback, auditory feedback, or tactile feedback and input from the user may be received in any form, including acoustic, voice, or tactile input. In addition, the computer can send and receive documents to and from the device that the user uses. For example, by receiving and sending documents from a web browser on a User's device. In response to a received request, the web browser transmits a web page to the user. The computer can also communicate with the user through text messages or other forms of communication. Send messages to a personal device, e.g. by running a messaging application and then receiving a response message from the user. The user can then interact with the device.

[0126] A data processing device for implementing a machine learning model may also be used for, e.g., machine learning training or production. Dedicated hardware to process the typical numerical part of the industry, i.e. inference, workloads. The device may include an accelerator unit.

[0127] Machine learning models are built using machine learning frameworks, such as the TensorFlow framework, crosoft Cognitive Toolkit framework, Apache Singa framework, or Apache It can be implemented and executed using the MXNet framework.

[0128] Implementations of the subject matter described herein may be implemented in a computer system, The computer system may include back-end components, such as data servers. , or middleware components, such as application servers or front-end End components, e.g., a client computer with a graphical user interface A computer, web browser, or other device that allows a user to interact with an implementation of the subject matter described herein. or one or more such backends, middleware, or frameworks. The components of the system may be of any type. or media, such as digital data communication, interconnected by a communications network; Examples of communication networks include local area networks (LANs) and wide area networks (WALs). Networks (WANs), such as the Internet.

[0129] The computer system may include clients and servers. The clients are generally remote from each other and typically interact through a communications network. The relationship between the client and the server is that they run on their own computers and are treated as clients and servers. In some embodiments, the server relationship is generated by a computer program. The server, for example, communicates with a device acting as a client, providing data to the user. and receiving user input from said user. Send messages to the user device. Data generated by the user device, e.g., Results of the interaction may be received at the server from the device.

[0130] Although the specification contains many specific implementation details, these may not be included within the scope or claims of any invention. should not be construed as limitations on the scope of what may be sought, but rather as limitations on the nature of any particular invention. It should be construed as a description of features that may be specific to a particular embodiment. Certain features which are described in this specification in the same context may also be used in combination in a single embodiment. Conversely, various features that are described in the context of a single embodiment may also be implemented in combination. may also be implemented in multiple embodiments separately or in any suitable subcombination. Moreover, features may be described above as functioning in certain combinations, even if Although initially it may be so claimed, one or more features of the claimed combination may be In some cases, it may be possible to remove from the combination, and the combination claimed would be: Subcombinations or variations of subcombinations may be applicable.

[0131] Similarly, although operations may be shown in the drawings and claimed in a particular order, This means that such operations must be performed in the particular order or sequences shown to achieve desired results. It does not require that the operations be performed in any particular order or sequence, or that all of the operations shown be performed. In certain circumstances, multitasking and parallel processing may be advantageous. In addition, the various system modules and components in the above embodiments may be Separation should not be construed as requiring such separation in all embodiments. The components and systems of the programs described herein are generally part of a single software product. It is understood that these may be integrated into one software product or packaged into multiple software products. should be interpreted as follows.

[0132] Although particular embodiments are described, the present disclosure is not limited to such embodiments.

[0133] As an example, certain bacteria are disclosed above, however, in some embodiments, the bacteria is For example, it may be associated with a sexually transmitted disease such as gonorrhea, syphilis, or chlamydia. In the microbial classification system, bacterial species are classified into subgroups, e.g., Treponema pallidum. idium, Mycoplasma genitalium, Neisseri a gonorrhoeae, Ureaplasma parvum, Mycoplasma Mycoplasma hominis, Ureaplasma urealyticum yticum, Gardnerella vaginalis, Chlamydia trachomatis Chlamydia trachomatis, Trichomonas vaginalis ), Candida albicans, or Haemophilus ducreyi emophilus ducreyi. Thus, the present disclosure provides a comparative The present invention provides a rapid and efficient diagnosis of infections, which may allow for relatively rapid treatment of such infections. do.

[0134] Certain embodiments of the subject matter have been described. Other embodiments are within the scope of the following claims. For example, the actions recited in the claims may be performed in a different order. By way of example, the process shown in the accompanying drawings may be does not necessarily require the particular sequence or order shown to achieve desirable results. No. In some cases, multitasking and parallel processing can be advantageous.

Claims

1. generating a three-dimensional quantitative phase image of one or more microorganisms using a phase contrast microscope; wherein the three-dimensional quantitative phase image comprises a three-dimensional representation of the microorganism; Processing the three-dimensional quantitative phase image using a neural network: The neural network is a convolutional neural network including one or more three-dimensional convolution layers. a real network; and The neural network performs a learning process according to the learned parameter values ​​of the neural network. and processing the three-dimensional quantitative phase images using a neural network to characterize the microorganism. configured to generate an output; and using the neural network output to identify a predicted species of the microorganism. Includes, a method.

2. The neural network output is a respective one for each of a predetermined number of microbial types. and The probability value for a given type of microorganism indicates the likelihood that the microorganism is of the given type of microorganism.

2. The method of claim 1.

3. using the neural network output to identify a predicted type of microorganism, comprising: The predicted species of the microorganism is selected from the group having the highest probability value in the neural network output.

3. The method of claim 2, further comprising identifying the species of microorganism present.

4. The neural network output indicates the likelihood that the microorganism is of the type of microorganism.

2. The method of claim 1, wherein the probability value of the type of microorganism is a probability value of the type of microorganism.

5. The predicted type of the microorganism is determined by genus, species, strain, Gram staining, metabolism, morphology, and motility.

2. The method of claim 1, wherein the compound is selected from the group consisting of:

6. 2. The method of claim 1, wherein the three-dimensional quantitative phase image of the microorganism is a three-dimensional refractive index tomography. 。

7. The microorganism is selected from the group consisting of bacteria, viruses, fungi, parasites, and microalgae. The method of claim 1 .

8. The method of claim 7, wherein the microorganism comprises a bacterium.

9. The method of claim 8, wherein the bacteria is present in a patient blood sample.

10. and administering an antibiotic therapy to the patient based on the predicted type of the bacteria.

10. The method of claim 9.

11. generating a three-dimensional quantitative phase image of a microorganism using the phase contrast microscope, The phase contrast microscope is used to capture phase and amplitude images of the microorganism at each of a plurality of illumination angles. generating an image; and and reconstructing a three-dimensional refractive index tomogram using the phase and amplitude images. The method described.

12. 10. The method of claim 1, wherein the method takes up to 1 hour.

13. According to the learned parameter values ​​of the neural network, by processing a three-dimensional quantitative phase image of one or more microorganisms using A method comprising identifying predicted species of a plurality of microorganisms.

14. 14. The method of claim 13, wherein the neural network includes one or more three-dimensional convolutional layers. How to.

15. a neural network output characterizing the microorganism; and The step of identifying the predicted species of the microorganism comprises: and identifying a likely species of the microorganism based at least in part on the 14. The method of claim 13, wherein the method takes up to 1 hour.

16. Providing a biological sample from a patient infected with a bacteria; Identifying the likely type of bacteria in the patient within one hour of obtaining a biological sample from the patient The process comprising: According to the learned parameter values ​​of the neural network, one or more 3D convolutions are performed. The neural network includes a convolutional layer, and one or more of the following are used to detect the isolated chromatin from the biological sample: The neural network processes 3D quantitative phase images of multiple bacteria to characterize the bacteria. generating a work output; and using the neural network output to identify a predicted species of the bacterium. and

17. 16. The method of claim 15, wherein the biological sample comprises a blood sample.

18. Within 45 minutes of providing a biological sample from the patient, a predicted bacterial infection rate for the patient is determined.

16. The method of claim 15, wherein the species is identified.

19. Within 30 minutes of providing a biological sample from the patient, a predicted bacterial infection rate for the patient is determined.

16. The method of claim 15, wherein the species is identified.

20. Within 15 minutes of preparing a biological sample from the patient, a predicted bacterial infection rate for the patient is determined. The method of claim 13, wherein the species is identified.

21. Providing a biological sample from a patient suspected of having a bacterial infection; Within one hour of obtaining a biological sample from the patient, the predicted species of bacteria in the patient is identified. Identifying a class, comprising: According to the learned parameter values ​​of the neural network, one or more 3D convolutions are performed. The neural network includes a convolutional layer, and one or more of the following are used to detect the isolated chromatin from the biological sample: The neural network processes 3D quantitative phase images of multiple bacteria to characterize the bacteria. generating a work output; and using the neural network output to identify a predicted species of the bacterium. and

22. A phase contrast microscope configured to generate three-dimensional quantitative phase images of one or more microorganisms. said phase contrast microscope, wherein said three-dimensional quantitative phase image comprises a three-dimensional representation of said microorganism; and a microbial classification system configured to perform the operations: The operation is processing the three-dimensional quantitative phase image using a neural network, The neural network is a convolutional neural network including one or more three-dimensional convolution layers. a global network; and The neural network is a trained parameter value of the neural network. A neural network that processes the three-dimensional quantitative phase images to characterize the microorganism according to the process being configured to generate a work output; using the neural network output to identify a predicted species of the microorganism; The system comprising:

23. When executed by one or more computers, the program causes one or more computers to operate One or more non-transitory computer-readable storage media that store instructions for causing a computer to execute a program. And the operation is: Receive a three-dimensional quantitative phase image of one or more microorganisms generated using a phase contrast microscope. wherein the three-dimensional quantitative phase image comprises a three-dimensional representation of the microorganism; Processing the three-dimensional quantitative phase image using a neural network: The neural network is a convolutional neural network including one or more three-dimensional convolution layers. a real network; and The neural network is adapted to the trained parameter values ​​of the neural network. The three-dimensional quantitative phase images are then processed to generate neural networks that characterize the microorganisms. generating a pulse output; using the neural network output to identify a predicted species of the microorganism; the one or more non-transitory computer-readable storage media.

Images