Machine learning for otitis media diagnosis

Abstract

Systems and methods for classifying tympanic membranes by using a classifier are disclosed herein. The classifier is a machine learning algorithm. A method of classifying a tympanic membrane includes the steps of: receiving one or more data sets related to the tympanic membrane from an interrogation system; determining a set of parameters from the one or more data sets, wherein at least one parameter in the set of parameters is related to a dynamic property or a static position of the tympanic membrane; and outputting a classification of the tympanic membrane based on a classifier model derived from the set of parameters. The classification includes one or more of a state, condition, or mobility metric of the tympanic membrane.

Description

[0001] This application is a divisional application of the Chinese patent application with the application date of January 24, 2020, application number 202080024219.9, and invention name “Machine Learning for Diagnosis of Otitis Media” (the application date of the corresponding PCT application is January 24, 2020, and the application number is PCT / US2020 / 015101).

[0002] Cross-references

[0003] This application claims the benefit of U.S. Provisional Application No. 62 / 796,762, filed January 25, 2019, which is incorporated herein by reference. Background Art

[0004] The present disclosure relates to systems and methods for characterizing tissue and related materials (such as for diagnosis). In particular, the present disclosure relates to measuring tissue mobility or position, such as the mobility or position of a tympanic membrane, for diagnosing and characterizing otitis media (i.e., ear infection). For example, embodiments of the present disclosure relate to machine learning, for example, using a set of training data consisting of optical and / or acoustic and / or pressure data sets of the tympanic membrane to predict tympanic membrane mobility or position. A predictive model can be generated using the training set and used to characterize the tympanic membrane mobility or position of a patient, thereby diagnosing and characterizing any otitis media.

[0005] Otitis media (OM) is a group of inflammatory diseases of the inner ear and is the most common cause of childhood health care problems, commonly referred to as "ear infections". OM is defined as the presence of fluid accumulation in the middle ear and includes two main types: acute otitis media (AOM), otitis media with effusion (OME), chronic otitis media (COM), and chronic suppurative otitis media (CSOM). The rapid onset of infection, often accompanied by ear pain, is the hallmark of AOM middle ear effusion (MEE), while OME is not usually associated with symptoms because MEE fluid is non-infectious.

[0006] OM diagnosis most commonly relies on otoscopy to visually assess the tympanic membrane (TM), i.e., the "ear drum." The otoscope illuminates and magnifies the TM to allow the physician to qualitatively interpret visual indicators associated with OM infection, such as coloration, translucency, and bulging. However, the accuracy of OM diagnosis via otoscopy varies due to equipment and physician skill—pediatricians and otolaryngologists, in particular, who are familiar with OM, have an estimated diagnostic accuracy ranging from 50-70% using otoscopy. Given the difficulty in appropriately diagnosing OM via otoscopy, only 40% of junior pediatricians are confident in their otoscopy findings, and AOM is frequently misdiagnosed.

[0007] Therefore, there is a need for improved systems, devices, and methods for classifying tympanic membranes and / or diagnosing otitis media.

[0008] References that may be relevant to the disclosure herein may include U.S. Patents 5,235,510, 5,331,550, 6,090,044, 7,058,441, 7,283,653, 7,771,356, 8,115,934, 8,798,345, 8,996,098, 9,053,536, 9,445,713, 9,536,054, 9,589,374, 9,636,007, 9,684,960, 9,750,450, 9,867,528, 9,922,242, and 1,001,757; U.S. Patents 2,003,026,470, 2,004,067,167, 2,006,0 and International Publications WO2000 / 010451, WO2001 / 026026, WO2009 / 157825, WO2018 / 045269, WO2018 / 106005, WO2018 / 140014, and WO2018 / 198253.

[0009] The following non-patent publications may be relevant to the disclosure herein: Vertan, C. et al., “Eardrum color content analysis in video-otoscopy images for the diagnosis support of pediatric,” Int. Symp. on Signals, Circuits and Systems, Bucharest, Romania, July 2011; Shaikh, N. et al., “Development of an algorithm for the diagnosis of otitis media,” Academic Pediatrics, Vol. 12, No. 3 (May-June 2012); Kuruvilla, A. et al., “Automated diagnosis of otitis media: vocabulary and grammar,” International Journal of Biomedical Imaging: NA. Hindawl Limited. (2013); Rappaport, K. et al., “Comparative Assessment of a Smartphone Otoscope for the Diagnosis and Management of Acute Otitis Media,” Provisional Section on Advances in Therapeutics and Technology Poster Presentation (October 28, 2013); Shie, CK et al., "A hybrid feature-based segmentation and classification system for the computer aided self-diagnosis of otitis media," Conf Proc IEEE Eng Med BiolSoc., 2014; 2014:4655-8; Somrak, M. et al., "Tricorder: consumer medical device for discovering common medical conditions," Informatica 38.1:81(8). SlovenianSociety Informatika. (March 2014); Myburgh, H.et al., "Towards low cost automated smartphone-and cloud-based otitis media diagnosis," Biomedical Signal Processing and Control 39(2018)34-52; Umea University. "Diagnosing ear infection using smartphone." ScienceDaily. ScienceDaily, March 30, 2016. <www.sciencedaily.com / releases / 2016 / 03 / 160330102850.htm>; Senaras, C. et al., "Detection of eardrum abnormalities using ensemble deep learning approaches," Proc. SPIE 10575, Medical Imaging 2018: Computer-Aided Diagnosis, 105751A (February 27, 2018); Kasher, M.S., "Otitis Media Analysis: An Automated Feature Extraction and Image Classification System," Helsinki Metropolia University of Applied Sciences, Bachelor of Engineering, Degree Programme in Electronics, Bachelor’s Thesis (25.04.2018); and Tran, T. et al., "Development of an Automatic Diagnostic Algorithm for Pediatric Otitis Media," Otology & neurotology: official publication of the American Otological Society, American Neurotology Society [and] European Academy of Otology and Neurotology 39.8:1060-1065. (September 2018). Summary of the Invention

[0010] The present disclosure relates to systems and methods for measuring the mobility and position of the tympanic membrane to diagnose and characterize otitis media (i.e., ear infection). Machine learning can be used with a set of training data consisting of optical and / or ultrasonic and / or pressure data sets of the tympanic membrane to generate a predictive model of tympanic membrane mobility and / or position.

[0011] Aspects of the present disclosure provide a method for classifying a tympanic membrane. The method includes: receiving one or more data sets related to the tympanic membrane from a medical consultation system; determining a set of parameters from the one or more data sets, wherein at least one parameter in the set of parameters is related to a dynamic property or a static position of the tympanic membrane; and outputting a classification of the tympanic membrane based on a classifier model derived from the set of parameters, wherein the classification includes one or more of a state, a condition, or a mobility measure of the tympanic membrane.

[0012] In some embodiments, the medical consultation system includes an imaging system, and wherein the one or more data sets include one or more images of the tympanic membrane. In some embodiments, the classifier model includes a machine learning algorithm. In some embodiments, the machine learning algorithm includes one or more of linear regression, logistic regression, classification and regression tree algorithms, support vector machines (SVM), naive Bayes, K nearest neighbor algorithms, random forest algorithms, boosting algorithms such as XGBoost and LightGBM, neural networks, convolutional neural networks, and recursive neural networks. In some embodiments, the machine learning algorithm is a supervised learning algorithm, an unsupervised learning algorithm, or a semi-supervised learning algorithm.

[0013] In some embodiments, the one or more images of the tympanic membrane include one or more ultrasound images. In some embodiments, the one or more ultrasound images include images from a pneumaticultrasound tympanoscope. In some embodiments, the one or more ultrasound images are measured in response to pneumatic excitation. In some embodiments, the one or more images of the tympanic membrane include one or more optical coherence tomography images. In some embodiments, the images include optical images.

[0014] In some embodiments, the at least one parameter related to the dynamic property or static position of the membrane is responsive to a pneumatic excitation. In some embodiments, the pneumatic excitation comprises a puff of gas. In some embodiments, the pneumatic excitation has a frequency greater than 10 Hz. In some embodiments, the dynamic properties of the tympanic membrane include one or more of the following: an indication of membrane movement or membrane mobility; a minimum or maximum displacement of the tympanic membrane; an abnormal displacement; a difference or ratio between minimum and maximum displacements; a slope of displacement or a slope of a difference or ratio between minimum and maximum displacements relative to the pressure of the pneumatic excitation; a response of measured pressure to applied pressure; visual movement of the tympanic membrane in response to aerodynamic excitation; one or more statistical components generated from singular value decomposition, principal component analysis, and K-means clustering; and an ultrasonic pulse echo amplitude or an ultrasonic echo phase or its derivative or its moving average. In some embodiments, the dynamic properties of the tympanic membrane are normalized relative to the pressure of the pneumatic excitation.

[0015] In some embodiments, the method further comprises generating one or more of an ordinal readout, a categorical readout, or a continuous numerical output of the tympanic membrane mobility. In some embodiments, the ordinal readout comprises a numerical level related to the degree of membrane mobility. In some embodiments, the numerical level comprises a classification from 0 to 4+. In some embodiments, the categorical readout comprises an indication of the degree of membrane mobility as at least one of highly mobile, moderately mobile, semi-mobile, or non-mobile. In some embodiments, the categorical readout comprises a binary classification. In some embodiments, the continuous numerical output comprises measuring one or more of membrane displacement, membrane movement speed, or membrane recovery speed.

[0016] In some embodiments, the state or condition of the tympanic membrane comprises one or more of acute otitis media, acute otitis media with effusion, middle ear effusion, chronic otitis media, chronic suppurative otitis media, bacterial infection, viral infection, no effusion, and unknown classification.

[0017] In some embodiments, the one or more data sets include an m-mode ultrasound data set. In some embodiments, the one or more data sets include infrared images. In some embodiments, the one or more data sets include a pneumatic data set. In some embodiments, the one or more data sets include one or more optical images acquired in response to a pneumatic excitation. In some embodiments, the static position includes an expanded membrane or a contracted membrane.

[0018] In another aspect, the present disclosure provides a system for classifying a tympanic membrane. The system includes: a computing system, the computing system including a memory, the memory including instructions for classifying the tympanic membrane, wherein the computing system is configured to execute the instructions to at least: receive one or more data sets related to the tympanic membrane from a consultation system; determine a set of parameters from the one or more data sets, wherein at least one parameter in the set of parameters is related to a dynamic property or a static position of the tympanic membrane; and output a classification of the tympanic membrane based on a classifier model derived from the set of parameters, wherein the classification includes a state, condition, or mobility measure of the tympanic membrane.

[0019] In some embodiments, the interrogation system comprises an imaging system, and wherein the one or more data sets comprise one or more images of the tympanic membrane. In some embodiments, the system additionally comprises a pneumatic ultrasonic tympanoscope.

[0020] In some embodiments, the classifier model includes a machine learning algorithm. In some embodiments, the machine learning algorithm includes one or more of linear regression, logistic regression, classification and regression tree algorithms, support vector machines (SVM), naive Bayes, K nearest neighbor algorithms, random forest algorithms, boosting algorithms such as XGBoost and LightGBM, neural networks, convolutional neural networks, and recursive neural networks. In some embodiments, the machine learning algorithm is a supervised learning algorithm, an unsupervised learning algorithm, or a semi-supervised learning algorithm.

[0021] In some embodiments, the one or more images of the tympanic membrane include one or more ultrasound images. In some embodiments, the one or more ultrasound images include images from a pneumatic ultrasonic tympanoscope. In some embodiments, the one or more ultrasound images are measured in response to pneumatic excitation. In some embodiments, the one or more images of the tympanic membrane include one or more optical coherence tomography images. In some embodiments, the images include optical images.

[0022] In some embodiments, the at least one parameter related to the dynamic properties or static position of the membrane is responsive to a pneumatic excitation. In some embodiments, the pneumatic excitation comprises a bolus of gas. In some embodiments, the pneumatic excitation comprises a frequency greater than 10 Hz. In some embodiments, the dynamic properties of the tympanic membrane include one or more of the following: an indication of membrane movement or membrane mobility; a minimum or maximum displacement of the tympanic membrane; an abnormal displacement; a difference or ratio between minimum and maximum displacements; a slope of displacement or a slope of a difference or ratio between minimum and maximum displacements relative to the pressure of the pneumatic excitation; a response of measured pressure to applied pressure; visual movement of the tympanic membrane in response to aerodynamic excitation; one or more statistical components generated from singular value decomposition, principal component analysis, and K-means clustering; and an ultrasonic pulse echo amplitude or an ultrasonic echo phase or its derivative or its moving average. In some embodiments, the dynamic properties of the tympanic membrane are normalized relative to the pressure of the pneumatic excitation.

[0023] In some embodiments, the computing system is further configured to execute the instructions to generate one or more of an ordinal readout, a categorical readout, or a continuous numerical output of the tympanic membrane mobility. In some embodiments, the ordinal readout includes a numerical level related to the degree of membrane mobility. In some embodiments, the numerical level includes a classification from 0 to 4+. In some embodiments, the categorical readout includes an indication of the degree of membrane mobility as at least one of highly mobile, moderately mobile, semi-mobile, or non-mobile. In some embodiments, the categorical readout includes a binary classification. In some embodiments, the continuous numerical output includes one or more of a measurement of membrane displacement, a membrane movement speed, or a membrane recovery speed.

[0024] In some embodiments, the state or condition of the tympanic membrane comprises one or more of acute otitis media, acute otitis media with effusion, middle ear effusion, chronic otitis media, chronic suppurative otitis media, bacterial infection, viral infection, no effusion, and unknown classification.

[0025] In some embodiments, the one or more data sets include an m-mode ultrasound data set. In some embodiments, the one or more data sets include infrared images. In some embodiments, the one or more data sets include a pneumatic data set. In some embodiments, the one or more data sets include one or more optical images acquired in response to a pneumatic excitation. In some embodiments, the static position includes an expanded membrane or a contracted membrane.

[0026] On the other hand, the present disclosure provides a non-transitory computer-readable medium comprising machine executable code that, when executed by a computing system, implements a method for classifying a membrane, the method comprising: receiving one or more data sets related to the tympanic membrane from a consultation system; determining a set of parameters from the one or more data sets, wherein at least one parameter in the set of parameters is related to dynamic properties or static position of the tympanic membrane; and outputting a classification of the tympanic membrane based on a classifier model derived from the set of parameters, wherein the classification includes a state, condition, or mobility measure of the tympanic membrane.

[0027] In some embodiments, the interrogation system comprises an imaging system, and wherein the one or more data sets comprise one or more images of the tympanic membrane. In some embodiments, the method may further comprise the method of any aspect or embodiment disclosed herein.

[0028] In another aspect, a method for training a computer-implemented classifier is provided. The method includes: receiving a set of parameters, the set of parameters being based on one or more data sets related to one or more tympanic membranes and one or more classification data sets related to the one or more tympanic membranes, wherein the classification includes a state, condition, or mobility metric of the tympanic membranes, and wherein the set of parameters includes at least one parameter related to a dynamic property or a static position of the one or more tympanic membranes; storing the set of parameters and the one or more classification data sets in a database; building a classifier model based on the set of parameters and the one or more classification data sets, wherein the classifier model is derived from the set of parameters, and wherein the classifier model outputs a classification based on a data set of the one or more classification data sets; and using the classifier model to provide a classification of an unclassified data set.

[0029] In some embodiments, the method further comprises updating the database based on a second one or more classified data sets; and updating the classifier model based on the second one or more classified data sets. In some embodiments, the method further comprises providing classification of the unclassified data set using the classifier model by the method for classifying a tympanic membrane according to any aspect or embodiment.

[0030] In another aspect, a system for training a computer-implemented classifier is provided. The system comprises: a computing system comprising a memory, the memory comprising instructions for training the data set, wherein the computing system is configured to execute the instructions to at least: receive a set of parameters, the set of parameters being based on one or more data sets related to one or more tympanic membranes and one or more classification data sets related to the one or more tympanic membranes, wherein the classification comprises a state, condition, or mobility metric of the tympanic membrane, and wherein the set of parameters comprises at least one parameter related to a dynamic property or a static position of the one or more tympanic membranes; storing the set of parameters and the one or more classification data sets in a database; constructing a classifier model based on the set of parameters and the one or more classification data sets, wherein the classifier model is derived from the set of parameters, and wherein the classifier model outputs a classification based on a data set of the one or more classification data sets; and using the classifier model to provide a classification of an unclassified data set.

[0031] In some embodiments, the system is configured to execute the instructions to at least: update the database based on a second one or more classification data sets; and update the classifier model based on the second one or more classification data sets. In some embodiments, the system also includes a system for classifying a tympanic membrane as described in any aspect or embodiment.

[0032] On the other hand, the present disclosure provides a non-transitory computer-readable medium comprising machine executable code that, when executed by a computing system, implements a method for training a computer-implemented classifier, the method comprising: receiving a set of parameters, the set of parameters being based on one or more data sets related to one or more tympanic membranes and one or more classification data sets related to the one or more tympanic membranes, wherein the classification comprises a state, condition, or mobility metric of the tympanic membrane, and wherein the set of parameters comprises at least one parameter related to a dynamic property or a static position of the one or more tympanic membranes; storing the set of parameters and the one or more classification data sets in a database; constructing a classifier model based on the set of parameters and the one or more classification data sets, wherein the classifier model is derived from the set of parameters, and wherein the classifier model outputs a classification; and using the classifier model to provide a classification of an unclassified data set.

[0033] In some embodiments, the method further comprises updating the database based on a second one or more classified data sets; and updating the classifier model based on the second one or more classified data sets. In some embodiments, the method further comprises providing classification of the unclassified data set using the classifier model by the method for classifying a tympanic membrane according to any aspect or embodiment.

[0034] Incorporation by reference

[0035] All publications, patents, and patent applications mentioned in this specification are incorporated herein by reference to the same extent as if each individual publication, patent, or patent application was specifically and individually indicated to be incorporated by reference. In the event that publications and patents or patent applications incorporated by reference conflict with the disclosure contained in this specification, this specification is intended to supersede and / or take precedence over any such conflicting material. BRIEF DESCRIPTION OF THE DRAWINGS

[0036] The novel features of the present invention are particularly set forth in the appended claims. A better understanding of the features and advantages of the present invention will be obtained by referring to the following detailed description and accompanying drawings (also referred to herein as "figures") which set forth illustrative embodiments in which the principles of the present invention are utilized, in which:

[0037] Figure 1 is a flow chart of an example method of characterizing a tympanic membrane according to some embodiments.

[0038] Figure 2 is a schematic diagram illustrating an example system 200 for classifying a tympanic membrane in accordance with some implementations.

[0039] Figure 3 is a schematic diagram illustrating an example of a medical consultation system as disclosed herein.

[0040] Figure 4 An example data set provided by the medical inquiry device of the present disclosure is illustrated.

[0041] Figure 5A and Figure 5B According to some embodiments, a schematic diagram of a set of parameters derived from pressure, ultrasound, and optical measurements may be included as input to a classifier model to output a classification.

[0042] Figure 6 is a diagram illustrating an example 600 of a classifier model including a machine learning algorithm according to some implementations.

[0043] Fig. 7A , Figure 7B and Figure 7C is a diagram illustrating an example decision tree that may be generated by a machine learning algorithm according to some implementations.

[0044] Figure 8 is a flow chart of an example method of training a computer-implemented classifier according to some embodiments.

[0045] Fig. 9 is a schematic diagram illustrating an example of a system for training a computer-implemented classifier.

[0046] Fig.10 is a schematic diagram illustrating an example system for classifying a tympanic membrane, the example system including a digital processing device and a user-viewable display, according to some implementations. DETAILED DESCRIPTION

[0047] Methods, systems and media disclosed herein can improve existing methods of classifying tissues by providing improved classification (e.g., diagnosis) of tissues. For example, methods and systems provided herein can use machine learning methods to build a classifier that improves the classification of tympanic membranes. Machine learning methods can utilize large data sets to obtain new insights into data sets. Classifier models can improve the characterization of tympanic membranes, which may lead to better patient outcomes. Classifier models can provide information for clinicians to be used for more accurate otitis media management (e.g., tympanic membrane mobility information and / or the state or condition of tympanic membranes), while reducing deviations due to physician training and experience with pneumatic otoscope examinations. In some cases, methods and systems provided herein can directly provide disease states from trained and validated algorithms.

[0048] The methods, systems, and media disclosed herein can improve existing methods of classifying tissue by utilizing parameters related to the mobility of the tympanic membrane. A normal tympanic membrane can move in response to pressure, and lack of or reduced mobility is an indication of fluid, perforation, or tympanic sclerosis, etc. in the middle ear. Adding mobility measurements may be important for diagnostic accuracy because the predictive value of visible tympanic membrane properties in OM diagnosis may vary. Therefore, even in cases where the appearance of the tympanic membrane does not give an indication of middle ear pathology, the additional information of mobility can provide an indication of the presence of effusion.

[0049] The methods, systems, and media disclosed herein can be used, for example, in combination with apparatus and methods for characterizing ductile films, surfaces, and subsurface properties, such as those described in commonly owned U.S. Patent No. 7,771,356 and U.S. Patent Publication Nos. 2019 / 0365292, 2018 / 0310917, and 2017 / 0014053, each of which is incorporated by reference in its entirety. The methods, systems, and media disclosed herein can be used, for example, in combination with apparatus and methods using optical coherence tomography (OCT), such as those disclosed in commonly assigned U.S. Patent Publication No. 2019 / 0200873 and U.S. Patent Publication No. 2017 / 0360302, each of which is incorporated by reference in its entirety.

[0050] The methods, systems and media disclosed herein can be used to characterize a variety of biological tissues to provide various diagnostic information. The biological tissues can include patient organs. The speculum can be arranged in a body cavity to characterize the patient tissue. The patient organ or body cavity can include, for example, muscles, tendons, ligaments, oral cavity, tongue, pharynx, esophagus, stomach, intestine, anus, liver, gallbladder, pancreas, nose, larynx, trachea, lung, kidney, bladder, urethra, uterus, vagina, ovary, testicles, prostate, heart, arteries, veins, spleen, glands, brain, spinal cord, nerves, etc., to name a few.

[0051] The methods, systems and media disclosed herein can be used to classify tympanic membranes. For example, the membrane can be classified to determine the condition of the ear, such as acute otitis media (AOM), chronic otitis media, otitis media with effusion, and / or chronic suppurative otitis media. The classification of the ear showing AOM can include detecting the presence of effusion and characterizing the type of effusion as one of serous, mucoid, purulent, or a combination of these. In AOM, middle ear effusion (MEE) can be induced by an infectious agent and may be thin or serous in bacterial infection, and thicker and purulent in bacterial infection. Therefore, determining various properties of the fluid adjacent to the tympanic membrane can provide information that can be used to characterize the membrane.

[0052] Reference will now be made in detail to various embodiments, examples of which are illustrated in the accompanying drawings. In the detailed description that follows, many specific details are set forth to provide a thorough understanding of the present disclosure and the described embodiments. However, the embodiments of the present disclosure are optionally practiced without these specific details. In other cases, known methods, steps, components, and circuits are not described in detail to avoid unnecessarily making the various aspects of the embodiments unclear. In the accompanying drawings, the same reference numerals refer to the same or similar steps or components.

[0053] The terms used herein are only for describing specific embodiments and are not intended to limit the claims. As used in the description of the embodiments and the appended claims, the singular forms "one", "a kind of" and "the" are also intended to include plural forms unless otherwise expressly stated herein. It should also be understood that the term "and / or" as used herein refers to and encompasses any and all possible combinations of one or more associated listed items. It will be further understood that when the terms "include" and / or "comprising" are used in this specification, they specify the presence of the features, integers, steps, operations, elements and / or components, but do not exclude the presence or addition of one or more other features, integers, steps, operations, elements, components, and / or groups thereof.

[0054] As used herein, the term "if" is optionally interpreted to mean "when", "after..." or "in response to determination" or "according to determination" or "in response to detection", the precondition is true, depending on the context. Similarly, the phrase "if it is determined that [the precondition is true]" or "if [the precondition is true]" or "when [the precondition is true]" is optionally interpreted to mean "after determination" or "in response to determination" or "according to determination" or "according to detection" or "in response to detection", the conditional precondition is true, depending on the context.

[0055] As used herein, and unless otherwise indicated, the term "about" or "approximately" means an acceptable error for a particular value determined by one of ordinary skill in the art, which may depend in part on how the value is measured or determined. In certain embodiments, the term "about" or "approximately" means within 1, 2, 3, or 4 standard deviations. In certain embodiments, the term "about" or "approximately" means within 30%, 25%, 20%, 15%, 10%, 9%, 8%, 7%, 6%, 5%, 4%, 3%, 2%, 1%, 0.5%, 0.1%, or 0.05% of a given value or range.

[0056] As used herein, the terms "comprises," "comprising," or any other variations thereof, are intended to cover a non-exclusive inclusion such that a process, method, article, or apparatus that comprises a list of elements includes not only the elements stated but may also include other elements not expressly listed or inherent to such process, method, article, or apparatus.

[0057] As used herein, the terms "subject" and "patient" are used interchangeably. As used herein, the term "subject" refers to animals (e.g., birds, reptiles, and mammals), including primates (e.g., monkeys, chimpanzees, and humans) and non-primates (e.g., camels, donkeys, zebras, cattle, pigs, horses, cats, dogs, rats, and mice). In certain embodiments, the mammal is 0 to 6 months old, 6 to 12 months old, 1 to 5 years old, 5 to 10 years old, 10 to 15 years old, 15 to 20 years old, 20 to 25 years old, 25 to 30 years old, 30 to 35 years old, 35 to 40 years old, 40 to 45 years old, 45 to 50 years old, 50 to 55 years old, 55 to 60 years old, 60 to 65 years old, 65 to 70 years old, 70 to 75 years old, 75 to 80 years old, 80 to 85 years old, 85 to 90 years old, 90 to 95 years old, or 95 to 100 years old.

[0058] Figure 11 is a flow chart of an example method 100 for classifying a tympanic membrane according to some embodiments. At operation 110, method 100 includes receiving one or more data sets related to the tympanic membrane from a consultation system. At operation 120, method 100 includes determining a set of parameters from the one or more data sets. At least one parameter in the set of parameters may be related to a dynamic property of the tympanic membrane. At operation 130, method 100 includes outputting a classification of the tympanic membrane based on a classifier model derived from the set of parameters. The classification may include one or more of a state, a condition, or a mobility metric of the tympanic membrane.

[0059] Although the above operations illustrate a method 100 for classifying a tympanic membrane according to some embodiments, a person of ordinary skill in the art will recognize many variations based on the teachings described herein. The steps can be completed in any order. Steps can be added or deleted. Many steps can include sub-steps. Many steps can be repeated as often as is beneficial to the manufacturing method.

[0060] One or more steps of method 100 may be performed using circuitry as described herein, such as one or more of a digital processing device or a processor or logic circuitry such as programmable array logic for a field programmable gate array. The circuitry may be programmed to provide one or more steps of method 100, and the program may include program instructions stored on a computer readable memory or programmed steps of logic circuitry such as, for example, programmable array logic or a field programmable gate array. Embodiments, variations, and examples of digital processing devices operable to perform one or more steps of method 100 are described elsewhere herein, such as with respect to the "Digital Processing Device" section and the descriptions thereof. Fig.10 .

[0061] Figure 2 is a schematic diagram illustrating an example system 200 for classifying a tympanic membrane according to some embodiments. System 200 may include one or more data sets 210 received from a consultation system. One or more data sets 210 may be stored in a memory of system 200. System 200 may include a classifier model 220 operable to classify a tympanic membrane based on one or more data sets 210 and output a classification 230. Classification 230 may include one or more of a state, condition, or mobility metric of the tympanic membrane. An output (e.g., a classification) of the classifier model may be generated. The classification may include information about TM mobility, which may help a physician diagnose an OM infection or directly diagnose a patient's disease state.

[0062] For example, a system for classifying a tympanic membrane is disclosed herein. The system includes: a computing system, the computing system including a memory, the memory including instructions for classifying the tympanic membrane, wherein the computing system is configured to execute the instructions to at least: receive one or more data sets related to the tympanic membrane from a consultation system; determine a set of parameters from the one or more data sets, wherein at least one parameter in the set of parameters is related to the dynamic properties of the tympanic membrane; and output a classification of the tympanic membrane based on a classifier model derived from the set of parameters, wherein the classification includes a state, condition, or mobility measure of the tympanic membrane. Implementations, variations, and examples of computing devices for a system for classifying a tympanic membrane are described elsewhere herein, such as with respect to the "Digital Processing Device" section and the descriptions therein. Fig.10 .

[0063] Consultation system

[0064] Figure 3 is a schematic diagram illustrating an example of an interrogation system 300 as disclosed herein. An interrogation system 300 as disclosed herein can collect one or more data sets 210 related to a biomembrane (e.g., a tympanic membrane) of a subject 320 disclosed herein. The one or more data sets related to the tympanic membrane can include data related to the physical properties of the biomembrane (e.g., a tympanic membrane) to be characterized. In some examples, the interrogation system can include an imaging system. In some examples, the interrogation system does not include an imaging system. The interrogation system can collect ultrasound data, such as reflected ultrasound data. The interrogation system can collect optical coherence tomography data. The interrogation system can collect infrared data. The interrogation system can collect optical data. The interrogation system can collect data in response to pneumatic excitation. The interrogation system can collect data sets related to membrane movement, for example, in response to pneumatic excitation. The pneumatic excitation can include a pressure excitation, such as a bolus of air. The interrogation system can measure the pressure in the ear canal as a function of applied pressure. In one example, the interrogation system can measure the pressure in the ear canal after the pneumatic excitation.

[0065] Any example or combination of examples of the medical consultation system as disclosed herein can be used independently or in combination to provide a data set from which parameters are generated for inclusion in a classifier model as disclosed herein. For example, any example or combination of examples of the medical consultation system as disclosed herein can be used independently or in combination as input to a machine learning algorithm to provide a classification output.

[0066] The inquiry system as disclosed herein can be remote from the classification system disclosed herein. The inquiry system can collect one or more data sets (e.g., data) that can be sent to the classification system as disclosed herein. In some cases, one or more data sets can be received from a remote server. However, in some cases, the inquiry system can be local to the classification system as disclosed herein. For example, the classification system can be part of the on-board logic on the processor of the classification system, such as on the logic within the ultrasound tympanoscope.

[0067] In some cases, the interrogation system does not include an imaging system. For example, the interrogation system may include an ultrasound system. The ultrasound system may collect ultrasound data sets, such as Doppler ultrasound data. In some cases, the interrogation system collects frequency-dependent data, such as frequency-dependent optical coherence tomography data, absorption spectra, depth-dependent ultrasound tomography data, etc. The ultrasound data may include data from a pneumatic ultrasound tympanoscope.

[0068] The medical inquiry system may include ultrasound data. The ultrasound data may include amplitude mode (a-mode) ultrasound data. The ultrasound data may include two-dimensional (b-mode) ultrasound data. The ultrasound data may include m-mode (motion mode) ultrasound data. The ultrasound data may include Doppler ultrasound data. The ultrasound data may include ultrasound m-mode phase with fixed gate depth and time. The ultrasound data may include m-mode amplitude with fixed gate depth and time.

[0069] In some examples, the interrogation system includes an imaging system. The imaging system can collect one or more data sets including one or more images of the tympanic membrane. In an example, the image can include an optical image such as from a camera. In an example, the image can include an ultrasound image. The image can include an optical coherence tomography image. The imaging system can include any other imaging mode or combination of imaging modes such as magnetic resonance imaging (MRI) or computed tomography (CT) or positron emission tomography (PET). Optionally, any image mode can be used with pneumatic excitation. For example, images can be collected in response to pneumatic excitation. Many images can be collected before and after pneumatic excitation. One or more images can include ultrasound images. One or more ultrasound images include images from a pneumatic ultrasound tympanoscope. One or more ultrasound images can be measured in response to pneumatic excitation. In an example, the interrogation system can measure the dynamic visual movement of the tympanic membrane using a CCD camera. The optical image data set in response to pneumatic excitation can include an optical integrated image intensity and time waveform form. The optical image data set in response to pneumatic excitation can include a light cone intensity and time waveform. The optical image data set in response to pneumatic excitation can include a light cone position and time waveform. The optical image data set in response to the aerodynamic excitation may include an image sharpness index and a temporal waveform.

[0070] In an example, a consultation system as disclosed herein may include implementations, variations, or examples of the methods and systems disclosed in U.S. Patent No. 7,771,356 and U.S. Patent Publication Nos. 2019 / 0365292, 2018 / 0310917, and 2017 / 0014053, each of which is incorporated herein by reference in its entirety. Methods and systems for obtaining information about the movement of the tympanic membrane using ultrasonic echo signals as disclosed in the incorporated references can be used to generate one or more parameters related to the dynamic properties of the tympanic membrane. A system for measuring ultrasonic echo signals can cause movement of the tympanic membrane by applying a system pressure pulse, and then extracting a Doppler shift signal from the ultrasound to analyze the displacement of the TM and / or classify the viscosity of the ear effusion.

[0071] In an example, a medical consultation system as disclosed herein may include embodiments, variations or examples of the methods and systems disclosed in commonly assigned U.S. Patent Publication No. 2019 / 0200873 and U.S. Patent Publication No. 2017 / 0360302, each of which is incorporated herein by reference in its entirety. Methods and systems for characterizing membranes using optical coherence tomography (OCT) as disclosed in U.S. Patent Publication No. 2019 / 0200873 and U.S. Patent Publication No. 2017 / 0360302 can be used to generate one or more parameters related to the dynamic properties of the tympanic membrane. For example, the dynamic properties of the tympanic membrane may include a phase delay or time delay of a reflected light signal in response to an applied pneumatic excitation. OCT can be used to collect depth-related data related to the tympanic membrane. OCT can be used to collect frequency-related data, such as the absorption wavelength of a membrane or a fluid adjacent to the membrane.

[0072] The interrogation system can collect data in response to pneumatic excitation. The interrogation system can collect data related to the movement of the membrane, for example, in response to the pneumatic excitation. The pneumatic excitation can include a pressure excitation, such as a bolus of air. The pneumatic excitation can change the response of the membrane to the ultrasonic excitation. For example, the pneumatic excitation can cause the membrane to deflect, which can change the phase of the reflected ultrasound relative to the membrane not exposed to the pneumatic excitation. The deflection of the membrane can include damped simple harmonic motion. The motion can be affected by changes in the elasticity of the membrane. For example, if water, bacterial growth or other foreign matter is adjacent to the membrane, the membrane elasticity may change.

[0073] In some examples, pneumatic excitation can produce movement of a surface or membrane within a certain time interval. The interval can be consistent with the sound wave transmitted to the surface or membrane by the ultrasonic transmitter. Pneumatic excitation can be continuous, can be pulsed, and so on. The ultrasound reflected from the surface can be received at the transducer. The transducer can be the same transducer that produces the incident sound wave. The displacement of the surface or membrane can be related to the phase change of the received signal (compared with the transmitted signal). The movement of the membrane can affect the phase change of the received ultrasound. The displacement can change with time. Analysis of the time displacement of the surface or membrane can be used to determine the mechanical properties of the surface or membrane, such as measured by the phase shift of the reflected ultrasound in response to the pneumatic excitation coupled to the surface or membrane.

[0074] Analysis of the time information can be used in conjunction with time displacements measured from templates of other membrane responses to create comparisons. Analysis of the time information can be used in conjunction with other metrics associated with the delay of the reflected ultrasound amplitude that characterizes the response of the surface or membrane. The measured mechanical properties can include ductility, elasticity, hardness, etc. Non-contact measurements of mechanical properties of the membrane surface or fluid below the membrane surface can be determined.

[0075] In some embodiments, the elasticity of the surface can be measured. The phase and / or amplitude of the ultrasound reflected from the membrane can be analyzed to produce an elasticity metric. The elasticity measurement can characterize a series of measurements in response to an applied excitation. The elasticity metric can be derived from the response of the surface and can provide one or more indications of several different phenomena. For example, the elasticity metric can indicate whether the surface adjacent to the membrane has a gaseous boundary or a fluid boundary. For example, if the membrane has a fluid boundary, the membrane may move less, move slower, or not move at all. In one example, for the case of characterizing the fluid behind the membrane, the elasticity metric can indicate the extent or characteristics of the fluid boundary, the fluid. In some examples, the elasticity metric can be used to measure the characteristics of elastic fluids with or without response hysteresis. In a fluid with a hysteresis response, the fluid may exhibit an offset in the displacement response or "memory" so that the response behavior in one direction is similar to the response behavior in the opposite direction, but only after a specific displacement distance. For a hysteresis response, it may be necessary to characterize the linear behavior of the response after a specific measured displacement associated with the system hysteresis. The fluid elasticity metric can be determined based on the ultrasonic characterization of the characteristic response and reflection of the surface or membrane to the surface excitation.

[0076] In some embodiments, the surface deflection can be estimated. For example, the estimate of the surface deflection can be derived from an estimate measured by velocity, acceleration, or any other relevant measure of the change in deflection over time. For example, a displacement of the surface will cause the path from the transducer to the surface to shorten, and the reflected signal from the surface back to the transducer will return with a phase shift. Therefore, the phase shift of the reflected ultrasound relative to the excitation gives information about the amount of deflection. Using the estimate of the force applied by the excitation, an estimate of the elasticity of the membrane can be estimated.

[0077] In one example, the excitation is a step or impulse response with a rising edge, a falling edge, or an impulse excitation. The impulse excitation causes an oscillatory deflection of the membrane. The reflected ultrasound can be measured based on the time from the excitation to the damping period of the membrane oscillation. In some embodiments, the estimation of position, elasticity, or viscosity can be performed by examining the ringing properties. For example, the ringing properties may include at least one of an exponential decay time or a ringing period interval or frequency, for example, decomposing the response into ringing properties, such as:

[0078] φ(t)=e -t / τ cos(2π∫t)

[0079] in:

[0080] φ(t) is the capture phase of a series of measurements;

[0081] τ is the exponential decay coefficient;

[0082] f is the ringing cycle frequency; and

[0083] t is time.

[0084] The damping constant of the oscillator may be related to the energy lost from the membrane to the surrounding environment. In one example, if the membrane is adjacent to a fluid, the fluid may dampen the oscillations of the membrane. The viscosity of the fluid may be related to the damping of the oscillator. The ringing cycle frequency may be related to the restitution constant of the elastic membrane. The restitution constant may be related to the elasticity of the membrane. The restitution constant may be related to the viscosity of the fluid adjacent to the membrane. The lower the viscosity of the fluid adjacent to the membrane, the higher the ringing cycle frequency may be.

[0085] Each excitation event may cause a new deflection of the membrane. For example, a pulse excitation may pull the membrane in or push the membrane out for a limited time. For example, a square wave excitation may pull the membrane in or push the membrane out for a longer time. For example, a sine wave or other more complex excitation may be applied, and the ringing observed at the transducer may be the cross-correlation of the excitation field with the response field. Pneumatic excitation may be applied at a frequency of less than 100kHz, less than 1kHz, less than 100Hz, less than 10Hz, less than 1Hz or less, or within a range given by any two of the aforementioned values. Pneumatic excitation may be applied at a frequency greater than 1Hz, greater than 10Hz, greater than 100Hz, greater than 1kHz, greater than 100kHz or greater, or within a range given by any two of the aforementioned values. Pneumatic excitation may be applied in the range of 10Hz to 100Hz.

[0086] Figure 4 An example data set provided by the inquiry device of the present disclosure is illustrated. In section A, Figure 4The intensity levels of the m-mode ultrasound signal are shown. Part A uses ultrasound measurements to characterize the position of the tympanic membrane during the application time course of the pressure pulses applied within a specified time range. The horizontal time level of Part A is the same level as in Parts B, C, and D (where Part D has a shortened time range compared to Parts B and C). At Part B, Figure 4 A graph of a horizontal segment of the heat map from part A is shown (indicated by the dashed line). While part A shows the signal amplitude at all depths, part B shows the change in membrane position based on the change in signal phase over time. Part B indicates the tympanic membrane position relative to the measurement time. At part C, Figure 4 An applied pressure pulse is shown over the same time frame as in section B. At section D, Figure 4 The measured values ​​of the pressure in the ear canal during the pressure pulse in the first half of the time level in section C are shown. The measured values ​​of the pressure in the ear canal during the second half of the time level in section C are plotted as a second curve in section D, which is shifted in time to overlap with the data from the first half. These sensed pressure profiles in section D are very close in value and difficult to distinguish from each other, indicating that the movement of the eardrum during repeated pressure stimulation is very similar. At section E, Figure 4 An example optical image recorded by a CCD camera is shown, where the dynamic movement of the white area, ie the light cone, (highlighted by the black frame with arrows) during the pressure pulse indicates the mobility of the TM.

[0087] In conclusion, Figure 4 As pressure pulses are applied (C), the generated and converted ultrasound signals provide detailed information about the location of the tympanic membrane (A and B) and the unique pressure and optical responses from images (D) and (E), respectively. The information contained in these measurements can be used to generate a set of parameters for training a classifier model, such as a machine learning model, to output a classification.

[0088] Model parameters

[0089] A set of parameters can be derived from one or more data sets. At least one parameter in the set of parameters can be related to the dynamic properties of the tympanic membrane. A set of parameters can also include parameters related to the static characteristics of the tympanic membrane (for example, parameters that are independent of membrane movement). A set of parameters can include parameters related to membrane movement and parameters that are independent of membrane movement. A set of parameters can include static parameters and dynamic parameters. A set of parameters can include one or more ultrasonic features, one or more pressure features, and / or one or more optical features. A set of parameters or a subset of the set of parameters can be used as input to a classifier model such as a trained machine learning algorithm. The classifier model can output a classification. The classification can include a mobility output. The classification can include a disease state.

[0090] At least one parameter in the set of parameters may be related to a dynamic property of the tympanic membrane. The dynamic property of the tympanic membrane may vary over time. For example, the dynamic property of the tympanic membrane may vary over time in response to a pneumatic stimulus, such as a puff of air or a bolus of gas. A non-exhaustive list of potential parameters related to dynamic properties is as follows: an indication of membrane movement or membrane mobility; a minimum or maximum displacement of the tympanic membrane; an abnormal displacement; a difference or ratio between the minimum and maximum displacements; a slope of the displacement or a slope of the difference or ratio between the minimum and maximum displacements relative to the pressure of the pneumatic stimulus; a measured pressure versus an applied pressure response; visual movement of the tympanic membrane in response to a pneumatic stimulus; one or more statistical components generated from singular value decomposition, principal component analysis, and K-means clustering; and an ultrasonic pulse echo amplitude or ultrasonic echo phase or its derivative or its moving average. The dynamic parameters may be derived from normalized or averaged dynamic properties of the tympanic membrane in response to a dynamic stimulus, such as a normalized or averaged value of the parameters listed above.

[0091] Table 1 shows an example list of parameters related to the dynamic properties of the eardrum. The displacement of the eardrum in response to an applied pneumatic excitation can be expressed by Figure 4 Alternatively, the displacement of the tympanic membrane in response to an applied pneumatic stimulus can be measured by optical coherence tomography.

[0092] Parameters related to the dynamic properties of the tympanic membrane may include the minimum or maximum displacement of the tympanic membrane. The absolute distance the tympanic membrane moves may vary from patient to patient. In general, larger displacements may relate to higher mobility, while lower displacements may relate to lower mobility. These displacements may indicate various classifications. For example, low mobility may relate to a viscous fluid behind the membrane.

[0093] Parameters related to the dynamic properties of the tympanic membrane may include abnormal displacements. In some cases, the measured displacements may be so large or small that they move outside the measurement range of the ultrasound device. This "abnormal" displacement may be treated as a categorical yes / no feature. Large abnormal values ​​generally indicate a highly mobile tympanic membrane. Small abnormal values ​​generally indicate a highly immobile tympanic membrane.

[0094] Parameters related to the dynamic properties of the tympanic membrane may include the difference or ratio between minimum and maximum displacements. The difference between minimum and maximum displacements may relate to the static contraction or expansion of the tympanic membrane. The static contraction or expansion may be measured by measuring the relationship between the distance traveled by the membrane during + and - pressure pulses. An expanded membrane (toward the interrogation device) may be indicated by greater mobility during a positive pressure cycle and less mobility during a negative pressure cycle. A contracted membrane may be indicated by less mobility during a positive pressure cycle and greater mobility during a negative pressure cycle. Such measurements may be used to extract static membrane parameters, such as an expanded membrane or a contracted membrane.

[0095] Parameters related to the dynamic properties of the tympanic membrane may include changes in displacement characteristics, including normalizing any displacement characteristics by the amplitude of the measured pressure pulse, and using any other linear or nonlinear transformation of the displacement and measured pressure since the movement will depend at least in part on the applied pressure stimulus. The metric may be bidirectional, for example, the characterization of tympanic membrane mobility may be more complete if it is known that the metric is under positive pressure stimulus as well as under negative pressure stimulus.

[0096] Parameters related to the dynamic properties of the tympanic membrane can include the slope of the displacement, the slope of the difference between the pressure minimum and maximum displacement relative to the pneumatic excitation, or the slope of the ratio between the pressure minimum and maximum displacement relative to the pneumatic excitation. The speed at which the membrane returns to its original position after being displaced by a pressure pulse can indicate the membrane mobility. The velocity can be measured by the derivative of the displacement measurement described herein. Second or higher order derivatives can also be used. As described herein, the damping coefficient of the oscillation characteristic can also be related to the restoring force of the membrane.

[0097] Parameters related to the dynamic properties of the tympanic membrane may include the response of the measured pressure to the applied pressure. A measured pressure that is less than the applied pressure may indicate membrane mobility. It may also indicate a failure of the instrument seal, which would allow identification of errors in the procedure. Parameters related to the dynamic properties of the tympanic membrane may include the slope of the pressure response. The slope of the pressure may be directly related to the mobility of the tympanic membrane.

[0098] During pressure stimulation, the tympanic membrane can be viewed via a CCD camera. These visual movements and displacements (rates, amplitudes) can be converted into input parameters or features for training a classifier. For example, a user (e.g., a medical expert) can indicate the observed membrane mobility. The user can indicate that the membrane is qualitatively moving or immobile. The user can indicate the mobility of 0-4 points. For example, after applying pressure in the ear canal, a normal tympanic membrane can move more briskly and be classified as 4+, while a partially or completely damaged tympanic membrane can be classified as 3+, 2+, 1+ or no mobility to indicate the degree of mobility reduction at the ordinal level. For example, a medical provider can indicate the flexibility of the tympanic membrane, for example, the speed at which the membrane qualitatively rebounds quickly in response to an applied stimulus. Pressure assessment can be performed by a clinician during a pneumatic otoscope examination.

[0099] Other parameters may be derived from other types of ultrasound signals, which may result from different pre-processing of the ultrasound signal, including but not limited to: bandpass filters of different frequencies, different pulse sequences and measurements, (I,Q) Doppler signal transforms, and data processing steps. Various non-limiting examples include: bandpass filters can be applied to the "fast time domain" captured by the ultrasound signal to remove noise artifacts outside the passband of the ultrasound transducer, thereby improving the signal-to-noise ratio; the phase of the (I,Q) Doppler signal at the depth of the tympanic membrane can be integrated to derive parameters of membrane displacement versus time; the parameters of displacement versus time can be resolved to reveal the response during the change in stimulation pressure of the external auditory canal, and the parameters are evaluated for slope (flexibility or slowness) and position (informing the total change in position) at the beginning and end of the stimulation pressure change; the Hilbert transform can be used to evaluate the amplitude of the ultrasound reflection from the tympanic membrane, and can be evaluated during the stimulation pressure change, to detect changes in light cone orientation under pressure stimulation as a further indication of tympanic membrane mobility; the directionality of membrane motion can be assessed as a quantification of an a priori understanding, for example, the tympanic membrane may move towards the ultrasound transducer in the event of a decrease in pressure in the external auditory canal, and vice versa in the event of an increase in pressure in the external auditory canal; the signal-to-noise ratio of the reflection from the tympanic membrane can be assessed to quantify whether the resulting mobility measurement is valid; and multiple adjacent gates covering a wider depth range than the tympanic membrane signal can be identified and can be monitored simultaneously in order to optimally track the tympanic membrane signal during the period when it is positioned in front of the transducer, and to assess mobility from the optimal gate depth with a qualified tympanic membrane signal.

[0100] Parameters related to the dynamic properties of the tympanic membrane can be derived from Doppler ultrasound data. For example, the Doppler signal of the tympanic membrane as a function of time can include the ultrasonic echo amplitude and the ultrasonic echo phase. These parameters can depend on the specific pulse echo gate depth. Parameters related to the ultrasonic amplitude can include data related to the tympanic membrane sound cross-section and the direction of the tympanic membrane surface normal. Parameters related to the ultrasonic echo phase can include the time derivative of the ultrasonic echo phase. The ultrasonic echo phase can be proportional to the membrane velocity.

[0101] Parameters related to the dynamic properties of the tympanic membrane may include features derived from principal component analysis, K-means, or similar clustering algorithms of any of the above-mentioned data sets. For example, any linear or nonlinear transformation of static data (e.g., principal component analysis, K-means clustering) used to reduce the collinearity of features or cluster similar sample groups based on their respective features can be used to extract one or more parameters. PCA and K-means can be used on various data sets to create parameters. PCA and K-means can be tried when the collinearity between features creates redundant information in the data set. PCA and K-means can reduce the variance of the prediction, or can provide parameters that improve accuracy. In the case of PCA, the principal components can be used as training features by themselves or in combination with untransformed features. Any number of principal components can be selected (e.g., only the first 2-3 or n components, until the explained variance, such as 95%) is met. For K-means clustering, features such as "clusters" can be derived, where the category training features consisting of the cluster group into which each sample falls can be used as parameters in the training model.

[0102] Additional features are generated with respect to time, including moving averages and exponential moving averages over different window lengths.

[0103] Table 1 - Example parameters related to the dynamic properties of the eardrum

[0104]

[0105]

[0106] A set of parameters can include parameters related to the static properties of the film. For example, a medical inquiry system that produces optical measurements can allow extraction of parameters based on visual features. Visual indicators of otitis media can include coloring, transparency and / or projections. Other static parameters can include color, opacity, translucency and position. Visual features from optical measurements can be indicated by the user and can be represented by parameters, such as visible projections or no visible projections.

[0107] In another example, parameters related to static positional properties of the tympanic membrane can be derived from the optical coherence tomography method. For example, the parameters can be related to the depth characteristics of the tympanic membrane. For example, the ear contains a low viscosity infectious effusion, and the initial peak of the tympanic membrane optical coherence tomography signal can produce an axial region of extended reflection, whose amplitude decreases due to optical attenuation of the self-reflected signal. For example, the ear contains a bacterial infection, and a bacterial film can be present on the opposite surface of the tympanic membrane, which can produce a greater axial extent of reflection and then a high scattering coefficient and corresponding increased attenuation. For example, the parameters can be related to the wavelength of the absorption feature of the tympanic membrane. If the tympanic membrane is found to absorb at a wavelength corresponding to a bacterial effusion or a viral effusion or an absence of an effusion, the wavelength of absorption or the identity of the absorption feature can include a parameter in the set of parameters.

[0108] Classification

[0109] The methods and systems of the present disclosure can output a classification of the tympanic membrane (e.g., an output). The classification can be based on a classifier model as disclosed herein, which can be a machine learning algorithm. A set of parameters or a subset of the set of parameters can be used as an input to a classifier model such as a trained machine learning algorithm. The classifier model can output a classification. The classification can include a mobility output. The classification can include a disease state. The classification can include one or more of a state, a condition, or a mobility metric of the tympanic membrane. An output of the classifier model can be generated. The output can include information about TM mobility, which can help a physician diagnose an otitis media infection or directly diagnose a patient's disease state.

[0110] Figure 5A and Figure 5B According to some embodiments, a schematic diagram of an input 510 to a classifier model 520 to output a classified set of parameters derived from pressure, ultrasound and optical measurements may be included. Figure 5A As shown, the amplitude of reflected ultrasound, the slope of the ultrasound response after pressure excitation, the measured pressure and the applied pressure, the slope of the pressure response, and the optical flexibility may comprise inputs to the classifier. The classifier may output a mobility metric 530. Figure 5A As shown, the mobility metric 530 may include a binary classification, such as mobile or not mobile. Figure 5B As shown, the amplitude of reflected ultrasound, the slope of the ultrasound response after pressure excitation, the measured pressure and the applied pressure, the slope of the pressure response, and the optical flexibility can include inputs to the classifier. The classifier can output a state or condition 531, such as acute otitis media, otitis media with effusion, and no effusion, as shown.

[0111] In some cases, the classification may include a measure of the mobility of the tympanic membrane. The measure of mobility may include one or more of a binary classification, an ordinal output, a categorical ordinal output, or a continuous numerical value. The classification may include various forms of tympanic membrane information that may be provided to a medical provider (e.g., a physician). The classification may be based on one or more of ultrasound, pressure, and optical measurement features.

[0112] In some examples, the classification includes a class readout. The class readout can include indicating at least one of a degree of membrane mobility as being highly mobile, moderately mobile, semi-mobile, or non-mobile. For example, the class classification can include highly mobile, moderately mobile, semi-mobile, or non-mobile. In some examples, the class readout includes a binary classification. The binary classification can include mobility or no mobility. The binary classification can include normal mobility or abnormal mobility. In an example, a simple classification can be a binary classifier, where a binary choice based on the maximum probability of "mobility" or "non-mobility" is displayed to the physician.

[0113] In some examples, the classification includes a category ordinal classification. The category ordinal classification may include a 0-4+ scale as described above in this document. In another example, the category ordinal output may include a 0-10 scale, for example, 10.0 is highly mobile, 5.0 is somewhat mobile, and 0 is no movement (or reverse). The user can indicate whether the membrane is qualitatively mobile or immobile. The user can indicate mobility on a 0-4 scale. For example, after pressure is applied in the ear canal, a normal tympanic membrane may move briskly and be classified as 4+, while a partially or completely damaged tympanic membrane may be classified as 3+, 2+, 1+, or no mobility to indicate the degree of reduced mobility on an ordinal scale. The pressure assessment can be performed by a clinician during a pneumatic otoscope examination.

[0114] In some examples, the classification includes a continuous numerical output, which includes one or more of a measured membrane displacement, a membrane movement speed, or a membrane recovery speed. In some examples, the ordinal readout includes a numerical level related to the degree of membrane mobility. The continuous metric can include a numerical level with a tenth increment. In an example of a continuous numerical output, a continuous variable regression algorithm can be used to generate a continuous prediction, for example, a probability percentage of a particular state or condition. In some examples, the state or condition can be normalized to certain values ​​or levels. For example, the continuous metric can be normalized so that the maximum value is 10 and the minimum value is zero, where 10 relates to a highly mobile membrane and 0 relates to a non-mobile membrane, or vice versa.

[0115] In some examples, a multi-class classification can be used. For example, a binary classification can be determined (e.g., mobile or not mobile), and if not mobile is most likely, various degrees of immobility can be classified (e.g., 0 vs. 1+ vs. 2+ vs. 3+ on a 4-point scale). In another example, the classification can include an expected probability for each possible class. The expected probability can include the maximum probability of the classification selected as displayed to the medical provider. For example, the probabilities can be calculated using a softmax function or a strategy such as one-vs-rest classification.

[0116] The classification can indicate the state or condition of the tympanic membrane to the user (e.g., a medical provider). In some cases, the classification can include the state or condition of the tympanic membrane. The state or condition of the tympanic membrane can include one or more of the following: acute otitis media, acute otitis media with effusion, middle ear effusion, bacterial infection, viral infection, no effusion, and unknown classification. The classification of the ear showing acute otitis media can include detecting the presence of effusion and characterizing the type of effusion as one of serous, mucoid, purulent, or a combination of these. In acute otitis media, middle ear effusion can be induced by an infectious agent and may be thin or serous in bacterial infection, and thicker and purulent in bacterial infection.

[0117] Classifier Model

[0118] Figure 6 6 is a schematic diagram illustrating an example 600 of a classifier model including a machine learning algorithm according to some embodiments. The classifier model can be generated based on the set of parameters 610. The set of parameters 610 can include ultrasound features and / or pressure features and / or optical features. For example, the set of parameters can include inputs to the classifier model, and the classification can be an output of the classifier model. The classifier model can output a classification 630. The classification can include a mobility output. The classification can include a disease state.

[0119] The classifier model may include a machine learning algorithm 620. For example, the machine learning algorithm may include one or more of linear regression, logistic regression, classification and regression tree algorithms, support vector machines (SVMs), naive Bayes, K-nearest neighbor algorithms, random forest algorithms, boosting algorithms such as XGBoost and LightGBM, neural networks, convolutional neural networks, and recursive neural networks. The machine learning algorithm may be a supervised learning algorithm, an unsupervised learning algorithm, or a semi-supervised learning algorithm.

[0120] A machine learning algorithm can be used to make predictions using a set of parameters. One type of machine learning algorithm, artificial neural network (ANNS), can include a portion of a classifier model. For example, feedforward neural networks (such as convolutional neural networks or CNNs) and recurrent neural networks (RNNs) can be used. A neural network binary classifier is trained by comparing the predictions made by its underlying machine learning model with the ground truth. The error function calculates the difference between the predicted value and the true value, and the error is iteratively back-propagated through the neural network over multiple cycles or periods to change a set of weights that affect the predicted output value. When the predicted value meets the convergence condition (such as obtaining a small calculation error), the training stops. A multi-layer neural network can be used to create a deep neural network. Using a deep neural network can increase the predictive power of the neural network algorithm. In some cases, a machine learning algorithm using a neural network can also include Adam optimization (e.g., adaptive learning rate), regularization, etc. The number of layers, the number of nodes in a layer, the stride length in a convolutional neural network, padding, filters, etc. can be adjustable parameters in a neural network.

[0121] Additional machine learning algorithms and statistical models can be used to gain insights from the parameters disclosed herein. Additional machine learning methods that can be used are logistic regression, classification and regression tree algorithms, support vector machines (SVM), naive Bayes, K nearest neighbors, and random forest algorithms. These algorithms can be used for many different tasks, including data classification, clustering, density estimation, or dimensionality reduction. Machine learning algorithms can be used for active learning, supervised learning, unsupervised learning, or semi-supervised learning tasks. In the present disclosure, various statistics, machine learning, or deep learning algorithms can be used to generate outputs based on the group of parameters.

[0122] Machine learning algorithms can use supervised learning methods. In supervised learning, the algorithm generates a function or model from training data. The training data can be labeled. The training data may include metadata associated with it. Each training example of the training data may be a pair consisting of at least an input object and a desired output value. Supervised learning algorithms may require the user to determine one or more control parameters. These parameters can be adjusted by optimizing performance on a subset of the training data (e.g., a validation set). After parameter adjustment and learning, the performance of the resulting function / model can be measured on a test set, which can be separate from the training set. Regression methods can be used for supervised learning methods.

[0123] In some embodiments, the supervised machine learning algorithm may include, but is not limited to, a neural network, a support vector machine, a nearest neighbor interpolator, a decision tree, a boosted decision stump, a boosted version of such an algorithm, a derivative version of such an algorithm, or a combination thereof. In some embodiments, the machine learning algorithm may include one or more of the following: a Bayesian model, a decision diagram, an inductive logic programming, a Gaussian process regression, a genetic programming, a kernel estimation, a minimum message length, a multilinear subspace learning, a naive Bayes classifier, a maximum entropy classifier, a conditional random field, a minimum complexity machine, a random forest, an ensemble of classifiers, and a multi-criteria classification algorithm.

[0124] Machine learning algorithms can use semi-supervised learning methods. Semi-supervised learning can combine labeled and unlabeled data to generate appropriate functions or classifiers.

[0125] In some embodiments, the machine learning algorithm may use an unsupervised learning method. In unsupervised learning, the algorithm may generate a function / model to describe a hidden structure from unlabeled data (i.e., a classification or category that cannot be directly observed or calculated). Since the examples given to the learner are unlabeled, there is no evaluation of the accuracy of the structure of the relevant algorithm output. Methods of unsupervised learning include: clustering, anomaly detection, and neural networks.

[0126] Machine learning algorithms can use reinforcement learning methods. In reinforcement learning, the algorithm learns policies about how to act given observations of the world. Each action can have some effect on the environment, and the environment can provide feedback that guides the learning algorithm.

[0127] Although the machine learning algorithms discussed previously can generate more complex relationships between input features and output labels, some simple examples of relational algorithms are shown by the illustrative decision trees in FIG7 .

[0128] Fig. 7A , Figure 7B and Figure 7C is a schematic diagram illustrating an example decision tree that can be generated by a machine learning algorithm according to some embodiments. In some embodiments, the machine learning algorithm constructs a decision tree based on the set of parameters. The decision tree may include thresholds for parameters within the set of parameters. The thresholds may determine which branch of the tree a data set should be classified into. The decision tree may include a maximum information gain at the top node of the tree. The decision tree may be constructed in part by searching for the most informative nodes (e.g., parameters) for a given data set.

[0129] In some examples, the decision tree is pruned. Pruning can include reducing the number of parameters within a set of parameters to a subset of the most relevant parameters. The parameter subset can be the minimum number of parameters that classifies the data set within a specified sensitivity or specificity (e.g., 90% sensitivity or specificity). In some embodiments, the decision tree includes J48, C4.5, or ID3. In some embodiments, the decision tree includes ADABoost or DecisionStump.

[0130] like Fig. 7A As shown, the first example tree includes a single parameter (eg, the absolute value of the displacement). If the parameter is greater than a threshold value of 0.2 mm, the membrane is classified as moving. If the parameter is less than the threshold value of 0.2 mm, the membrane is classified as not moving.

[0131] like Figure 7B As shown, the second example tree includes three parameters (e.g., the absolute value of the displacement with a fixed pressure stimulation amplitude, the absolute value of the displacement with a fixed time measurement window duration, and the abnormal displacement). The classification of the model relates to the mobility of the tympanic membrane. If the absolute value of the displacement with a fixed pressure stimulation amplitude is greater than a threshold of 0.1, and the abnormal displacement is greater than 0.5, the membrane is highly mobile. If the absolute value of the displacement with a fixed pressure stimulation amplitude is greater than a threshold of 0.1, and the abnormal displacement is less than 0.5, the membrane is semi-mobile. If the absolute value of the displacement with a fixed pressure stimulation amplitude is less than a threshold of 0.1, and the absolute value of the displacement with a fixed time measurement window duration is greater than 1.5, the membrane is basically not moving. If the absolute value of the displacement with a fixed pressure stimulation amplitude is less than a threshold of 0.1, and the absolute value of the displacement with a fixed time window duration is less than 1.5, the membrane is completely not moving.

[0132] like Figure 7C As shown, the third example tree includes three parameters (e.g., the absolute value of the displacement with a fixed pressure stimulation amplitude, the absolute value of the displacement with a fixed time measurement window duration, and the abnormal displacement). The classification of the model relates to the state or condition of the tympanic membrane. If the absolute value of the displacement with a fixed pressure stimulation amplitude is greater than a threshold of 0.1, and the abnormal displacement is greater than 0.5, the membrane is characterized as having no effusion. If the absolute value of the displacement with a fixed pressure stimulation amplitude is greater than a threshold of 0.1, and the abnormal displacement is less than 0.5, the membrane is characterized as having otitis media with effusion. If the absolute value of the displacement with a fixed pressure stimulation amplitude is less than a threshold of 0.1, and the absolute value of the displacement with a fixed time window duration is greater than 1.5, the membrane is characterized as having otitis media with effusion. If the absolute value of the displacement with a fixed pressure stimulation amplitude is less than a threshold of 0.1, and the absolute value of the displacement with a fixed time window duration is less than 1.5, the membrane is characterized as having acute otitis media.

[0133] Training a computer-implemented classifier

[0134] Figure 8 A flow chart of an example method of training a computer-implemented classifier according to some embodiments is shown. At operation 810, method 800 may include receiving a set of parameters based on one or more data sets related to one or more tympanic membranes and one or more classified data sets related to one or more tympanic membranes. The classification may include a state, condition, or mobility metric of the tympanic membrane, and wherein the set of parameters includes at least one parameter related to the dynamic properties of one or more tympanic membranes. At operation 820, method 800 may include storing the set of parameters and the one or more classified data sets in a database. At operation 830, method 800 may include building a classifier model based on the set of parameters and the one or more classified data sets. The classifier model is derived from the set of parameters, and wherein the classifier model outputs a classification based on a data set of the one or more classified data sets. At operation 840, method 800 may include using the classifier model to provide a classification of an unclassified data set.

[0135] In some cases, the method further includes updating the database based on the second one or more classified data sets. In some cases, the method further includes updating the classifier model based on the second one or more classified data sets. The method also includes using the classifier model to provide classification of the unclassified data set by the method for classifying the tympanic membrane described herein. The classifier can be continuously updated. For example, the method can use an online gradient descent or stochastic gradient descent method to continuously update the model parameters for improved detection. In some cases, the classifier can be updated with each data set. The classifier can be updated with the most recent sample in the batch.

[0136] Although the above operations illustrate a method 800 for training a computer-implemented classifier according to some embodiments, those of ordinary skill in the art will recognize many variations based on the teachings described herein. These steps may be completed in any order. Steps may be added or deleted. Some steps may include sub-steps. Multiple steps may be repeated as much as possible, as long as it is beneficial to the method of manufacture.

[0137] One or more steps of method 800 may be performed using circuits described herein, such as one or more of a digital processing device or a processor or a logic circuit (e.g., a programmable array logic for a field programmable gate array). The circuit may be programmed to provide one or more steps of method 800, and the programming may include program instructions stored on a computer-readable memory or programming steps of a logic circuit (e.g., a programmable array logic or a field programmable gate array). Embodiments, variations, and examples of operable digital processing devices for performing one or more steps of method 800 are described elsewhere herein, such as with respect to the "Digital Processing Device" section and the descriptions thereof. Fig.10 .

[0138] Fig. 9 is a schematic diagram illustrating an example of a system 900 for training a computer-implemented classifier according to some embodiments. Fig. 9 The system may include a system operable to perform Figure 8 An example of a system of method 800 shown in . System 900 may include one or more classification data sets (e.g., historical data). The one or more classification data sets may include one or more data sets related to one or more tympanic membranes. The classification data set may include data with a known classification. For example, the classification data set may include a data set from a tympanic membrane that has been biopsied. The classification data set may have an associated set of parameters, including at least one parameter based on a dynamic property of the tympanic membrane. The classification data set may have an associated classification. In some cases, a set of parameters (e.g., features) may be extracted from the one or more classification data sets.

[0139] In the next loop of the system 900, all or part of one or more classification data sets can be used as training data. A set of parameters for the classification data set can be used as part of a guided learning algorithm to train a classifier model to find a classification. For example, all or part of the classification data can be used as a training data set for a guided learning algorithm.

[0140] As shown in the upper loop of system 900, a portion of the classified data set can be retained as a validation data set (e.g., test data). For example, through experiments, the validation data can have a set of established classifications. The validation data set can be used to test the predictions of the classifier model. The validation data set can be used to verify the model trained on the training data set. The training data can be used as part of the engineering test loop. For example, a validation process can be used to determine a model or a set of models (e.g., a "set"), where the model is trained on a subset of available data (e.g., training data) and then used to predict the remaining subset of available data (e.g., test data). Accuracy metrics (e.g., root mean square error for numerical data, precision and recall for category data, area under the receiver operating curve (AUC), etc.) can be defined as the accuracy of classifying a given model for a provided data set. In some examples, over- or under-sampling (e.g., artificial minority class oversampling method (SMOTE)) can be used as part of modeling during model training.

[0141] By using as large a representative training dataset as possible, the risk of a machine learning model being unable to generalize to data that the model has never encountered before can be reduced. In addition, utilizing techniques such as hyperparameter tuning with cross-validation to maximize model generalization can improve model generalization. In short, hyperparameter tuning can change aspects of a given model to prevent bias (i.e., underfitting) or variance (i.e., overfitting) with respect to the training and validation data. In addition, cross-validation can be used to continuously train on different subsets of data (e.g., K-Fold and Leave-one-out), and the accuracy predictions generated from each subset can be averaged to create an overall accuracy metric for a given model.

[0142] The trained classifier model can then be subjected to testing on an unclassified data set. For example, the unclassified data set may include one or more tympanic membranes that are not in the classified data set. The classifier model can be used to generate a classification based on a set of parameters extracted from the unclassified data set. The classifier model can be used to classify a membrane as, for example, one of highly mobile, slightly mobile, less mobile, or not mobile.

[0143] A system for training a computer-implemented classifier may include a system described elsewhere herein (e.g., with respect to the "Digital Processing Device" section and the Fig.10 ) embodiments, variations and examples of digital processing devices.

[0144] Digital processing equipment

[0145] In some embodiments, the apparatus, system and method of using the same described herein include a digital processing device or its use. For example, a digital processing device can be used to control various aspects of the apparatus and method disclosed herein. For example, a digital processing device can be used to perform a method for classifying a tympanic membrane. The digital processing device may include a computing system, for example, a computing system including a memory, the memory including instructions for classifying a tympanic membrane. The digital processing device may be configured to perform one or more steps of a method for classifying a tympanic membrane. The digital processing device may be configured to perform one or more steps of method 100 or method 800, as disclosed herein. The digital processing device may be configured to control a medical consultation system, such as any example, variation or example of a medical consultation system disclosed herein. The digital processing device may receive and / or retrieve one or more data sets from the medical consultation system. The digital processing device may include a database management system for one or more data sets. As described herein, the digital processing device may perform one or more steps for training a computer-implemented classifier.

[0146] In other embodiments, the digital processing device includes one or more hardware central processing units (CPUs), general purpose graphics processing units (GPGPUs), or field programmable gate arrays (FPGAs) that perform the functions of the device. In other embodiments, the digital processing device also includes an operating system configured to execute executable instructions. In some embodiments, the digital processing device may be optionally connected to a computer network. In other embodiments, the digital processing device is optionally connected to the Internet so that it can access the World Wide Web. In other embodiments, the digital processing device is optionally connected to a cloud computing infrastructure. In other embodiments, the digital processing device is optionally connected to an intranet. In other embodiments, the digital processing device is optionally connected to a data storage device.

[0147] According to the description herein, suitable digital processing devices include, as non-limiting examples, server computers, desktop computers, portable computers, notebook computers, sub-notebook computers, netbook computers, online tablet computers, set-top box computers, media streaming devices, handheld computers, Internet devices, mobile smart phones, tablet computers, personal digital assistants, video game consoles and vehicles. Those skilled in the art will recognize that a variety of smart phones are suitable for the system described herein. Those skilled in the art will also recognize that selected televisions, video players and digital music players with optional computer network connectivity are suitable for the system described herein. Suitable tablet computers include those with flip cover (booklet) construction, plate construction and convertible construction known to those skilled in the art.

[0148] In some embodiments, the digital processing device includes an operating system configured to execute executable instructions. The operating system is software, including programs and data, that manages the hardware of the device and provides services for executing application programs.

[0149] In some embodiments, the device includes a storage and / or memory device. A storage and / or memory device is one or more physical devices for temporarily or permanently storing data or programs. In some embodiments, the device is a volatile memory and requires power to maintain the stored information. In some embodiments, the device is a non-volatile memory and retains the stored information when the digital processing device is not powered on. In other embodiments, the non-volatile memory includes flash memory. In some embodiments, the non-volatile memory includes a dynamic random access memory (DRAM). In some embodiments, the non-volatile memory includes a ferroelectric random access memory (FRAM). In some embodiments, the non-volatile memory includes a phase change random access memory (PRAM). In other embodiments, as a non-limiting example, the device is a storage device including a CD-ROM, a DVD, a flash memory device, a disk drive, a tape drive, an optical drive, and a cloud computing-based memory. In other embodiments, the storage and / or memory device is a combination of devices such as those disclosed herein.

[0150] In some embodiments, the digital processing device includes a display to send visual information to a user. In some embodiments, the display is a cathode ray tube (CRT). In some embodiments, the display is a liquid crystal display (LCD). In other embodiments, the display is a thin film transistor liquid crystal display (TFT-LCD). In some embodiments, the display is an organic light emitting diode (OLED) display. In various other embodiments, the OLED display is a passive matrix OLED (PMOLED) or an active matrix OLED (AMOLED) display. In some embodiments, the display is a plasma display. In other embodiments, the display is a video projector. In other embodiments, the display is a combination of devices such as those disclosed herein.

[0151] In some embodiments, the digital processing device includes an input device for receiving information from a user. In some embodiments, the input device is a keyboard. In some embodiments, the input device is a pointing device, which includes, as non-limiting examples, a mouse, a trackball, a trackpad, a joystick, a game controller, or a stylus. In some embodiments, the input device is a touch screen or a multi-touch screen. In other embodiments, the input device is a microphone to capture voice or other sound input. In other embodiments, the input device is a camera or other sensor to capture motion or visual input. In other embodiments, the input device is a Kinect, Leap Motion, etc. In other embodiments, the input device is a combination of devices such as those disclosed herein.

[0152] refer to Fig.10 In certain embodiments, an example digital processing device 1001 is programmed or otherwise configured to control or implement a system and method for classifying a tympanic membrane, and a system and method for training a computer-implemented classifier, as described herein. The device 1001 can adjust various aspects of the disclosed system and method for classifying a tympanic membrane, and a system and method for training a computer-implemented classifier, such as, for example, performing processing steps. In this embodiment, the digital processing device 1001 includes a central processing unit (CPU, also referred to herein as a "processor" and a "computer processor") 1005, which can be a single-core or multi-core processor, or a plurality of processors for parallel processing. The digital processing device 1001 also includes a memory or memory location 1010 (e.g., random access memory, read-only memory, flash memory), an electronic storage unit 1015 (e.g., a hard disk), a communication interface 1020 (e.g., a network adapter) for communicating with one or more other systems, and a peripheral device 1025 (e.g., a cache, other memory, data storage, and / or an electronic display adapter). Memory 1010, storage unit 1015, interface 1020 and peripheral device 1025 communicate with CPU 1005 via a communication bus (solid line) (such as a motherboard). Storage unit 1015 may be a data storage unit (or data repository) for storing data. Digital processing device 1001 may be effectively coupled to a computer network ("network") 1030 by means of communication interface 1020. Network 1030 may be the Internet, the Internet and / or an extranet, or an intranet and / or an extranet that communicates with the Internet. In some cases, network 1030 is a telecommunications and / or data network. Network 1030 may include one or more computer servers, which may implement distributed computing, such as cloud computing. In some cases, network 1030 may implement a peer-to-peer network by means of device 1001, which may enable devices coupled to device 1001 to act as clients or servers.

[0153] Continue to refer Fig.10 , CPU 1005 may execute a series of machine-readable instructions that may be embodied in a program or software. The instructions may be stored in a memory location, such as memory 1010. The instructions may be directed to CPU 1005, which may then be programmed to perform the methods of the present disclosure or otherwise configured to perform the methods of the present disclosure. Examples of operations performed by CPU 1005 may include acquisition, decoding, execution, and writeback. CPU 1005 may be part of a circuit (such as an integrated circuit). One or more other components of device 1001 may be included in the circuit. In some cases, the circuit is an application specific integrated circuit (ASIC) or a field programmable gate array (FPGA).

[0154] Continue to refer Fig.10 , storage unit 1015 may store files, such as drivers, libraries, and saved programs. Storage unit 1015 may store user data, such as user preferences and user programs. In some cases, digital processing device 1001 may include one or more additional external data storage units, such as located on a remote server communicating via an intranet or the Internet. Digital processing device 1001 may communicate with one or more remote computer systems via network 1030. For example, device 1001 may communicate with a user's remote computer system.

[0155] Examples of remote computer systems include personal computers (e.g., portable PCs), tablet computers (e.g., iPad, Galaxy Tab), phones, smartphones (e.g. iPhone, Android-enabled devices, ) or a personal digital assistant.

[0156] The methods described herein may be performed by machine (e.g., computer processor) executable code stored on an electronic storage location (e.g., memory 1010 or electronic storage unit 1015) of digital processing device 1001. Machine executable code or machine readable code may be provided in the form of software. During use, the code may be executed by processor 1005. In some cases, the code may be retrieved from storage unit 1015 and stored in memory 1010 for ready access by processor 1005. In some cases, electronic storage unit 1015 may be excluded and machine executable instructions may be stored in memory 1010.

[0157] The digital processing device 1001 may include or communicate with an electronic display 1035, which includes a user interface (UI) 1040. Examples of UIs include, but are not limited to, graphical user interfaces (GUIs) and web-based user interfaces. In some cases, the electronic display 1035 may be connected to the computer system 1001 via a network (e.g., via the network 1030).

[0158] In some embodiments, the platforms, systems, media, and methods disclosed herein include one or more non-transitory computer-readable storage media, which use program coding including instructions that can be executed by an operating system of an optionally networked digital processing device. In other embodiments, the computer-readable storage medium is a tangible component of the digital processing device. In other embodiments, the computer-readable storage medium is optionally removable from the digital processing device. In some embodiments, the computer-readable storage medium includes, as non-limiting examples, CD-ROMs, DVDs, flash memory devices, solid-state memories, disk drives, tape drives, optical drives, cloud computing systems and servers, etc. In some cases, the programs and instructions are permanently, substantially permanently, semi-permanently, or non-temporarily encoded on the medium.

[0159] In some embodiments, the platforms, systems, media, and methods disclosed herein include at least one computer program, or its use. The computer program includes an instruction sequence that can be executed in the CPU of a digital processing device, and the instruction sequence is written to perform a specified task. Computer-readable instructions can be implemented as program modules that perform specific tasks or implement specific abstract data types, such as functions, objects, application programming interfaces (APIs), data structures, etc. Based on the disclosure provided herein, those skilled in the art will recognize that computer programs can be written in various versions of various languages.

[0160] The functionality of the computer readable instructions may be combined or distributed as desired in various environments. In some embodiments, the computer program comprises one sequence of instructions. In some embodiments, the computer program comprises multiple sequences of instructions. In some embodiments, the computer program is provided from one location. In other embodiments, the computer program is provided from multiple locations. In various embodiments, the computer program comprises one or more software modules. In various embodiments, the computer program comprises, in part or in whole, one or more web applications, one or more mobile applications, one or more stand-alone applications, one or more web browser plug-ins, extensions, add-ons or accessories, or a combination thereof.

[0161] In some embodiments, the platform, system, medium and method disclosed herein include software modules, server modules and / or database modules, or use thereof. In view of the disclosure provided herein, the software module is created by using machines, software and languages ​​known in the art through techniques known to those skilled in the art. The software modules disclosed herein are implemented in a variety of ways. In various embodiments, the software module includes a file, a piece of code, a programming object, a programming structure or a combination thereof. In further various embodiments, the software module includes multiple files, multiple pieces of code, multiple programming objects, multiple programming structures or a combination thereof. In various embodiments, the one or more software modules include, by way of non-limiting example, a network application, a mobile application and a standalone application. In some embodiments, the software module is in a computer program or application. In other embodiments, the software module is in more than one computer program or application. In some embodiments, the software module is hosted on a machine. In other embodiments, the software module is hosted on more than one machine. In further embodiments, the software module is hosted on a cloud computing platform. In some embodiments, the software module is hosted on one or more machines located in one location. In other embodiments, the software module is hosted on one or more machines located in more than one location.

[0162] In some embodiments, the platform, system, medium and method disclosed herein include one or more databases or their use. In view of the disclosure provided herein, those skilled in the art will recognize that many databases are suitable for storing and retrieving data sets from the medical inquiry system, storing classified databases, determining parameters from one or more databases, storing parameters associated with classified databases, etc. In various embodiments, suitable databases include, for example, relational databases, non-relational databases, object-oriented databases, object databases, entity relationship model databases, associated databases, and XML databases. Further non-limiting examples include SQL, PostgreSQL, MySQL, Oracle, DB2, and Sybase. In some embodiments, the database is based on the Internet. In further embodiments, the database is based on a network. In further embodiments, the database is based on cloud computing. In other embodiments, the database is based on one or more local computer storage devices.

[0163] The present invention provides embodiments including but not limited to the following:

[0164] 1. A method for classifying a tympanic membrane, the method comprising:

[0165] receiving one or more data sets related to the tympanic membrane from a consultation system;

[0166] determining a set of parameters from the one or more data sets, wherein at least one parameter in the set of parameters is related to a dynamic property or a static position of the tympanic membrane; and

[0167] A classification of the tympanic membrane is output based on a classifier model derived from the set of parameters, wherein the classification includes one or more of a state, a condition, or a mobility measure of the tympanic membrane.

[0168] 2. The method of embodiment 1, wherein the consultation system comprises an imaging system, and wherein the one or more data sets comprise one or more images of the tympanic membrane.

[0169] 3. A method according to embodiment 1, wherein the classifier model includes a machine learning algorithm.

[0170] 4. A method according to embodiment 3, wherein the machine learning algorithm includes one or more of linear regression, logistic regression, classification and regression tree algorithm, support vector machine (SVM), naive Bayes, K nearest neighbor algorithm, random forest algorithm, boosting algorithms such as XGBoost and LightGBM, neural network, convolutional neural network and recursive neural network.

[0171] 5. A method according to embodiment 3, wherein the machine learning algorithm is a supervised learning algorithm, an unsupervised learning algorithm, or a semi-supervised learning algorithm.

[0172] 6. A method according to embodiment 2, wherein the one or more images of the tympanic membrane include one or more ultrasound images.

[0173] 7. The method of embodiment 6, wherein the one or more ultrasound images include images from a pneumatic ultrasonic tympanoscope.

[0174] 8. The method of embodiment 6, wherein the one or more ultrasound images are measured in response to aerodynamic excitation.

[0175] 9. A method according to embodiment 2, wherein the one or more images of the tympanic membrane include one or more optical coherence tomography images.

[0176] 10. The method of embodiment 2, wherein the image comprises an optical image.

[0177] 11. The method of embodiment 2, wherein the at least one parameter related to the dynamic property or static position of the membrane is responsive to aerodynamic excitation.

[0178] 12. The method of embodiment 11, wherein the pneumatic excitation comprises a puff of gas.

[0179] 13. A method according to embodiment 11, wherein the pneumatic excitation has a frequency greater than 10 Hz.

[0180] 14. The method of embodiment 1, wherein the dynamic properties of the tympanic membrane include one or more of the following:

[0181] An indication of membrane movement or membrane mobility;

[0182] the minimum or maximum displacement of the eardrum;

[0183] Abnormal displacement;

[0184] the difference or ratio between minimum and maximum displacements;

[0185] The slope of displacement or the slope of the difference or ratio between minimum and maximum displacements relative to the pressure of the aerodynamic excitation;

[0186] Measuring pressure and response to applied pressure;

[0187] The tympanic membrane moves visually in response to pneumatic actuation;

[0188] One or more statistical components generated from singular value decomposition, principal component analysis, and K-means clustering; and

[0189] Ultrasonic pulse echo amplitude or ultrasonic echo phase or its derivative or its moving average.

[0190] 15. A method according to embodiment 14, wherein the dynamic properties of the tympanic membrane are normalized relative to the pressure of the pneumatic excitation.

[0191] 16. A method according to embodiment 1, wherein the method also includes generating one or more of an ordinal readout, a categorical readout, or a continuous numerical output of the tympanic membrane mobility.

[0192] 17. A method according to embodiment 16, wherein the ordinal readout includes a numerical level related to the degree of membrane mobility.

[0193] 18. A method according to embodiment 17, wherein the numerical scale includes a classification from 0 to 4+.

[0194] 19. A method according to embodiment 16, wherein the category readout includes an indication of the degree of membrane mobility as at least one of highly mobile, moderately mobile, semi-mobile, or non-mobile.

[0195] 20. A method according to embodiment 16, wherein the category readout includes binary classification.

[0196] 21. The method of embodiment 16, wherein the continuous numerical output comprises measuring one or more of membrane displacement, membrane movement speed, or membrane recovery speed.

[0197] 22. A method according to embodiment 1, wherein the state or condition of the tympanic membrane includes one or more of acute otitis media, acute otitis media with effusion, middle ear effusion, chronic otitis media, chronic suppurative otitis media, bacterial infection, viral infection, no effusion, and unknown classification.

[0198] 23. The method of embodiment 1, wherein the one or more data sets include an m-mode ultrasound data set.

[0199] 24. The method of embodiment 1, wherein the one or more data sets include infrared images.

[0200] 25. The method of embodiment 1, wherein the one or more data sets include an aerodynamic data set.

[0201] 26. The method of embodiment 1, wherein the one or more data sets include one or more optical images acquired in response to aerodynamic excitation.

[0202] 27. The method of embodiment 1, wherein the static position comprises an expanded membrane or a contracted membrane.

[0203] 28. A system for classifying a tympanic membrane, the system comprising:

[0204] A computing system comprising a memory including instructions for classifying the tympanic membrane, wherein the computing system is configured to execute the instructions to at least:

[0205] receiving one or more data sets related to the tympanic membrane from a consultation system;

[0206] determining a set of parameters from the one or more data sets, wherein at least one parameter in the set of parameters is related to a dynamic property or a static position of the tympanic membrane; and

[0207] A classification of the tympanic membrane is output based on a classifier model derived from the set of parameters, wherein the classification includes a state, condition, or mobility measure of the tympanic membrane.

[0208] 29. A system according to embodiment 28, wherein the consultation system includes an imaging system, and wherein the one or more data sets include one or more images of the tympanic membrane.

[0209] 30. The system of embodiment 28, wherein the system additionally comprises a pneumatic ultrasonic tympanoscope.

[0210] 31. A system according to embodiment 28, wherein the classifier model comprises a machine learning algorithm.

[0211] 32. A system according to embodiment 31, wherein the machine learning algorithm includes one or more of linear regression, logistic regression, classification and regression tree algorithms, support vector machines (SVM), naive Bayes, K-nearest neighbor algorithms, random forest algorithms, boosting algorithms such as XGBoost and LightGBM, neural networks, convolutional neural networks, and recursive neural networks.

[0212] 33. A system according to embodiment 31, wherein the machine learning algorithm is a supervised learning algorithm, an unsupervised learning algorithm, or a semi-supervised learning algorithm.

[0213] 34. A system according to embodiment 29, wherein the one or more images of the tympanic membrane include one or more ultrasound images.

[0214] 35. A system according to embodiment 34, wherein the one or more ultrasound images include images from a pneumatic ultrasonic tympanoscope.

[0215] 36. A system according to embodiment 34, wherein the one or more ultrasound images are measured in response to pneumatic excitation.

[0216] 37. A system according to embodiment 29, wherein the one or more images of the tympanic membrane include one or more optical coherence tomography images.

[0217] 38. A system according to embodiment 29, wherein the image comprises an optical image.

[0218] 39. A system according to embodiment 29, wherein at least one parameter related to the dynamic properties or static position of the membrane is responsive to aerodynamic excitation.

[0219] 40. The system of embodiment 39, wherein the pneumatic excitation comprises a puff of gas.

[0220] 41. A system according to embodiment 39, wherein the pneumatic excitation comprises a frequency greater than 10 Hz.

[0221] 42. The system of embodiment 28, wherein the dynamic properties of the tympanic membrane include one or more of:

[0222] An indication of membrane movement or membrane mobility;

[0223] the minimum or maximum displacement of the eardrum;

[0224] Abnormal displacement;

[0225] the difference or ratio between minimum and maximum displacements;

[0226] The slope of displacement or the slope of the difference or ratio between minimum and maximum displacements relative to the pressure of the aerodynamic excitation;

[0227] Measuring pressure and response to applied pressure;

[0228] The tympanic membrane moves visually in response to pneumatic actuation;

[0229] One or more statistical components generated from singular value decomposition, principal component analysis, and K-means clustering; and

[0230] Ultrasonic pulse echo amplitude or ultrasonic echo phase or its derivative or its moving average.

[0231] 43. A system according to embodiment 42, wherein the dynamic properties of the tympanic membrane are normalized relative to the pressure of pneumatic excitation.

[0232] 44. A system according to embodiment 28, wherein the computing system is further configured to execute the instructions to generate one or more of an ordinal readout, a categorical readout, or a continuous numerical output of the tympanic membrane mobility.

[0233] 45. A system according to embodiment 44, wherein the ordinal readout includes a numerical level related to the degree of membrane mobility.

[0234] 46. ​​A system according to embodiment 45, wherein the numerical scale includes a classification from 0 to 4+.

[0235] 47. A system according to embodiment 44, wherein the category readout includes an indication of the degree of membrane mobility as at least one of highly mobile, moderately mobile, semi-mobile, or non-mobile.

[0236] 48. A system according to embodiment 44, wherein the category readout includes binary classification.

[0237] 49. A system according to embodiment 44, wherein the continuous numerical output includes one or more of measuring membrane displacement, membrane movement speed, or membrane recovery speed.

[0238] 50. A system according to embodiment 28, wherein the state or condition of the tympanic membrane includes one or more of acute otitis media, acute otitis media with effusion, middle ear effusion, chronic otitis media, chronic suppurative otitis media, bacterial infection, viral infection, no effusion, and unknown classification.

[0239] 51. A system according to embodiment 28, wherein the one or more data sets include an m-mode ultrasound data set.

[0240] 52. A system according to embodiment 28, wherein the one or more data sets include infrared images.

[0241] 53. A system according to embodiment 28, wherein the one or more data sets include a pneumatic data set.

[0242] 54. A system according to embodiment 28, wherein the one or more data sets include one or more optical images acquired in response to pneumatic excitation.

[0243] 55. The system of embodiment 28, wherein the static position comprises an expanded membrane or a contracted membrane.

[0244] 56. A non-transitory computer readable medium comprising machine executable code that when executed by a computing system implements a method for classifying films, the method comprising:

[0245] receiving one or more data sets related to the tympanic membrane from a consultation system;

[0246] determining a set of parameters from the one or more data sets, wherein at least one parameter in the set of parameters is related to a dynamic property or a static position of the tympanic membrane; and

[0247] A classification of the tympanic membrane is output based on a classifier model derived from the set of parameters, wherein the classification includes a state, condition, or mobility measure of the tympanic membrane.

[0248] 57. A non-transitory computer-readable storage medium according to embodiment 56, wherein the consultation system includes an imaging system, and wherein the one or more data sets include one or more images of the tympanic membrane.

[0249] 58. A non-transitory computer-readable storage medium according to embodiment 56, wherein the method also includes the method of any one of embodiments 2-27.

[0250] 59. A method of training a computer-implemented classifier, the method comprising:

[0251] receiving a set of parameters based on one or more data sets related to one or more tympanic membranes and one or more classification data sets related to the one or more tympanic membranes, wherein the classification comprises a state, a condition, or a mobility measure of the tympanic membranes, and wherein the set of parameters comprises at least one parameter related to a dynamic property or a static position of the one or more tympanic membranes;

[0252] storing the set of parameters and the one or more classification data sets in a database;

[0253] constructing a classifier model based on the set of parameters and the one or more classification data sets, wherein the classifier model is derived from the set of parameters, and wherein the classifier model outputs a classification based on a data set of the one or more classification data sets; and

[0254] The classifier model is used to provide classification of the unclassified data set.

[0255] 60. The method of embodiment 59, wherein the method further comprises updating the database based on a second one or more classified data sets; and

[0256] The classifier model is updated based on the second one or more classification data sets.

[0257] 61. A method according to embodiment 59 or 60, wherein the method further comprises using the classifier model to provide classification of an unclassified data set by a method for classifying a tympanic membrane according to any one of embodiments 1 to 27.

[0258] 62. A system for training a computer-implemented classifier, the system comprising:

[0259] A computing system comprising a memory including instructions for training the data set, wherein the computing system is configured to execute the instructions to at least:

[0260] receiving a set of parameters based on one or more data sets related to one or more tympanic membranes and one or more classification data sets related to the one or more tympanic membranes, wherein the classification comprises a state, a condition, or a mobility measure of the tympanic membranes, and wherein the set of parameters comprises at least one parameter related to a dynamic property or a static position of the one or more tympanic membranes;

[0261] storing the set of parameters and the one or more classification data sets in a database;

[0262] constructing a classifier model based on the set of parameters and the one or more classification data sets, wherein the classifier model is derived from the set of parameters, and wherein the classifier model outputs a classification based on a data set in the one or more classification data sets; and

[0263] The classifier model is used to provide classification of the unclassified data set.

[0264] 63. A system according to embodiment 62, wherein the system is configured to execute the instructions to at least: update the database based on a second one or more classified data sets; and update the classifier model based on the second one or more classified data sets.

[0265] 64. The system of embodiment 62 or 63, wherein the system further comprises the system for classifying the tympanic membrane of any one of embodiments 28-55.

[0266] 65. A non-transitory computer readable medium comprising machine executable code that when executed by a computing system implements a method for training a computer implemented classifier, the method comprising:

[0267] receiving a set of parameters based on one or more data sets related to one or more tympanic membranes and one or more classification data sets related to the one or more tympanic membranes, wherein the classification comprises a state, a condition, or a mobility measure of the tympanic membranes, and wherein the set of parameters comprises at least one parameter related to a dynamic property or a static position of the one or more tympanic membranes;

[0268] storing the set of parameters and the one or more classification data sets in a database;

[0269] constructing a classifier model based on the set of parameters and the one or more classification data sets, wherein the classifier model is derived from the set of parameters, and wherein the classifier model outputs a classification; and

[0270] The classifier model is used to provide classification of the unclassified data set.

[0271] 66. A non-transitory computer-readable medium according to embodiment 65, wherein the method further comprises updating the database based on a second one or more classified data sets; and updating the classifier model based on the second one or more classified data sets.

[0272] 67. A non-transitory computer-readable medium according to embodiment 65 or 66, wherein the method further comprises using the classifier model to provide classification of an unclassified data set by a method for classifying a tympanic membrane according to any one of embodiments 1 to 28.

[0273] Although preferred embodiments of the present disclosure have been shown and described herein, it is readily understood by those skilled in the art that these embodiments are provided by way of example only. Without departing from the present disclosure, those skilled in the art will now appreciate that many variations, changes, and substitutions are possible. It should be understood that various alternatives to the embodiments of the present disclosure described herein may be used to implement the invention of the present disclosure. It is intended that the scope of the present invention be limited by the following claims, and that methods and structures and their equivalents within the scope of these claims are encompassed thereby.

Claims

1. A method for classifying a tympanic membrane, the method comprising: receiving one or more data sets related to the tympanic membrane from a consultation system; determining a set of parameters from the one or more data sets, wherein at least one parameter in the set of parameters is related to a dynamic property or a static position of the tympanic membrane; as well as A classification of the tympanic membrane is output based on a classifier model derived from the set of parameters, wherein the classification includes one or more of a state, a condition, or a mobility measure of the tympanic membrane. 2 . The method of claim 1 , wherein the interrogation system comprises an imaging system, and wherein the one or more data sets comprise one or more images of the tympanic membrane.

3. The method of claim 1, wherein the classifier model comprises a machine learning algorithm.

4. The method of claim 3, wherein the machine learning algorithm comprises one or more of linear regression, logistic regression, classification and regression tree algorithms, support vector machines (SVM), naive Bayes, K-nearest neighbor algorithms, random forest algorithms, boosting algorithms such as XGBoost and LightGBM, neural networks, convolutional neural networks, and recursive neural networks.

5. The method of claim 3, wherein the machine learning algorithm is a supervised learning algorithm, an unsupervised learning algorithm, or a semi-supervised learning algorithm.

6. A system for classifying a tympanic membrane, the system comprising: A computing system comprising a memory including instructions for classifying the tympanic membrane, wherein the computing system is configured to execute the instructions to at least: receiving one or more data sets related to the tympanic membrane from a consultation system; determining a set of parameters from the one or more data sets, wherein at least one parameter in the set of parameters is related to a dynamic property or a static position of the tympanic membrane; as well as A classification of the tympanic membrane is output based on a classifier model derived from the set of parameters, wherein the classification includes a state, condition, or mobility measure of the tympanic membrane.

7. A non-transitory computer readable medium comprising machine executable code that when executed by a computing system implements a method for classifying films, the method comprising: receiving one or more data sets related to the tympanic membrane from a consultation system; determining a set of parameters from the one or more data sets, wherein at least one parameter in the set of parameters is related to a dynamic property or a static position of the tympanic membrane; as well as A classification of the tympanic membrane is output based on a classifier model derived from the set of parameters, wherein the classification includes a state, condition, or mobility measure of the tympanic membrane.

8. A method of training a computer-implemented classifier, the method comprising: receiving a set of parameters based on one or more data sets related to one or more tympanic membranes and one or more classification data sets related to the one or more tympanic membranes, wherein the classification comprises a state, a condition, or a mobility measure of the tympanic membranes, and wherein the set of parameters comprises at least one parameter related to a dynamic property or a static position of the one or more tympanic membranes; storing the set of parameters and the one or more classification data sets in a database; constructing a classifier model based on the set of parameters and the one or more classification data sets, wherein the classifier model is derived from the set of parameters, and wherein the classifier model outputs a classification based on a data set of the one or more classification data sets; and The classifier model is used to provide classification of the unclassified data set.

9. A system for training a computer-implemented classifier, the system comprising: A computing system comprising a memory including instructions for training the data set, wherein the computing system is configured to execute the instructions to at least: receiving a set of parameters based on one or more data sets related to one or more tympanic membranes and one or more classification data sets related to the one or more tympanic membranes, wherein the classification comprises a state, a condition, or a mobility measure of the tympanic membranes, and wherein the set of parameters comprises at least one parameter related to a dynamic property or a static position of the one or more tympanic membranes; storing the set of parameters and the one or more classification data sets in a database; constructing a classifier model based on the set of parameters and the one or more classification data sets, wherein the classifier model is derived from the set of parameters, and wherein the classifier model outputs a classification based on a data set in the one or more classification data sets; and The classifier model is used to provide classification of the unclassified data set.

10. A non-transitory computer readable medium comprising machine executable code that when executed by a computing system implements a method for training a computer-implemented classifier, the method comprising: receiving a set of parameters based on one or more data sets related to one or more tympanic membranes and one or more classification data sets related to the one or more tympanic membranes, wherein the classification comprises a state, a condition, or a mobility measure of the tympanic membranes, and wherein the set of parameters comprises at least one parameter related to a dynamic property or a static position of the one or more tympanic membranes; storing the set of parameters and the one or more classification data sets in a database; constructing a classifier model based on the set of parameters and the one or more classification data sets, wherein the classifier model is derived from the set of parameters, and wherein the classifier model outputs a classification; and The classifier model is used to provide classification of the unclassified data set.

Images