System for detecting otitis media acoustically

Abstract

A system for detecting otitis media (OM) acoustically is provided. The system comprising a first acoustic receiver, configured to be placed in a patient 's first ear for receiving utterances of the patient; an acoustic sampling circuitry, configured to connect with the first acoustic receiver and io sample the utterances of the patient as first sound signals; a memory, configured to store one or more trained ML or NN (machine learning or neural network) models; and one or more processors, configured, to inference the one or more ML or NN models with the first sound signals as inputs to determine whether the first ear is with OM or not.

Description

SYSTEM FOR DETECTING OTITIS MEDIA ACOUSTICALLYCROSS REFERENCE TO RELATED PATENT APPLICATION10001 | This patent application is based on a U.S. provisional patent application No. 63 / 540,137 filed on September 25, 2023.FIELD OF THE INVENTION

[0002] The present invention relates to machine learning, and more particularly, to machine learning for detection of otitis media with effusion using in-ear acoustic receiver for patient’s utterances.BACKGROUND OF THE INVENTION

[0003] Otitis media (OM) is a common disease and infection of the middle ear in children. A previous report showed that the annual medical cost of OM treatment was approximately $4 billion in the United States. Otitis media with effusion (OME) is a specific type of OM characterized by middle ear effusion but lacks signs of acute infection. Persistent OME for three months or more, also known as chronic OME, may lead to hearing loss, sleep disruption and balance issues. Additionally, children may have decreased learning efficiency, signs of inattention, and delayed speech and developmental skills. Chronic OME without spontaneous resolution may also cause eardrum structural damage. A typical examination tool is pneumatic otoscopy, which is used to visually assess the tympanic membrane and middle ear. However, a clinician using otoscopy has an accuracy of 50%-70% for diagnosing OM. In particular, OME is often underdiagnosed in clinical environments.

[0004] Various assessment technologies have been applied to support clinical diagnosis, including image-based and acoustic-based approaches. Telescope with video cameras, an image-based approach, is the most common assistive method in clinical environments. It can enhance the diagnostic accuracy, but it still requires professional interpretation, and the high-cost video system is not affordable for all primary clinics. Tympanometry, the acousticbased approach, is a dynamic documentation of middle ear impedance, using increased and decreased air pressure in the external auditory canal and then measuring the associated reflected response recorded by a microphone . However, an airtight seal is required between the probe and the ear canal This may cause manual errors or bias and decrease usability in clinical settings. Moreover, tympanometry cannot be used alone as a diagnostic tool. For example, the result will di ffer if cerumen or a perforation is present. In summary, several challenges and issues limit the capabilities and usability of these assessment technologies. First, OM assessment tools rely on specialists’ interpretation. The diagnostic accuracy of otolaryngologists, pediatricians, and general practitioners is different and depends on their experience and skills. The second challenge is that the high-cost equipment decreases the penetration rate and availability in primary clinics: cost can be the main barrier for primary clinics to use tympanometry in diagnosis. For example, only 7% to 33% of caregivers have pneumatic otoscopy. Therefore, it is essential to develop low- cost, easy-to-use and objective approaches that help primary clinics detect and identify OM.

[0005] In recent years, machine learning (ML) techniques or artificial intelligence (Al) technologies have been widely applied to support clinicalassessment and objectively classify the status of the middle ear. Clinically, ML can enhance current predictions and be a novel tool for helping clinicians assess diseases. In this application. ML is a kind of Al technology. However, the present application does not limit to the ML techniques.[0006 J A. OM Detection Based on Images

[0007] Several assessments based on image detection have been proposed for otitis media. Optical coherence tomography, a cross-sectional imaging technique, combined with ML methods may provide a promising platform with more than 90% accuracy. Viscaino et al, successfully applied ML approaches for external and middle ear assessment with computer vision (digital otoscopy). Their experimental results show that diagnosing external and middle ear conditions using a SVM can achieve 93.9% accuracy. Crowson et al. reported that they employed ML to assist with the diagnosis of otitis media using tympanic images obtained from children. The proposed model achieved an accuracy of 83.8%. Kashani et al. integrated shortwave infrared into an otoscope along with a machine-learning model. The random forest method yielded the h ighest accuracy of 90.3%). In general, the accuracy of image-based detection with ML has satisfactory results. The accuracy of machine learning approaches for diagnosing middle ear disorders using tympanic membrane images has ranged from 76.00% to 98.26%. However, these image-based approaches require high-cost equipment.[00081 B. OM Detection Based on Acoustics

[0009] Compared to image-based detection of OME, research on acoustics-based OME detection is less common. Chan et al. developed an acoustic-based OME classification system using smartphones and logisticregression classifiers. A funnel was designed and placed between the smartphone’s speak er / microph one and the participant’s ear canal, and the raw acoustic waveform was obtained from the ear canal after chirps were played into an ear, with or without middle ear fluid. The svstem used to assess eardrum mobility can achieve 85% sensitivity and 82% specificity. Binol el al. combined otoscopy imaging and tympanogram to detect OME automatically. The proposed decision fusion method using the random forest classifier has an accuracy of 84.9%. Grais el al. employed machine learning techniques to examine the differences between normal ears and those with OME using wideband tympanometry analysis. They used a random forest model to create class activation maps for interpreting diagnostic decisions. These works demonstrated the superiority of ML models in supporting OME assessment in medical practice.

[0010] Therefore, it exists a need for applying artificial intelligence technologies such as machine learning and neural network to provide low-cost and easy-to-use acoustic-based system and method for diagnoses of otitis media and otitis media with effusion, especially for primary clinics and home environment.SUMMARY OF THE INVENTION

[0001] 1 An objective of the present invention is to provide system and method for detecting otitis media acoustically, especially for detecting otitis media with effusion.

[0012] According io an embodiment of the present application, a system for detecting otitis media (OM) acoustically is provided. The system comprising a first acoustic receiver, configured to be placed in a patient’s first ear for receiving utterances of the patient; an acoustic sampling circuitry, configured to connect with the first acoustic receiver and to sample the utterances of the patient as first sound signals; a memory, configured to store one or more trained ML or NN (machine learning or neural network) models; and one or more processors, configured to inference the one or more ML or NN models with the first sound signals as inputs to determine whether the first ear is with OM or not.

[0013] According to an embodiment of the present application, a network system for detecting otitis media (OM) acoustically is provided. The network system comprising a system and a detection server, wherein the system comprises a first acoustic receiver, configured to be placed in a patient’s first ear for receiving utterances of the patient; an acoustic sampling circuitry, configured to connect with the first acoustic receiver and to sample the utterances of the patient as first sound signals; a first network interface configured for connecting with a network; and a first processor, configured to have the first network interlace transmit the first sound signals to the detection server via the network, wherein the detection server comprises a second network interface configured to receive the first sound signals via the network; a memory, configured to store one or more trained ML or NN (machine learning or neural network) models; and one or more second processors, configured to inference the one or more ML or NN models with the first sound signals as input to determine whether the first ear is with OM or not.

[0014] According to an embodiment of the present application, a method for detecting otitis media (OM) acoustically is provided. The method comprising: receiving utterances of a patient by a first acoustic receiver which is configured to be placed in the patient’s first ear; sampling the uterances of the patient as first sound signals: and inferencing one or more trained ML or NN (machine learning or neural network) models with the first sound signals as input to determine whether the first ear is with OM or not.

[0015] The system and method for detecting otitis media acoustically provided by the present application fulfills the needs of applying artificial intelligence technologies such as machine learning and neural network to provide low-cost and easy-to-use acoustic-based system and method for diagnoses of otitis media and otitis media with effusion, especially tor primary clinics and home environment.BRIEF DESCRIPTION OF THE DRAWINGS

[0016] The advantages and spirit related to the present invention can be further understood via the follow ing detailed description and drawings.

[0017] Fig. 1 depicts a schematic framework of the provided OM detection system 100 in accordance with an embodiment of the present application.

[0018] Figs. 2A~2C illustrate the hardware and microphone placements in accordance with an embodiment of the present application.

[0019] Fig. 3 is a diagram shows how to apply Short Termed Fourier Transform to extract acoustic features in accordance with an embodiment of the present application.

[0020] Fig. 4 A shows a health ear. Fig. 4B shows an ear with OME. Figs. 4C and 4D show the magnitude spectrograms of the vowel sound signals from a unilateral OME patient corresponding to the health ear and the ear with OME, respectively.

[0021] Figs. 5A--5D depict diagrams of the accuracy, sensitivity, precision, and specificity of the proposed OME detection approach using ML models and vowel recordings with different / ? values are presented, respectively.

[0022] Fig. 6 depicts a block diagram of a system 600 for detecting Otitis Media (OM) in accordance with an embodiment of the present application,

[0023] Fig. 7 depicts a flowchart diagram of a method 700 for detecting OM in accordance with an embodiment of the present application.

[0024] Fig. 8 depicts a block diagram of an OM detection network system 800 in accordance with an embodiment of the present application.

[0025] Fig. 9 depicts a block diagram of an OM detection network system 900 according to another embodiment of the present application.

[0026] Fig. 10 is a flowchart diagram illustrates a method 1000 for remote sampling in-ear sound signals in accordance with an embodiment of the present application,

[0027] Fig. 11 is a flowchart diagram illustrates a method 1100 for detecting OM based on remote in-ear sound signals in accordance with an embodiment of the present application.DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENT

[0028] Some embodiments of the present application are described in detail below. However, in addition to the description given below, the present invent ion can be applicable to other embodiments, and the scope of the present invention is not limited by such rather by the scope of the claims. Moreover, for better understanding and clarity of the description, some compo nents in the drawings may not necessary be drawn to scale, in which some may be exaggerated related to others, and irrelevant. If no relation of two steps is described, their execution order is not bound by the sequence as shown in the fl owchart diagram.

[0029] Traditionally, the diagnostic accuracy of otitis media with effusion (OME) depends on a clinician's experience and evaluation tools. Various assessment technologies have been applied to support clinical diagnosis, such as digital otoscopy and tympanometry. However, several challenges and issues limit the capabilities and usability of these assessment technologies, including high costs and needing to rely on specialists’ interpretations. In this application, the Inventors designed and validated OME detection using a machine learning model and in-ear microphones.

[0030] In the present application, the microphone may be considered as a type of acoustic receiver or a type of acoustic transducer, which may include piezoelectric materials for transforming characteristics of acoustic waves into electric signals, such as voltage and / or current. Generally, the acoustic receiver is supposed to receive a band in between around 20 hertz and 20,000 hertz. However, the present application does not l imit the resonance frequency range of the acoustic receiver. In the present application, the term “microphone” is equivalent to and interchangeable with the term “acoustic receiver.” Theacoustic receiver may be connected to the acoustic sampling circuitry via physical wires or wireless. For example, the acoustic receiver may have a bluetooth wifeless interface to be connected to the acoustic sampling circuitry. The present application does not limit how the acoustic receiver connects to the acoustic sampling circuitry.

[0031] In one embodimen t, two off-the-shelf microphones were placed in the bilateral ear canals to record the voice when participants pronounced five 3 -second sustained vowel sounds. Person having ordinary skill in the art can understand that the number of vowel and the length of sustained sounds may be varied in different embodiments. Moreover, the sounds may be altered. For examples, the sounds may include consonant and vowel. In the present application, the sustained vowel sounds may represent any combination sound of consonant and vowel. In main propagation paths, the sounds are produced inside the skull of the patient and being transmitted through the middle ear to the ear canals. If there is fluid in one or two of the middle ears of the patient, the propagation of the sounds would be affected by the fluid. Some energy of the sound in a specific frequency spectrum would be absorbed by the fluid.

[0032] Various signal processing and machine learning techniques were applied to the recordings, and the magnitude spectrograms of the sound recording from in-ear microphones can distinguish ears with OME from healthy ears according to the differences in high-frequency response. Results of an embodiment provided by the present application using in-ear microphones and ML algorithms had an accuracy of 80.65% in detecting OME, similar to that of typical OME detection approaches. This applicationdemonstrates the embodiment is an enabler to provide healthcare practitioners with a simple, safe, and more reliable expert-level diagnostic tool.

[0033] One of the aims of this present application is to provide an OME detection system using in-ear microphones and ML techniques to assist clinicians in diagnoses. In one embodiment, the system records the voices of the user using in-ear microphones, extracting acoustic features (e.g., using short-time Fourier transform, STFT, or some analogous extracting methods) and estimating the health status of the middle ear with ML classifiers and weighted-threshold postprocessing. The developed low-cost, easy-to-use and objective approaches have chances to help clinicians or even patients themselves detect and identify OME in primary clinics or in home-based environments.

[0034] In summary, the main contributions of this present application are as follows: (1 ) the provided detection system using in-ear microphones directly records the voices o f the user transmitted through the middl e ear, in contrast to typical acoustic-based assessment tools, which require additional speakers to generate a specific tone for the recording and analysis; (2) off-the-shelf microphones are used for voice recording ins tead of customized design s, which decreases the cost of the equipment and lowers the barrier to accessing OME screening tools for caregivers and physicians; and (3) this application discloses the effectiveness of various ML classification models in automatic OME detection, including support vector machine (SVM), Gaussian Naive Bayes (GNB), adaptive boosting (AdaBoost), random forest (RF), and convolutional neural networks (CNN). Person having ordinary skill in the art can understandthat the disclosed ML models are exemplary. Any suitable ML or Neural Network models may be applicable in the detection system.

[0035] The rest description of this application i ncludes the experimental setups and the processes of the OME detection systems. Moreover, the application presents the experimental results using the provided approach. The performance analysis, limitations, and futures are discussed, and a conclusion is provided.

[0036] The schematic framework of the provided OM detection system 100 in accordance with an embodiment of the presen t application is shown in Error! Reference source not found.. The OM detection system 100 comprises two similar stages. Left hand side is a training stage. Right hand side is an analogous testing stage. Initially, one or more signal preprocessing and feature extraction approaches were applied to the recorded vowel sounds, including Z score normalization and STFT. The OM detection models were trained and developed with one or more various ML techniques and training vowel sounds. Then, in the testing stage, the testing data were input into the trained ML models, and preliminary results were obtained. Finally, the weighted threshold function of postprocessing was employed for the preli minary results to make the final OME decision.

[0037] Following descriptions related to the OM detection system 100 are provided. If there is no direct or indirect causal relationship between any two steps, the present application does not limit the executing sequence of these two steps. Although the OM detection system 100 was used to detect OME, person having ordinary skill in the art can understand that the OM detection system 100 can be used to detect OME, acute OM, chronic OM, and any other kindsof middle ear disease, especially which affects voice transmission through the middle ear.

[0038] Step 110: receiving voice signals and corresponding labels of disease. The voice signals are in a training set.

[0039] Step 120: pre-processing the received voice signals.

[0040] Step 130: extracting features from the pre-processed voice signals.

[0041] Step 140: training one or more ML / Neural Network models according to the extracted features and the corresponding labels of disease.

[0042] Step 150: recei ving voice signals. The voice signals are in a testing set. The voice signals in the testing set are processed through the steps 120 and 130. consequently.[00431 Step 160: inferencing the one or more trained models according to the extracted features to generate outputs.

[0044] Step 170: post-processing the outputs to generate detection results.

[0045] Participants and Experimental Protocol

[0046] A cross-sectional study related to the present application was completed in a tertiary and academic medical center. Adults above 20 years of age who were diagnosed with unilateral or bilateral 0M.E at outpatient departments were enrolled. All patients underwent pure tone audiometry, tympanometry, video telescopy, and nasopharyngoscopy. Patients with a history* of head and neck cancer or midd le ear surgery* were excluded. The study was conducted under a protocol approved by the Taipei Veterans General Hospital Review Boards. All participants provided written informed consent.

[0047] Thirty-one adults diagnosed with OME (20 men and 11 women with a mean age of 60 years (range 26-79 years)) participated in this study.Twenty-five patients had unilateral OME (15 right ear and 1.0 left ear), and 6 had bilateral OME. In summary, there were 25 healthy ears and 37 ears with OM E. Of the 37 ears with OME, 27 had type B tympanograms and 10 had type C tympanograms, with a mean air-bone gap of 25.81 dB HL (Standard Deviation 13.26 dB) in pure tone audiograms (see the appendix paragraphs of the present application). The patient demographics are summarized in Table I.

[0048] Table I: Patient Demographics

[0049] Hardware and Data Recording

[0050] Each participant was invited to pronounce five 3-second sustained vowel sounds, namely, / e: / , / i: / , / aj / 5 / o: / , and / a: / . Two off-the-shelf microphones (Sony Electronic Inc., Tokyo, Japan) sampled at 44.1 kHz were placed at the ear canal entrance to record the voice in the ears. Roland OCTACAPTURE (Roland Cor., Shizuoka, Japan) was applied to synchronize the two microphone signals and transmit the data to a laptop via a USB connection. The participants sat in a quiet office room with an average noise level of 30 dBAand were asked to wear earmuffs (3M1MPELTOR™ X3A) to reduce the environmental noise from the recordings and ensure the acoustic signal quality.

[0051] An illustration of the hardware and microphone placements is shown in Error! Reference source not found. A~2C. Figs. 2A and 2B depict acoustic receivers for right ear and left ear, respectively. Fig. 2C depicts two acoustic receivers, an acoustic sampling circuitry module, and a computer for receiving the sampled data from the sampling circuitry module.

[0052] In an embodiment of the present application, each of the in-ear microphone or acoustic recei ver may include a piece of piezoelectric material as a sensor for transforming characteristics of acoustic waves into electric signals, such as voltage and / or current. The piezoelectric material of the acoustic receiver may be placed in the ear canal of the patient for receiving the voice pronounced by the patient. In another embodiment, the acoustic recei ver may include a one-dimensional array or a two-dimensional array of acoustic sensors. The array of the acoustic receiver can be configured to adjust electronically the receiving direction or focus to the inner ear.

[0053] The acoustic sampling module may be independently designed as shown in Fig. 2C, However, the acoustic sampling module may be bundled w ith the computer in an alternative embodiment. In order to sample the electric signals provided by the piezoelectric material of the acoustic receiver, the acoustic sampling module may include a combination of following well-known components: filter, integral circuit, analog-to-digital converter, DC-.DC converter, adder, oscillator, frequency divider, frequency multiplier, power amplifier, operational amplifier, amplifier etc. fbr fulfilling the acoustic sampling operation and an interface for connecting the computer. The interfacemay be proprietary or compliant with industry standards such as USB series, RS-232, RS-422, IEEE 1394 and etc. Person having ordinary skill in the art can understand that the sampled voice sounds of patient would be transferred to the computer via the interface. The sampled voice sounds may be digitally formatted in one or more waveforms.

[0054] Although the computer as shown in Fig. 2C is a laptop computer, the computer may be any other kinds of computers such as smartphone, tablet, desktop computer, personal digital assistant, or any tailor-made electronics which equips an interface for connecting the acoustic sampling module or the computer itsel f equips the acoustic sampling module.

[0055] Person having ordinary skill in the art can understand the computer may comprise a CPU (central processing unit), a memory, a storage device, input and output devices such as keyboard, mouse, touch panel, touch screen, display screen, speaker. The CPU is responsible for executing an operating system to manage the computer. And one or more application programs are run under the operating system. At least one of the application programs may be supposed to receive the sampled voice sounds in real time from the acoustic sampling module and process the sampled voice sounds to determine the possibility of OME. The application program may be also responsible for showing visual prompts to the patient to remind the patient to pronounce the vowels at right moments. This may help to synchronize the vowel pronouncement and the receiving time duration.

[0056] Alternatively, at least one of the application programs may be supposed to retrieve the sampled voice sounds from the storage device and process the sampled voice sounds to determine the possibility' of OME.

[0057] In order to accelerate the OME detection operations provided by the present application, the computer may optionally include at least one hardware acceleration circuitry such as Graphics Processing Unit (GPU), Auxiliary Processing Unit (APU), Neural -network Processing Unit (NPU) etc. to accelerate the OME detection operations. Collaborating with the CPU, the hardware acceleration circuitry is supposed to accelerate training and inference operations of the ML and / or the neural network models.

[0058] Alternatively, the OME detection operations may be forwarded to a remote computer via a network. Especially, the OME detection embodiments may be improved from time to time when the OME detection embodiments learn more and more cases. Therefore, the computer placed in the primary clinics can only be responsible for the acoustic receiving and sampling. Via the network, the sampled sound signals can be transmitted to a remote server which performs the OME detection operations. After that, the detection result is returned to the computer placed in the primary clinics for reference.

[0059] Data Preprocessing and Acoustic Feature Extraction

[0060] A series of data processing steps were employed to obtain satisfactory acoustic features for OME detection. In the data preprocessing, the raw sound signals were resampled first at 16 kHz (the frequency of a human speaking voice is typically up to 4 kHz). Then, the researchers labeled the starting and ending points for each vowel sound recording. Finally, because of the unstable voice quality, we removed the first and last 0.70 second from the recorded waveform.

[0061] This study related to the present application applied STFT to extract acoustic features (Error! Reference source not found.). The STFT hasbeen widely used in many speech processing applications. For each uterance in the U recording set, the framing technique was first applied to split the input waveform into a series of temporal frames, where the frame size and the hop length for the framing process were 64 and 32 ms, respectively.10(1621 In addition, the Flamming window function was used to smooth the frame boundary. Then, a Fourier transform was applied to provide magnitude spectra for each temporal frame signal. Next, we collected and stacked all the magnitude spectra to generate the acoustic features. Finally, we normalized this spectrogram to zero mean and identify variance to enhance the energy in the high-frequency range.

[0063] ML-based OME detection

[0064] ML classification models were appl ied to acoustic feature vectors to detect OME. As shown in Fig. 1 , ML-based OME detection involves two stages: training and testing stages. In the training stage, the training feature set was composed of pairs of a feature frame vector andthe associated OME label ctrain, which was a binary value. Then, a classification modelwas leveraged to predict the OME label withrespect to the input - The error between thepredicted and was then minimized to optimizeIn thetesting stage, a testing acoustic feature was placed at the input side ofand the classification output was obtained.

[0065] In this study related to the present application, five popular ML classification models were applied to acoustic feature vectors to detect OME: SVM, GNB, AdaBoost, RF, and CNN classification models. These models have been widely used in acoustic-based medical applications, achievingreliable performance. A brief introduction and the important parameters of the applied models are as follows:

[0066] SVM: SVM. is a general classification method in many healthcare and medical diagnosis applications. The effectiveness and usability of SVM have been validated in previous studies. The main objective of SVM is to observe a hyperplane in the feature space to make decisions. In the training stage, the hyperplane is optimized with the maximum margin between two classes. The trained SVM classifies the testing data according to the decision hyperplane. This work applied a linear kernel function and Bayesian optimization to train the SVM model.

[0067] GNB: GNB is a typical identi fication approach based on the Bayes theorem, which assumes that the input features are independent. In the training stage, GNB calculates the mean and the standard deviation of the training data and applies them to the conditional likelihood function. GNB observes the input feature with the largest posterior probability and outputs the class label.

[0068] AdaBoost: AdaBoost aims to build an ensemble of weak classifiers and apply them to final decision making. During the training, each weak classifier literately fits the training data in a specific feature domain to minimize weighted errors. Then, more weights are updated for the classifier that has a lower false prediction rate. The final classification model is a linear combination of the trained weak classifiers. In this study related to the present application, the number of weak classifiers are investigated in a range of 80 to 120 with a step of 10. The best detection performance with 100 weak classifiers is shown in the final results.

[0069] RF: RF is an ensemble learning approach for classification. This approach trains k decision trees with k different training subsets, where the training subset data are randomly sampled from the training set by the bootstrap method. The RF classification model is then constructed based on these trained k decision trees. In the testing stage, the testing data are input into each decision tree, and the final classification output is generated by the majority voting of* k decision trees. In this work, a range of k from 30 to 60 with a step of 10 was tested, and the best performance using k = 50 is presented in the results.

[0070] CNN: C INN is a common type of deep neural network that has been applied in different applications, including image recognition and audio processing. The CNN-based classification model mainly contains ID convolution, batch normalization, dropout, rectified linear unit (ReLU), max pooling, full connection, and softmax. layers. In. the beginning, we apply two CNN blocks to process the inputs, where a CNN block is composed of a I D convolution layer with a kernel number of 32, kernel size of 30, stride size of I , batch normalization, ReLU, max pooling with a size of 2, and dropout (p ~ 0.2). A fter the process of the CNN blocks, we employ the flattening process to the feature maps and connect to one full connection with 32 filters and softmax layers for the detection. The softmax layer calculates the possibility of the target classes. Finally, the classification model outputs the class with the highest probability.

[0071] The Adam optimizer is utilized during the training process. The loss function is cross-entropy. The learning weight and weight decay are 0.0001 and 0.000001, respectively. A total of 40 epochs are applied for model training.and the batch size is 32. The CNN model is implemented on Python 3.9, Py Torch 1.9.1 , and CUDA 1.1.1.

[0072] Postprocessing

[0073] W< sighted threshold postprocessing was applied to the outputs of the ML classifiers for the final 0M.E decision. An efficient postprocessing mechanism can enhance performance in the field of computer-aided diagnostic systems, such as for the detection of Parkinson's disease and lung function diagnosis. The weighted threshold function f was achieved via the following steps: (1 ) The number of positive labels in an utterance was represented as the score Sc(2) The threshold / ?, which varied from 5 to 40, was compared with to assess the positive OME hypothesis of

[0074] Performance evaluati on

[0075] Several evaluation metrics were used to evaluate the performance of the proposed OME detection systems, such as accuracy (ACC), sensitivity (SEN), precision (PRE) and specificity (SPE). The definitions of these metrics are as follows:where TP and TN represent that the number of OME and the healthy ears are correctly identified, while FP and FN determine the misidentification number of OME and healthy ears.

[0076] This study related to the present application applied the leave-one- subject-out (LOSO) cross-validation approach to validate the effectiveness of the proposed OME detection approach. LOSO divided all the data into K subsets based on K subjects involved in the experiments. Then, one subset and the remaining K — 1 subsets were used to test and train the OME detection model. This validation approach was repeated K times until all subsets were tested. Finally, the testing results were averaged over all folds / subjects.

[0077] The proposed OME detection approach was implemented in the Python environment. The data preprocessing (e.g., FFT and feature extraction) and ML models were realized with librosa 0.8.0 music and audio analysis package and scikit-leam 0.24.1 ML toolkit.

[0078] Experimental Results

[0079] The magnitude spectrograms of the vowel sound recording from a unilateral OME patient are shown in Error! Reference source not found.C and 4D. The results showed that the frequency response >2 kHz was lost in OME patients, while it was maintained in healthy ears. Thus, the voices recorded from the healthy ear retained most high-frequency responses to voice, as shown in Error! Reference source not found.A and 4C.

[0080] The high-frequency features of sounds were eliminated in the ear with OME, as shown in Error! Reference source not feund.B and 4D, because effusion in the middle ear space had a considerable impact on voice transmission, which led to the loss of consonant components. Such distinguished differences in high-frequency regions enabled the proposed system to distinguish ears with OME from healthy cars.

[0081] The accuracy, sensitivity, precision, and specificity of the proposed OME detection approach using ML models and vowel recordings with differentvalues are presented in Error! Reference source not found.A~5D. In general, most ML models, including SVM, AdaBoost, RF, and CNN, usingfrom 10 to 23 achieved the best accuracy, and their sensitivity increased when was greater than 5. In contrast, their precision andspecificity decreased notably withft , particularly for SVM and AdaBoost. For the GNB model, the accuracy and the sensitivity improved when ft was larger than 20.

[0082] Table II presents the best accuracy with the corresponding ft values of the ML models using all vowel sounds and five single vowel sounds ( / e: / , / i : / , / aj / , / o: / , and / a: / ). The proposed OME detection approach using the CNN model with ft = 23 achieved the best accuracy of 80.65%, while those achieved using typical ML models were lower than 75.00%. Similarly, the CNN-based OME detection system achieved the best specificity of 87.8%. The RF model with ft = 13 achieved the best sensitivity of 94.59% but the worst specificity of 16.32%. Additionally, the detection system achieved the best accuracy of 77.42% with the single vowels / i:A For the OME detection systems using single vowels, the RF model had the best performance. Overall, most ML models using single vowel sounds had high sensitivity and moderate precision, while their specificity was relatively low.[00831 Table 11: The Best Detection Performance of Different ML and / or NN Models

[0084] DISCUSSION

[0085] Accurate diagnosis of otitis media is important; however, the diagnostic accuracy depends on the experience of the clinician and evaluation tools. Otolaryngologists had a better correct diagnostic rate than pediatricians and general practitioners. The correct diagnosis rates of video-presented OME, acute otitis media and retracted tympanic membrane among otolaryngologists, pediatricians, and general practitioners in the US were 74%, 51%, and 46%, respectively. Pichichero and Poole evaluated 188 otolaryngologists and 514 pediatricians for their diagnostic accuracy of otitis media, and the correct diagnosis rates by otolaryngologists and pediatricians were 73% and 50%, respectively. Jones and Kaleida reported that pneumatic otoscopy accuracy for OME was 76% and that static otoscopy accuracy for OME was 61% in 34 pediatric residents, 4 pediatricians and 2 pediatric otolaryngologists. Both pneumatic otoscopy and video-telescopy require professional interpretation. The experiment using in-ear acoustic receivers and ML algorithms had an accuracy of 80.65% in detecting OME, similar to that of typical OME otoscopy detection approaches. Such detection accuracy shows the low-cost and easy-to- use benefits of the provided approach in primary clinical practice and in home environment.11)0861 Tympanometry is a rapid and simple examination to assess the middle ear condition. According to the American Academy of Otolaryngology- Head and Neck Surgery clinical practice guidelines, tympanometry should be performed when the diagnosis of OME is uncertain after performing pneumaticotoscopy. However, the equipment for tympanometry is relatively expensive and requires a referral to a clinician. The OME detection system provided by the present application records the voices of the user using commercial in-ear microphones, extracts acoustic features with STFT, and estimates the health status of the middle ear with ML classifiers and rule-based postprocessing. This low-cost and easv-to-use OME detection system is usable and beneficial for nonmedical caregivers and primary clinicians.

[0087] To support the clinical assessment of OME. this study proposed an OME detection system using in-ear microphones and ML models. Our analysis showed that the proposed OME detection system using an ensemble learning approach could achieve better detection performance than that using a single strong classification model. For example, the system using the CNN model and all vowel sounds achieved the best accuracy of 80.65%. Similar trends showed that the CNN model was suitable for the sy stem using single vowel records. Several previous studies have shown the superiority of ensemble learning methods for classification problems. The effectiveness of ensemble models has been validated in different applications. Additionally, we demonstrated that the weighted threshold function can improve the detection performance since the detection system is sensitive to OME diseases and avoids misclassifying healthy ears as OME. Suitable weight tuning can enhance the correct rejection ability of the system.

[0088] The present application may be the first work to provide an OME detection system using in-ear acoustic receivers and ML techniques to support clinical diagnosis and assessment. Previous works have demonstrated the feasibility of ML models for acoustic-based OME detection. However, they hadto use both acoustic receivers and speakers for disease detection. In contrast to previous works, the provided approach required microphones only for voice recording and analysis. Furthermore, similar to previous work, this application preliminarily validated the feasibility of off-the-shelf acoustic receivers in assessing the health status of the middle ear. These advantages and experimental results show that the provided OME detection system has the potential to provide a low-cost, easy-to-use, and user-friendly screening tool for clinical evaluation outside of the examination room.

[0089] C urrently, the provided proof-ol-concept prototype of OME detection system has several limitations. First, individual differences in pronunciation, sex, and age caused high variance and diversity in the voice recording. For example, the fundamental frequency was different for females and males. This challenge limits the OME detection performance. Second, previous works have demonstrated that different design and environmental factors affect system reliability, including background noise and wearing position. Furthermore, it is necessary to validate the intra- and interrater reliability for OME detection. Finally, the provided prototype did not classify the types of middle ear fluid, such as purulent, serous, and mucoid. These detailed disease types would support clinical professionals for more robust assessment and evaluation.

[0090] In an embodiment of the present application, more types of OM (e.g., acute OM and chronic OM) and their subtypes would be distinguishable in a robust disease detection system. Various acoustic feature extraction techniques (e.g., wavelet transform and empirical mode decomposition) and advanced deep learning techniques (e.g., transformers and self-supervisedlearning) may be applied to improve the detection accuracy and address the technical issues of individual differences. Furthermore, the impacts of extrinsic factors on the system performance, including wearing positions, background noise, and microphone type would be also considered in the ear disease detection system.

[0091] Telehealth increased during the coronavirus disease 2019 (COVID-19) pandemic, with many clinicians using video or telephone visits to provide medical consultation and to reduce the risk of exposure to the virus. Our ultimate goal is to develop a smartphone-based OM detection system using derived algorithms and models. The use of smartphone-based telemedicine can provide health care practitioners with a simple, safe, and reliable tool to detect middle ear disease.

[0092] CONCLUSION

[0093] To develop 0.ME diagnostic tools for caregivers and primary clinicians in home / clinical environments, this application provides a low-cost, easy-to-use, and reliable OME detection system using off-the-shelf microphones and ML models. The experimental results show that the proposed OME detection system can achieve 80.65% ACC, 66,67% SEN, 73.68% PRE, and 87.8% SPE. It is demonstrated that in-ear acoustic receiver is sufficient to support OME detection, and it can help to develop telemedicine in pandemic and post-pandemic health care delivery.

[0094] APPENDIX

[0095] Appendix A

[0096] Tympanometry with a 226 Hz stimulus is a standard audiological procedure to evaluate middle ear function. In the absence of any objects in theear canal and with an intact eardrum, there are three main types of tympanograms: type A, type B, and type C. A tympanogram provides information regarding the compliance of the middle ear system. The horizontal x-axis of a tympanogram measures the pressure in the ear (dPa) while the vertical y-axis represents the compliance (ml), which is the ability of sound to pass through the eardrum to the middle ear. The tympanogram also provides information about the ear canal volume (cm3.)

[0097] 1. A normal, or type A, tympanogram typically displays a peak in compliance that falls within the range of 0 to 100 dPa in the ear canal. Both the location and height of this peak must fall within the normal range on the pressure dimension for the recording to be considered normal.

[0098] 2. Type B tympanogram exhibits a flat pattern with little or no apparent change in compliance as pressure is applied to the ear canal, as opposed to a distinct peak in compl iance. This pattern is frequently indicative of fluid within the middle ear space, known as otitis media.

[0099] 3. Type C tympanogram exhibits a compliance peak similar to that of type A recordings, except that the peak falls within the negative pressure region beyond approximately 100 dPa. This pattern is typically observed in patients with eustachian tube dysfunction and insufficient ventilation of the middle ear space. It often precedes the acquisition of a type B tympanogram during the development of otitis media.

[0100] Appendix B

[0101] The mean of pure tone frequencies refers to the average of hearing threshold levels at a specific set of frequencies, typically including 500, 1000, 2000, and 4000 Hz.

[0102] Please refer to Fig. 6, which depicts a block diagram of a system 600 for detecting Otitis Media (OM) in accordance with an embodiment of the present application. The system 600 may be a mobile computer, such as smartphone, laptop computer, tablet computer, or any computer. Preferably, the cost of the system 600 is low enough to be owned by ordinary person in modern world.

[0103] The system 600 may comprise a processor 610, a memory 620, a storage device 624, an acoustic sampling circuitry 630, a first in-ear acoustic receiver 641, a second in-ear acoustic receiver 642, a first earmuff 651, a second earmuff 652, a touch screen 660, a wireless telecommunication device 670, and an optional battery 680. The processor 610 may be a CPU which runs an operating system (OS) and application programs under the OS for implementing embodiments of the present application. For example, the system 600 may be an Android smartphone or an iPhone which run Android series and iOS series operating systems, respecti vely.

[0104] The memory 620 may include a dynamic random access memory for system operation. The storage device 625 may comprise a non-volatile memory such as flash memory, EEPROM memory, hard disk drive, optical disk drive for storing the OS, application programs, one or more abovementioned machine learning ML models, and the computer instructions which implement the method provided by the present application.

[0105] The acoustic sampling circuitry 630 and the first in-ear acoustic receiver 641 is supposed to record utterances of a patient in the patient’s first ear. Similarly and optionally, the acoustic sampling circuitry 630 and the second in-ear acoustic receiver 642 is supposed to recorded the utterances inthe patient’s second ear. The in-ear acoustic receivers 641 and 642 and the acoustic sampling circuitry 630 collectively perform transducing and sampling acoustic waves into electronic signals or voice sounds represented in a digital waveform. The sampling rate may be 44 J kHz or below which may be one of commercial or off-the-shelf standard sampling rates,

[0106] While recording the utterances, the first and the second earmuff 651 and 652 may be configured to cover the first and the second ears of the patient for reducing environmental noises. Optionally, the system 600 may comprises a touch screen 660 as a major input / output device, and a wireless telecommunication device 670 which is compliant to modem 2G / 3G / 4G / 5G and any other industrial standards.

[0107] The touch screen 660 may be used to show- prompt to the patient for pronouncing vowels. After seeing the visual prompts, the patient may touch the touch screen 660 and begin the corresponding vowel pronunciations. The visual prompt may also show a count-down number of seconds to tell how long the vowel pronunciation should be lasted. After the number of seconds reduces to zero, the patient can stop the vowel pronunciation. Meanwhile, the acoustic receivers 641 and 642 as well as the acoustic sampling circuitry 630 are supposed to sample the utterances of patients. The samples may be saved in the storage device 625 for record,

[0108] Alternatively, the system 600 may have other input and output devices instead of the touch screen 660. Person having ordinary skill in the art can understand that the prompting and sampling procedure can be done by other kinds of input and output devices such as lights and buttons.

[0109] In one embodiment, the system 600 can detect whether the samples of the utterances are good enough. If the utterance is interrupted or the patient does not pronounce in the prompted time duration, the system 600 would repeat the prompting and sampling procedure until the samples are ready.

[0110] The battery 680 may be included in the system 600 for providing energy for the operations for the system 600. Therefore, the system 600 can be transportable.

[0111] The processor 610 and / or optional GPU, NPU, APU processing units are configured to execute the application program which implements the provided invention.

[0112] Please refer to Fig. 7, which depicts a flowchart diagram of a method 700 for detecting OM in accordance with an embodiment of the present application. The method 700 may be embodied in the system 600 as shown in Fig. 6. The method 700 may be implemented as a plurality of computer instructions which are executable by the processor 610 and / or optional GPU, NPU, APU. If any two steps have no causal relation directly or indirectly, the present application does not limit the execution order of these two steps. The method 700 may begin at step 710 when the application program is executed by the processor 610.

[0113] Step 710: providing prompts to a patient and / or a clinician. The prompts may comprise visual and audio prompts provided by the touch screen 660 and speakers, respectively. The content of prompts may indicate one of the expected vowels and its counting seconds to pronounce.

[0114] Step 720: sampling first sound signals which are corresponding to utterances of the patient by a first acoustic receiver which is placed in thepatient’s first ear. Preferably, the uterances are the expected vowels, respectively. Preferably, the first ear is covered with a first earmuff for reducing environmental noise. Preferably, the first sound signals are sampled at 44.1 kHz or below.

[0115] Optional step 725: sampling second sound signals which are corresponding to utterances of the patient by a second acoustic receiver which is placed in the patient’s second ear. Preferably, the utterances are the expected vowels, respectively. Preferably, the second ear is covered with a second earmuff for reducing environmental noise. Preferably, the second sounds are sampled at 44. 1 kHz or below.

[0116] In one embodiment, the first sound signals and the second sound signals are sampled at the same time. Therefore, the flow may proceed to the optional step 730. If only one acoustic receiver is applied, the flow may proceed to the step 735

[0117] Optional Step 730: synchronizing the first and the second sound signals.

[0118] Step 735: determining whether the first and / or the second sound signals are good. For example, the determining step may inspect whether the volume of the first and / or the second sound signals are large enough. If so, the flow1proceeds to the step 740. Otherwise, the flow may return to the step 710.

[0119] Step 740: preprocessing the first and / or the second sound signals. Preferably, the step 740 may comprises resampling the firsV'second sound signals at 16 kHz. Preferably, the step 740 may comprises retrieving a middle section of each of the first / second sound signals as stable quality of the inputs. Preferably, the preprocessing may include one or any combination of following:Z score normalization; and STFT (short-term Fourier Transformation). Preferably, the preprocessing may include framing the first sound signals into a series of temporal frames; applying Flamming window function to the series of temporal frames; applying Fourier transform to generate magnitude spectra of each of the temporal frames; stacki ng all the magnitude spectra to generate a spectrogram with acoustic features; normalizing the spectrogram to zero mean; and identi fying variance to enhance the energy in a high-frequency range.

[0120] Step 750; inferencing one or more ML / Neural Network models with the first and / or the second sound signals as inputs and labels as outputs of their corresponding classifiers, respectively. Preferably, the ML / Neural Network models are supposed to detect at least one of following OM: Otitis Media with Effusion; Acute Otitis Media; Acute Otitis Media with Effusion; Chronic Otitis Media; and Chronic Otitis Media with Effusion. Preferably, the ML model is supposed to detect a type of middle ear fluid, wherein the type of middle ear fluid is one of following: purulent fluid; serous fluid; and mucoid fluid. Preferably, the ML / Neural Network models are a combination of following types: SVM; GNB; AdaBoost; RF; and CNN.

[0121] Step 760: post-processing the outputted labels to determine the fi rst and / or the second ears is with OM or not. The step may include calculating a score according to each of classified labels outputted from the ML Neural Network models corresponding to each of the utterances; comparing the score with a threshold; and when the score is larger than the threshold, it is determined that the first and / or the second ears are with OM according to the classified labels. Preferably, each of the labels are weighted.

[0122] Optional step 770: display the detection results.

[0123] Please refer to Fig. 8, which is an OM detection network system 800 in accordance with an embodiment of the present application. The OM detection network system 800 includes a centralized OM detection server 820 which embodies the trained ML / Neural Network models. The OM detection server 820 is reachable by one or more systems 600 and at least one clinician computer 840 via a network 810. The network 810 may be a combination of a telecommunication network (e.g., fixed line networks and wireless mobile communication networks compliant with 3G, 4G, 5G, 6G and any other industrial standards) and wired or wireless access networks (e.g., compliant with Wireless LAN, WiFi, IEEE 802.1.1 series, and IEEE 802.3 series and any other industrial standards).

[0124] Similar to the system 600, the OM detection server 820 may includes a CPU, a memory, a storage device, and a network interface for connecting to the network 810. Optionally, the OM. detection server 820 may further comprise GPU, APU, N.PU and any other kinds of acceleration hardware for inferencing and / or training the ML / Neural Network models. In an alternative embodiment, the OM detection networking system 800 may further comprises one or more Al computational server 825 which connects to the OM detection server 820 via the network 810 or an internal network which is invisible from outside. Equipped with the GPU, APU, NPU and any other kinds of acceleration hardware, the Al computational server 825 may be served to provide computation services for the OM detection server 820, especially provide computation resources for inferencing and / or training the ML / Neural Network models.

[0125] Furthermore, the OM detection network system 800 may further comprises a database server 830, which may be a database management system such as relational DBMS or non-relational DBMS, The database server 830 may be configured to store samples of the first and / or the second sound signals of the patients. Moreover, the database server 830 may further store labels of diseases corresponding to the samples. These samples and their corresponding labels may be further inputted into the training stage of the OM detection system 100 as shown in Fig. 1 for refining the trained ML / Neural Network models. In addition, these samples and their corresponding labels may be further inputted into the testing stage of the OM detection system 100 as shown in Fig. 1 for testing the trained ML / Neural Network models.

[0126] Similar to the system 600, the clinician computer 840 may include a CPU, a memory, a storage device, a display, a speaker, a printer, and a network interface for connecting to the network 810. The clinician computer 840 may be used to recei ve the historical detection results from the detection server 820 or the database server 830. Moreover, the clinician computer 840 may further receives the first and / or the second sound signals from the detection server 820 or the database server 830 for replaying. The clinician computer 840 can show the detection results on the display or print it out by the printer.

[0127] Person having ordinary skill in the art can understand the access of the patient’s information and the detection results by the detection server 820, the database server 830 and / or the clinician computer 840 must be authenticated and authorized via secure means or procedures.

[0128] Telehealth function can be achieved if the system 600 (i.e., patient) and the clinician computer 840 (i.e.. clinician) are located at different locations.>5The ML / Neural Network models stored in the detection server 820 and / or the Al computational server 825 may be improved because new pairs of samples and labels of diseases are collected for further trai nings.

[0129] However, it takes a higher cost to establish the OM detection network system 800. Please refer to Fig. 9, which depicts of an OM detection network system 900 according to another embodiment of the present application. Comparing with the OM detection network system 800, the cost of the OM detection network system 900 can be reduced significantly because only the system 600 and the clinician computer 840 are involved. The trained ML / Neural Network models can be stored in the clinician computer 840. The system 600 is only responsible tor sampling the first and / or the second sound signals of the patient. Telehealth function can be achieved if the system 600 (i.e., patient) and the clinician computer 840 (i.e., clinician) are located at different locations, too.

[0130] Please refer to Fig. 10, which illustrates a method 1000 for remote sampling in-ear sound signals in accordance with an embodiment of the present application. The method 1000 may be applicable to the system 600 of the OM detection network 800 as shown in Fig. 8 or the system 600 of the OM detection network 900 as shown in Fig. 9. Especially, the method 1000 may be implemented as data and instructions of the processor 610 of the system 600 stored in a non-volatile memory. The method 1000 may begin at step 710. When the step 735 determines that the sampled first and / or the second sound signals are good, the flow proceeds to step 1040.

[0131] Step 1040: transmitting the first and / or the second sound signals via a network. When the method 1000 is applied to the OM detection networksystem 800, the first and / or the second sound signals are transmitted to the OM detection server 820. When the method 1000 is applied to the OM detection network system 900, the first and / or the second sound signals are transmitted to the clinician computer 840. After that, the flow may end here. Otherwise, the flow may further proceed to the optional steps 1050 and 770.

[0132] Optional step 1050: receiving the detection results corresponding to the transmitted first and / or the second sound signals via the network. When the method 1000 is applied to the OM detection network system 800, the detection results are transmitted from the OM detection server 820. When the method 1000 is applied to the OM detection network system 900, the detection results are transmitted from the clinician computer 840. After that, the flow may end here. Otherwise, the flow may further proceed to the optional step 770.

[0133] Please refer to Fig. 11, which is a flowchart diagram illustrates a method 1100 for detecting OM based on remote in-ear sound signals in accordance with an embodiment of the present application. The method 1100 may be applicable to the detection server 820 as shown in Fig. 8 or to the clinician computer 840 as shown in Fig. 9. Especially, the method 1 .100 may be implemented as data and instructions of a processor of the detection server 820 or the clinician computer 840 stored in a non-volatile memory. The method 1100 may begin at step 1110.

[0134] Step 1110: receiving first and / or second sound signals of utterances of a patient via a network. The flow may proceed to steps 740, 750, 760, and 770 subsequently. The flow may proceed to optional step 1120 or optional step 1130.

[0135] Optional step 1120: transmitting the detection result generated at step 760 via the network to the system 600.

[0136] Optional step 1130: saving the first and / or the second sound signals and their corresponding detection results to a database server.

[0137] According to an embodiment of the present application, a system for detecting otitis media (OM) acoustically is provided. The system comprising a first acoustic receiver, configured to be placed in a patient’s first ear for receiving utterances of the patient; an acoustic sampling circuitry, configured to connect with the first acoustic receiver and to sample the utterances of the patient as first sound signals; a memory, configured to store one or more trained ML or NN (machine learning or neural network) models; and one or more processors, configured to inference the one or more ML or NN models with the first sound signals as inputs to determine whether the first ear is with OM or not.

[0138] Preferably, in order to detect various kinds of OM, wherein the OM to be determined is selected from one of following: otitis media with effusion (OME); acute otitis media (AOM); acute otitis media with effusion (AOM'E); chronic otitis media (COM); and chronic otitis media with effusion (COME).

[0139] Preferably, in order to detect the type of the middle ear fluid, wherein the one or more trained ML or NN models are trained to determine a type of first ear’s fluid, wherein the type includes one of following: purulent fluids; serous fluid; and mucoid fluid.

[0140] Preferably, in order to detect the OM, the one or more ML or NN models is selected from one of following: SVM; GNB; AdaBoost; RF; andCNN.

[0141] Preferably, in order to provide accurate and powerful enough sound sources, the utterances are one of followings: vowels; and a combination of consonant and vowels.

[0142] Preferably, in order to reduce environmental interference, the system further comprises a first earmuff wh ich is configured to cover the first ear.

[0143] Preferably, in order to provide realistic sound signals, the acoustic sampling circuitry is configured to sample the utterances at 44.1 kHz or below as the first sound signals.

[0144] Preferably, in order to fit into the ML or NN models, the processor is further configured to resample the first sound signals at 16 kHz.

[0145] Preferably, in order to have a stable quality of input, the processor is further configured to preserve a middle section of each of the first sound signals,

[0146] Preferably, in order to pre-process the first sound signals, the processor is further configured to pre-process the first sound signals to extract acoustic features by one or any combination of following: Z score normalization; and short-term Fourier transform (STFT.)

[0147] Preferably, in order to apply STFT to extract the acoustic features, the processor is further configured for: framing the first sound signals into a series of temporal frames; applying Hamming window function to the series of temporal frames; applying Fourier transform to generate magnitude spectra ofeach of the temporal frames; stacking all the magnitude spectra of each of the temporal frames to generate a spectrogram with acoustic features; normalizing the spectrogram to zero mean; and identifying variance to enhance the energy in a high-frequency range.

[0148] Preferably, in order to post-process the outcomes of inferencing, the processor is further configured for: calculating a score according to each of classified labels outputted from the one or more trained ML or NN models corresponding to each of the utterances; comparing the score wi th a threshold; and when the score is larger than the threshold, determining that the first ear is with OM.

[0149] Preferably, in order to accurately determine the outcome of inferencing, wherein each of the classified labels are weighted.

[0150] Preferably, in order to have bilateral ear detection at one time, the system further comprises: a second acoustic receiver configured to be placed in a patient’s second ear for recei ving the utterances of the patient, wherein the acoustic sampling circuitry is configured to connect with the second acoustic receiver and to sample the utterances of the patient as second sound signals, and wherein the one or more processors, configured to inference the one or more ML or NN models with the second sound signals as input to determine whether the second ear is with OM or not.

[0151] Preferably, in order to provide prompts to the patient and / or his or her clinician, wherein the system further comprises at least one of following: a display configured to provide visual prompt to have the patient pronounce the utterance; and a speaker configured to provide audio prompt to have the patient pronounce the utterances and to replay the utterances.

[0152] According to an embodiment of the present application, a network system for detecting otitis media (OM) acoustically is provided. The network system comprising a system and a detection server, wherein the system comprises a first acoustic receiver, configured to be placed in a patient’s first ear for receiving utterances of the patient; an acoustic sampling circuitry, configured to connect with the first acoustic receiver and to sample the utterances of the patient as first sound signals; a first network interface configured for connecting with a network; and a first processor, configured to have the network interface transmit the first sound signals to the detection server via the network, wherein the detection server comprises a second network interface configured to receive the first sound signals via the network; a memory, configured to s tore one or more trai ned ML or NN (mach ine learning or neural network) models; and one or more processors, configured to inference the one or more ML or NN models with the first sound signals as input to determine whether the first ear is with OM or not.

[0153] Preferably, in order to have the patient informed, the system further comprises a display, wherein the detection server is further configured to transmit a detection result back to the system, wherein the first processor of the system is further configured to have the display show the detection result.

[0154] Preferably in order to have a remote clinician of the patient informed, the network system further comprises a clinician computer with a display, wherein the detection server is further configured to transmit a detection result to the clinician computer, wherein the clinician computer is further configured to have the display show the result.

[0155] Preferably, in order to preserve the samples for further training, the network system further comprises a database management system, wherein the detection server is further configured to transmit the first sounds and a detection result to the database management system for storage.

[0156] According to an embodiment of the present application, a method for detecting otitis media (OM) acoustically is provided. The method comprising: receiving utterances of a patient by a first acoustic receiver which is configured to be placed in the patient’s first ear; sampling the utterances of the patient as first sound signals; and inferencing one or more trained ML or NN (machine learning or neural network) models with the first sound signals as input to determine whether the first ear is with OM or not.

[0157] The system and method for detecting otitis media acoustically provided by the present application fulfills the needs of applying artificial intelligence technologies such as machine learning and deep neural network to provide low-cost and easy-to-use acoustic-based system and method for diagnoses of otitis media with effusion, especially for primary clinics and home environment.

[0158] While the invention has been described in terms of what is presently considered to be the most practical and preferred embodiments, it is to be understood that the invention needs not to be limited to the above embodiments. On the contrary, it is intended to cover various modifications and similar arrangements included within the spirit and scope of the appended claims which are to be accorded with the broadest interpretation so as to encompass all such modifications and similar structures.

Claims

WHAT IS CLAIMED IS;1 . A system for detecting otitis media (OM) acoustically, comprising: a first acoustic receiver, configured to be placed in a patient’s first ear for receiving utterances of the patient: an acoustic sampling circuitry, configured to connect with the first acoustic receiver and to sample the utterances of the patient as first sound signals; a memory, configured to store one or more trained ML or NN (machine learning or neural network) models; and one or more processors, configured to inference the one or more ML or NN models with the first sound signals as inputs to determine whether the first ear is with OM or not.

2. The system of claim L wherein the OM to be determined is selected from one of foliowring: otitis media with effusion (OME); acute otitis media (AOM); acute otitis media with effusion (A.OM.E); chronic otitis media (COM); and chronic otitis media with effusion (COME).

3. The system of claim 1 , wherein the one or more trained ML or NN models are trained to determine a type of first ear’s fluid, wherein the type includes one of following: purulent fluids: serous fluid; and mucoid fluid.

4. The system of claim 1 , wherein the one or more ML or NN models is selected from one of following: SVM; GNB; AdaBoost; RF; and CNN.

5. The system of claim 1, wherein the utterances are one of followings: vowels; and a combination of consonant and vowels.

6. The system of claim L further comprises a first earmuff which is configured to cover the first ear.

7. The system of claim I , wherein the acoustic sampl ing circuitry is con figured to sample the utterances at 44, 1 kHz or below as the first sound signals.

8. The system of claim 1, wherein the processor is further configured to resample the first sound signals at 16 kHz.

9. The system of claim I, wherein the processor is further configured to preserve a middle section of each of the first sound signals.

10. The system of claim 1, wherein the processor is further configured to pre- process the first sound signals to extract acoustic features by one or any combination of following:Z score normalization; and short-term Fourier transform (STFT.)11 . The system of clai m 1, wherein the processor is further configured for: framing the first sound signals into a series of temporal frames; applying Mamming window function to the series of temporal frames; applying Fourier transform to generate magnitude spectra of each of the temporal frames; stacking all the magnitude spectra of each of the temporal frames to generate a spectrogram with acoustic features; normalizing the spectrogram to zero mean; and identifying variance to enhance the energy in a high-frequency range.

12. The system of claim 1, wherein the processor is further configured for: calculating a score according to each of classified labels outputted from the one or more trained ML or NN models corresponding to each of the utterances; comparing the score with a threshold; and when the score is larger than the threshold, determining that the first ear is with OM.

13. The system of claim 12, wherein each of the classified labels are weighted.

14. The system of claim L further comprises: a second acoustic receiver con figured to be placed in a patient’s second ear for recei ving the utterances of the patient,wherein the acoustic sampling circuitry is configured to connect with the second acoustic receiver and to sample the utterances of the patient as second sound signals, and wherein the one or more processors, configured to inference the one or more ML or NN models with the second sound signals as input to determine whether the second ear is with OM or not.

15. The system of claim I, further comprises at least one of following: a display configured to provide visual prompt to have the patient pronounce the utterance; and a speaker configured to provide audio prompt to have the patient pronounce the utterances and to replay the utterances.

16. A network system for detecting otitis media (OM) acoustically, comprising: a system and a detection server, wherein the system comprises: a first acoustic receiver, configured to be placed in a patient’s first ear for receivi ng utterances of the patient; an acoustic sampling circuitry, configured to connect with the first acoustic receiver and to sample the utterances of the patient as first sound signals; a first network interface configured for connecting with a network; and a first processor, configured to have the first network interface transmit the first sound signals to the detection server via the network, wherein the detection server comprises:a second network interface configured to receive the first sound signals via the network; a memory, configured to store one or more trained ML or NN (machine learning or neural network) models; and one or more second processors, configured to inference the one or more ML or NN models with the first sound signals as input to determine whether the first ear is with OM. or not,17. The network system of claim 16, wherein the system further comprises a display, wherein the detection server is further configured to transmit a detection result back to the system, wherein the first processor of the system is further configured to have the display show the detection result.

18. The network system of claim 16, further comprises a clinician computer with a display, wherein the detection server is further configured to transmit a detection result to the clinician computer, wherein the clinician computer is further configured to have the display show the result.

19. The network system of claim 16, further comprises a database management system, wherein the detection server is further configured to transmit the first sounds and a detection result to the database management system for storage.

20. A method for detecting otitis media (OM) acoustically, comprising; receiving utterances of a patient by a first acoustic receiver which is configured to be placed in the patient’s first ear;sampling the utterances of the patient as first sound signals; and inferencing one or more trained ML or NN (machine learning or neural network) models with the first sound signals as input to determine whether the first ear is with OM or not.

Images