Calibrating a machine learning model trained to determine a location and shape of a sound source

Abstract

Methods, systems, and apparatus, including computer programs encoded on computer-storage media, for calibrating a machine learning model that has been trained to determine a location and shape of a sound source based on input data generated by a plurality of sensors installed on a device. In one aspect, the method can include obtaining, by one or more computers, data generated by a device, the obtained data corresponding to an utterance of a calibration script by an entity, generating, by one or more computers, calibration data based on the obtained data corresponding to the utterance of the calibration script, and calibrating, by one or more computers, the trained machine learning model using the generated calibration data.

Description

CALIBRATING A MACHINE LEARNING MODEL TRAINED TO DETERMINE A LOCATION AND SHAPE OF A SOUND SOURCECROSS-REFERENCE TO RELATED APPLICATIONS

[0001] This application claims priority to U.S. Patent Application No. 63 / 544,632, filed October 17, 2023, entitled “CALIBRATING A MACHINE LEARNING MODEL TRAINED TO DETERMINE A LOCATION AND SHAPE OF A SOUND SOURCE,” which is incorporated herein by reference in its entirety.BACKGROUND

[0002] Sleep apnea is a disorder that causes a person to stop breathing while the person sleeps. The degrees of sleep apnea for any particular person may vary, but in some people, sleep apnea can cause them to stop breathing hundreds of times per night. A common treatment for sleep apnea is a doctor prescribing a patient to use a continuous positive airway pressure (CPAP) machine. Additionally, upper airway stimulation (UAS) is a common treatment for sleep apnea. However, effective patient selection for UAS treatment, which is dependent on reliable and precise anatomical assessments, is critical for optimizing outcomes and minimizing risks related to UAS treatment.

[0003] Dysphagia, or difficulty swallowing, is another significant concern, particular in an aging population. Dysphagia often results from age-related physiological changes or comorbid conditions such as stroke and dementia. The diagnosis and management of dysphagia are challenged by a limited understanding of mechanisms underlying swallowing function. Current approaches often focus on symptoms rather than the underlying physiological issues, highlighting a need for advanced tools that provide detailed and personalized assessments of swallowing mechanics.SUMMARY

[0004] CPAPs are commonly prescribed as a treatment for sleep apnea. However, the effectiveness of a CPAP may be muted for persons having particular anatomical features. Similarly, upper airway stimulation (UAS) has emerged as a promising treatment alternative to surgical solutions, as it improves muscle tone and significantly reduces apnea severity. However, effective treatment selection for an individual is difficult due to a limited understanding of complex mechanisms of relevant anatomical features. Accordingly,knowledge of the particular anatomical features of a person’s body such as, e.g., their respiratory tract can have an impact on a doctor’s diagnosis for sleep apnea, dysphagia, and other health challenges related to the upper airway anatomy. In some respects, outputs of the present disclosure can be used to generate a model of an interior portion of the person’s anatomy. Though the present disclosure has particular applications regarding the diagnosis of sleep apnea, the present disclosure is not so limited. For example, in some implementations, the present disclosure can be used to model other aspects of a person’s anatomy such as, e.g., chambers of a heart, model blood flow or gas flow through the body, or the like.

[0005] Yet, even other applications of the present disclosure exist. For example, in some implementations, the present disclosure can be used a treatment for sleep-disordered breathing, speech, & tongue movement disorder. For example, the present disclosure can be used for the diagnosis of dysphagia. In such implementations, for example, output of the systems described herein can be used as biofeedback to a patient user, which the user can use to strengthen their tongue muscles, as in myofunctional therapy.

[0006] According to one innovative aspect of the present disclosure, a method for calibrating a machine learning model that has been trained to determine a source anatomical location and shape of a sound source based on input data generated by a plurality of sensors installed on a device is disclosed. In one aspect, the method can include obtaining, by one or more computers, vibration data corresponding to a generated signal in response to a detection of vibrations due to a calibration sound source, generating, by one or more computers, calibration data based on the obtained data corresponding to the calibration sound source. In one aspect, generating the calibration sound source includes accessing label data specifying a set of pre-defined sounds associated with a portion of the source anatomical location and shape, identifying at least a portion of the vibration data corresponding to a particular predefined sound of the set of pre-defined sounds, and based on the identifying, associating the portion of the vibration data to one or more portions of the source anatomical location and shape. The method further includes calibrating, by one or more computers, the trained machine learning model based on the association of the portion of the vibration data to the one or more portions of the source anatomical location and shape.

[0007] Other versions include corresponding apparatus, methods, and computer programs to perform the actions of methods defined by instructions encoded on computer readable storage devices.

[0008] These and other versions may optionally include one or more of the following features. For instance, in some implementations, the calibration sound source is a result of an utterance of a calibration script generated by an entity, in which the device is disposed on the entity’s body.

[0009] In some implementations, the utterance of the calibration script includes instructions for the entity to produce sounds corresponding to a vocalization of one or more consonant sounds.

[0010] In some implementations, the calibration sound source is a result of a naturally occurring anatomical process within an entity’s body, in which the device is disposed on the entity’s body.

[0011] In some implementations, the data corresponding to the calibration sound source as the result of the naturally occurring anatomical process within the entity’s body is obtained while the user is in a state of rest (e.g., sleeping). In some implementations, the data corresponding to the calibration sound source as the result of the naturally occurring anatomical process within the entity’s body is obtained while the user is in a state of routine activity. In some implementations, the data corresponding to the calibration sound source as the result of the naturally occurring anatomical process within the entity’s body is obtained while the user is in a state of stress.

[0012] In some implementations, the calibration sound source is a sound source external to the entity’s body, in which the device is disposed on the entity’s body.

[0013] In some implementations, the plurality of features includes one or more of a time delay, an amplitude difference, and multipath effects.

[0014] In some implementations, the time delay includes a time delay between signals received by the plurality of microphones of the device, in which the time delay between the plurality of microphones is determined by performing operations including receiving, by a processor from a first microphone of the device, a first signal corresponding to a detection of a vibration from the calibration sound source, receiving, by the processor from a second microphone of the device, a second signal corresponding to a detection of a vibration from the calibration sound source, and determining, based on the first signal and the second signal, an output of a generalized cross correlation (GCC) function, the output indicative of a degree of similarity between the first signal and the second signal.

[0015] In some implementations, the amplitude difference includes a difference in signal amplitude between signals received by a plurality of microphones of the device, in which the amplitude difference between the plurality of microphones is determined by performingoperations including receiving, by a processor from a first microphone of the device, a first signal corresponding to a detection of a vibration from the calibration sound source, receiving, by the processor from a second microphone of the device, a second signal corresponding to a detection of a vibration from the calibration sound source, and determining, based on the first signal and the second signal, an output of a relative transfer function (RTF), the output indicative of a degree of similarity between the first signal and the second signal.

[0016] In some implementations, calibrating, by the one or more computers, the trained machine learning model using the generated calibration data includes adjusting, by one or more computers, one or more parameters of the trained machine learning model based on the plurality of feature values of the data corresponding to the calibration sound source, in which the adjusted parameters represent particular aspects of the entity’s anatomy.

[0017] In some implementations, the obtained data includes a data field indicative of whether the obtained data corresponds to calibration data or run-time data.

[0018] Implementations of the systems and methods of this disclosure can provide various technical benefits. Calibration of a trained machine learning model, based on data that is specific to a particular environment and a particular implementation result in more precise predictions and more precise evaluation of the location and shape of a sound source. Due to variations in the placement of sensors between uses for a particular individual, and variations of the anatomy between multiple users, a modification or calibration of the baseline trained machine learning model is required to achieve suitable predictive capabilities across individuals and across individual implementations. By precisely identifying a location and configuration of a sound source that may be indicative of an obstructive airway collapse during natural sleep enables patients to be matched to an effective, personalized treatment, improving outcomes of individuals that suffer from sleep apnea and other related health challenges.

[0019] Additionally, the systems and method of this disclosure relate to a diagnostic approach that can be implemented during natural sleep at home, improving upon current standards of awake patient visual evaluation and drug-induced sleep endoscopy. By performing diagnostic evaluations at home in a natural setting, a more accessible and less costly solution is made available, which addresses a serious problem of severe underdiagnosing and undertreatment in sleep apnea and other related health challenges.

[0020] In some cases, the anatomical effects related to sleep apnea and dysphagia can be linked. For example, sleep apnea can contribute to swallowing difficulties throughmechanisms such as reduced upper airway sensation, neuromuscular changes, and inflammation. This intersection of symptoms between sleep apnea and dysphagia indicates a need for a comprehensive assessment of airway and swallowing function. The systems and methods of this disclosure contribute to a generation of insights for managing both conditions, particularly in an older population.

[0021] The details of one or more implementations of these systems and methods are set forth in the accompanying drawings and the description below. Other features, objects, and advantages of these systems and methods will be apparent from the description and drawings, and from the claims.BRIEF DESCRIPTION OF THE DRAWINGS

[0022] FIG. l is a diagram of an example of a system for calibrating a machine learning model that has been trained to determine a location and shape of a sound source.

[0023] FIG. 2 is a diagram of an example wearable device.

[0024] FIG. 3 is a flowchart of an example process for calibrating a machine learning model that has been trained to determine a location and shape of a sound source.

[0025]

[0026] FIG. 4 is an example user interface for displaying locations of airway collapse during sleep.

[0027] FIG. 5 illustrates example signal paths from an anatomical vibration source to vibration sensors.

[0028] FIG. 6 illustrates example data from example vibration sources.

[0029] FIG. 7 illustrates example machine learning model performance data.

[0030] FIG. 8 illustrates example time series data collected from a wearable device.

[0031] FIG. 9 is an example user interface for obtaining calibration data.

[0032] FIG. 10 is an example user interface for obtaining calibration data.

[0033] FIG. 11 is an example user interface for displaying data indicative of a sleep summary.

[0034] FIG. 12 is an example user interface for displaying data indicative of a breathing summary.

[0035] FIG. 13 is an example user interface for visualizing data indicative of sleep metric ratings.

[0036] FIG. 14 illustrates example user interface components for comparing predicted values of airway obstruction parameters and measured values of airway obstruction parameters.

[0037] FIG. 15 is a block diagram of system components that can be used to implement a system for calibrating a machine learning model that has been trained to determine a location and shape of a sound source.

[0038] Like reference numbers and designations in the various drawings indicate like elements.DETAILED DESCRIPTION

[0039] The present disclosure is directed towards systems, methods, and computer programs, for calibrating a machine learning model that has been trained to determine a location and shape of a sound source. In some implementations, the sound source is in a complex acoustic environment such as, e.g., an entity’s head, an entity’s respiratory tract (or a portion thereof), an entity’s chest cavity, an entity’s heart, an entity’s lungs, or the like. An entity may include, e.g., a human or other animal. Aspects of the present disclosure employ one or more sensors packaged into a wearable device that are used to measure sounds traveling through an entity’s tissue. The present disclosure describes sounds as vibrations that propagate through the entity from the sound source. The sensors packaged into the wearable device detect the vibrations by converting mechanical motion due to the vibrations into electrical signals. The electrical signals generated by each sensor can be processed as data to generate calibration data, input data to the machine learning model, or both.

[0040] Calibration data can correspond to, for example, any sound originating within the body of an entity that corresponds to a known sound associated with a known anatomical location within the body. In some implementations, such calibration data can be generated based on an utterance by the entity in response to the wearable device (or associated application) prompting the entity to utter pronunciations of a calibration script. In other implementations, calibration data can be generated based on sounds originating naturally within the body of an entity such as sounds of blood flow through coronary arteries, sounds generated by the opening and closing of different heart valves, sounds of gas flow in an entity’s abdomen (e.g., in GI tract or renal tract), gas (e.g., oxygen or carbon dioxide) flowing inside an entity’s lungs, or the like. In other implementations, calibration data can be generated based on sounds originating from external vibration sources. Though these sounds are provided as examples upon which calibration data is based, any sound within or near the surface of the body can be used to generate calibration data.

[0041] Determining a location and shape of such sound sources comprise multiple challenges. In particular, one of the most significant challenges to determining a location andshape of such sound sources using a machine learning model is developing a machine learning model that is able to account for the variations between the airways of different entities. For example, each individual entity who may be diagnosed using techniques of the present disclosure typically has a different head and respiratory tract shape. This variation in head and respiratory tract shape introduces variability into data that is generated by sensors of the wearable device and the analysis, by one or more machine learning models of feature data extracted from the generated data.

[0042] Further complicating matters is that there is no precise way for a patient to repeatably attach the wearable device of the present disclosure. Accordingly, even assuming a single entity uses the same device for multiple sessions, the entity (or person assisting the entity) likely will not be able to attach the wearable device in the same exact position for each session. This variability in attachment of the wearable device to the entity also introduces variability into the data generated by one or more sensors of the wearable device and the analysis, by one or more machine learning models of feature data extracted from the detected data.

[0043] FIG. 1 is a diagram of a system 100 for calibrating a machine learning model that has been trained to determine a location and shape of a sound source. The system 100 can include a wearable device 110, a network 120, an application server 130, one or more sound sources 140, sources of sleep data 150, and a user device 140. The application server 130 can include one or more processors that can execute operations associated with an input engine 131, a decisioning engine 132, a calibration engine 133, a trained machine learning model 134, and a sound source generation engine 135.

[0044] The wearable device 110 includes one or more sensors for detecting sounds (vibrations) and generating data corresponding to the detected sounds to the system 100. Processors of the system 100 can then process the generated data to determine the location and shape of the source of the detected sounds. In some implementations, the wearable device 110 can include multiple contact microphones that are configured to operate in the acoustic range of 100Hz to 3.5kHz. However, other contact microphones can be used that are configured to operation in an acoustic range outside of 100 Hz. To 3.5kHz. A contact microphone is similar to a regular microphone in that a contact microphone is able to record, or otherwise detect, sound. However, a contact microphone differs from a regular microphone in that a contact microphone is immune to air-borne sounds and, instead, only pick up the vibrations caused by the sound waves travelling through tissue, bone, or both. In addition, contact microphones can also be based on different physical mechanisms comparedto regular microphones. For example, contact microphone can include piezo diaphragms, accelerometers, or regular microphones (electret or MEMS types, which are specially packaged or modified to only detect contact sounds) that respond to vibrations on a surface (e.g., an entity’s skin) to which the contact microphone is attached.

[0045] An example of a wearable device that can be used in system 100 as wearable device 110 is wearable device 200 as shown in FIG. 2. The wearable device 200 is configured in a small form factor (e.g., in a band-aid like) design and includes multiple contact microphones 210, 212, 214. In addition to the contact microphones, the wearable device 200 can include other hardware 220 such as a printed circuit board (PCB) or flexPCB, battery (disposable or rechargeable), and one or more accelerometers. In some implementations, the PCB of the wearable device 200 can include necessary hardware to charge a rechargeable battery or otherwise power the device. In addition, the PCB of the wearable device 200 include hardware necessary to communicate data (e.g., a Bluetooth Low Energy (BLE) interface) corresponding to sounds detected by the contact microphones 210, vibrations detected by the accelerometer, or both, to another component of the system 100 like the server 130 or the user device 140. In some implementations, for example, the PCB can be configured with hardware necessary to wirelessly transmit data corresponding to sounds, vibrations, or both, detected by the wearable device 200. In other implementations, however, the PCB can be configured to include hardware necessary to transmit data corresponding to sounds, vibrations, or both, detected by the wearable device 200 using a wired network. In some implementations, the wearable device 200 can be configured to transmit data both wired and wirelessly. The wearable device 200 may have at least one side coated with an adhesive such that the wearable device 200 can be attached to the cheek, chin, forehead, chest, or other portion of the entity’s skin.

[0046] The wearable device 110 can be attached to a portion of the entity’s body such as, e.g., the entity’s facial cheek or chest. After the wearable device 110 is attached to the entity and prior to using the wearable device 110 to diagnose a sleep apnea treatment or a treatment related to other relevant conditions, the entity can initiate a calibration process that is configured to calibrate the trained machine learning model 134 so that it can more accurately determine a location and shape of a sound source associated with the entity. Calibrating the trained machine learning model 134 can include tuning the parameters of the trained machine learning model 134 to account for the specific dimensions of the entity’s anatomy such as, e.g., the entity’s head, respiratory tract, chest cavity, or lungs.

[0047] The wearable device 110 is configured to detect sounds, vibrations, or both from sound sources 140 within and / or near the entity’s body. The sound sources 140 include runtime sound sources 140a, which originate from within the entity and correspond to anatomical features like the uvula, epiglottis, etc. The sound sources 140 can also include calibration sound sources 140b. The calibration sound sources 140b can include sound sources that originate from within the entity and / or sound sources that originate from a region external to the entity. For example, the calibration sound source 140b can be a vibration source disposed external to the entity in contact with the skin.

[0048] In some implementations, the calibration process begins with the entity speaking a calibration script. In some implementations, the calibration script includes a plurality of predetermined words that cause the user to utter a pronunciation of consonants. The calibration script is designed to cause the entity to utter different consonants whose sounds are generated at different parts of the entity’s airway by voluntary constriction or obstruction of the airway. For example, a / t / consonant is generated closer to the front of the mouth with the tip of the tongue, while a / k / sound is generated further back in the mouth with the base of the tongue. All consonants have known locations and anatomical shapes associated with them. Thus, the entity’s utterance of the calibration script causes the entity to pronounce different consonants that produces sound waves from different anatomical locations within the patient’s head, respiratory tract, etc. These data corresponding to these sound waves that are detected by the wearable device 110 during the calibration process correspond to the calibration sound source 140b that can be used to calibrate the trained machine learning model 110, as described below.

[0049] In some implementations, the calibration sound source 140b includes a set of predefined sounds associated with a portion of a source anatomical location and shape. In the case of the entity uttering a script, the pre-defined sounds correspond to particular consonant sounds that occur due to a vibration at an anatomical location. In some implementations, the calibration sound source 140b produces vibrations that result in a generation of data by the wearable device 110, and the generated data includes a label indicative of the type, location, and shape of the associated calibration sound source 140b.

[0050] The wearable device 110 can include one or more sensors such as one or more contact microphones, an accelerometer, or a combination thereof. The sensors generate electrical signals that can be processed as data indicative of sound waves traveling through the entity’s tissue proximate to the wearable device 110 based on the entity’s utterance of the calibration script or other calibration sound sources 140b. The wearable device 110 can transmitcalibration data 112 corresponding to the calibration sound source 140b to the application server 130 via the network 120.

[0051] Though an example is discussed herein of a wearable device 110 that generates calibration data by detecting speech utterances, the present disclosure is not so limited. Instead, in some implementations, the wearable device 110 can be used to generate other types of calibration data based on other calibration sound sources 140b. For example, in some implementations, the calibration sound source 140b can originate in other parts of the entity’s body such as, e.g., an entity’s heart, the entity’s lungs, the entity’s abdomen (e.g., GI tract or a portion thereof), or the like. In such instances, the wearable device 110 can be used to obtain data corresponding to sounds originating from inside an entity’s body in one of these locations while the entity is in a particular state such as a state of rest (e.g., sleeping), a state of routine activity, or a state of stress. In such instances, data obtained by the wearable device 110 during a calibration period can be generated based on the sounds output by the entity’s body and transmitted to the application server 130 as obtained calibration data 112 or run-time sounds 114.

[0052] In some implementations, the calibration sound source 140 corresponds to vibrations from an external vibration source, in which the external vibration source is external to the entity’s body.

[0053] The processors of the application server 130 execute operations of the input engine 131 to obtain data corresponding to sound waves detected by the sensors of the wearable device 110 that are transmitted to the application server 130 by the wearable device 110 via the network 120. The obtained data corresponding to sound waves can include calibration data 112 or run-time data 114 generated by the sensors of the wearable device 110 after calibration such as breathing sounds, snores, chokes, and snore-like gasps. The run-time data 114 corresponds to sounds generated by the entity during normal operation (not during calibration). The input engine 131 includes a decisioning engine 132 to analyze the obtained data 112, 114 and to generate an output that is indicative of whether the obtained data 112, 114 is the calibration data 112 or the run-time data 114.

[0054] In some implementations, the sensors of the wearable device 110 detect vibrations from a run-time sound source 140a to generate the run-time data 114 during critical periods of time. In some implementations, one or more processors communicatively coupled to the wearable device 110 execute instructions according to one or more algorithms that causes the wearable device 110 to collect and transmit the calibration data 112 and / or run-time data 114 during pre-defined periods of time or during periods of time determined by the one or morealgorithms. The non-continuous generation of run-time data 114 results in reduced data storage requirements, battery consumption, and data transfer requirements. Further detail about specific strategies for implementing non-continuous collection of the run-time data 114 is discussed in relation to FIG. 8.

[0055] In some implementations, the one or more processors communicatively coupled to the wearable device 110 are local to the wearable device 110. In some other implementations, the one or more processors are remote from the wearable device 110 and communicate with the wearable device 110 through a network adapter or some other means of connectivity.

[0056] In some implementations, the application server 130 receives data from one or more sleep data sources 150. In some cases, a sleep data source corresponds to data from a wearable device, e.g., a smartwatch. The sleep data sources 150 are communicatively coupled to the application server 130 or the user device 140, in which the sleep data sources 150 transmit sleep data 152 to the application server 130 or to the user device 140 over the network 120. For example, data associated with the sleep data sources 150 can include data indicative of an apnea-hypopnea index (AHI), oxygen saturation, heart rate, heart rhythm, muscle activity through electromyography (EMG), electrical activity of the brain through an electroencephalogram (EEG), activity and rest cycles through actigraphy, sleep position, snoring, temperature, respiratory airflow, respiratory effort, and sleep staging.

[0057] The processors of the application server 130 can execute operations of the decisioning engine 132 to determine whether obtained data corresponds to calibration data 112 or runtime data 114 in a number of different ways. In some implementations, e.g., the obtained data 112, 114 can include a field with a flag set to a particular value that indicates whether the obtained data 112, 114 is calibration data 112 or run-time data 114. For example, in some implementations, a value in the field can be set to “1” if the obtained data is calibration data or “0” if the obtained data is run-time data 114. In other implementations, e.g., the decisioning engine 132 can be configured to detect that the obtained data includes utterances corresponding to a calibration script or vibrations from an external vibration source.Regardless of the decisioning logic employed by the decisioning engine 132, if an output of the decisioning engine 132 is indicative of calibration data, the calibration engine 133 can receive the calibration data 112 from the input engine 131. Alternatively, if the output of the decisioning engine 132 is indicative of run-time data 114, the trained machine learning model 134 can receive the run-time data 114 from the input engine 131.

[0058] In the example of FIG. 1, the input engine 131 receives the obtained data 112, 114. The output of the decisioning engine 132 is indicative of whether the obtained data 112, 114is the calibration data 112 or the run-time data 114. Here, the output of the decisioning engine 132 determines if the obtained data 112, 114 comprises sounds corresponding to the calibration sound source 140b and is, thus, calibration data 112. Based on determining that the obtained data 112, 114 includes calibration data 112, the input engine 131 can generate feature data 112a based on the obtained calibration data 112. In some implementations, generation of the feature data 112a can include analyzing three key features of the sound signals: time delay, amplitude difference, multipath effects, as described in more detail below.

[0059] The time delay feature is a value corresponding to how long it takes for the sound to reach each microphone (sensor) of the wearable device 110. By comparing the time it takes for the sound to reach each of the microphones (relative to the other microphones of the wearable device 110), the processors of the application server 130 can execute operations of a trained machine learning model to estimate a location of a sound source. In some implementations, a processor of the application server 130 can implement operations associated with a Generalized Cross Correlation (GCC) method that can be used to estimate a time delay between the one or more microphones of the wearable device 110. GCC is a mathematical technique with an output that is indicative of a similarity between signals received by different microphones of the wearable device 110, allowing for accurate time delay estimation.

[0060] The amplitude difference feature is a value corresponding to a difference in loudness between each of the plurality of microphones of the wearable device 110. If one microphone picks up a louder sound than the others, it is likely closer to the source. The processors of the application server 130 can also execute operations to determine variations of the sound waves as it propagates through the head, which reveal information about the environment and the location of the sound source. In some implementations, the processors of the application server 130 can execute operations to determine a Relative Transfer Function (RTF), in which the RTF is associated with an output value that is indicative of an amplitude difference between signals detected by each microphone. An output of the RTF is a measure associated with variations in amplitude and phase as the sound waves propagate through an environment. An evaluation of the RTF contributes to a determination of a relationship between the locations of the microphones of the wearable device 110 and the sound source, providing insights into the location of the source.

[0061] In some implementations, outputs of the input engine 131 reflect multipath effects and reflections of sound waves. In a complex environment such as the head, sound can bounceoff interfaces between bone, soft tissue, and air before reaching the microphones. This creates multiple paths for the sound to travel between two points, which can make it harder to determine where the sound originates. By executing operations of the input engine 131 that account for these multipath effects and reflections, the system 100 can more accurately pinpoint the location and shape of the sound source. In some implementations, in addition to or separate from the RTF, the processors of the application server 130 can execute operations associated with Cepstrum to account for these multipath effects. Cepstrum is a representation of the sound signal that emphasizes the patterns created by echoes and reflections in the environment. By account for the Cepstral information in the operations executed by the processors of the application server 130, the system 100 can more accurately determine the head’s acoustic properties and how the sound waves interact with the tissue environment. Additional detail in relation to multipath effects is provided in relation to the discussion of FIG. 5.

[0062] In some cases, the output of the RTF represents a spatial fingerprint of an acoustic environment within the entity. The output of the RTF provides source-independent spatial information by representing variations in amplitude and phase of sound waves as they propagate through tissue and regions within the entity’s head. In some cases, the output of an RTF is not ideal for classification tasks due to a high dimensional space upon which the RTF is represented. In other words, the output of the RTF is represented in a high dimensional space in comparison with the common low dimensional space associated with most classification tasks (e.g., black or white, big or small, etc.) To overcome this challenge, in some implementations, the processors of the application server 130 can execute instructions associated with dimensionality reduction techniques that map the outputs of an RTF onto a lower dimensional space indexed by classification parameters.

[0063] In some implementations, depending on training data upon which the machine learning model 134 is trained, the predictions of the system 100 can suffer from elevated prediction error rates as the distance between training data points is increased. Training data points are points in a region of interest in which measurements and / or predictions are made. In other words, as the distance between sampling points within a sample entity’s head (in some cases, the sample entity is a simulated anatomical environment) increases, the nonlinearity of the RTF causes prediction errors for regions between sampling points. The RTF cannot be linearly interpolated between sampling points. In some implementations, the processors of the application server 130 can implement instructions to linearize the nonlinearRTF space into an isometric space in a latent representation to allow for linear interpolation between sampling points.

[0064] In some implementations, the processors of the application server 130 implement instructions associated with a localized sampling strategy, e.g., burst sampling. A burst consists of samples taken from a local neighborhood, providing information on a local variability of the RTF near each sampling point. By sampling a region around each sampling point directly, local nonlinear relationships can be determined. In some implementations, a processor trains a conformal autoencoder using whitening and reconstruction loss functions and on training data obtained with the burst sampling method. In some implementations, an artificial acoustic environment with test sound sources is used to generate training data. In some implementations, the conformal autoencoder is a neural network, in which the neural network learns to map data in a nonlinear space onto a linear space (encoding) and to map the data represented in the linear space back to a nonlinear space (decoding). Model parameters of the neural network of the conformal autoencoder are determined through an iterative algorithm (e.g., gradient descent) by calculating an error between a decoded representation and an input representation.

[0065] Turning to a specific implementation of generating calibration data by instructing an entity to utter a calibration script, the entity generates multiple voluntary airway obstructions. For example, consonants are a category of sounds characterized by the obstruction or restriction of airflow in the upper airway. They are produced by altering the shape or position of structures like the tongue, palate, or glottis, which in turn create different consonants. There are several types of consonants, including plosives, fricatives, and affricates.

[0066] Plosives involve a complete closure of the upper airway, followed by a sudden release of air. Examples of plosives include / t / , / d / , / k / , and / g / , produced by raising the soft palate to block the nasal cavity and placing the tongue against the hard palate or alveolar ridge to fully obstruct the oral cavity. The burst of sound results from the abrupt release of air after this closure.

[0067] Fricatives are consonants created through partial obstruction of the upper airway, generating a turbulent airflow. Examples of fricatives include / f / , / v / , / s / , / z / , / J7, and 73 / , which are produced when the tongue, teeth, or lips form a narrow opening that forces air to create friction noise. Affricates, on the other hand, begin as plosives and transition into fricatives. Examples fricatives include / tj7 and / d3 / , which involve an initial complete closure of the vocal tract, followed by a gradual release to create a narrow opening for turbulent airflow.

[0068] The obstructive nature of consonants and the turbulence they generate in the airway are particularly relevant to Obstructive Sleep Apnea (OSA). By utilizing consonants with known anatomical locations and shapes, a machine learning system can be trained to identify tissue-borne sound features associated with different regions and types of obstruction in the upper airway. This could lead to more accurate and precise determination of the location and severity of airway obstruction during OSA.

[0069] Since consonants are voluntary obstructions produced during wakefulness, they offer an opportunity to collect a diverse and comprehensive dataset of tissue-borne sounds from various speakers with different anatomical variations. This can improve the generalizability and robustness of the machine learning model, which is crucial for its application in clinical settings. However, it is essential to validate models trained with consonants and similar sounds during wakefulness against real OSA cases. Additionally, further training may be required using data from obstructions that occur specifically during OSA to ensure the model's accuracy and effectiveness.

[0070] In some implementations, the machine learning model is trained for a specific application, e.g., dysphagia or OSA. Although OSA and dysphagia have similar impacted muscles, slight differences between the data associated with each function can result in differences in the training data and determined model weights. In some cases, a trained machine learning model based on a first use case (e.g., OSA) can be trained (fine-tuned) with additional training data associated with a second use case (e.g., dysphagia). In some cases, due to the similar anatomical effects of both use cases, the parameters of the respective trained machine learning models are similar, so a fine-tuning approach using use casespecific training data can be implemented.

[0071] In response to a determination that the obtained data 112, 114 include calibration data 112, the input engine 131 can provide the generated feature data 112a as an input to the calibration engine 133. The calibration engine 133 can use the feature data 112a, based on the calibration data 112, to calibration the trained machine learning model 134. Calibrating the trained machine learning model based on the feature data 112a generated based on the calibration data can include, for example, adjusting one or more parameters of the trained machine learning model 134 based on the generated feature data 112a. In some implementations, the calibration engine 133 can iteratively adjust the parameters of the trained machine learning model 134 based on the feature data 112a to optimize an output of the model 134.

[0072] In some implementations, after calibrating the trained machine learning model 134, the entity, with the wearable device 110 attached, can go to sleep. In some other implementations, in which the use case is not related to sleep (e.g., dysphagia), the user remains awake. The wearable device 110 can detect sounds within the entity’s head, respiratory tract, or chest cavity, generate sound data such as data 114 that corresponds to the detected sounds, and then transmit the sound data 114 to the application server 130. The application server 130 can execute instructions associated with decisioning logic 132 to determine whether sound data 114 is calibration data or run-time sound data. Here, based on a determination by the decisioning logic 132 that the obtained data 112, 114 is run-time sound data originating from the run-time source 140a, the decisioning engine 132 can cause the input engine 131 to generate feature data 114a based on the run-time sound data 114. In some implementations, generation of the feature data 114a can include analyzing the three key features of the sound signals: time delay, amplitude difference, multipath effects, as previously described in relation to the calibration data 112a.

[0073] The input engine 131 can provide the feature data 114a as input to the trained, and now calibrated, machine learning model 134. This machine learning model 134 is trained based on baseline training data (e.g., not specific to a particular entity) to determine a location and shape of a sound source based on processing of input data corresponding to features of sound data of an entity while sleeping. In addition, the machine learning model 134 includes modified parameters that are tuned to the particular entity that generated run-time sound data 114 (and corresponding feature data 114a) by using the calibration data 112a. The trained and calibrated machine learning model 134 can include any type of neural network including, e.g., one or more neural networks, one or more deep learning models, one or more generative artificial intelligence models, or any combination thereof.

[0074] The trained and calibrated machine learning model 134 can process the feature data 114a corresponding to the run-time sound data 114 through each layer (in the case of a neural network) of the machine learning model 134 in order to generate output data 134a indicative of a location and shape of the source of the run-time sound data 114. The machine learning model 134 can provide the output data 134a as an input to the sound source generation engine 135.

[0075] The sound source generation engine 135 is configured to obtain output data of the trained and calibrated machine learning model 134 such as output data 134a and generate output data 135a based on processing of the output data 134a. Output data 135a can include, for example, a heatmap of a portion of an entity’s body such as the entity’s head, respiratorytract, or chest cavity indicating the shape of the portion of the entity’s body and an origin location of the run-time sound data 114. The sound source generation engine 135 can transmit the output data 135a to the user device 140 and / or the application server 130 over the network 120. In some implementations, the output data 135a can include rendering data that, when rendered by the user device 140, causes the user device 140 to output, on its display, the heatmap.

[0076] However, the present disclosure is not so limited. In some implementations, for example, output data 134a generated by the machine learning model 134 can be used in a treatment for sleep-disordered breathing, speech, & tongue movement disorder. In such implementations, for example, the output data 134a can be a model the tongue position and shape. Then, this output can be used to provide myofunctional therapy with biofeedback. For some use cases, the myofunctional therapy is related to tongue position. For some other use cases, the myofunctional therapy is related to muscle position beyond tongue position like swallowing muscles including the velum, cheeks, lips, etc.

[0077] Providing biofeedback can be performed in one or more ways including, for example, by (i) providing real-time feedback on tongue placement and movement, (ii) providing a visualization of tongue motion, (iii) providing strength training, and / or (iv) use of progress tracking. Each of these aspects of biofeedback are described in more detail below.

[0078] In some implementations, providing biofeedback includes providing real-time feedback on tongue placement and movement. This can include, for example, identifying the location and movement of the tongue in real-time and notifying a patient regarding the tongue location and / or movement. Based on this notification, the patient can adjust their tongue's position. For example, if a patient's tongue is consistently positioned on one side or the other during certain exercises, a biofeedback notification can be generated and provided to the patient in order to alert the patient to correct it. This alert can be output via the wearable device 110 (e.g., via haptic feedback, audio feedback, and / or visual feedback) or a nonwearable device such as a smartphone (e.g., via haptic feedback, audio feedback, and / or visual feedback). In some implementations, the biofeedback alert may be designed to direct the patient’s attention to another device (e.g., a display of a smartphone, tablet, or other display) that enables the patient to view information (e.g., a visual model of tongue placement and / or movement) and providing an indication to the patient on how to change the patient’s tongue location and / or movement. In other implementations, the biofeedback alert includes an indication as to the tongue’s current location and / or movement as well as an indication as to how the patient should change the patient’s tongue location and / or movement(e.g., an audio notification describing the patient’s tongue location and / or movement and then additional audio indicating how the patient should change the patient’s tongue location and / or movement).

[0079] In some implementations, providing biofeedback to a patient includes providing a visualization of tongue motion. As part of, separate, or in addition to, the real-time feedback on tongue placement and / or motion described above, a device paired with an app or software interface can visually display the tongue's movement and position at a single point in time or over a period of time. This can be especially beneficial for exercises that focus on controlling tongue movement, like sliding the tongue side to side against the upper palate. While the above example regarding real-time feedback describes a particular implementation where visualization of tongue motion can be provided in real-time, there is not requirement that such visualizations be provided in real-time. Thus, in this example of biofeedback, the visualizations may be generated based on output data of the machine learning model and provided in non-real time (e.g., at a later date, at a doctor’s appointment, etc.).

[0080] In some implementations, providing biofeedback includes providing instructions for strength training. In some implementations one or more myofunctional exercises are provided that aim to strengthen the tongue. For example, an exercise can be provided that instructs the patient to push their tongue against their palate and a wearable device worn by the patient can be configured to measure how strongly the patient pushes their tongue against the palate. Then, over time, the patient can aim to increase this force, ensuring the strengthening of the tongue muscles.

[0081] In some implementations, providing biofeedback includes providing progress tracking. Providing progress tracking can include, for example, continuously monitoring and saving data about the patient’s tongue placement, tongue movement, and / or tongue strength training over time, thus enabling the patient to track their progress over time. This can be beneficial for patient compliance and / or continued use of the wearable device implementing the systems of the present disclosure, and also provide valuable information for a therapist to adjust the therapy as needed.

[0082] In some implementations, providing biofeedback can include, for example, generation of customized therapeutic plans. For example, using the data collected from the output of the machine learning model and / or from one or more of the prior examples of providing biofeedback, a therapist can customize exercise plans for each patient. For instance, if the device detects that a patient struggles more with lateral tongue movements, the therapeutic plan for that patient can be configured to focus more on exercises that target this weakness.

[0083] In yet other implementations, other types of visualizations can be generated that model features of a patient’s body other than the mouth, tongue, and / or respiratory tract other than the described heatmap or visualizations for myofunctional therapy with biofeedback. For example, a visualization can be generated, based on the output data 135a, that indicates a flow of a liquid or gas through a portion of the entity’s body such as, e.g., the entity’s respiratory tract (or a portion thereof), the entity’s heart, one or more of an entity’s arteries, an entities GI tract (or portion thereof), or one or more of the entity’s lungs. This visualization can be an animated visualization that generates, in a sonar-like manner, a flow of the liquid or gas through one or more portions of the entity’s body.

[0084] FIG. 3 is a flowchart of an example of a process 300 for calibrating a machine learning model that has been trained to determine a source anatomical location and shape of a sound source based on input data generated by multiple sensors installed on a device (e.g., the wearable device 110). The process 300 will be described as being performed by a system such as the system 100 of FIG. 1 that includes one or more computers.

[0085] The system obtains (302), by one or more computers, vibration data generated by a device (e.g., the wearable device 110), in which the vibration data corresponds to a detection of vibrations due to a calibration sound source. In some implementations, the device is a wearable device like the device 110 of FIG. 1. In some other implementations, the device is a non-wearable device.

[0086] The system generates (304), by one or more computers, calibration data based on the obtained data corresponding to the calibration sound source. In some implementations, the generated calibration data includes feature values for each feature of data corresponding to the obtained data associated with the calibration sound source. In some implementations, the features include one or more of a time delay between sounds captured from multiple microphones of a device (e.g., a wearable or non-wearable device), an amplitude difference between sounds captured from multiple microphones of a wearable device, and multipath effects between sounds captured from multiple microphones of a wearable device.

[0087] In some implementations, the time delay between sounds captured from multiple microphones of a device (e.g., a wearable device or non-wearable device) are determined using a Generalized Cross Correlation (GCC). In some implementations, the amplitude difference between sounds captured from multiple microphones of a wearable device are determined using a Relative Transfer Function (RTF).

[0088] In some implementations, the generation of calibration data includes accessing label data specifying a set of pre-defined sounds associated with a portion of the source anatomicallocation and shape. The label data can include a label for each pre-defined sound that indicates an anatomical location of the source that produces the pre-defined sound. For example, a sound originating from an utterance of a particular consonant can be correlated with a source anatomical location of a particular part of the mouth and tongue orientation. As another example, a sound originating from a particular external vibration source can be correlated with a known position of the external sound source.

[0089] In some implementations, a signal that is generated due to a calibration sound source (e.g., utterance, external source, etc.), is identified as corresponding to a particular predefined sound of a set of pre-defined sounds. In some cases, a portion of vibration data corresponds to a particular pre-defined sound. In some implementations, based on the identification of the portion of the signal that corresponds to a pre-defined sound, the portion of the vibration data is associated to one or more portions of the source anatomical location and shape of the calibration sound source.

[0090] In some implementations, the calibration sound source is generated by an utterance of a calibration script by an entity. The utterance of the calibration script can include instructions for the entity to produce sounds corresponding to a vocalization of one or more consonant sounds. Each consonant sound originates from a well-defined source anatomical location (e.g., back of the mouth, front of the mouth, etc.).

[0091] In some implementations, the calibration sound source is a result of a naturally occurring anatomical process within the entity’s body. In some implementations, the vibration data associated with the calibration sound source due to a naturally occurring anatomical is generated while the entity’s body in a state of rest. A state of rest may be, e.g., an entity’s state after laying down on a surface for more than a threshold period of time or a sleeping state. In other implementations, the system generates the calibration data while the entity’s body in a state of daily, routine activity. A state of daily, routine activity may be, e.g., an entity’s state in the middle of a typical day of work or errands. In other implementations, the system generates the calibration data while the entity’s body in an elevated state of stress. The entity can be placed into an elevated state of stress by causing the entity to perform, e.g., a stress test (e.g., walking on a treadmill for a predetermined period of time).

[0092] In some implementations, the calibration sound source that originates from naturally occurring processes within the entity’s body originate from within the body of an entity such as sounds of blood flow through coronary arteries, sounds of gas flow in an entity’s abdomen (e.g., in GI tract or renal tract), gas (e.g., oxygen or carbon dioxide) flowing inside an entity’slungs, or the like. Though these sounds are provided as examples, any sound within the body that is output by an entity’s body can be used to generate calibration data.

[0093] In some implementations, the calibration sound source originates from a source external to the entity’s body. In some implementations, the external sound source is disposed on or near the skin of the entity’s body at a known location and produces vibrations. The known location refers to a location of the external sound source with a known distance from a wearable device (e.g., the wearable device 110 of FIG. 1). In some implementations, the generated calibration data includes feature values for each feature of data corresponding to the obtained sound output by the external sound source.

[0094] The system calibrates (306), by one or more computers, a trained machine learning model based on an association of a portion of vibration data to the one or more portions of the source anatomical location and shape. In some implementations, using one or more computers to calibrate the trained machine learning model includes re-training or fine-tuning the machine learning model using the calibration data generated at stage 304.

[0095] In some implementations, the system adjusts one or more parameters of the trained machine learning model based on the feature values of the data that corresponds to the calibration sound source. In some cases, the adjusted parameters represent particular aspects of the entity’s anatomy. For example, a time delay that is evaluated due to a detection of vibration data corresponding to a calibration sound source can be affected by particular anatomical dimensions and configurations of the entity’s body. The feature values corresponding to the time delay evaluation can affect the adjustment of the parameters of the trained machine learning model, resulting in a machine learning model calibrated to specific features of the entity’s anatomy.

[0096] FIG. 4 is an example user interface 400 for displaying locations of airway collapse during sleep. The user interface 400 includes a display of data generated from the system 100. For example, the user interface 400 includes temporal distributions 402 of a degree of obstruction of an airway. In some cases, the system derives the degree of obstruction displayed by the temporal distributions 402 from a predicted shape and location of a vibration source by the sound source generation engine 135.

[0097] The temporal distributions 402 include a horizontal axis indicative of a time while a subject is sleeping. The distributions 402 include a vertical axis indicative of an obstruction level. The temporal distributions 402 include four temporal distributions 404-410. The temporal distribution 404 is indicative of a degree of obstruction as a function of time associated with the velum, a soft tissue constituting the back of the roof of the mouth. Thetemporal distribution 406 is indicative of a degree of obstruction as a function of time associated with the oropharynx, the middle section of the throat, located behind the mouth and below the soft palate. The temporal distribution 408 is indicative of a degree of obstruction as a function of time associated with the tongue base. The temporal distribution 410 is indicative of a degree of obstruction as a function of time associated with the epiglottis, a flap of cartilage in the throat the prevents food and liquid from entering the lungs while swallowing. Each of the anatomical components (e.g., the velum, oropharynx, tongue base, and epiglottis) are positioned uniquely within a body.

[0098] A top indication bar 412 of the temporal distributions 402 graphically indicates periods of rapid eye movement (REM) sleep and periods of non-REM sleep. A bottom indication bar 414 of the temporal distributions 402 indicates periods of supine sleep and periods of non-supine sleep. An analysis of the temporal distributions 402 by an entity, doctor, medical professional, etc., facilitate a review of sleep patterns as they relate to a degree of obstruction associated with different anatomical locations.

[0099] Four anatomical locations 422 are displayed in the visualization of the human head 420. For each location 422, a degree of obstruction 426 (minimal, partial, or complete) is illustrated along with an obstruction profile 424 (antero-posterior, concentric, or latero- lateral). The anatomical location, profile, and degree of obstruction are determined and displayed for various sleeping conditions (REM, non-REM, supine, non-supine). The reported data is indicative of anatomical drivers of sleep apnea and other airway obstruction- related health challenges, including the location and configuration of airway obstructions.

[0100] In some implementations, the user interface 400 is implemented on a display, in which the display receives data from a computer that is communicatively coupled to the application server 130. In some implementations, the data displayed on the user interface 400 is collected during a period of time and displayed to a user after the period of time. In some other implementations, the data displayed on the user inface 500 is displayed to a user during a sleep session, in which a sensor is communicatively coupled to one or more processors like the application server 130, which processes and displays data as it is being collected.

[0101] FIG. 5 illustrates example vibrational (sound) signal paths from an anatomical vibration source to vibration sensors. The example signal paths are illustrated in a first anatomical configuration 500 that includes a single obstruction within an entity. In addition, the example signal paths are illustrated in a second anatomical configuration 550 that includes multiple obstructions within an entity. 1

[0102] Turning to the first configuration 500, an anatomical vibration source 502 emits a vibrational signal. The vibrational signal is detected by a first vibration sensor 508 by way of a first propagation path 506. The vibrational signal is detected by a second vibration sensor 510 by way of a second propagation path 504.

[0103] Turning to the second configuration 550, an anatomical vibration source 552 emits a vibrational signal. The vibrational signal is detected by a first vibration sensor 558 by way of a set of multiple propagation paths 556. The vibrational signal is detected by a second vibration sensor 560 by way of a set of multiple propagation paths 554. The sets of multiple propagation paths are a result of multiple obstructions positioned between the vibration source 552 and the vibration sensors 558-560.

[0104] In some cases, the multiple propagation paths illustrated in the second configuration 550 creates circumstances that prohibit a precise evaluation of the location of the vibration source 552. However, to account for potential multi-path effects, a transfer function quantifies changes in amplitude and phase of sound waves as they propagate through the human head. The transfer function captures the multi-path effects as the sound wave propagates from the source 552 to the sensors 558-560. A processor that receives data from the sensors 558-560 can determine a relative transfer function (RTF) from the combined data. To further improve accuracy, the system can include multiple vibration sensors (more than two) and determine the RTF between each possible sensor-pair combination.

[0105] FIG. 6 illustrates example data collected by a wearable device in response to vibrations that originate from example vibration sources. A first vibration source 610, illustrated by a bright spot in a depiction of the human head, corresponds to an entity producing a / t / sound. For this example, the front of the entity’s tongue restricts the airway towards the front of the mouth, as illustrated in configuration 606. A second vibration source 612 corresponds to an entity producing a / k / sound, in which the back of the tongue restricts the airway towards the back of the mouth, as illustrated in configuration 608.

[0106] In this example, a measurement device (e.g., the wearable device 110) is disposed on the cheek of an entity, e.g., a person associated with the depiction of the human head in configurations 606-608. The measurement device records vibration data as a function of time. A sample vibrational signal is illustrated in plot 620 which overlays data from the first vibration source 610 and the second vibration source 612. A difference in arrival time between a first signal 614 and a second signal 616 is indicative of the two vibration sources being spatially distinct, and the relative time for the vibrational energy (e.g., the sound) to travel from the source to the sensor is different between the two configurations 606 and 608.In this example, a sensor of the measurement device is closer to the point of restriction related to the first vibration source 610 compared to the second vibration source 612, so the relative signal associated with the source 610 precedes the signal associated with the source 612 in time.

[0107] A plot 602 illustrates a correlation between the actual distance between the sensor and the example vibration sources and a predicted distance between the sensor and example vibration sources, as predicted with a non-calibrated machine learning model. A plot 604 illustrates a correlation between the actual distance between the sensor and the example vibration sources and a predicted distance between the sensor and example vibration sources, as predicted with a calibrated machine learning model. The example vibration sources correspond to utterances with known airway restriction locations (e.g., / t / and / k / ).

[0108] As illustrated in plot 602 and 604, a calibrated machine learning model, as described in relation to previous figures, yields more precise predictions. The plot 604 illustrates data with a stronger correlation (closer to the one-to-one correlation line) in comparison with the data illustrated in plot 602.

[0109] FIG. 7 illustrates example machine learning model performance data. In some implementations, a machine learning model that processes vibration and / or sound data from a sensor to predict a classification parameter associated with the sound data requires reliable training data to achieve high accuracy. In some cases, a classification task includes a classification of a sound as a consonant or a vowel. In some other cases, a classification task includes identifying a period of time that includes an apneic episode (i.e., a period of time in which a person is affected by the effects of sleep apnea).

[0110] The example data 702 is displayed in a plot and is indicative of a signal representing a consonant sound 704 during a first time period and a signal representing a vowel sound 706 during a second time period. The table 710 displays data that represent performance metrics for real-time labeling of ten different consonants by a trained machine learning model.Metrics are calculated based on detecting the presence of a consonant within a time period, rather than on a frame-by-frame basis. In some implementations, a training system utilizes a TIMIT-based method, in which the TIMIT -based method employs training data from the TIMIT database which is an Acoustic-Phonetic Continuous Speech Corpus that includes time-aligned transcripts of English language recordings.

[0111] In the case of identifying a time period that includes an apneic episode or some other sleep-related event, in some implementations, a machine learning model training system implements a Gaussian soft labeling approach. The Gaussian soft labeling approachincludes assigning a probability distribution around an annotated start and end time of an event, e.g., an apneic event. The soft labeling approach can mitigate effects of inaccurate hard labeling in cases that include inherent mis-labeling of event start and end times. Soft labeling approaches allow a machine learning model to account for temporal uncertainty inherent in manual annotations by recognizing a continuum between relevant and non-relevant sounds (e.g., apnea and non-apnea sounds), aligning with probabilistic real-world clinical assessments, in which an exact demarcation of an event is not always clear.

[0112] In some implementations, the training system processes sleep sound recordings by segmenting them into overlapping temporal windows to ensure coverage of transition regions between events and non-events (e.g., apnea and non-apnea). For each annotated event, a training system generates a Gaussian distribution centered at the annotated start and end times, with a standard deviation that captures an expected annotation error. The Gaussian distributions produces a soft transition from event to non-event periods of time. The training system trains a machine learning model with the soft labels using a sigmoid output activation function optimized for binary cross-entropy loss, while treating the classification problem as a binary classification with soft target values.

[0113] FIG. 8 illustrates example time series data 800 collected from an example wearable device 802. The time series data 800 includes time series data 804-808 collected by a respective sensor of three sensors of the wearable device 802. The time series data 800 also illustrates two time series data 810-812 that each correspond to a signal from a voice activity detector (VAD) associated with a respective distinct threshold value. The VAD is integrated into an analog-to-digital (ADC) chip local to the wearable device 802.

[0114] In some implementations, a context-adaptive approach to data acquisition provides a non-continuous collection of run-time data (e.g., the run-time data 114). In some cases, critical data (e.g., data indicative of an apneic event) occurs sporadically within a continuous acoustic stream of data. By primarily capturing anatomical information during periods of airflow limitation like hypopneas and / or respiratory events that generate snoring and postapnea snores, the data storage, battery consumption, and data transfer requirements are decreased. In some cases, high collection rates are required during specific apneic events.

[0115] In some implementations, the context-adaptive approach employs one or more voice activity detectors (VAD) integrated into analog-to-digital (ADC) chips. When no voice is detected by the VAD, a sleep mode of the wearable device 802 can be activated. The ADC continuously monitors input channels that receive data from the VAD and signals the host (a circuit associated with the wearable device 802) to “wake up” and perform energy -intensivetasks such as data processing, switching to higher sampling rates, and recording the processed data. In some implementations, the VAD algorithm that determines if the wearable device 802 should “wake up” includes a decision tree classification and a 16-bit IIR filter bank for feature extraction, with decision-tree parameters, feature selection, and filter bank settings adjustable offline through coefficient-memory writes.

[0116] As depicted in the time series data 810 that corresponds to a signal output from the VAD according to a first threshold value, the VAD outputs an elevated signal in response to voice activity above a first threshold. As depicted in the time series data 812 that corresponds to a signal output from the VAD according to a second threshold value, the VAD outputs an elevated signal in response to voice activity above a second threshold.

[0117] The example data 800 demonstrates a scenario in which one sensor of the three sensors (in this case, the sensor that records the time series data 804) of the wearable device 802 is always active and continuously records audio data at a low sampling rate (e.g., 200 Hz). The continuously active sensor serves as a snore detector by capturing a sound envelope or loudness level. The continuous data provides an overview of the overnight sound profile that can be combined with higher-fidelity data from the other two sensors of the wearable device 802. The other two sensors (in this case, the sensors that record the time series data 806 and 808) are activated if the VAD signals indicate activity in the time series data 804. For example, the other two sensors are active when the VAD outputs an elevated signal indicating activity recorded by the continuously monitoring sensor. When the VAD outputs indicate activity, all three sensors sample at a high sampling rate (e.g., 48 kHz) for brief intervals (during the period of elevated VAD output signal).

[0118] The context-adaptive approach in which a subset of sensors of the wearable device 802 are inactive during times of received signal inactivity provides an energy efficient monitoring strategy. However, the approach may generate false positives or false negatives depending on a choice of the VAD threshold. In some instances, false positives are preferable over false negatives, and the VAD threshold can be chosen accordingly. In some cases, the wearable device 802 includes more than one VAD, each having a distinct threshold value.

[0119] FIG. 9 is an example user interface 900 for obtaining calibration data from a user. The example user interface 900 includes a first side 902 that includes control elements and indicators related to data collection. A second side 904 includes measured and analyzed data received, by a processor that implements functions of the user interface, from a wearable device (e.g., the wearable device 110).

[0120] The first side 902 includes a feedback control element 908 in which a user can approve or reject a calibration measurement. The first side 902 also includes a re-do control element 906 in which a user can repeat a calibration measurement. The first side 902 also includes an indicator that provides a visualization of a number of calibration measurements as a function of a total number of expected, required, or requested number of calibration measurements.

[0121] The second side 904 includes two time series data plots of measured vibration data. A first time series data plot, illustrated on top, represents a time trace indicative of vibrations produced by a user that utters the word “try.” The second time series plot, illustrated on the bottom, represents the same time trace as the first time series data plot with a highlighted region that corresponds to a detected portion of the time trace associated with a consonant sound / t / .

[0122] In some implementations, the user views the data displayed on the second side 904 in relation to the detection of the correct consonant sound / t / in the time trace associated with the utterance of the word “try,” to inform actions taken on the first side 902 (e.g., re-do the calibration measurement and / or approve or reject the calibration measurement).

[0123] FIG. 10 is an example user interface 1000 for obtaining calibration data. The user interface 1000 includes a first side 1002 that includes a visualization of a time series data associated with calibration data. In some implementations, the calibration data is a result of an utterance of a calibration script by a user. The time series data depicts a vibrational amplitude obtained by a wearable device as a function of time.

[0124] The user interface 1000 includes a second side 1004. The second side 1004 depicts a visualization of a human head with an overlay of a heatmap. The heatmap indicates a location of a sound source associated with the generation of calibration data that is visualized on the first side 1002. As depicted by the heatmap overlay ed on the human head, a bright spot is apparent towards the back of the mouth and the back of the tongue, which indicates the calibration script included a large number of consonant sounds associated with the / k / consonant.

[0125] FIG. 11 is an example user interface 1100 for displaying data indicative of a sleep summary. The user interface 1100 includes indicators for communicating sleep-related data to a user. The user interface 1100 includes a total sleep time indicator 1102 and a breakdown table 1104 that indicates a percentage of sleep time spent in each of four categorical stages of REM sleep, non-REM sleep, supine position, and non-supine position.

[0126] The user interface 1100 includes an indicator of a sleep quality score 1106 as a percentage value. In some implementations, the sleep quality score 1106 is based on measurable data values that can include sleep metrics from standard sleep tests along with anatomical parameters. In addition, the user interface 1100 includes temporal analysis plots. A first temporal analysis plot 1108 displays data indicative of sleep stages. The first temporal analysis plot 1108 illustrates a differentiation between periods of time in which a user was awake, lightly sleeping, deeply sleeping, and REM sleeping. Similarly, a second temporal analysis plot 1110 displays data indicative of sleeping positions. The second temporal analysis plot 1110 illustrates a differentiation between periods of time in which a user is sleeping in a supine position, in a prone position, on a right side, and on a left side.

[0127] FIG. 12 is an example user interface 1200 for displaying data indicative of a breathing summary. The user interface 1200 includes three temporal analysis plots.

[0128] A first temporal analysis plot 1202 depicts data associated with respiratory events. The plot 1202 illustrates data indicative of discrete periods of time in which the system detects a respiratory event. For example, the data represented in plot 1202 indicate times in which the user experiences hypopnea, obstructive sleep apnea, a mixed respiratory event, and central sleep apnea.

[0129] A second temporal analysis plot 1204 depicts data associated with arousal events. The plot 1204 illustrates data indicative of discrete periods of time in which the system detects an arousal event. For example, the data represented in plot 1204 indicate times in which the user is awake.

[0130] A third temporal analysis plot 1206 depicts data associated with snoring events. The plot 1206 illustrates data indicative of discrete periods of time in which the system detects a snoring event.

[0131] The user interface 1200 also includes a sleep data table 1208. The sleep data table 1208 depicts sleep metrics that include a respiratory disturbance index (RD I), an apnea- hypopnea index (AHI), and a measure of hypoxic burden displayed in units of %min / h. Each of the three sleep metrics are evaluated as a percentage of total sleep time during a full sleep session, periods of REM sleep, periods of non-REM sleep, periods of sleep in a supine position, and periods of sleep in the non-supine position.

[0132] FIG. 13 is an example user interface 1300 for visualizing data indicative of sleep metric ratings. The user interface 1300 includes a respective rating associated with each of four sleep metrics 1302. The four sleep metrics 1302 include airway collapsibility, loop gain, arousal threshold, and muscle responsiveness. Each sleep metric of the four sleep metrics1302 is determined to be in one of three rating ranges 1304 including “good”, “moderate,” and “bad”. For each sleep metric, a ratings bar, e.g., the ratings bar 1306, indicates a position in the rating range for the corresponding sleep metric. For example, the ratings bar 1306 indicates the sleep metric of airway collapsibility is evaluated to be in the “bad” range.

[0133] FIG. 14 illustrates example user interface components 1400 for comparing predicted values of airway obstruction parameters and measured (evaluated by a surgeon during a procedure) values of airway obstruction parameters. The user interface components 1400 include a first predicted obstruction configuration 1402 associated with four anatomical locations (velum, oropharynx, tongue base, and epiglottis) at the start of a drug-induced sleep endoscopy (DISE) procedure. The user interface components 1400 include a second predicted obstruction configuration 1404 associated with the four anatomical locations eight minutes after the DISE procedure begins. In the example illustration, the associated DISE procedure employs a propofol sedative that induces a shift in the anatomical obstruction configuration, as evaluated by an attending surgeon. A comparison of the first configuration 1402 and the second configuration 1404 indicates a shift from a concentric velar obstruction to an anteroposterior velar obstruction.

[0134] FIG. 15 is a block diagram of system components that can be used to implement a system for calibrating a machine learning model that has been trained to determine a location and shape of a sound source.

[0135] Computing device 1500 is intended to represent various forms of digital computers, such as laptops, desktops, workstations, personal digital assistants, servers, blade servers, mainframes, and other appropriate computers. Computing device 1550 is intended to represent various forms of mobile devices, such as personal digital assistants, cellular telephones, smartphones, and other similar computing devices. Additionally, computing device 1500 or 1550 can include Universal Serial Bus (USB) flash drives. The USB flash drives can store operating systems and other applications. The USB flash drives can include input / output components, such as a wireless transmitter or USB connector that can be inserted into a USB port of another computing device. The components shown here, their connections and relationships, and their functions, are meant to be exemplary only, and are not meant to limit implementations of the inventions described and / or claimed in this document.

[0136] Computing device 1500 includes a processor 1502, memory 1504, a storage device 1506, a high-speed interface 1508 connecting to memory 1504 and high-speed expansion ports 1510, and a low-speed interface 1512 connecting to low-speed bus 1514 and storagedevice 1506. Each of the components 1502, 1504, 1506, 1508, 1510, and 1512, are interconnected using various busses, and can be mounted on a common motherboard or in other manners as appropriate. The processor 1502 can process instructions for execution within the computing device 1500, including instructions stored in the memory 1504 or on the storage device 1508 to display graphical information for a GUI on an external input / output device, such as display 1516 coupled to high-speed interface 1508. In other implementations, multiple processors and / or multiple buses can be used, as appropriate, along with multiple memories and types of memory. Also, multiple computing devices 1500 can be connected, with each device providing portions of the necessary operations, e.g., as a server bank, a group of blade servers, or a multi -processor system.

[0137] The memory 1504 stores information within the computing device 1500. In one implementation, the memory 1504 is a volatile memory unit or units. In another implementation, the memory 1504 is a non-volatile memory unit or units. The memory 1504 can also be another form of computer-readable medium, such as a magnetic or optical disk.

[0138] The storage device 1508 is capable of providing mass storage for the computing device 1500. In one implementation, the storage device 1508 can be or contain a computer- readable medium, such as a floppy disk device, a hard disk device, an optical disk device, or a tape device, a flash memory or other similar solid state memory device, or an array of devices, including devices in a storage area network or other configurations. A computer program product can be tangibly embodied in an information carrier. The computer program product can also contain instructions that, when executed, perform one or more methods, such as those described above. The information carrier is a computer- or machine-readable medium, such as the memory 1504, the storage device 1508, or memory on processor 1502.

[0139] The high-speed controller 1508 manages bandwidth-intensive operations for the computing device 1500, while the low-speed controller 1512 manages lower bandwidth intensive operations. Such allocation of functions is exemplary only. In one implementation, the high-speed controller 1508 is coupled to memory 1504, display 1516, e.g., through a graphics processor or accelerator, and to high-speed expansion ports 1510, which can accept various expansion cards (not shown). In the implementation, low-speed controller 1512 is coupled to storage device 1508 and low-speed expansion port 1514. The low-speed expansion port, which can include various communication ports, e.g., USB, Bluetooth, Ethernet, wireless Ethernet can be coupled to one or more input / output devices, such as a keyboard, a pointing device, microphone / speaker pair, a scanner, or a networking device such as a switch or router, e.g., through a network adapter. The computing device 1500 can beimplemented in a number of different forms, as shown in the figure. For example, it can be implemented as a standard server 1520, or multiple times in a group of such servers. It can also be implemented as part of a rack server system 1524. In addition, it can be implemented in a personal computer such as a laptop computer 1522. Alternatively, components from computing device 1500 can be combined with other components in a mobile device (not shown), such as device 1550. Each of such devices can contain one or more of computing device 1500, 1550, and an entire system can be made up of multiple computing devices 1500, 1550 communicating with each other.

[0140] The computing device 1500 can be implemented in a number of different forms, as shown in the figure. For example, it can be implemented as a standard server 1520, or multiple times in a group of such servers. It can also be implemented as part of a rack server system 1524. In addition, it can be implemented in a personal computer such as a laptop computer 1522. Alternatively, components from computing device 1500 can be combined with other components in a mobile device (not shown), such as device 1550. Each of such devices can contain one or more of computing device 1500, 1550, and an entire system can be made up of multiple computing devices 1500, 1550 communicating with each other.

[0141] Computing device 1550 includes a processor 1552, memory 1564, and an input / output device such as a display 1554, a communication interface 1566, and a transceiver 1568, among other components. The device 1550 can also be provided with a storage device, such as a micro-drive or other device, to provide additional storage. Each of the components 1550, 1552, 1564, 1554, 1566, and 1568, are interconnected using various buses, and several of the components can be mounted on a common motherboard or in other manners as appropriate.

[0142] The processor 1552 can execute instructions within the computing device 1550, including instructions stored in the memory 1564. The processor can be implemented as a chipset of chips that include separate and multiple analog and digital processors.Additionally, the processor can be implemented using any of a number of architectures. For example, the processor 1510 can be a CISC (Complex Instruction Set Computers) processor, a RISC (Reduced Instruction Set Computer) processor, or a MISC (Minimal Instruction Set Computer) processor. The processor can provide, for example, for coordination of the other components of the device 1550, such as control of user interfaces, applications run by device 1550, and wireless communication by device 1550.

[0143] Processor 1552 can communicate with a user through control interface 1558 and display interface 1556 coupled to a display 1554. The display 1554 can be, for example, aTFT (Thin-Film-Transistor Liquid Crystal Display) display or an OLED (Organic Light Emitting Diode) display, or other appropriate display technology. The display interface 1556 can comprise appropriate circuitry for driving the display 1554 to present graphical and other information to a user. The control interface 1558 can receive commands from a user and convert them for submission to the processor 1552. In addition, an external interface 1562 can be provide in communication with processor 1552, so as to enable near area communication of device 1550 with other devices. External interface 1562 can provide, for example, for wired communication in some implementations, or for wireless communication in other implementations, and multiple interfaces can also be used.

[0144] The memory 1564 stores information within the computing device 1550. The memory 1564 can be implemented as one or more of a computer-readable medium or media, a volatile memory unit or units, or a non-volatile memory unit or units. Expansion memory 1574 can also be provided and connected to device 1550 through expansion interface 1572, which can include, for example, a SIMM (Single In Line Memory Module) card interface. Such expansion memory 1574 can provide extra storage space for device 1550 or can also store applications or other information for device 1550. Specifically, expansion memory 1574 can include instructions to carry out or supplement the processes described above and can include secure information also. Thus, for example, expansion memory 1574 can be provide as a security module for device 1550 and can be programmed with instructions that permit secure use of device 1550. In addition, secure applications can be provided via the SIMM cards, along with additional information, such as placing identifying information on the SIMM card in a non-hackable manner.

[0145] The memory can include, for example, flash memory and / or NVRAM memory, as discussed below. In one implementation, a computer program product is tangibly embodied in an information carrier. The computer program product contains instructions that, when executed, perform one or more methods, such as those described above. The information carrier is a computer- or machine-readable medium, such as the memory 1564, expansion memory 1574, or memory on processor 1552 that can be received, for example, over transceiver 1568 or external interface 1562.

[0146] Device 1550 can communicate wirelessly through communication interface 1566, which can include digital signal processing circuitry where necessary. Communication interface 1566 can provide for communications under various modes or protocols, such as GSM voice calls, SMS, EMS, or MMS messaging, CDMA, TDMA, PDC, WCDMA, CDMA2000, or GPRS, among others. Such communication can occur, for example, throughradio-frequency transceiver 1568. In addition, short-range communication can occur, such as using a Bluetooth, Wi-Fi, or other such transceiver (not shown). In addition, GPS (Global Positioning System) receiver module 1570 can provide additional navigation- and location- related wireless data to device 1550, which can be used as appropriate by applications running on device 1550.

[0147] Device 1550 can also communicate audibly using audio codec 1560, which can receive spoken information from a user and convert it to usable digital information. Audio codec 1560 can likewise generate audible sound for a user, such as through a speaker, e.g., in a handset of device 1550. Such sound can include sound from voice telephone calls, can include recorded sound, e.g., voice messages, music files, etc. and can also include sound generated by applications operating on device 1550.

[0148] The computing device 1550 can be implemented in a number of different forms, as shown in the figure. For example, it can be implemented as a cellular telephone 1580. It can also be implemented as part of a smartphone 1582, personal digital assistant, or another similar mobile device.

[0149] Various implementations of the systems and methods described here can be realized in digital electronic circuitry, integrated circuitry, specially designed ASICs (application specific integrated circuits), computer hardware, firmware, software, and / or combinations of such implementations. These various implementations can include implementation in one or more computer programs that are executable and / or interpretable on a programmable system including at least one programmable processor, which can be special or general purpose, coupled to receive data and instructions from, and to transmit data and instructions to, a storage system, at least one input device, and at least one output device.

[0150] These computer programs (also known as programs, software, software applications or code) include machine instructions for a programmable processor and can be implemented in a high-level procedural and / or object-oriented programming language, and / or in assembly / machine language. As used herein, the terms "machine-readable medium" "computer-readable medium" refers to any computer program product, apparatus and / or device, e.g., magnetic discs, optical disks, memory, Programmable Logic Devices (PLDs), used to provide machine instructions and / or data to a programmable processor, including a machine-readable medium that receives machine instructions as a machine-readable signal. The term "machine-readable signal" refers to any signal used to provide machine instructions and / or data to a programmable processor.

[0151] To provide for interaction with a user, the systems and techniques described here can be implemented on a computer having a display device, e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor for displaying information to the user and a keyboard and a pointing device, e.g., a mouse or a trackball by which the user can provide input to the computer. Other kinds of devices can be used to provide for interaction with a user as well; for example, feedback provided to the user can be any form of sensory feedback, e.g., visual feedback, auditory feedback, or tactile feedback; and input from the user can be received in any form, including acoustic, speech, or tactile input.

[0152] The systems and techniques described here can be implemented in a computing system that includes a back end component, e.g., as a data server, or that includes a middleware component, e.g., an application server, or that includes a front end component, e.g., a client computer having a graphical user interface or a Web browser through which a user can interact with an implementation of the systems and techniques described here, or any combination of such back end, middleware, or front end components. The components of the system can be interconnected by any form or medium of digital data communication, e.g., a communication network. Examples of communication networks include a local area network ("LAN"), a wide area network ("WAN"), and the Internet.

[0153] The computing system can include clients and servers. A client and server are generally remote from each other and typically interact through a communication network. The relationship of client and server arises by virtue of computer programs running on the respective computers and having a client-server relationship to each other.

[0154] In addition to the embodiments described above, the following embodiments are also innovative:

[0155] Embodiment l is a method for calibrating a machine learning model that has been trained to determine a source anatomical location and shape of a sound source based on input data generated by a plurality of sensors installed on a device, the method comprising:

[0156] obtaining, by one or more computers, vibration data generated by the device, the vibration data corresponding to a generated signal in response to a detection of vibrations due to a calibration sound source;

[0157] generating, by one or more computers, calibration data based on the obtained data corresponding to the calibration sound source, wherein generating the calibration data comprises:

[0158] accessing label data specifying a set of pre-defined sounds associated with a portion of the source anatomical location and shape;

[0159] identifying at least a portion of the vibration data corresponding to a particular pre-defined sound of the set of pre-defined sounds;

[0160] based on the identifying, associating the portion of the vibration data to one or more portions of the source anatomical location and shape; and

[0161] calibrating, by one or more computers, the trained machine learning model based on the association of the portion of the vibration data to the one or more portions of the source anatomical location and shape.

[0162] Embodiment 2 is the method of embodiment 1, wherein the calibration sound source is a result of an utterance of a calibration script generated by an entity, wherein the device is disposed on the entity’s body.

[0163] Embodiment 3 is the method of any of embodiment 1-2, wherein the utterance of the calibration script comprises instructions for the entity to produce sounds corresponding to a vocalization of one or more consonant sounds.

[0164] Embodiment 4 is the method of any of embodiment 1-3, wherein the calibration sound source is a result of a naturally occurring anatomical process within an entity’s body, wherein the device is disposed on the entity’s body.

[0165] Embodiment 5 is the method of any of embodiment 1-4, wherein the data corresponding to the calibration sound source as the result of the naturally occurring anatomical process within the entity’s body is obtained while the user is in a state of rest.

[0166] Embodiment 6 is the method of any of embodiment 1-5, wherein the data corresponding to the calibration sound source as the result of the naturally occurring anatomical process within the entity’s body is obtained while the user is in a state of routine activity.

[0167] Embodiment 7 is the method of any of embodiment 1-6, wherein the data corresponding to the calibration sound source as the result of the naturally occurring anatomical process within the entity’s body is obtained while the user is in a state of stress.

[0168] Embodiment 8 is the method of any of embodiment 1-7, wherein the calibration sound source is a sound source external to an entity’s body, wherein the device is disposed on the entity’s body.

[0169] Embodiment 9 is the method of any of embodiment 1-8, wherein the plurality of features comprises one or more of a time delay, an amplitude difference, and multipath effects.

[0170] Embodiment 10 is the method of any of embodiment 1-9, wherein the time delay comprises a time delay between signals received by a plurality of microphones of the device, wherein the time delay between the plurality of microphones is determined by performing operations comprising:

[0171] receiving, by a processor from a first microphone of the device, a first signal corresponding to a detection of a vibration from the calibration sound source;

[0172] receiving, by the processor from a second microphone of the device, a second signal corresponding to a detection of a vibration from the calibration sound source; and

[0173] determining, based on the first signal and the second signal, an output of a generalized cross correlation (GCC) function, the output indicative of a degree of similarity between the first signal and the second signal.

[0174] Embodiment 11 is the method of any of embodiment 1-10, wherein the amplitude difference comprises a difference in signal amplitude between signals received by a plurality of microphones of the device, wherein the amplitude difference between the plurality of microphones is determined by performing operations comprising:

[0175] receiving, by a processor from a first microphone of the device, a first signal corresponding to a detection of a vibration from the calibration sound source;

[0176] receiving, by the processor from a second microphone of the device, a second signal corresponding to a detection of a vibration from the calibration sound source; and

[0177] determining, based on the first signal and the second signal, an output of a relative transfer function (RTF), the output indicative of a degree of similarity between the first signal and the second signal.

[0178] Embodiment 12 is the method of any of embodiment 1-11, wherein calibrating, by the one or more computers, the trained machine learning model using the generated calibration data comprises:

[0179] adjusting, by one or more computers, one or more parameters of the trained machine learning model based on the plurality of feature values of the data corresponding to the calibration sound source, wherein the adjusted parameters represent particular aspects of the entity’s anatomy.

[0180] Embodiment 13 is the method of any of embodiment 1-12, wherein the obtained data comprises a data field indicative of whether the obtained data corresponds to calibration data or run-time data.

[0181] Embodiment 14 is a system comprising:

[0182] one or more computers; and

[0183] one or more computer-readable storage devices storing instructions that, when executed by the one or more computers, cause the one or more computers to perform the operations of method embodiments 1-13.

[0184] Embodiment 15 is one or more computer-readable storage devices storing instructions that, when executed by one or more computers, cause the one or more computers to perform the operations of method embodiments 1-13.

[0185] While this specification contains many specific implementation details, these should not be construed as limitations on the scope of any invention or on the scope of what may be claimed, but rather as descriptions of features that may be specific to particular embodiments of particular inventions. Certain features that are described in this specification in the context of separate embodiments can also be implemented in combination in a single embodiment. Conversely, various features that are described in the context of a single embodiment can also be implemented in multiple embodiments separately or in any suitable subcombination. Moreover, although features may be described above as acting in certain combinations and even initially be claimed as such, one or more features from a claimed combination can in some cases be excised from the combination, and the claimed combination may be directed to a subcombination or variation of a subcombination.

[0186] Similarly, while operations are depicted in the drawings in a particular order, this should not be understood as requiring that such operations be performed in the particular order shown or in sequential order, or that all illustrated operations be performed, to achieve desirable results. In certain circumstances, multitasking and parallel processing may be advantageous. Moreover, the separation of various system modules and components in the embodiments described above should not be understood as requiring such separation in all embodiments, and it should be understood that the described program components and systems can generally be integrated together in a single software product or packaged into multiple software products.

[0187] Particular embodiments of the subject matter have been described. Other embodiments are within the scope of the following claims. For example, the actions recited in the claims can be performed in a different order and still achieve desirable results. As one example, the processes depicted in the accompanying figures do not necessarily require the particular order shown, or sequential order, to achieve desirable results. In certain implementations, multitasking and parallel processing may be advantageous.

Claims

WHAT IS CLAIMED IS:

1. A method for calibrating a machine learning model that has been trained to determine a source anatomical location and shape of a sound source based on input data generated by a plurality of sensors installed on a device, the method comprising: obtaining, by one or more computers, vibration data generated by the device, the vibration data corresponding to a generated signal in response to a detection of vibrations due to a calibration sound source; generating, by one or more computers, calibration data based on the obtained data corresponding to the calibration sound source, wherein generating the calibration data comprises: accessing label data specifying a set of pre-defined sounds associated with a portion of the source anatomical location and shape; identifying at least a portion of the vibration data corresponding to a particular pre-defined sound of the set of pre-defined sounds; based on the identifying, associating the portion of the vibration data to one or more portions of the source anatomical location and shape; and calibrating, by one or more computers, the trained machine learning model based on the association of the portion of the vibration data to the one or more portions of the source anatomical location and shape.

2. The method of claim 1, wherein the calibration sound source is a result of an utterance of a calibration script generated by an entity, wherein the device is disposed on the entity’s body.

3. The method of claim 2, wherein the utterance of the calibration script comprises instructions for the entity to produce sounds corresponding to a vocalization of one or more consonant sounds.

4. The method of any one of claims 1-3, wherein the calibration sound source is a result of a naturally occurring anatomical process within an entity’s body, wherein the device is disposed on the entity’s body.

5. The method of claim 4, wherein the data corresponding to the calibration sound source as the result of the naturally occurring anatomical process within the entity’s body is obtained while the user is in a state of rest.

6. The method of claim 4, wherein the data corresponding to the calibration sound source as the result of the naturally occurring anatomical process within the entity’s body is obtained while the user is in a state of routine activity.

7. The method of claim 4, wherein the data corresponding to the calibration sound source as the result of the naturally occurring anatomical process within the entity’s body is obtained while the user is in a state of stress.

8. The method of any one of claims 1-7, wherein the calibration sound source is a sound source external to an entity’s body, wherein the device is disposed on the entity’s body.

9. The method of any one of claims 1-8, wherein the plurality of features comprises one or more of a time delay, an amplitude difference, and multipath effects.

10. The method of claim 9, wherein the time delay comprises a time delay between signals received by a plurality of microphones of the device, wherein the time delay between the plurality of microphones is determined by performing operations comprising: receiving, by a processor from a first microphone of the device, a first signal corresponding to a detection of a vibration from the calibration sound source; receiving, by the processor from a second microphone of the device, a second signal corresponding to a detection of a vibration from the calibration sound source; and determining, based on the first signal and the second signal, an output of a generalized cross correlation (GCC) function, the output indicative of a degree of similarity between the first signal and the second signal.

11. The method of claim 9, wherein the amplitude difference comprises a difference in signal amplitude between signals received by a plurality of microphones of the device, wherein the amplitude difference between the plurality of microphones is determined by performing operations comprising:receiving, by a processor from a first microphone of the device, a first signal corresponding to a detection of a vibration from the calibration sound source; receiving, by the processor from a second microphone of the device, a second signal corresponding to a detection of a vibration from the calibration sound source; and determining, based on the first signal and the second signal, an output of a relative transfer function (RTF), the output indicative of a degree of similarity between the first signal and the second signal.

12. The method of any one of claims 1-11, wherein calibrating, by the one or more computers, the trained machine learning model using the generated calibration data comprises: adjusting, by one or more computers, one or more parameters of the trained machine learning model based on the plurality of feature values of the data corresponding to the calibration sound source, wherein the adjusted parameters represent particular aspects of the entity’s anatomy.

13. The method of any one of claims 1-12, wherein the obtained data comprises a data field indicative of whether the obtained data corresponds to calibration data or run-time data.

14. A system comprising: one or more computers; and one or more computer-readable storage devices storing instructions that, when executed by the one or more computers, cause the one or more computers to perform the operations of method claims 1-13.

15. One or more computer-readable storage devices storing instructions that, when executed by one or more computers, cause the one or more computers to perform the operations of method claims 1-13.

16. A method for analyzing an airway based on an output of a calibrated machine learning model, the method comprising: obtaining, by one or more computers and from a device worn by a patient, first vibration data representing vibrations in the patient’s airway, wherein a plurality of sensorsare installed on the device, the vibrations generated by a first anatomical sound source of the patient, wherein the vibrations are generated either by the patient in response to an instruction provided to the patient or by an external sound source disposed on or near the patient’s body; generating, by one or more computers, calibration data based on the first vibration data corresponding to the vibrations, wherein generating the calibration data comprises: accessing label data specifying a set of pre-defined sounds associated with a portion of the location and shape of the first anatomical sound source; identifying at least a portion of the first vibration data corresponding to a particular pre-defined sound of the set of pre-defined sounds, wherein the particular predefined sound is associated with the provided instruction; based on the identifying, associating the portion of the first vibration data to one or more portions of the first source anatomical location and shape; calibrating, by one or more computers, the trained machine learning model based on the association of the portion of the first vibration data to the one or more portions of the first anatomical sound source location and shape; receiving, by one or more computers and from the device worn by the patient, second vibration data representing vibrations in the patient’s airway, the vibrations generated by a second anatomical sound source of the patient, wherein the patient generates the vibrations during a sleep cycle; and processing, by the calibrated trained machine learning model, the second vibration data to generate output data indicative of a predicted location and shape of the second anatomical sound source of the patient.

Images