Lung health sensing by sound analysis

Abstract

A patient monitoring system includes a microphone that collects audio data from a patient. The audio data is used to generate audio characteristics to classify the audio data and analyze the audio data to determine a patient health status. The audio characteristics and the patient health status are tracked over time and used to monitor a patient respiratory condition. Based on a comparison between the audio characteristics and a database of audio characteristics associated with recorded health statuses of various patients, the system determines the patient health status. Based on the database of audio characteristics and associated health statuses, the system generates a report of the current patient health status.

Description

[0001] Cross Reference to Related Applications

[0002] This application claims priority to U.S. Provisional Patent Application No. 63 / 073,561, filed September 2, 2020, the entirety of which is incorporated herein by reference. TECHNICAL FIELD

[0003] The present application relates to medical equipment, and in particular, to systems and methods associated with determining pulmonary health information, patient rehabilitation status, and / or other parameters. BACKGROUND

[0004] Many respiratory diseases and respiratory conditions are associated with symptoms that affect a patient’s respiratory tract (e.g., nasal cavity, oral cavity, trachea, bronchi, lungs, etc.). Additionally, based on the symptoms associated with respiratory diseases and respiratory conditions, a patient’s ability to speak and breathe can be impaired, altered, or otherwise affected. Further, based on the symptoms associated with respiratory diseases and respiratory conditions, the symptoms a patient is experiencing can cause audio events, audible or inaudible to a human, to occur. Such diseases can include asthma, chronic obstructive pulmonary disease (COPD), bronchitis, emphysema, lung cancer, cystic fibrosis, pneumonia, and pleural effusion. Symptoms resulting from respiratory diseases and respiratory conditions can include inflammation of the respiratory tract, changes and / or reductions in lung capacity, mucus production, coughing, difficulty breathing, and other respiratory symptoms. As such, a physician can diagnose a patient based on one or more of the above symptoms and can provide medical treatment for the respiratory disease.

[0005] However, while a physician can observe the above symptoms through examination and inspection of a patient, the physician can only observe the patient’s symptoms when the patient is actively receiving diagnosis and / or treatment within a medical environment. Additionally, monitoring of a patient’s symptoms during treatment and rehabilitation is generally achieved through observation by a physician within a medical environment and / or using monitoring equipment. Such equipment typically includes physical leads and / or attachments coupled to a patient in order to effectively monitor the patient’s respiratory activity. Such equipment can be cumbersome and difficult to use for a physician. The leads or other attachments of such equipment can also cause discomfort if coupled to a patient for an extended period of time. The continuous passive monitoring received by a patient involves the use of technology and equipment local to the medical environment, and thus the use of such equipment can also be limited.

[0006] Various examples of the present disclosure are directed to addressing one or more of the problems set forth above. SUMMARY

[0007] In an example of the present disclosure, a system can include a microphone operable to generate audio data from a patient and / or individual. The audio data captured by the microphone associated with the system can originate from a human vocal cord, a human lung, a human movement, and / or other sources associated with the patient and / or individual. In particular, a patient inhaling and exhaling air can affect the speech production of the patient's vocal cord. Moreover, when the patient has a respiratory disease, a respiratory illness, and / or a respiratory condition, the patient's vocal cord and lung can be affected, thereby changing the speech produced by the patient. Based on the patient's voice and other respiratory sounds of the patient, the microphone associated with the system can generate sound parameters. The sound parameters characterize at least a pitch, a timbre, a rhythm, a volume, and a speech rate associated with the patient. The system can utilize a sound recognition algorithm to analyze the sound parameters associated with the patient to match and compare the current state of the patient with a previous state of the patient and / or an additional state associated with a second patient.

[0008] An example system can include a memory, one or more processors, and computer- executable instructions stored in the memory and executable by the one or more processors to perform various operations. For example, the operations can include receiving, from a user device, current speech data associated with a patient and identifying, based at least on the current speech data, at least one of a vocal utterance and a respiratory sequence of the patient. Moreover, the operations can include determining, based on the at least one of the vocal utterance and the respiratory sequence and based on at least one of a previous vocal utterance associated with the patient and a previous respiratory sequence associated with the patient, one or more audio events. Additionally, the operations can include determining one or more first audio characteristics associated with the one or more audio events and determining, based at least on the one or more first audio characteristics, an audio event category associated with the one or more audio events, wherein the audio event category associated with the second audio characteristics indicates a respiratory condition of the patient. In turn, a current health status of the patient associated with the respiratory condition can be determined based at least on a comparison of the one or more audio events to the audio event category.

[0009] An example method performed by a patient monitoring system can include causing a microphone to record current audio data from a patient, the current audio data including a patient vocal utterance and a patient respiratory sequence and receiving, from the microphone, the current audio data. Moreover, the method can include identifying, based at least on previous audio data, a current vocal characteristic associated with the patient vocal utterance and a current respiratory characteristic associated with the patient respiratory sequence and training a machine learning algorithm using the previous audio data, a previous vocal characteristic associated with the patient, and a previous respiratory sequence associated with the patient. Additionally, the method can include determining, using the machine learning algorithm, a current health status of the patient based on a comparison of the current vocal characteristic to the previous vocal characteristic.

[0010] An additional example patient monitoring system can perform operations including causing a microphone associated with a user device to collect a first audio recording, where the first audio recording includes at least one of a patient speaking or a patient breathing; receiving the first audio recording from the user device; determining, based at least on the first audio recording, an audio category associated with the first audio recording, the audio category being associated with one or more human voice utterances that share one or more audio characteristics. Further, based at least on the audio category, the operations can include determining a first patient health state, the first patient health state being associated with one or more respiratory symptoms of the patient. Additionally, the operations can include causing the microphone associated with the user device to collect a second audio recording; identifying, based at least on the second audio recording, that the one or more audio characteristics are associated with the first audio recording and the second audio recording; and determining, based at least on the one or more audio characteristics, a second patient health state, where the second patient health state indicates one or more health state changes relative to the first patient health state. BRIEF DESCRIPTION OF DRAWINGS

[0011] The features, nature, and various advantages of the present disclosure will be more apparent from the following detailed description considered in conjunction with the accompanying drawings.

[0012] Figure 1 An example system of the present disclosure is shown. In certain embodiments, Figure 1 The components of the example system shown can be used to monitor a current health state of a patient.

[0013] Figure 2 Another example system of the present disclosure is shown.

[0014] Figure 3 A data processing workflow of the present disclosure is shown. In certain embodiments, Figure 1 and Figure 2 The example system shown can utilize a series of algorithms configured to identify and classify patient data from incoming audio data.

[0015] Figure 4 A flowchart showing an example method of the present disclosure is provided.

[0016] Figure 5 A flowchart showing another example method of the present disclosure is provided.

[0017] Figure 6 Example user devices and servers of the present disclosure that can operate to monitor a current health state of a patient are provided.

[0018] In the drawings, the left-most digit(s) of each component reference number identifies the first occurrence of the component in the drawing. The use of same reference numbers in different drawings indicates like or similar components or features. The drawings are not to scale. DETAILED DESCRIPTION

[0019] The present disclosure relates, in part, to a patient monitoring system and corresponding method. Such an exemplary patient monitoring system can be configured to collect speech data and audio data associated with a patient or individual, analyze the speech data and audio data, and output the results of the analysis to a user of the device, such as a physician or physician’s assistant. For example, the patient monitoring system can utilize one or more microphones associated with a medical device, a user device, a patient device, or other device configured to collect speech data and audio data. The system can receive the speech data and audio data and perform a comparison and matching analysis through a voice recognition algorithm. Further, the system can determine, based on the comparison and matching analysis, whether the speech data and audio data match previous speech data and previous audio data for the patient and determine a comparison result of the speech data and audio data to the previous speech data and previous audio data. As such, the system can be used to monitor a patient’s status by analyzing various vocal characteristics and additional audio characteristics produced by the patient. Further, the system can generate a recommendation / diagnosis associated with the patient for display to a user of the patient monitoring system. For example, utilizing standard test data and / or machine learning techniques, the system can evaluate the measurements determined by the system to provide a recommendation to the user regarding the health of the patient (e.g., whether the patient meets one or more thresholds, whether additional monitoring of the patient will be completed, etc.). As such, the system described herein can provide automated diagnostic recommendations to assist a physician or other user of the patient monitoring system.

[0020] In any of the examples described herein, the patient monitoring system can be configured to monitor a patient diagnosed with one or more respiratory conditions. For example, the patient can be diagnosed with asthma, chronic obstructive pulmonary disease (COPD), bronchitis, emphysema, lung cancer, cystic fibrosis, pneumonia, pleural effusion, or other respiratory conditions. Further, the patient monitoring system can be configured to track one or more voice characteristics that can change as a result of the patient’s one or more respiratory conditions. For example, the patient monitoring system can be configured to determine one or more voice characteristics from audio data collected from the patient, the one or more voice characteristics including pitch, timbre, rhythm, volume, speech rate, audio events, and other voice characteristics associated with the patient. Additionally, the patient monitoring system can be configured to monitor one or more voice characteristics associated with one or more respiratory conditions. The one or more voice characteristics can be passively collected from the patient (e.g., without requiring a triggering event or phrase) to monitor a recovery or treatment progress associated with the respiratory condition. For example, the one or more respiratory conditions can be associated with one or more symptoms that can be related to a change in voice characteristics of the patient. In certain examples, asthma can be associated with inflammation in the patient’s airways; COPD can be associated with an inability to fully exhale and / or exhale properly; bronchitis can be associated with excess mucus in the patient’s airways and chronic coughing; emphysema can be associated with the patient’s difficulty in exhaling air from the patient’s lungs; lung cancer can be associated with chronic coughing, changes in the patient’s voice, a harsh respiratory sound, and coughing up blood; cystic fibrosis can be associated with chronic coughing, frequent lung infections, mucus buildup in the patient’s airways, frequent respiratory infections, wheezing, and shortness of breath; pneumonia can be associated with shortness of breath; pleural effusion can be associated with chest tightness and shortness of breath; and so on. Accordingly, the patient monitoring system can determine whether the patient is responding to treatment, whether the patient’s condition is improving, and how the patient’s current state is based at least on the one or more voice characteristics of the patient.

[0021] Further, in any of the examples described herein, the patient monitoring system can utilize one or more algorithms, neural networks, machine learning components, and / or other components configured to track one or more voice characteristics over time. In particular, the patient monitoring system can be configured such that changes in the one or more voice characteristics can be associated with a patient state. Further, the patient monitoring system can receive a patient state indication and associate the patient state with one or more voice characteristics captured at a time associated with the indication. Accordingly, the patient monitoring system can identify, at a second time, that the patient is associated with the state based on the one or more voice characteristics or that the patient is no longer associated with the state based on the one or more voice characteristics. Additionally, the patient monitoring system can be provided with a database of patient state indications, where patient state indications from the database can be associated with a set of voice characteristics that the patient monitoring system uses to determine whether a current patient state matches the patient state indication.

[0022] The following reference Figures 1 to 6 Additional details regarding the above-described techniques are described. It should be understood that these figures depict exemplary systems and devices that can utilize the methods claimed in this disclosure, and the methods, processes, functions, operations, and / or techniques described herein can also be applied to other devices, systems, etc.

[0023] Figure 1 An exemplary system 100, such as a patient monitoring system, is shown according to certain embodiments. Figure 1 As shown, an exemplary patient monitoring system can be used to monitor the current state of patient 102 based at least on the breathing sounds 104 generated by patient 102. For example, medical device 106 can be configured to provide treatment for respiratory illness, airway injury, and / or one or more other respiratory conditions. In at least some instances, medical device 106 can be a vest or other device providing respiratory assistance, a ventilator, a continuous positive airway pressure (CPAP) device, a drug delivery device, an airway clearance device, and other respiratory devices that can be used to treat, monitor, or otherwise assist a patient. As described herein, medical device 106 may include or be associated with microphone 108. Furthermore, control unit 110 can be configured to interface with microphone 108 such that audio data recorded by the microphone can be transmitted to control unit 110. Additionally, microphone 108 can be mounted, attached, attached, or otherwise physically connected to medical device 106.

[0024] In some instances, microphone 112 may be configured to monitor the breathing sounds 104 generated by the patient and generate audio data from the breathing sounds 104. Additionally, user equipment 114 may include microphone 112 and be configured to receive and / or record the audio data generated by microphone 112. Furthermore, user equipment 114 may be a personal device associated with patient 102, an additional device associated with medical device 106, and / or another device configured to monitor patient 102. Microphone 112 may be an internal microphone associated with user equipment 114 or an external microphone already communicatively associated with user equipment 114.

[0025] It should be understood that Figure 1The system 100 is depicted as including a single medical device 106 and a single user device 114, in certain additional examples, the system 100 can include any number of local or remote medical devices and / or user devices substantially similar to the medical device 106 and the user device 114, configured to operate independently and / or in combination, and configured to communicate over the network 116. In still certain examples, the system 100 can include one or more databases 118 and one or more audio analysis systems 120, including one or more processors 122, one or more network interfaces 124, and / or an audio screening component 126. The audio screening component 126 can include one or more programs, modules, engines, instructions, algorithms, neural networks, machine learning components, and / or other processor 122 executable audio screening components.

[0026] As described herein, the medical device 106, the control unit 110, the user device 114, and / or the audio analysis system 120 can be configured to monitor the patient 102 before, during, or after treatment of the patient 102. For example, monitoring the patient 102 can include collecting respiratory sounds 104 from the patient 102 over a period of time, analyzing audio data generated from the respiratory sounds 104, determining one or more vocal / human voice characteristics in the audio data, tracking one or more vocal / human voice characteristic variants over the period of time, and determining patient health information in the one or more vocal / human voice characteristic variants. Specifically, the medical device 106, the control unit 110, and / or the user device 114 can be configured to receive audio data generated from the respiratory sounds 104 from at least one of the microphone 108 and / or the microphone 112. Further, the medical device 106, the control unit 110, and / or the user device 114 can be configured to analyze the audio data locally to determine one or more vocal characteristics and track the one or more vocal characteristics over time. Alternatively or additionally, the medical device 106, the control unit 110, and / or the user device 114 can include wireless and / or wired communication capabilities that enable the audio data generated from the respiratory sounds 104 to be transmitted to other devices associated with the patient monitoring system (i.e., the medical device 106, the control unit 110, the user device 114, and / or the audio analysis system 120). It should be noted that the microphone 108 and / or the microphone 112 can include a recording assembly that includes a transducer (e.g., a component that is capable of converting a sound or mechanical motion into an electrical signal), a diaphragm that converts sound into mechanical motion, a signal output (e.g., a port, a cable, or other signal carrying component), a microphone housing, and / or any other audio device configured to at least collect the respiratory sounds 104 and provide a signal that can be recorded as audio data. In still other instances, the microphone 108 and / or the microphone 112 can be a condenser microphone, a radio frequency (RF) condenser microphone, an electret microphone, a moving coil microphone, an aluminum ribbon microphone, a piezoelectric microphone, an optical fiber microphone, or other microphone capable of capturing the respiratory sounds 104. Regardless of the type of microphone used by the aforementioned devices, the microphone used in association with the patient monitoring system can be selected based on the ability to make a high-fidelity recording (e.g., the ability to record human audible and inaudible soundings while accurately capturing the audio profile of the audible and inaudible soundings).

[0027] In some examples, the memory associated with the medical device 106, the control unit 110, the user device 114, and / or the audio analysis system 120 can be configured to store and / or access data associated with the patient 102. For example, the patient 102 and / or a user of the patient monitoring system can provide data (referred to herein as "patient data") at the initiation of patient monitoring. For example, when the medical device 106, the control unit 110, the user device 114, and / or the audio analysis system 120 are used to monitor the respiratory sounds 104 associated with the patient 102, a patient data file can be created for the audio data generated from the patient and include the patient data. Additionally, the patient 102 can provide patient data or a user can request patient data including demographic information, physical characteristics, preferences, and similar information about the patient 102. For example, the patient 102 can provide demographic information such as name, age, race, gender, and the like. The patient 102 can also provide physical characteristics information such as the height of the patient 102. In these examples, the patient 102 can be monitored concurrently or prior to the initiation of monitoring, and the user can request the patient data. In certain examples, the user can be provided with predetermined categories associated with the patient 102, such as a predetermined age range (e.g., 6 to 12 months, 1 to 5 years, etc.), and the user can request the patient data in order to select the appropriate category associated with the patient 102. In other examples, the user can provide free-form input associated with the patient data. In yet other examples, input elements can be provided directly to the patient 102.

[0028] In certain examples, the medical device 106, the control unit 110, the user device 114, and / or the audio analysis system 120 can be configured to generate audio data associated with the patient 102 at the initiation of the patient monitoring process. For example, the medical device 106 and / or the user device 114 can include one or more microphones, audio sensors, or other audio capture devices configured to generate audio data from the respiratory sounds 104 of the patient 102 over time. Alternatively or additionally, the medical device 106 and / or the user device 114 can include one or more microphones, audio sensors, or other audio capture devices configured to generate additional audio data from speech produced by the patient over time. Additionally, one or more processors of the medical device 106 and / or the user device 114 can analyze the collected audio data to determine one or more audio characteristics including at least one of pitch, timbre, rhythm, volume, and speech rate associated with the patient. Alternatively or additionally, the medical device 106 and / or the user device 114 can analyze the collected additional audio data to determine one or more audio characteristics including at least one of pitch, timbre, rhythm, volume, and speech rate associated with the patient.

[0029] Alternatively or additionally, the medical device 106, the control unit 110, and / or the user device 114 can be configured to transmit the audio data, the additional audio data, and / or any other collected information to the audio analysis system 120 over the network 116 for analysis. In any of these instances, the audio analysis system 120 can store this information in the audio screening component 126 and / or the external database 118. For example, the database 118 can include a memory or computer-readable medium that is substantially similar to and / or the same as the computer-readable medium associated with the audio screening component 126. The audio analysis system 120 and / or the medical device 106, the microphone 108, the control unit 110, and / or the user device 114 can access the database 118 over the network 116. In any of these instances, the database 118 can be configured to store patient data in association with a patient ID (e.g., a name, a social security number, an alphanumeric code, etc.) or other unique patient identifier. When the patient 102 and / or the user enters the patient ID, the audio screening component 126 can access or receive the patient data stored in association with the patient ID.

[0030] As used herein, the network 116 is generally any type of wireless network or other communication network known in the art. Examples of the network 116 include the Internet, an intranet, a wide-area network (WAN), a local-area network (LAN), a virtual private network (VPN), a cellular network connection, and a connection established using protocols such as 802.11a, b, g, n, and / or ac, among others.

[0031] In certain examples, the medical device 106, the microphone 108, the control unit 110, and / or the user device 114 can include a microprocessor or control unit configured to execute instructions stored in a computer-readable medium to perform various operations and methods. For example, the medical device 106 and / or the control unit 110 can include one or more processors and / or other hardware and / or software components configured to operatively control the microphone 108. Similarly, the user device 114 can include one or more processors and / or other hardware and / or software components configured to operatively control the microphone 112. Moreover, one or more processors of the medical device 106, the microphone 108, the control unit 110, and / or the user device 114 can be configured to provide a user interface, a sound recognition algorithm, and other components of the patient monitoring system. In certain additional examples, the medical device 106, the microphone 108, the control unit 110, the user device 114, and / or the audio analysis system 120 can include a single processing unit (e.g., a uniprocessor) or multiple processing units (e.g., a multiprocessor), and can include a single computing unit or multiple computing units and / or multiple processing cores. The processor 122 of the medical device 106, the microphone 108, the control unit 110, the user device 114, and / or the audio analysis system 120 can be implemented as one or more microprocessors, microcomputers, microcontrollers, digital signal processors, central processing units, state machines, logic circuitries, and / or any devices that manipulate signals based on operational instructions. For example, the processor 122 of the medical device 106, the microphone 108, the control unit 110, the user device 114, and / or the audio analysis system 120 can be one or more hardware processors and / or logic circuitries of any suitable type that are specifically programmed or configured to perform the algorithms, operations, and methods described herein. The processor 122 of the medical device 106, the microphone 108, the control unit 110, the user device 114, and / or the audio analysis system 120 can be configured to fetch and execute computer-readable instructions stored in the audio screening component 126, which can program the processor 122 to perform the functions described herein. Additionally or alternatively, the processor 122 of the medical device 106, the microphone 108, the control unit 110, the user device 114, and / or the audio analysis system 120 can be configured to fetch and execute computer-readable instructions stored in a computer-readable medium and / or other memory local to the respective device.

[0032] As described herein, a processor (e.g., processor 122) can be a single processing unit or a plurality of processing units and can include single or multiple computing units, or processing cores. The processor 122 can be implemented as one or more microprocessors, microcomputers, microcontrollers, digital signal processors, central processing units, state machines, logic circuitries, and / or any devices that manipulate signals and operate on

[0033] The network interfaces associated with the medical devices 106, microphones 108, control unit 110, user device 114, and / or audio analysis system 120 can enable wired and / or wireless communication between and / or among components and / or devices shown in the patient monitoring system 100 and / or with one or more other remote systems and other networked devices. For example, at least some of the network interfaces can include a personal area network component to enable communication over one or more short-range wireless communication channels. Additionally, at least some of the network interfaces can include a wide area network component to enable communication over a wide area network. Such network interfaces can enable communication between the medical devices 106, microphones 108, control unit 110, user device 114, and / or audio analysis system 120 and / or other components of the system 100, e.g., over the network 116.

[0034] The audio screening component 126 can include volatile and nonvolatile memory and / or removable and non-removable media implemented in any type of technology for storing information such as computer readable instructions, data structures, program modules or other data. The memory can include, but is not limited to, RAM, ROM, EEPROM, flash memory or other memory technology, optical storage, solid state storage, magnetic tape, magnetic disk storage, RAID storage systems, storage arrays, network-attached storage, storage area networks, cloud storage, or any other medium that can be used to store the desired information and that can be accessed by a computing device. The audio screening component 126 can include a variety of computer readable media and / or can be a tangible non-transitory medium, when referred to as such, the non-transitory computer readable medium excludes media that can change the state of or otherwise alter the physical implementation of the computer readable medium, such as energy, carrier signals, electromagnetic waves, and signals per se.

[0035] The audio screening component 126 can include any number of function components executable by the processor(s) 122. In many embodiments, these function components include instructions or programs executable by the processor(s) 122 that, when executed, specifically configure the processor(s) 122 to perform actions associated with monitoring the health state of a patient.

[0036] Figure 1 Although not shown, in some instances, the audio screening component 126 can include a computer-readable medium configured to store the audio data analysis component. In these instances, the audio data analysis component can be configured to receive, access, and / or analyze audio data collected and / or detected by the medical device 106 and / or the user device 114 during a patient monitoring procedure. For example, the audio data analysis component can be configured to receive audio data over the network 116. The audio data can be generated from respiratory sounds 104 and / or speech produced by the patient during a patient monitoring procedure by the various devices and / or microphones described above. The audio data analysis component can analyze the audio data to determine one or more vocal, respiratory, and / or audio characteristics associated with the patient 102, such as pitch, timbre, rhythm, volume, and speech rate associated with the patient 102.

[0037] Additionally, Figure 1Although not shown, the audio screening component 126 can also include a computer-readable medium configured to store a diagnostic data component. The diagnostic data component can be configured to receive, access, and / or analyze diagnostic data associated with the patient 102 and a health status associated with the patient. For example, in these embodiments, the diagnostic data component can be configured to access or receive data from one or more additional databases (e.g., the database 118, third party databases, etc.) that store test data, physician assessments, prior measurement data, and / or a range of values that indicate thresholds or ranges that audio characteristics and vocal characteristics determined from the audio data and associated with the patient 102 should fall within. These thresholds or ranges can be associated with an initial health status of the patient 102, a target health status of the patient 102, and / or a current health status of the patient 102 relative to a prior health status of the patient 102. For example, the diagnostic data component can access or receive individual and / or standardized diagnostic data that can be used for comparison to the audio data / audio data characteristics generated and stored by the audio data analysis component described above. For example, the diagnostic data associated with the patient 102 can include standard measurements obtained when the patient 102 is assessed to be healthy and / or in a target health status (e.g., pitch, tone, rhythm, volume, and pace associated with the patient 102). Additionally or alternatively, the diagnostic data associated with the patient 102 can include standard measurements obtained when the patient 102 is assessed to have a respiratory ailment and / or respiratory condition (i.e., asthma, COPD, bronchitis, emphysema, lung cancer, cystic fibrosis, pneumonia, and pleural effusion) and determined to have an initial health status associated with the assessment. From this, the standard measurements can identify threshold ranges of audio characteristics and / or vocal characteristics that the current audio characteristics and / or vocal characteristics should reach (i.e., standard measurements associated with a target health status) and / or need to improve (e.g., standard measurements associated with an initial health status). Additionally, the standard measurements can incorporate diagnostic data from other patients to determine whether the patient 102 is recovering normally from an initial health status, whether the patient 102 is progressing toward a target health status, whether the health of the patient 102 has fallen below a target health status, and / or other auxiliary monitoring of the patient 102.

[0038] Figure 2 An additional example system 200 of the present disclosure is shown. In some instances, the system 200 can include one or more of the same components included in the system 100. In some additional instances, the system 200 can include different components that provide similar functionality to the components included in the system 100. As shown, the system 200 can include a microphone 202, a processor 204, a memory 206, a display 208, and a speaker 210. In some instances, the microphone 202, the processor 204, the memory 206, the display 208, and the speaker 210 can be included in a single device (e.g., a smartphone, a tablet, a computer, etc.). In some additional instances, the microphone 202, the processor 204, the memory 206, the display 208, and the speaker 210 can be included in separate devices that are communicatively coupled to one another (e.g., a smartphone, a tablet, a computer, etc.). Figure 2As shown, system 200 can be used to determine a patient 202 within a medical environment 208 or a non-medical environment 214 over a period of time. Within medical environment 208, patient 202 can be associated with a medical device 204 and a microphone 206. Within non-medical environment 214, patient 202 can be monitored through an auxiliary device 210 and a personal device 212. It should be noted that auxiliary device 210 and personal device 212 can include one or more additional microphones that are the same or different from microphone 206. Further, medical device 206, auxiliary device 210, and personal device 212 can transmit audio data to an audio analysis environment 218 through a communication network 216. Audio analysis environment 218 can include a database configured to store audio data generated from patient 202, a sound recognition algorithm 222, an audio feature algorithm 224, and a patient state component 226.

[0039] In Figure 2 the exemplary system 200, as Figure 1 described, microphone 206 associated with medical device 204 can monitor patient 202 during treatment. For example, microphone 206 can collect and record one or more vocal utterances, breath samples, audio events, and / or other vocalizations produced by patient 202. The one or more vocal utterances can include conversations, request phrases, and / or other vocalizations produced at least in part by the vocal cords of patient 202. Similarly, the one or more breath samples can include long periods of patient breathing, breathing patterns in a sleep cycle of patient 202, breathing patterns during treatment of patient 202, and / or other vocalizations produced at least in part by patient 202 inhaling and exhaling air from the lungs. Additionally, the one or more audio events can include sighs, moans, sharp inhalations / exhalations, gasps, choking events, and / or other audio events that can be indicative of the airway status of patient 202.

[0040] In Figure 2In the example system of FIG. 2, the microphone 206 can be used to collect audio data from the patient 202 within the medical environment 208. Further, the audio data collected by the microphone 206 can be transmitted to the audio analysis environment 218 via the communication network 216. Additionally, the audio data can be stored in the database 220 in association with contextual information of the audio data. The contextual data can include device information, time information, medical device associations, location information, medical data, and other information associated with the patient 202 or the medical environment 208. The audio analysis environment 218 can be configured to determine one or more audio characteristics from the audio data collected within the medical environment 208. For example, the audio characteristics can include vocal characteristics associated with one or more human voice utterances, where the vocal characteristics can include pitch, timbre, rhythm, volume, and / or speech rate as data points. Further, the vocal characteristics can be determined from the audio data obtained from the patient 202 by the voice recognition algorithm 222. Additionally, the patient status component 226 can be configured to determine an initial health status of the patient 202 within the medical environment 208 from the vocal characteristics.

[0041] In Figure 2 In the example system of FIG. 2, the microphone 206 can be used to collect audio data from the patient 202 within the medical environment 208. Further, the audio data collected by the microphone 206 can be transmitted to the audio analysis environment 218 via the communication network 216. Additionally, the audio data can be stored in the database 220 in association with contextual information of the audio data. The contextual data can include device information, time information, medical device associations, location information, medical data, and other information associated with the patient 202 or the medical environment 208. The audio analysis environment 218 can be configured to determine one or more audio characteristics from the audio data collected within the medical environment 208. For example, the audio characteristics can include vocal characteristics associated with one or more human voice utterances, where the vocal characteristics can include pitch, timbre, rhythm, volume, and / or speech rate as data points. Further, the vocal characteristics can be determined from the audio data obtained from the patient 202 by the voice recognition algorithm 222. Additionally, the patient status component 226 can be configured to determine an initial health status of the patient 202 within the medical environment 208 from the vocal characteristics.

[0042] In at least one example system, the sound recognition algorithm 222 can be configured to receive audio data from the patient 202 and determine whether the audio data includes one or more vocal utterances. For example, the sound recognition algorithm 222 can be configured to analyze the audio data and identify audio patterns associated with human speech, where the audio patterns can include phonemes and / or patterned variations in human voice characteristics (e.g., pitch, timbre, cadence, volume, and / or speech rate) produced by the patient 202. Further, the sound recognition algorithm 222 can be configured to utilize neural network algorithms and machine learning algorithms to identify the audio patterns within the audio data and associate them with the patient 202 within the database 220. As such, the sound recognition algorithm 222 can be configured to parse the audio data recorded by the microphone 206 and determine whether the patient 202 is speaking. Additionally, when the patient 202 is determined to be speaking, the sound recognition algorithm 222 can be configured to identify individual phonemes within the vocal utterances of the patient 202, where the phonemes represent individual components of human speech that can be combined into words and other vocal utterances associated with human speech and communication. Alternatively or additionally, the sound recognition algorithm 222 can be configured to identify patterned variations in the human voice characteristics of the patient 202 identified within the audio data.

[0043] For example, the sound recognition algorithm 222 can utilize machine learning algorithms, neural networks, artificial intelligence (AI), and / or other techniques to identify patterns associated with sound samples and / or sound patterns within the audio data captured by the microphone 206 from the patient 202. For example, the sound recognition algorithm 222 can be configured to determine human voice characteristics from the audio characteristics of the audio data. Further, the sound recognition algorithm 222 can be trained to identify and / or recognize human voice patterns / human voice characteristics within the audio data based on a predefined library and / or a set of historical patient audio characteristics. When identifying and / or receiving audio characteristics such as from the audio feature algorithm 224 (e.g., only receiving audio data identified as being associated with the patient’s 202 voice), the sound recognition algorithm 222 can determine whether the audio characteristics of the audio data match audio samples stored by the predefined library and / or the set of historical patient audio characteristics. Alternatively or additionally, the sound recognition algorithm 222 can be configured to identify and / or recognize audio characteristics associated with the audio data by analyzing a spectrogram associated with the audio data. Additionally, the sound recognition algorithm 222 can be configured to record human voice characteristics (e.g., pitch, timbre, vocal clarity, speech cadence, etc.) associated with the human speech generated by the patient 202 based on the audio samples of the audio data matching the predefined library and / or the set of historical patient audio characteristics.

[0044] In at least one additional example system, the audio feature algorithm 224 can be configured to receive audio data from the patient 202 and determine whether the audio data includes audio events and / or breath samples. For example, the audio feature algorithm 224 can be configured to analyze the audio data and identify audio patterns associated with breathing patterns and audio events associated with the patient 202. Specifically, the audio feature algorithm can be configured to identify audio events such as sighs, moans, sharp inhalations / exhalations, gasping, and choking events. The audio events can be identified and recorded in a database, marking potential indicators of the patient’s health status. Additionally or alternatively, the audio feature algorithm 224 can be configured to identify breathing patterns and audio characteristics associated with the patient’s 202 breaths. Additionally, the audio characteristics of the patient’s 202 breaths can include the patient’s 202 pitch, timbre, rhythm, volume, and / or breathing rate. Similarly, the audio feature algorithm 224 can be configured to identify one or more breathing patterns associated with the patient’s 202 initial health status within the medical environment 208.

[0045] For example, the audio feature algorithm 224 can analyze audio data captured by the microphone 206 or other devices associated with the system 200. The audio data can be captured from the environment and / or the patient 202 substantially continuously and / or passively (e.g., continuously collecting audio data independent of a start signal, continuously collecting audio data when the system 200 is powered on, etc.). It should be noted that the audio data can also be collected periodically, aperiodically, in response to an indication, and / or based on other indications that audio data is to be collected from the patient 202. The microphone 206 can provide the audio data to the audio feature algorithm 224, including decibel data, energy data, spectrograms, and / or other audio data representations. The audio feature algorithm 224 can perform a multi-stage analysis (e.g., pre-processing, analysis, post-processing, etc.). The initial stage (e.g., pre-processing) of the audio feature algorithm 224 can include receiving the audio data from the microphone 206 and quantizing, sampling, and filtering the audio data to denoise from decibel data, energy data, spectrograms, and / or other audio data representations. Further, the initial stage of the audio feature algorithm can utilize one or more additional microphones associated with the system 200. Additionally, the audio data from the microphone 206 and the additional microphones can be merged and normalized to further enhance the quality of the audio data analyzed by the audio feature algorithm 224.

[0046] In certain instances of the audio feature algorithm 224, the audio feature algorithm 224 can utilize signal processing techniques and natural language processing to classify portions of audio data as "verbal" audio characteristics (e.g., human speech) and "non-verbal" audio characteristics (e.g., moans, cries, etc.). For example, the natural language processing can prioritize the identification of common phonemes and audio data patterns associated with the most common words and sounds produced by human voices. The identified high-priority audio patterns can be utilized to identify "verbal" audio data associated with verbal audio characteristics. Similar to the sound recognition algorithm 222 described above, the audio feature algorithm 224 (or the sound recognition algorithm 222) can utilize stored audio samples of common phonemes and audio data patterns to train the natural language processing algorithm to identify high-priority audio patterns to identify verbal audio data and verbal audio characteristics extracted from the verbal audio data. As such, the audio feature algorithm 224 can classify audio data having relatively high-priority / common audio patterns as verbal audio data having verbal audio characteristics.

[0047] In certain instances of the audio feature algorithm 224, the non-verbal audio characteristics can be further analyzed to detect instances of breathing patterns and pain noises. For example, breathing patterns can be identified from the non-verbal audio characteristics based on identifying periodic and / or regular audio characteristics within the non-verbal audio data. Further, the identified breathing patterns can be analyzed to obtain breathing parameters, such as peak bits of inhalation and exhalation, noise amplitude during breathing, length of inhalation and exhalation, and other breathing characteristics that can be used to identify respiratory system disorders and / or diseases. Additionally, monitoring breathing frequency and detecting other indicators from the breathing patterns can provide additional indicators of respiratory diseases and / or conditions. By comparison, instances of pain noises (e.g., moans, cries, gasps, grunts, etc.) can be identified from the non-verbal audio characteristics based on non-periodic, independent, and / or isolated audio characteristics in the non-verbal audio data. At a final stage of the audio feature algorithm 224, the various verbal audio characteristics, non-verbal audio characteristics, breathing patterns, instances of pain noises, and other detected audio features can be stored for further analysis and use in the system (e.g., the sound recognition algorithm 222 can utilize the verbal audio characteristics).

[0048] In certain additional instances of the audio feature algorithm 224, non-verbal audio characteristics and / or verbal audio characteristics can be used to create a sound filter to remove and / or reduce various audio features from the audio data. For example, the audio data waveform can include periodic audio features determined to be associated with the breathing pattern of the patient 202. Further, the audio data can include audio noise and / or extraneous audio features that should not be included in the audio data. In this regard, a preliminary analysis of the audio data can reveal audio frequencies that are not relevant or minimally relevant to the audio data of interest (e.g., the breathing pattern). Additionally, a sound filter can be created based on the preliminary analysis to remove the audio frequencies that are not relevant or minimally relevant to the audio data of interest. For example, it can be determined through the preliminary analysis that the breathing pattern is primarily associated with audio features within the frequency range 150-1000 Hz. In this regard, a sound filter can be generated to remove audio features outside the 150-1000 Hz range and reduce the amount of noise and / or extraneous audio data within the audio data that the audio feature algorithm 224 is analyzing.

[0049] In still certain instances of the audio feature algorithm 224, the audio feature algorithm 224 can be configured to identify variations in a standard audio profile of an audio feature. For example, a cough can be identified based on an initial phase (e.g., explosive phase) associated with high frequency high amplitude audio data, a middle phase of high frequency low amplitude audio data, and a vocalization phase of medium frequency medium amplitude. Further, the audio feature (e.g., cough) can be associated with a sound length. However, based on one or more symptoms, conditions, and / or diseases associated with the patient, the three phases of the cough associated with the patient 202 can change. An increase or decrease in the amplitude, frequency, and sound length of the three phases can be associated with symptoms of a respiratory condition / disease. Such changes in the audio data of the patient 202 can be indicative of symptoms of airway constriction, airway obstruction, airway dilation, lung trauma, lung capacity limitations, fluid filled lungs, and other respiratory diseases and / or conditions.

[0050] In at least one further example system, the patient status component 226 can be configured to retrieve audio data from the medical device 204 and / or the microphone 206 to determine an initial patient status of the patient 202. Specifically, the patient status component 226 can be configured to determine an initial health status of the patient 202 based at least on contextual data stored in the database 220. For example, the patient status component 226 can determine an initial health status of the patient 202 based at least on a health assessment provided by a physician, an indication provided by the patient 202, or health information associated with other patients. Alternatively or additionally, the patient status component can determine a patient health status range, where the patient health status range can include an optimal health status (e.g., a health status when the patient is not suffering from a respiratory illness or experiencing negative symptoms associated with a respiratory condition), a target health status (e.g., a health status when the patient is experiencing the least negative impact due to a respiratory illness and / or respiratory condition), an initial health status (e.g., a health status indicative of a respiratory condition prior to treatment of the respiratory illness and / or respiratory condition), a threshold health status (e.g., a health status indicative of a respiratory condition that can trigger further action to be taken), and / or other health statuses representative of patient conditions reported and / or flagged for future resolution. The initial health status can be determined prior to the medical device 204 providing treatment to the patient 202 within the medical environment 208 and can be associated with one or more vocal utterances, breathing patterns, and / or audio events recorded by the microphone 206. Similarly, the additional health statuses can be determined after the medical device 204 provides treatment to the patient 202 within the medical environment 208 and can be associated with one or more additional vocal utterances, additional breathing patterns, and / or additional audio events recorded by the microphone 206. Additionally, the health statuses can be established by the patient status component based on indications provided by a physician, the patient 202, the medical device 204, or other systems associated with the patient monitoring system to the audio analysis system.

[0051] In Figure 2 In the example system 200, one or more microphones associated with the auxiliary device 210, the personal device 212, and / or other devices in the system 200 can be configured to collect audio data from the patient 202 in a non-medical environment 214 (e.g., a residence, a workplace, a public place, a telemedicine, etc.). For example, the system 200 can be configured to monitor a health status of the patient 202 in the non-medical environment 214. Specifically, the one or more microphones are associated with at least one of the auxiliary device 210 and / or the personal device 212. It should be noted that the microphone can include a microphone associated with the auxiliary device 210, the personal device 212, and / or other devices as described with reference to FIG. 1. Figure 1Similar components are described. Additionally, the auxiliary devices 210 and / or the personal devices 212 can operate in conjunction with each other and / or independently of each other to collect audio data from the patient 202 within the non-medical environment 214 and transmit the audio data to the audio analysis environment 218. Additionally, the audio data can be stored in the database 220 in association with contextual information of the audio data. The contextual data from the non-medical environment 214 can include device information, time information, location data, and other information associated with the patient 202 or the non-medical environment 214. The audio analysis environment 218 can be configured to determine one or more audio characteristics in the audio data from the audio data collected within the non-medical environment 214. For example, the audio analysis environment 218 can be configured to determine one or more vocal utterances, breathing patterns, audio events, and / or other utterances from the audio data generated by the patient 202. It should be noted that the audio characteristics of the audio data can be determined in a similar manner as discussed above for the audio data captured in the medical environment 208.

[0052] In Figure 2 In the exemplary system, the communication network 216 can be configured to enable continuous monitoring of the patient within the medical environment 208 and the non-medical environment 214. Additionally, the audio analysis environment 218 can be configured to monitor the patient 202 in a substantially continuous manner, independent of the patient 202 moving between the medical environment 208 and the non-medical environment 214 and entering or leaving the range of the various devices associated with the system 200. Additionally, the audio analysis environment 218 can be configured to determine whether the incoming audio data from the medical devices 204, the auxiliary devices 210, the personal devices 212, and / or additional devices matches any recorded health states stored in the database 200. For example, the database 220 can include one or more health state thresholds, recorded patient health states, and / or associations between audio characteristics and patient 202 health states. Additionally, when the audio analysis environment 218 determines that the audio data received by the audio analysis environment 218 satisfies one or more health state thresholds and / or matches one or more recorded health states, the audio analysis environment 218 can be configured to transmit an indication that the audio data satisfies the health state threshold and / or matches the recorded health state.

[0053] In certain instances, the audio analysis environment 218 can receive audio data from the medical device 204, the microphone 206, the auxiliary device 210, the personal device 212, and / or other devices associated with the system 200 via the communication network 216. Further, the audio analysis environment 218 can perform comparative and matching analysis of the audio data and patient health state data (e.g., patient health state thresholds, recorded patient health states, patient health state data, etc.) stored in the database 220. For example, the database 220 can be populated with initial health states, target health states, optimal health states, and / or other health states associated with the patient and determined from audio data in a similar manner as described above. It should be noted that although the above is described with respect to audio data collected from the patient within the medical environment 208, the audio analysis environment can determine a patient health state based at least in part on audio data collected from the non-medical environment 214. Further, through comparative and matching analysis of current audio data received from devices associated with the system 200 and previous audio data associated with a patient health state, the patient state component 226 can determine a current health state of the patient 202.

[0054] In certain additional instances, the audio analysis environment 218 can determine a current health state of the patient 202 based at least in part on current audio data received from one or more devices associated with the system 200. As described above, the patient state component 226 can be configured to identify a current health state of the patient 202 from the current audio data and determine whether to transmit a notification to the patient 202 and / or a physician. For example, the patient state component 226 can compare audio characteristics of the current audio to audio characteristics of previous audio data associated with patient health state data stored in the database 220. Further, the patient state component 226 can determine that the current audio characteristics substantially match the previous audio characteristics stored by the database 220. Additionally or alternatively, the patient state component 226 can determine that the current audio characteristics satisfy one or more audio characteristic thresholds associated with patient health data stored by the database 220. It should be noted that the patient state component 226 can be configured to utilize machine learning algorithms and / or neural network techniques to form categories of audio characteristic values associated with various respiratory illnesses and / or respiratory conditions. Further, the patient state component 226 can determine from the previous audio characteristics ranges and thresholds of audio characteristic values associated with symptoms of respiratory illnesses and / or respiratory conditions. In turn, the patient state component 226 can be configured to determine a current health state of the patient 202 and / or indicate one or more respiratory illnesses and / or respiratory conditions that the patient 202 is experiencing based at least on the current audio characteristics.

[0055] In Figure 2 In the exemplary system, the communication interface can be configured to transmit the audio data to the audio analysis environment via the communication network by referencing the audio data to the patient 202 and the time at which the audio data was collected. Figure 1The communication network 116 provides data connectivity and network communication for the medical devices 204, ancillary devices 210, personal devices 212, audio analysis devices 218, and other devices associated with the system 200. For example, the communication network 216 can enable various devices to connect to an external database (e.g., database 118) to receive, access, and / or transmit audio data using wireless connectivity. Wireless connectivity can include cellular network connectivity and connectivity established using 802.1 la, b, g, and / or ac protocols, among others. In other instances, wireless connectivity between the audio analysis environment 218 and an external display can be achieved directly using one or more wired or wireless protocols such as Bluetooth, WiFi Direct, radio frequency identification (RFID), infrared signals, and / or Zigbee, among others. Other configurations are also possible. Data communication to an external database 118 or external system can enable printing of reports of the audio data and / or health status of the patient 202, or further evaluation thereof. For example, collected data and analysis results can be transmitted wirelessly and stored in a remote database accessible to authorized medical professionals.

[0056] Figure 3 An example data processing workflow of the present disclosure is shown. Figure 3 Although not shown, in certain instances, the workflow can be performed by the example system 100 and / or the example system 200. In Figure 3 In certain additional instances, the workflow can be initiated when the example system receives incoming audio data 302. Further, upon receipt of the incoming audio data 302, a classification algorithm 304 can be invoked, activated, or otherwise initiated to analyze the incoming audio data 302 and identify relevant portions of the incoming audio data 302. For example, the classification algorithm can be configured and / or trained to distinguish between human speech utterances 306, respiratory data 308, and / or other audio data contained in the incoming audio data set 302. When the incoming audio data set 302 is parsed and the relevant portions are classified as human speech utterances 306, respiratory data 308, and other audio data types, the example system can activate additional algorithms to further analyze the data. For example, a voice recognition algorithm 310 and / or an audio event recognition algorithm 312 can be configured to receive the human speech utterance data 306 and identify voice characteristics 316 and / or audio event data 318 that can be used to identify and / or monitor the health status of a patient. Similarly, respiratory pattern analysis 314 can be performed to identify respiratory characteristics 320 that can be used to identify and / or monitor the health status of a patient.

[0057] In Figure 3In certain examples, the incoming audio data 302 can be unprocessed and / or raw audio data collected from the patient and / or the environment in which the patient is located. The incoming audio data 302 can include patient speech, patient breathing, and other audio events associated with the patient. However, the incoming audio can also include speech and breathing of other individuals, environmental sounds (e.g., objects moving within the environment, other individuals walking by, electronic audio sources, etc.), and other audio data unrelated to the patient's health status.

[0058] In Figure 3 In certain examples, the classification algorithm 304 can be configured to identify the human speech utterances 306 and the breathing data 308 in the incoming audio data 302. For example, the classification algorithm 304 can be configured to utilize machine learning techniques and neural networks to parse the incoming audio data 302 to identify relevant human speech utterances and breathing data. In particular, a database associated with the example system can include a plurality of recordings that sufficiently include all of the phonemes produced when the patient speaks. Additionally or alternatively, the database can include additional plurality of recordings that sufficiently represent the voice quality during the patient's speech. Additionally or alternatively, the database can include human voice recordings associated with a plurality of individuals, where the human voice recordings can be utilized to identify the speech audio data within the incoming audio data 302. As such, the classification algorithm 304 can include machine learning algorithms and / or neural networks that have been trained to recognize the phonemes, the voice quality associated with the patient, and / or the human speech within the incoming audio data 302.

[0059] In certain additional examples, similar to the above, the database can further include data representative of the breathing patterns of the patient and / or other individuals. The breathing patterns in the database can include passive breathing of the patient and / or other individuals, breathing patterns when the patient and / or other individuals speak, and other breathing patterns. As such, the classification algorithm can include machine learning algorithms and / or neural networks that have been trained to identify inhalations and exhalations recorded in the incoming audio data 302.

[0060] In Figure 3In certain instances, the classification algorithm 304 can be used to process the incoming audio data 302. For example, the classification algorithm 304 can be configured to utilize digital signal processing tools, such as a Fast Fourier Transform (FFT) filter, an audio signal filter, a spectral analysis tool, and other signal processing tools. For example, the incoming audio data 302 can be passed through an FFT filter configured to break down the incoming audio data 302 into amplitude-frequency components associated with a pitch amplitude and / or a pitch volume. Further, the rhythm of the audio features can be represented by the pattern of the produced sound. For example, by extracting the dense / crest and sparse / trough from the waveform of the audio data and comparing / analyzing the dense / crest and sparse / trough of the audio data, the rhythm of the audio features can be determined. Additionally, the classification algorithm 304 can extract the average duration of the dense and sparse associated with the audio sample to classify the audio sample. In additional instances, the timbre of the audio sample can be determined based on one or more specific frequencies of the audio waveform associated with a high amplitude relative to other frequencies of the audio waveform. When analyzed by FFT, the breakdown of the audio data into frequency and amplitude components can be used to determine the timbre associated with the audio data. In further instances, the classification algorithm can utilize techniques such as Linear Predictive Coding (LPC), Perceptual Linear Prediction (PLP), Mel Frequency Cepstral Coefficients (MFCC), and other techniques to extract audio features in the recorded audio sample. For example, the MFCCs can be determined based on the shape of the patient's vocal tract and can be used as an identifier for the patient based on the sound produced by the vocal tract shape represented by the MFCCs. Further, the MFCCs can be determined based on the breakdown of the audio data into overlapping frames and a series of FFTs and filters can be applied to the audio data to extract / identify the MFCCs of the audio data and the patient. As such, the LPCs, PLPs, and MFCCs can be used as indicators to classify the audio sample associated with the patient. Additionally, variations in the LPCs, PLPs, and MFCCs can also be classified as or associated with one or more symptoms / respiratory conditions. The audio data can include audio energy, audio spectrum, audio amplitude, audio intensity, and / or MFCCs (and / or other indicators) that can be used to identify features for extraction and classification by the classification algorithm 304. As such, the various indicators identified above can be tracked and used to identify whether a segment of audio data is to be classified as human speech data 306 or respiratory data 308.

[0061] In Figure 3In certain instances, the vocal utterance data 306 can be further analyzed by a sound recognition algorithm 310 following classification by the classification algorithm 304. For example, the sound recognition algorithm 310 can be configured to identify the vocal / human voice characteristics 316 of the patient within the vocal utterance 306. Similar to the classification algorithm, the sound recognition algorithm 310 can access a database containing a plurality of voice samples associated with the patient. Further, the sound recognition algorithm 310 can utilize a machine learning algorithm and neural network trained based on the plurality of patient voice samples to identify the partial vocal utterance associated with the patient. Additionally, the sound recognition algorithm can be configured to identify the human voice characteristics 316 that can include the tone, timbre, rhythm, volume, and pace of speech associated with the patient. It should be noted that the machine learning algorithm and neural network can be trained to also identify a deviation between the human voice characteristics 316 and stored patient human voice characteristics, and / or identify stored patient human voice characteristics that match the identified human voice characteristics 316.

[0062] In Figure 3 In certain instances, the vocal utterance data 306 can be further analyzed by an audio event recognition algorithm 312 following classification by the classification algorithm 304. For example, the audio event recognition algorithm 312 can be configured to identify the audio events 318 associated with the patient within the vocal utterance 306. Similar to the classification algorithm, the audio event recognition algorithm 312 can access a database containing a plurality of past audio events associated with the patient. As described above, the audio events can include sighs, moans, gasps, groans, coughs, and other utterances not associated with speech but indicative of the patient’s airway status. Further, the audio event recognition algorithm 312 can utilize a machine learning algorithm and neural network trained based on the plurality of past audio event samples to identify the partial vocal utterance as being the audio events generated by the patient. In turn, the audio event recognition algorithm 312 can identify the audio events 318 in the vocal utterance 306.

[0063] In Figure 3In some instances, the respiratory data 308 identified by the classification algorithm can be further analyzed by a respiratory pattern analysis algorithm 314. For example, the respiratory data 308 can include repeated inhalations and exhalations by the patient during speaking, resting, and activity. Further, the respiratory pattern analysis algorithm 314 can be configured to track the duration, rate, force, and other respiratory characteristics 320 associated with the respiratory data 308. Additionally, the respiratory pattern analysis algorithm 314 can be configured to identify respiratory patterns indicative of the patient's health status. For example, similar to the above, the respiratory pattern analysis algorithm 314 can utilize machine learning techniques and / or neural networks to identify respiratory characteristics 320 and / or associated respiratory patterns from the incoming audio data 302. The respiratory pattern analysis algorithm 314 can access a database including respiratory patterns associated with respiratory illnesses and / or respiratory conditions (e.g., recordings of individuals and / or patients experiencing an asthma attack and coughing due to illness, etc.), recordings indicative of previous respiratory characteristics associated with the patient, and other recordings associated with the patient's respiratory patterns.

[0064] Figure 4 A flowchart showing an example method 400 of monitoring a patient's health status is provided. The method 400 is shown as a collection of blocks in a logical flowchart, the blocks representing a sequence of operations that can be implemented in hardware, software, or a combination thereof. In the case of software, the blocks represent computer-executable instructions stored on one or more computer-readable storage media that, when executed by a computer processor (CPU), perform the recited operations. Generally, computer-executable instructions include routines, programs, objects, components, data structures, and the like that perform particular functions or implement particular abstract data types. The described operations do not imply a specific order of execution unless explicitly stated that a certain operation is performed before another operation. Any number of the described blocks can be combined in any order and / or in parallel to implement the method 400, eliminating possibly unnecessary ones. In certain embodiments, one or more of the blocks of the method 400 can be omitted completely.

[0065] At block 402, the CPU of the patient monitoring system can receive current audio data associated with the patient having a respiratory condition from a user device. As described above, the user device can be a medical device, a personal device, an auxiliary device, or other device associated with the patient monitoring system. In certain example methods, various devices associated with the patient monitoring system can further be associated with a microphone and can be configured to transmit the current audio data to the patient monitoring system over a communication network. The current audio data can be transmitted by the device to the patient monitoring system on a substantially periodic, non-periodic, or continuous basis. Transmission on a periodic basis can include transmitting the current audio data to the patient monitoring system at substantially regular intervals (e.g., after a number of seconds, minutes, hours, days, etc. have elapsed since a previous transmission of current audio data). Transmission on a non-periodic basis can include transmitting the current audio data when one or more audio data thresholds are met, when a request for information transmission is made, or when it would not otherwise occur on a substantially periodic basis. Transmission on a continuous basis can include transmitting the current audio data at substantially the same time that the user device captures / records the current audio data.

[0066] At block 402, in certain additional examples, the current audio data can be collected during treatment for the patient's respiratory condition or in a non-medical environment associated with the patient. For example, the current audio data can be captured by a microphone associated with a medical device when the patient is receiving treatment, advice, and / or consultation from a physician. Alternatively or additionally, the current audio data can be captured by a microphone associated with a personal device of the patient when the patient is at home, in public, and / or otherwise completing daily activities (e.g., conversing, eating, sleeping, etc.).

[0067] At block 404, the CPU of the patient monitoring system can parse the current audio data to identify vocal utterances and / or respiratory sequences associated with the patient. As described with reference to Figure 3 the patient monitoring system can utilize various algorithms (e.g., speech recognition, machine learning, neural networks, etc.) to identify portions of interest within the current audio data. It should be noted that parsing the current audio data to identify vocal utterances and respiratory sequences can include the CPU executing a classification algorithm configured to distinguish the vocal utterances and respiratory sequences produced by the patient within the current audio data from background audio of the current audio data. Further, identifying portions of interest (e.g., vocal utterances, respiratory sequences, and / or other vocalizations generated by the patient) from the current audio data can include the patient monitoring system invoking a trained to recognize spoken language and other noises produced by the patient within excess audio data (e.g., other individuals speaking and breathing) and environmental noise (e.g., objects moving, devices outputting audio content, mechanical noise, etc.) within the current audio data. Additionally, the patient monitoring system CPU can execute the classification algorithm while identifying the vocal utterances and respiratory sequences of the patient concurrently with Figures 1 to 3Operations in the various algorithms and methods described.

[0068] In certain embodiments of block 404, the algorithms can utilize machine learning techniques to identify portions of audio data as indicators of a current patient health status. Specifically, a machine learning algorithm can be trained based on a set of audio data features extracted from a plurality of audio samples stored in a database. Based on a comparison between current audio features and the set of audio features associated with past patient audio data and past patient health statuses, the set of audio data features can be used to determine a patient health status. For example, the machine learning algorithm can determine a current health status of a patient from a current audio sample matching one or more audio features associated with past audio data and past health statuses of the patient. Further, the set of features can include a pitch of a vocalization segment, an audio data amplitude, an audio energy, and other audio characteristics associated with past audio data. Alternatively or additionally, the set of features can be associated with MFCCs associated with a vocal tract shape during generation of the set of audio data features of past audio data. Thereby, based on a match between current audio features and one or more audio features in the set of audio features, the machine learning algorithm can determine a current health status of the patient. For example, the machine learning algorithm can compare audio characteristics / MFCCs of current audio data and past audio characteristics / past MFCCs associated with the set of audio features. Additionally, the machine learning algorithm can return one or more past audio features and a past health status associated with the one or more past audio features. Based on a closest match between the one or more past audio features and current audio features, the machine learning algorithm can be configured to identify a current health status from the past health status. Similarly, normalized audio pitch, amplitude, energy, intensity, and MFCCs from past audio samples can be used to train a classifier configured to predict a closest match between past audio samples and current audio samples.

[0069] At block 406, the CPU of the patient monitoring system can determine one or more audio events from the vocal utterances and / or respiratory sequences based at least on previous vocal utterances and previous respiratory sequences associated with the patient. In certain instances, the vocal utterances can include one or more audio events including a conversation, a request phrase, an audible pain event, and / or a scream associated with the patient. Alternatively or additionally, the respiratory sequences can include one or more additional audio events including an inhalation, an exhalation, a cough, a sneeze, and / or a breathing difficulty associated with the patient. Additionally, the one or more audio events can include additional audible sounds produced by the patient including a panting, a wheeze, a moan, a cry, and other sounds associated with breathing difficulties, pain, and respiratory conditions / illnesses of the patient.

[0070] At block 408, the CPU of the patient monitoring system can determine one or more audio characteristics associated with the one or more audio events. As described above with reference toFigures 1 to 3 The one or more audio characteristics can include a pitch of the human voice utterance, a timbre of the human voice utterance, a rhythm of the human voice utterance, a volume of the human voice utterance, a speech rate associated with the human voice utterance, an inhalation duration associated with the breath sequence, an exhalation duration associated with the breath sequence, and / or a breath rate associated with the breath sequence. Further, in certain instances, the audio characteristics can include additional audio characteristics associated with speaking and breathing difficulty, speaking and breathing distress, speaking and breathing effort, and / or other human audible / non-audible audio characteristics associated with the patient health state. Additionally, the various audio characteristics can be identified by a sound recognition algorithm configured to decompose human speech / sound into component elements, portions, and / or variables. Based on analyzing a spectrogram (e.g., a visual representation of recorded audio data) and / or other electronic data formats associated with the current audio data, various aspects of the human speech can be determined.

[0071] At block 410, the CPU of the patient monitoring system can determine a past audio event class associated with the one or more audio events based at least on the one or more audio characteristics. Further, the past audio event class can include a set of past audio recordings associated with the respiratory condition and the plurality of patient health states. In certain instances, the past audio event class can include a plurality of recorded audio events associated with a human voice utterance type and / or a breath sequence type. It should be noted that the human voice utterance type and / or the breath sequence type can be characterized at least as a set of audio characteristics selected from the various audio characteristics described above. For example, the past audio event class can be associated with a cry, the cry and the set of audio characteristics associated with the human voice utterance comprising the cry class. Further, the one or more audio event classes can be determined based at least on the one or more audio characteristics comprising at least the set of audio characteristics. For example, when the one or more audio characteristics associated with the one or more audio events match the set of audio characteristics associated with the cry class, the one or more audio events can be determined to comprise one or more cries. Additionally, a particular audio event of the one or more audio events can be flagged as a cry due to its association with the set of audio characteristics.

[0072] At block 412, based at least on the comparison of the one or more audio events to the past audio event categories, the CPU of the patient monitoring system can determine a current health status of the patient associated with the respiratory condition. In certain instances, determining the current health status of the patient can further include determining one or more recorded health statuses associated with the respiratory condition of the patient and determining the current health status associated with the respiratory condition of the patient. As described above, the past audio event categories can include a plurality of recorded audio events associated with a human voice utterance type and / or a respiratory sequence type. The plurality of recorded audio events can be previously analyzed to identify one or more recorded health statuses associated with audio characteristics of the plurality of recorded audio events. In this regard, based on the current respiratory symptoms associated with the one or more audio events of the current audio data, the CPU of the patient monitoring system can utilize the associations between various respiratory symptoms and the one or more recorded health statuses to determine the current health status of the patient.

[0073] At block 412, in certain instances, the CPU of the patient monitoring system can be configured to determine a correspondence between one or more audio characteristics associated with the one or more audio events of the audio data and respiratory symptoms of the patient. For example, the recorded health statuses associated with the past audio event categories can include one or more respiratory symptoms associated with the plurality of recorded audio events, the one or more respiratory symptoms including at least one of the following: patient airway inflammation, difficulty inhaling, difficulty exhaling, excess mucus in the patient airway, chronic cough, one or more altered audio characteristics produced by the patient, coughing up blood, shortness of breath, and chest tightness / chest pain. In this regard, by identifying a correspondence between the respiratory symptoms of the past audio event categories and one or more past audio characteristics applicable to the one or more audio characteristics, the CPU of the patient monitoring system can determine the current symptoms of the one or more audio events. Alternatively or additionally, the one or more recorded health statuses can include an initial health status associated with an initial audio characteristic and an initial patient symptom previously recorded for the medical respiratory condition; a target health status associated with a target audio characteristic and a target patient condition recorded when the patient is not associated with the respiratory condition or when the patient is effectively treated for the respiratory condition; and a threshold health status associated with a threshold audio characteristic, wherein the system transmits an indication to the patient or a physician based at least on the one or more audio characteristics satisfying the threshold audio characteristic.

[0074] Figure 5A flowchart illustrating an exemplary method 500 of monitoring a patient's health status as described herein is provided. The method 500 is shown as a collection of blocks in a logical flow graph, with each block representing a set of operations that can be implemented in hardware, software, or a combination thereof. In the context of software, the blocks represent computer-executable instructions stored on one or more computer-readable storage media that, when executed by a CPU, perform the recited operations. Generally, computer-executable instructions include routines, programs, objects, components, data structures, and the like that perform particular functions or implement particular abstract data types. The described operations do not imply a specific order of execution unless explicitly stated that a certain operation is conducted prior to another operation. Any number of the described blocks can be combined in any order and / or in parallel to implement the method 500, omitting one or more blocks in certain embodiments.

[0075] At block 502, the CPU of the patient monitoring system can activate a patient monitor or activate a patient monitoring process, where the patient monitor can be a microphone associated with a user device (e.g., a medical device, a personal device, an auxiliary device, etc.) and configured to collect current audio data associated with a patient's respiratory function. The patient's respiratory function can be associated with the patient's ability to breathe, speak, or otherwise utilize the patient's respiratory tract. As described above, depending at least in part on the severity of the patient's respiratory symptoms experienced by the patient due to a respiratory condition (e.g., a respiratory illness, a respiratory disease, a respiratory problem, etc.), the respiratory condition can impair the patient's respiratory function. In certain instances, the patient monitoring system can be configured to collect and store audio data from a plurality of patients.

[0076] At block 504, the CPU of the patient monitoring system can cause the microphone to record current audio data from the patient's vocal utterances and respiratory sequences. As described above, the patient's respiratory function can be affected by the severity of the patient's respiratory symptoms experienced by the patient due to a respiratory condition. Accordingly, causing the microphone to monitor and record the patient's speech (e.g., conversation, self-talk, responses to prompts, etc.), breathing, or otherwise generate human audible or inaudible sounds results in current audio data from the patient's vocal utterances and respiratory sequences.

[0077] At block 506, the CPU of the patient monitoring system can identify current vocal characteristics associated with the patient vocal utterance and current respiratory characteristics associated with the patient respiratory sequence. In certain instances, the identification of the current vocal characteristics and the current respiratory characteristics can be performed by the CPU in the manner described above. In certain additional instances, the identification of the current vocal characteristics and the current respiratory characteristics can be determined from one or more audio spectrograms. For example, the CPU can determine the pitch, timbre, rhythm, volume, and speech rate associated with the patient from one or more audio spectrograms associated with the patient speaking. Further, the CPU can determine the respiratory rate and average respiratory volume from one or more additional spectrograms associated with the patient respiratory sequence. Additionally, the CPU can identify one or more pain events from one or more additional spectrograms, where the one or more pain events are associated with one or more moans, one or more groans, or one or more grunts.

[0078] At block 508, the CPU of the patient monitoring system can analyze a plurality of previous vocal characteristics and respiratory sequences associated with the patient and previous audio data to train a machine learning algorithm. For example, the plurality of previous vocal characteristics and respiratory sequences can include previous patient vocal characteristics, previous patient respiratory sequences, vocal characteristics associated with one or more additional patients, and respiratory sequences associated with the one or more additional patients. The previous vocal characteristics and respiratory sequences can include spectrograms or other audio data recordings that reflect the ability of the patient or the one or more additional patients to speak and breathe while experiencing a set of respiratory symptoms. From this, each set of vocal characteristics and respiratory sequences can be associated with a patient health status and the set of respiratory symptoms. Additionally, the set of vocal characteristics and respiratory sequences can be further associated with context or demographic information to more accurately match the symptoms the patient is experiencing and the set of respiratory symptoms experienced by the patients associated with the plurality of previous vocal characteristics and respiratory sequences.

[0079] At block 508, in certain instances, analyzing the plurality of prior acoustic characteristics and respiratory sequences to train the machine learning algorithm can include training the machine learning algorithm from prior assessment audio recordings or prior diagnostic audio recordings. For example, the CPU of the patient monitoring system can receive one or more diagnostic audio recordings, where one of the one or more diagnostic audio recordings is associated with a respiratory condition, a recorded health status, and a prior set of acoustic characteristics. Further, the CPU can provide the recorded health status and the prior set of acoustic characteristics of one of the one or more diagnostic audio recordings to the machine learning algorithm. Additionally, the CPU can invoke the machine learning algorithm to correlate the plurality of prior acoustic characteristics and respiratory sequences based at least on the recorded health status and the prior set of acoustic characteristics of each individual diagnostic audio recording of the one or more diagnostic audio recordings. Generally, the one or more diagnostic audio recordings can include recordings of acoustic speech and / or respiratory sequences recorded from the patient and / or one or more additional patients, where the one or more additional patients can share respiratory symptoms, demographic information, contextual information, and / or other associations with the patient. The one or more diagnostic audio recordings can have been previously assessed, graded, and / or diagnosed, and associated with one or more respiratory conditions, one or more respiratory symptoms, a severity of the respiratory condition / symptom, and other information related to the recorded health status of the source of the diagnostic audio recordings (e.g., the patient and / or one or more additional patients). From this, the machine learning algorithm can receive digital information of the one or more diagnostic audio recordings and generate a correspondence between the audio characteristics of the diagnostic audio recordings and the symptoms of the source of the diagnostic audio recordings.

[0080] At block 510, the CPU of the patient monitoring system can utilize a machine learning algorithm to determine a current health status associated with the patient. For example, the current health status of the patient can be determined from a comparison of the current vocal characteristics to a plurality of prior vocal characteristics. In certain instances, determining the current health status associated with the patient can utilize a trained machine learning algorithm based on one or more health statuses associated with one or more diagnostic audio recordings. For example, the CPU can compare the current vocal characteristics of the patient to a plurality of prior vocal characteristics and respiratory sequences. Further, the CPU can identify a diagnostic audio recording associated with a set of prior vocal characteristics determined to substantially match the current vocal characteristics. Based on analyzing respective spectrograms associated with the diagnostic audio recording and the current audio data, the set of prior vocal characteristics can be determined to substantially match the current vocal characteristics. Alternatively or additionally, the CPU can utilize a recorded correspondence between a patient health status / symptom and one or more diagnostic audio recordings to determine a severity associated with a symptom indicated by the current audio data. In other words, the one or more diagnostic audio recordings can indicate a series of changes in data source (e.g., patient or one or more additional patients) vocal characteristics caused by each set of respiratory symptoms associated with different severities. From this, the CPU can determine the current health status of the patient based at least on the recorded health status (e.g., recorded respiratory symptom severity).

[0081] Figure 6 An example patient monitoring system as described herein is shown. The patient monitoring system can utilize a user device 602 in communication with a server 618 over a communication network 616. Specifically, the user device 602 can include a computer processing unit (CPU) 604, a memory 606, a communication interface 608, a power management component 610, a display 612, and a user interface 614. Further, the server 618 can include a CPU 620, a memory 622, a communication interface 624, a database 626, a machine learning component 628, and a neural network component 630.

[0082] In Figure 6 certain instances, the CPU 604 and the CPU 620 can be configured to perform or partially perform the methods described with reference to Figures 1 to 5 Additionally or alternatively, the CPU 604 can operate to control a microphone associated with the user device to capture audio data related to a patient. The CPU 604 can communicate with the CPU 320, and the CPU 620 can communicate with the CPU 640 through the communication interface 608 and the communication interface 624 to receive an indication of a routine and / or algorithm to be performed and to transmit captured audio data during performance of the routine and / or algorithm.

[0083] In Figure 6In the illustrated example, the CPU 604 of the user device 602 can include one or more controllers, processors, and / or other hardware and / or software components configured to operatively control a microphone associated with the user device 602, the communication interface 608, the power management component 610, the display 612, the user interface 614, and other components of the user device 602. For example, Figure 6 The illustrated CPU 604 can include a single processing unit (e.g., a single processor) or multiple processing units (e.g., a multiprocessor), and can include a single computing unit or multiple computing units or processing cores. Figure 6 The illustrated CPU 602 can be implemented as one or more microprocessors, microcomputers, microcontrollers, digital signal processors, central processing units, state machines, logic circuitries, and / or any devices that manipulate signals based on operational instructions. For example, Figure 6 The illustrated CPU 602 can be one or more hardware processors and / or logic circuitries of any suitable type that are specifically programmed or configured to perform the algorithms, operations, and methods described herein. Figure 6 The illustrated CPU 602 can be configured to fetch and execute computer-readable instructions stored in the memory 606, which can program the CPU 602 to perform the functions described herein. Additionally or alternatively, Figure 6 The illustrated CPU 602 can be configured to fetch and execute computer-readable instructions stored in the audio screening component 126 of the audio analysis system 120 Figure 1 ).

[0084] In certain aspects, Figure 6 The illustrated memory 606 can be similar to the audio screening component 126 described above with respect to the audio analysis system 120 Figure 1 ). For example, the memory 606 can include volatile and nonvolatile memory and / or removable and non-removable media implemented in any type of technology for storing information such as computer-readable instructions, data structures, program modules or other data. Such memory 606 can include, but is not limited to, RAM, ROM, EEPROM, flash memory or other memory technology, optical storage, solid state storage, magnetic tape, magnetic disk storage, RAID storage systems, storage arrays, network-attached storage, storage area networks, cloud storage, or any other medium that can be used to store the desired information and that can be accessed by a computing device. The memory 606 can be a computer-readable storage medium and / or can be a tangible, non-transitory medium, when referred to herein, a non-transitory computer-readable medium does not include energy signals, carrier waves, electromagnetic waves, and signals per se, among others.

[0085] Memory 606 can be used to store any number of executable functional components as well as audio data processed by CPU 604 and / or transmitted through communication interface 608. In many embodiments, these functional components include instructions or programs executable by CPU 602 that, when executed, cause one or more CPUs 602 to be specifically configured to perform actions described herein associated with monitoring a patient's current health state.

[0086] Other functional components stored in memory 606 can include, among other things, a graphical representation data component, a measurement data component, a threshold data component, a notification component, a microphone data component, a machine learning component, a neural network component, and / or other functional components associated with operation of a patient monitoring system.

[0087] In certain instances, Figure 6 The illustrated communication interface 608 can be configured to transmit information to communication interface 624 through a communication network 616. For example, communication interface 608 and communication information 624 can enable user device 602 and server 618 to exchange audio data indications, patient health states, audio data indications to be collected, and / or other information using wired or wireless connections. Wireless connections can include cellular network connections and connections established using protocols such as 802.11a, b, g, and / or ac. In other instances, wireless connections between an audio user device and a server can be achieved using one or more wired or wireless protocols such as Bluetooth, WiFi Direct, radio frequency identification (RFID), infrared signals, and / or Zigbee. Other configurations are also possible. Similarly, communication network 616 is generally any type of wireless network or other communication network known in the art. Examples of network 616 include the Internet, an intranet, a wide-area network (WAN), a local-area network (LAN), a virtual private network (VPN), a cellular network connection, and connections established using protocols such as 802.11a, b, g, n, and / or ac.

[0088] In any of the instances described herein, Figure 6The illustrated CPU 602 can be configured to receive various information, signals, and / or other inputs from a microphone associated with the user device 602, the display 612, the user interface 614, and / or other components of the user device 602. In certain instances, the user interface 614 can receive such inputs from a user, patient, and / or physician, one or more of which can include a command or request for the patient monitoring system to generate, display, provide, and / or otherwise output one or more patient health statuses, audio data, patient symptom reports, and / or other outputs generated during monitoring of the patient. In certain additional instances, the display 612 can include, for example, a liquid crystal display (LCD), a light-emitting diode (LED) display, an organic light-emitting diode (OLED) display, or an active-matrix organic light-emitting display (AMOLED). Moreover, the display 612 can be an interactive display configured to receive touch inputs for the user interface 614. Alternatively or additionally, the user interface 614 can receive user inputs through a keyboard, mouse, or other input device.

[0089] In Figure 6 In the illustrated example, the CPU 620 of the server 618 can include one or more controllers, processors, and / or other hardware and / or software components configured to operatively control the microphone, the communication interface 624, the database 626, the machine learning component 628, the neural network component, and other components of the server 618 associated with the server 618. For example, as Figure 6 The illustrated CPU 620 can be substantially similar to the CPU 602. Moreover, the memory 622 can include computer-executable instructions similar to the memory 606 of the user device 602. Additionally, the database 626 can take the configuration described above with reference to Figures 1 to 5 the database 626 of the user device 602.

[0090] In Figures 1 to 5 In the illustrated example, the machine learning component 628 can take the configuration described above with reference to ​The configuration described. Generally, the machine learning component 628 represents a series of instructions for causing the server to process a training data set to establish a correspondence between audio characteristics of the incoming audio data and previously recorded patient symptoms associated with various respiratory conditions based on recorded audio characteristics of the recorded patient symptoms. The larger the training data set, the more accurate the determination of the current health status of the patient based on the previously recorded patient symptoms and the recorded audio characteristics generally will be. As such, the machine learning component can provide an algorithm capable of monitoring the incoming audio data from the patient and tracking the current health status based on audio data previously analyzed and characterized for the patient and / or other patients sharing similar categorization information. In certain additional examples, the machine learning component 628 can include an algorithm that utilizes a neural network to categorize the incoming audio data from the patient and determine the current health status of the patient based on the audio characteristics of the incoming audio data. For example, the neural network can be developed from a training data set that establishes branched categories of audio data characteristics associated with individual respiratory symptoms, respiratory conditions, and types of vocalizations produced by the patient.

[0091] It should be noted that while the above primarily relates to analyzing audio data related to a user speaking, breathing, and / or otherwise generating speech through the patient's respiratory tract and vocal cords, the system can also analyze other vocalizations. For example, the patient's footsteps can be tracked by a microphone to identify gait changes that can be caused by pain, lack of breath, and other issues. Alternatively or additionally, audio characteristics of the patient's vocal speech can be indicative of potential health issues other than respiratory conditions. Additionally, other aspects of the current health status associated with the patient can be tracked through audio characteristics. For example, by tracking different breathing patterns of the patient, a sleep habit model can be tracked, and significant events or changes in breathing patterns during sleep can be used to indicate the current health status of the patient. As such, the present disclosure is not limited to the explicitly disclosed methods, systems, and apparatuses, but is intended to include alterations and modifications made within the spirit of the appended claims.

[0092] The foregoing merely illustrates the principles of the disclosure and various modifications can be made by those skilled in the art without departing from the scope of the disclosure. The examples described above are presented for purposes of illustration only and not limitation. The present disclosure can take many forms of implementation. Accordingly, the disclosure is not limited to the explicit disclosure made herein. Rather, it covers all modifications and variations falling within the spirit of the appended claims.

[0093] As further examples, as illustrated and described herein, device variants or process limitations (e.g., dimensions, configurations, components, process step orderings, etc.) can be made to further optimize the provided structures, apparatuses, and methods. Regardless, the structures and apparatuses described herein, as well as the related methods, have broad applications. Accordingly, the disclosed subject matter should not be limited to any single example described herein, but should be construed in accordance with the broadest possible interpretation of the appended claims.

Claims

1. A lung health sensing system based on sound analysis, comprising: Memory; One or more processors; as well as Computer-executable instructions stored in the memory and executable by the one or more processors to perform operations, the operations including: Receive patient-associated audio data from a user device; identify at least one of the patient's spoken words and breathing sequences based on the audio data; One or more audio events are determined based on at least one of the spoken words and breathing sequences and based on at least one of the previous spoken words and previous breathing sequences associated with the patient, the audio events including one or more respiratory symptoms; Based at least in part on the human speech or breathing sequence, determine one or more first audio features associated with the one or more audio events; Based at least on the one or more first audio characteristics, determine an audio event category associated with the one or more audio events, the audio event category being associated with one or more second audio events indicating the patient's respiratory status; Diagnostic data is retrieved from a database, including individual audio data and corresponding health statuses associated with at least one of a plurality of other patients; and Based at least on the diagnostic data, the one or more audio events and the audio event categories, the patient's current health status is determined.

2. The system according to claim 1, wherein, The user equipment includes a medical device, and the audio data is collected via a microphone operatively connected to the user equipment. The medical device is configured to transmit the audio data to the system on a substantially periodic, aperiodic, or continuous basis.

3. The system according to claim 1, wherein, The audio data is collected from the patient following treatment or diagnosis associated with the respiratory status; and Track the patient's current health status to monitor recovery from the respiratory condition.

4. The system according to claim 1, wherein, The one or more audio events are determined at least in part based on the spoken words, the one or more audio events including at least one of dialogues, request phrases, audible pain events and / or screams associated with the patient.

5. The system according to claim 1, wherein, The one or more audio events are determined at least in part based on the breathing sequence, the one or more audio events including at least one of inhalation, exhalation, coughing, sneezing, or dyspnea associated with the patient.

6. The system according to claim 1, wherein, Identifying at least one of the spoken words and the breathing sequence further includes: distinguishing at least one of the spoken words and the breathing sequence within the audio data from the background audio of the audio data.

7. The system according to claim 1, wherein, The one or more first audio features include: The tone of the human voice; The timbre of the human voice; The rhythm of the spoken words; The volume of the human voice; The speech rate associated with the spoken words; The duration of inspiratory respiration associated with the breathing sequence; The duration of exhalation associated with the respiratory sequence; and The respiratory rate associated with the respiratory sequence.

8. The system according to claim 1, wherein, The audio event category includes recorded audio events associated with a speech type or a breathing sequence type, wherein the speech type or the breathing sequence type is characterized by at least one or more of the second audio features; and The audio event category is determined based at least on one or more of the first audio characteristics, including at least one or more of the second audio characteristics.

9. The system according to claim 8, wherein, Determining the patient's current health status further includes: Based on audio data associated with at least one of several other patients, a match is determined with the audio event category; Based at least on the diagnostic data associated with the at least one patient, a recorded health status associated with the respiratory status of the at least one patient is determined; The current health status is based at least on the recorded health status.

10. The system according to claim 9, wherein, The recorded health status includes one or more respiratory symptoms associated with the at least one patient, and the one or more respiratory symptoms include at least one of the following: The patient has respiratory tract inflammation; Difficulty inhaling; Difficulty exhaling; The patient has excessive mucus in their respiratory tract; Chronic cough; One or more altered audio characteristics; Coughing up blood; Shortness of breath; and chest tightness.

11. The system according to claim 9, wherein, The recorded health status includes: Initial health status associated with the initial audio characteristics and initial patient symptoms recorded prior to the medical description of respiratory status; Target health status associated with target audio characteristics and target patient symptoms; or The actual health status recorded when treating the respiratory condition, in association with actual audio data and actual patient symptoms.

12. The system according to claim 1, wherein, The respiratory condition includes at least one of asthma, chronic obstructive pulmonary disease, bronchitis, emphysema, lung cancer, cystic fibrosis, pneumonia, or pleural effusion; and The current health status includes an indication of the severity of symptoms associated with the respiratory condition.

13. A method for sensing lung health through sound analysis, executed by a processor, the method comprising: The microphone is prompted to record audio data from the patient, including the patient's voice or breathing sequence. Receive the audio data from the microphone; Based at least on audio data, identify the vocal characteristics associated with the patient's speech and the respiratory characteristics associated with the patient's breathing sequence; Using a first machine learning model, based on human voice characteristics and breathing characteristics, the categories of audio events indicating the patient's breathing status are determined; Diagnostic data is retrieved from a database, including diagnostic audio recordings associated with multiple other patients and corresponding recorded health statuses; and The patient's current health status is determined at least in part based on the diagnostic data and the audio event category.

14. The method according to claim 13, wherein, The first machine learning model is trained using previous voice and breathing characteristics associated with one or more additional patients.

15. The method according to claim 13, wherein, The current health status is determined by a second machine learning model, and training the second machine learning model includes: The recorded health status and diagnostic audio recordings associated with the other patients are provided as training inputs to the second machine learning model.

16. The method of claim 13, wherein, Determining the current health status associated with the patient further includes: Identify at least one patient from the diagnostic audio recordings, wherein previous vocal characteristics associated with the at least one patient are determined to be substantially a match of the vocal characteristics; and The current health status of the patient is determined based at least on the recorded health status associated with the at least one patient.

17. The method according to claim 13, wherein, Identifying the vocal characteristics and the breathing characteristics further includes: Based at least on the audio spectrogram of the audio data, determine the pitch, timbre, rhythm, volume, and speech rate associated with the patient; Based at least on the audio spectrogram, determine the respiratory rate and average respiratory volume; and Based at least on the audio spectrogram, one or more pain events are identified, wherein the one or more pain events are associated with one or more groans, one or more sighs, or one or more snoring.

18. A lung health sensing system based on sound analysis, comprising: One or more processors; Memory; as well as Computer-executable instructions stored in the memory and executable by the one or more processors to perform operations, the operations including: The microphone associated with the user device is prompted to collect a first audio recording, wherein the first audio recording includes at least one of the patient speaking or the patient breathing; Receive the first audio recording from the user equipment; Based at least on the first audio recording, determine an audio category associated with the first audio recording, the audio category being associated with one or more human voice utterances in the first audio recording characterized by one or more first audio features; retrieve diagnostic data from a database, the diagnostic data including various audio data and corresponding health statuses associated with at least one of a plurality of other patients; determine a first patient's health status based at least on the audio category and the diagnostic data, the first patient's health status being associated with the patient's respiratory symptoms; This causes the microphone associated with the user equipment to collect a second audio recording; Based at least on the second audio recording, identify one or more second audio features associated with the second audio recording; and A second patient health status is determined based at least on the diagnostic data and by comparing the one or more first audio characteristics with the one or more second audio characteristics, wherein the second patient health status indicates a change in health status relative to the first patient health status.

19. The system according to claim 18, wherein, Determining the audio category associated with the first audio recording further includes: prompting a machine learning algorithm to identify the type of human speech associated with the first audio recording, wherein the type of human speech identifies at least one of the source, demographics, or content associated with the human speech.

20. The system of claim 18, wherein the operation further comprises: Based at least on a health status threshold, it is determined that the health status of the second patient meets the health status threshold, which is associated with a worsening of respiratory symptoms compared to the health status of the first patient; as well as The system transmits an indication of the second patient's health status to the patient's personal device.

Images