Systems, methods and computer-readable media for processing acoustic signals

Abstract

A system for processing acoustic signals comprises at least one sensor configured to obtain at least one acoustic signal over a period of time, a processing module configured to: determine signal data of the at least one acoustic signal obtained from the at least one sensor, partition the signal data into a plurality of intervals, generate, for each interval in the plurality of intervals, a bin associated with the interval by processing data points of the signal data within the interval; a user interface module configured to generate, for each bin, a responsive component on a user interface corresponding to the bin; an audio access module configured to: access, for each responsive component, the signal data within the interval associated with the bin that corresponds to the responsive component, wherein upon interaction with the responsive component, the audio access module is configured to output the audio signal.

Description

"Systems, methods and computer-readable media for processing acoustic signals"Cross-reference to related applications

[0001] This application claims the benefit of Australian Provisional Patent Application No. 2024903221, filed 4 October 2024, which is incorporated by reference herein in its entirety, and is hereby expressly made a part of this specification.Technical Field

[0002] The present disclosure generally relates to systems, methods and computer-readable media for processing acoustic signals, and in some embodiments, systems, methods and computer readable media for outputting segments of audio signals corresponding to the acoustic signals.Background

[0003] Various medical monitoring systems have been developed to provide observation metrics and data for patient conditions. These devices focus on metric-based reporting, indicating patient state, through the use of oscillometric and ECG signal inputs. They may be used in clinical settings to monitor vital signs and other key health indicators, offering healthcare professionals the ability to monitor patient conditions. However, these devices are primarily focused on the aggregation and presentation of metrics, rather than providing access to the underlying continuous data. In some medical monitoring systems, the design approach primarily focuses on the use case of a device that provides point-in-time observation metrics and data for patient condition as a primary source of value.

[0004] In clinical settings, the processing of acoustic signals is undertaken for various diagnostic and monitoring applications. However, existing systems face several significant challenges, particularly with the amount of data generating from continuous monitoring. Continuous monitoring, especially of vital signs in applications such as sleep monitoring, can generates hours of data per person. This amount of data significantly slows down the time it takes to access the data, particularly through user interfaces. The sheer volume of data requires extensive load times and bandwidth to process and access, often resulting in unresponsive user interfaces that fail to provide critical information promptly.

[0005] Additionally, the large amounts of data collected in continuous monitoring can cause slow loading times, making the user interface unresponsive for long periods, and make it difficult for users, such as clinicians, to interact with the data effectively. This can result in delays and errors in diagnosis and treatment, as critical information may not be readily accessible when needed. Furthermore, existing systems fail to provide responsive mechanisms for accessing and reviewing critical data points in the collected data. The extensive load times required to access large datasets can hinder the ability to quickly retrieve and analyse relevant segments of the data. In scenarios like sleep studies, clinicians often have to review the entire dataset, which could be hours of acoustic recordings, in order to diagnose issues, which is time-consuming and inefficient.

[0006] Moreover, the process of retrieving and converting large amounts of data into an output can further slow systems, affecting the overall efficiency and reliability of the data processing.

[0007] Any discussion of documents, acts, materials, devices, articles or the like which has been included in the present specification is not to be taken as an admission that any or all of these matters form part of the prior art base or were common general knowledge in the field relevant to the present disclosure as it existed before the priority date of each of the appended claims.Summary

[0008] Some embodiments relate to a system for processing acoustic signals, the system comprising: at least one sensor configured to obtain at least one acoustic signal over a period of time; a processing module, configured to: determine signal data of the at least one acoustic signal obtained from the at least one sensor; partition the signal data into a plurality of intervals; generate, for each interval in the plurality of intervals, a bin associated with the interval by processing data points of the signal data within the interval; a user interface module, configured to generate, for each bin, a responsive component on a user interface corresponding to the bin;an audio access module configured to: access, for each responsive component, the signal data within the interval associated with the bin that corresponds to the responsive component; wherein upon interaction with the responsive component, the audio access module is configured to: retrieve at least part of the signal data of the acoustic signal convert the retrieved signal data to an audio signal; and output the audio signal.

[0009] The processing module may be further configured to identify one or more events in the signal data of the acoustic signal. The processing module may be further configured to classify the one or more identified events based on at least one of frequency, amplitude, and duration.

[0010] In some embodiments, identifying the one or more events may comprise applying a machine learning model to detect deviations from a predicted pattern in the signal data. In some embodiments, identifying the one or more events may comprise applying a machine learning model to further refine the one or more events based on patterns in the signal data. The patterns may comprise resemblant patterns. The one or more events may be determined based on at least one of a spike in data values in the signal data and a drop in data values in the signal data.

[0011] The audio access module may include a playback component integrated with the user interface, and the audio access module may be configured to output the audio signal to the playback component. In some embodiments, the audio access module may be configured to provide a visual representation of the audio signal on the user interface based on the retrieved signal data. The audio access module may include a control module configured to control playback of the audio signal, the control module having one or more control components integrated with the user interface.

[0012] In some embodiments, the system may further comprise a haptic feedback device, configured to receive the output audio signal, and convert the audio signal into mechanical vibrations.

[0013] The at least one sensor may be a vibration sensor. The acoustic signal may include at least one biosignal.

[0014] The acoustic signal may include at least one primary context signal. The primary context signal may include at least one of: breath sounds, heart sounds, snoring, obstructions, wheezing, coughing, and crackles.

[0015] The user interface module may be configured to generate the bin for each interval based on the at least one primary context signal. In some embodiments, the processing module may be configured to classify one or more events in the signal data based on the primary context signal.

[0016] The acoustic signal includes at least one secondary signal. The at least one secondary signal may comprise a breathing movement signal and / or a movement signal.

[0017] In some embodiments, the processing module may be configured to filter the acoustic signal into one or more frequency ranges of interest to create a plurality of signals. Partitioning the signal data into a plurality of intervals may be based on a predetermined time duration and / or length of the acoustic signal.

[0018] In some embodiments, the bin may be a decimation bin. Processing the data points within the interval may include extracting a subset of data points in the interval. Processing the data points within the interval may include aggregating and / or condensing the data points in the interval. Extracting the subset of data points for each interval may comprise calculating a statistical measure, the statistical measure comprising at least one of count, sum, ratio, mean, median, mode, quantile and standard deviation of the data points within the interval.

[0019] In some embodiments, the decimation bin may provide a representation of the signal data within the interval. The responsive component may comprise a graphical element representative of the generated bin. The user interface module may be configured to generate responsive components in real time as bins are generated.

[0020] In some embodiments, the user interface module may be configured to generate a container component to arrange responsive components on the interface in a sequential array corresponding to a position of the corresponding intervals within the signal data. The userinterface module may be configured to generate, for a plurality of primary context signals, a container component corresponding to each primary context signal, the container component including an array of responsive components corresponding to intervals of a primary context signal.

[0021] The responsive components within a container component may enable interrogation of the signal data corresponding to the container component. In some embodiments, the retrieved signal data may include baseline data.

[0022] Some embodiments relate to a method for processing acoustic signals, the method comprising: determining signal data from at least one acoustic signal obtained from at least one sensor; partitioning the signal data into a plurality of intervals; generating, for each interval in the plurality of intervals, a bin associated with signal data in the interval by processing data points within the interval; generating, for each bin, a responsive component on a user interface corresponding to the bin; upon interaction with the responsive component, retrieving at least part of the signal data of the acoustic signal; converting the retrieved signal data to an audio signal; and outputting the audio signal.

[0023] Some embodiments relate to a user interface, comprising: a plurality of responsive components arranged on the user interface, each of the plurality of responsive components including an attached handler; wherein each responsive component corresponds to a bin associated with an interval of signal data of at least one acoustic signal received from at least one sensor; and wherein each responsive component is bound to interval data associated with the bin to provide access to the signal data; at least one container component configured to hold a plurality of the responsive components associated with the at least one acoustic signal, the container component arranging the responsive components on the user interface to provide an integrated graphical representation of the at least one acoustic signal;at least one handler attached to each responsive component, configured to, upon determining an interaction with a responsive component has occurred, retrieve at least part of the signal data of the acoustic signal bound to the responsive component; and a playback component integrated within the user interface, the playback component configured to receive an audio signal converted from the at least part of the signal data of the acoustic signal.

[0024] Some embodiments relate to a system for processing a target audio segment, the system comprising: a user interface having a plurality of responsive components; at least one extraction module, configured to: upon interaction with a responsive component of the plurality of responsive components, determine an interval of signal data associated with the responsive component; determine a first time point and a second time point; and extract the signal data between the first time point and the second time point to define a target data segment; an audio segment output module, configured to: convert the target data segment into an audio segment; and output the audio segment.In some embodiments, the extraction module may be further configured to: identify at least one event within the interval of signal data; and determine the first time point as a time point before the at least one event, and the second time point as a time point after the at least one event.

[0025] The at least one event may be determined based on at least one of a spike in data values in the signal data and a drop in data values in the signal data. The first time point may be determined based on a predefined buffer prior to a start of the at least one identified event in which there is no anomalous activity. The second time point may be determined based on a predefined buffer after an end of the at least one identified event in which there is no anomalous activity.

[0026] In some embodiments, the audio segment output module may be further configured to generate a visual alert on the user interface when playback of the audio segment reaches theat least one identified event. The first time point and the second time point may be determined based on a time interval.

[0027] In some embodiments, the audio segment output module may include a playback component integrated with the user interface, and the audio segment output module may be configured to initiate playback of the audio segment through the playback component. The audio segment output module may be configured to provide a visual representation of the audio segment on the user interface based on the retrieved signal data.

[0028] In some embodiments, the audio segment output module may include a control module configured to control the playback component, the control module having one or more control components integrated with the user interface. The audio segment output module may be further configured to synchronise the playback of the audio segment with a visual representation of the signal data.

[0029] The audio segment output module may be further configured to synchronise the playback of the audio segment with one or more external devices. In some embodiments, may further comprise a haptic feedback device, configured to receive the audio segment and convert the audio segment into mechanical vibrations.

[0030] Each of the plurality of responsive components may represent a bin associated with the interval of signal data. In some embodiments, the signal data may be data from an acoustic signal. The acoustic signal includes at least one primary context signal. The primary context signal may include at least one of: breath sounds, heart sounds, snoring, obstructions, wheezing, coughing, and crackles. The user interface may include a responsive component corresponding to each interval of the signal data of the at least one primary context signal.

[0031] In some embodiments, the acoustic signal may include at least one secondary signal. The at least one secondary signal may comprise a breathing movement signal and / or a movement signal.

[0032] In some embodiments, the audio segment output module may be configured to convert the target data segment into an audio segment by converting the target data segment to an analog signal to create an audio signal.

[0033] In some embodiments, the extraction module may be further configured to store the target data segment in a data store. The extraction module may be further configured to apply a compression algorithm to the target data segment to reduce a size of the target data segment before converting the target data segment into an audio segment. The extraction module may be further configured to generate metadata for the target data segment, wherein the metadata includes at least the first time point and the second time point.

[0034] The user interface may further comprise an annotation component configured to annotate the visual representation of the audio signal.

[0035] Some embodiments relate to a method for processing a target audio segment, the method comprising: providing a user interface with a plurality of responsive components; upon interaction with a responsive component of the plurality of responsive components, determining an interval of signal data bound with the responsive component; determining a first time point and a second time point; and extracting the signal data between the first time point and the second time point to define a target data segment; converting the target data segment into an audio segment; and outputting the audio segment.

[0036] Some embodiments relate to a user interface, comprising: a plurality of responsive components arranged on the user interface, each of the plurality of responsive components including an attached handler; wherein each responsive component corresponds to a bin associated with an interval of signal data of at least one acoustic signal received from at least one sensor; and wherein each responsive component is bound to the interval data associated with the bin to provide access to the signal data; at least one container component configured to hold a plurality of the responsive components associated with the at least one acoustic signal, the container component arranging the responsive components on the user interface to provide an integrated graphical representation of the at least one acoustic signal; and at least one handler attached to each responsive component, the handler configured to, upon determining an interaction with a responsive component has occurred, retrieve an audio segment;wherein the audio segment comprises extracted signal data between a first time point and a second time point within the interval.

[0037] Throughout this specification the word "comprise", or variations such as "comprises" or "comprising", will be understood to imply the inclusion of a stated element, integer or step, or group of elements, integers or steps, but not the exclusion of any other element, integer or step, or group of elements, integers or steps.Brief Description of Drawings

[0038] Various ones of the appended drawings merely illustrate example embodiments of the present disclosure and cannot be considered as limiting its scope.

[0039] Figure 1A is a block diagram of a system for processing acoustic signals, according to some embodiments.;

[0040] Figure IB is a process flow diagram of a method for processing acoustic signals, according to some embodiments;

[0041] Figure 2 is a schematic diagram of a user interface, according to some embodiments;

[0042] Figure 3 is a schematic diagram of the user interface of Figure 2, on which interaction with a responsive component has occurred, according to some embodiments;

[0043] Figure 4 is a schematic diagram of the user interface of Figure 2, showing control of the audio signal playback through the user interface, according to some embodiments;

[0044] Figure 5A illustrates a responsive component having a graphical representation in the form of a bar / line plot graphical representation, according to some embodiments;

[0045] Figure 5B illustrates a responsive component having a graphical representation in the form of a superimposed bar plot, according to some embodiments;

[0046] Figure 5C illustrates a responsive component having a graphical representation in the form of a line plot with a range, according to some embodiments;

[0047] Figure 5D illustrates a responsive component having a graphical representation in the form of a combined line plot, according to some embodiments;

[0048] Figure 6 is a block diagram of a system for processing a target audio segment, according to some embodiments;

[0049] Figure 7 is a process flow diagram of a method for processing a target audio segment, according to some embodiments;

[0050] Figure 8 is an image of an example user interface related to a clinical setting, according to some embodiments;

[0051] Figure 9 is an image of the example user interface of Figure 8, showing the interchange of a pathology explorer, according to some embodiments;

[0052] Figure 10 is an image of the example user interface of Figure 8, showing an apnea pathology explorer, according to some embodiments;

[0053] Figure 11 is an image of the example user interface of Figure 8, showing interaction with a responsive component in the apnea pathology explorer, according to some embodiments;

[0054] Figure 12 is an image of the example user interface of Figure 8, showing interaction with a responsive component in an obstruction rate pathology explorer, according to some embodiments; and

[0055] Figure 13 is an image of the example user interface of Figure 8, showing interaction with a responsive component in a heart rate pathology explorer, according to some embodiments.Description of Embodiments

[0056] The present disclosure generally relates to systems, methods and computer readable media for processing acoustic signals. In some embodiments, the present disclosure relates to a system for processing and providing access to acoustic signal data through a user interface and, in some embodiments, providing access to intervals or segments of acoustic signal datathrough a user interface. Some embodiments of the present disclosure relate to an improved user interface for providing access to partitioned signal data of acoustic signals. Some embodiments relate to providing access to an audio signal from a graphical representation of acoustic signal data. Some embodiments relate to auto-selection of a target audio segment of an acoustic signal through a graphical representation of the acoustic signal data.

[0057] In some embodiments, the present disclosure relates to a clinical data processing system, and may be designed to monitor, interrogate and analyse patient data, and produce patient assessments, such as observation charts for vital signs, by utilising the observed clinical data. For example, the system may capture and record the vibro-acoustic signals that are detected using a stethoscope-like sensor array (such as auscultatory signals) and then enables such signals to be available and accessible to be replayed in real-time so that a clinician, healthcare provider, or a patient, is able to reproduce the physical examination asynchronously at any later time, interrogate the original auscultatory signals, and facilitate palpation of the vibro-acoustic signals.

[0058] The present disclosure encompasses the processing of acoustic signals to monitor both normal data or normal event data, indicative of healthy or standard functioning (for example, a regular heart rate), and significant or anomalous event data, which highlights occurrences within the signal data such as snoring or apnea events that indicate non- normative functioning. This capability allows for a comprehensive assessment of a patient’s condition by continuously monitoring and analysing both normal and / or anomalous data.

[0059] In some embodiments, the system may classify the observed data into events and discretise these events into decimation bins. Such a process accounts for the time-elastic nature of the observed pathology, structuring the data into specific N-width inputs. These inputs may be fed into an Artificial Neural Network (ANN) or a similar machine learning model to generate patient assessments. The data may be captured by the system and / or include third-party devices that provide Application Programming Interfaces (APIs) for integration and may be suitable for continuous monitoring of vital signs such as breathing, heart rate, movement and auscultatory sounds.

[0060] Figure 1A is a block diagram of a system 100 for processing acoustic signals, according to some embodiments. The system 100 comprises one or more processor(s) 102 and / or memory 104. The processor(s) 102 may comprise one or more microprocessors,central processing units (CPUs), application specific instruction set processors (ASIPs), application specific integrated circuits (ASICs) or other processors capable of reading and executing instruction code.

[0061] The memory 104 may comprise one or more volatile or non-volatile memory types. For example, memory 104 may comprise one or more of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM) or flash memory. The memory 104 is configured to store program code accessible by the processor(s) 102. The program code comprises executable program code modules. In other words, memory is configured to store executable code modules which, when executed by the processors, cause the system to perform certain functionalities. For example, memory 104 may comprise a signal acquisition module 106, a processing module 108, a user interface module 110 and / or an audio access module 112. A module may be a collection of computer executable or computer intelligible code, that when executed by a processor, causes the processor to perform processes, action or functionalities, such as the processes, actions or functionalities of the methods described herein.

[0062] In some embodiments, memory 104 stores instructions (such as program code) which when executed by the processor(s) 102 causes the system 100 to perform methods for processing acoustic signals.

[0063] In some embodiments, the system 100 may be implemented as a distributed system comprising multiple server systems configured to communicate over a network to provide the functionality of the system 100. For example, one or more of the program code(s) (for example modules or engines) may be deployed on one or more disparate or remote servers, which may cooperate to provide the functionality of the system 100 described. In some embodiments, the system 100 may be in communication with a network 114, and include a network interface 116 to facilitate communication with additional components, including computing device(s) 118, and one or more data stores 120. The network interface 116 may comprise a combination of network interface hardware and network interface software suitable for establishing, maintaining and facilitating communication over a relevant communication channel.

[0064] The one or more data stores 120 may form part of or be local to the system 100, or may be remote from and accessible to the system 100, for example, through the network 114.The one or more data stores 120 may be relational or non-relational databases. In some embodiments, the one or more data stores 120 may comprise a database configured to store signal data of at least one acoustic signal.

[0065] Network 114 may comprise one or more local area networks or wide area networks that facilitate communication between elements of system 100. For example, according to some embodiments, network 114 may be the internet. However, network 114 may comprise at least a portion of any one or more networks having one or more nodes that transmit, receive, forward, generate, buffer, store, route, switch, process (or any combination thereof), one or more messages, packets, signals, and / or some combination thereof. Network 114 may include, for example, one or more of: a wireless network, a wired network, an internet, an intranet, a public network, a packet-switched network, a circuit-switched network, an ad hoc network, an infrastructure network, a public-switched telephone network (PSTN), a cable network, a cellular network, a satellite network, a fibre-optic network, and / or some combination thereof.

[0066] The system 100 may include a signal acquisition module 106 in communication with the one or more sensors 122. The signal acquisition module 106 may be configured to control the one or more sensors 122 to obtain at least one acoustic signal. In some embodiments, the sensors 122 are configured to transmit the received data through the network 114 to the signal acquisition module 106. In some embodiments, system 100 may be configured to receive data directly from the sensors 122, through a wired or wireless connection. In some embodiments, the at least one acoustic signal obtained by the sensors 122 may be stored in data store 120 and retrieved by the signal acquisition module 106.

[0067] In some embodiments, the system 100 may be configured to continuously collect acoustic signals from the one or more sensors 122. Acoustic signals may refer to the mechanical waves that propagate through a medium due to oscillations in pressure, particle displacement, or other physical quantities. For example, acoustic signals may include sounds waves, ultrasound, infrasound, and / or vibrations. The acoustic signals may comprise vital signs related to a subject, such as those detected by auscultation. For example, they may comprise vibroacoustic physical signals (or “physical signs”) from the one or more sensors 122. The acoustic signals may include, but are not limited to, breathing movements and sounds generated inside the human body by those physical acts of breathing (such as breath sounds and lung sounds, including vesicular sounds, crackles, wheezes, rhonchi, stridor and / or pleural friction rub), heart movements and heart sounds (such as beats, murmurs,clicks, rubs), and / or gastrointestinal (“gut”) movements and gut sounds (such as bowel sounds, borborygmi and bruits). In some embodiments, the sensors 122 may be configured to pick up a plurality of different acoustic signals over a number of frequency and / or amplitude ranges.

[0068] In some embodiments, the sounds and vibrations generated continuously internally by the normal pumping of the heart and breathing system include the dual heart sounds, and vesicular (“normal”) sounds and intermittent gut sounds confirming normal function, as well as identifying a large array of potential heart and lung pathological sounds.

[0069] In some embodiments, the sounds and vibrations generated continuously internally by the heart pump include a normal first and second heart sound, without additional sounds and vibrations, indicative of normal heart function. The sounds and vibrations may also include different relative amplitudes of the heart sounds and shifting with breathing indicative of high pulmonary arterial pressures. Additionally, the sounds and vibrations may include a third heart sound (with a "gallop rhythm") indicative of reduced left ventricular ejection fraction and heart failure, and a fourth sound, indicative of heart failure with preserved ejection fraction and / or diastolic heart failure. In some embodiments, vibrations and sounds may further include heart murmurs, different for each heart valve and either systolic and / or diastolic timing, continuous heart murmur (and palpable vibration) of a ventricular septal defect, fetal heart sounds (captured, for example, from a pregnant subject lying on the sensor mat) identified and enhanced by the haptic combination of visual display, and both auscultation and palpation detected vibrations, pericardial rub, indicative of pericarditis, and other vibroacoustic signals.

[0070] In some embodiments, the sounds and vibrations generated continuously internally by the breathing system include vesicular ("normal, white noise "blowing" sounds) indicative of normal breathing. The sounds and vibrations may include snoring indicative of upper airway obstruction, stertor, indicative of marked fixed obstruction in the upper airway, and / or croup, a coarse rasping sound indicative of marked tracheal obstruction. Additionally, in some embodiments, sounds and vibrations may include wheeze including multiple patterns of high- pitched, low-pitched expiratory times, inspiratory time indicative of a range of airway disorders, including bronchitis, pneumonia, asthma, crackles of several different characteristics indicative of pneumonia, interstitial lung diseases, and heart failure, pleural rub, indicative of pleurisy, fetal breathing movements and other episodic fetal movements (forexample, captured from a pregnant subject lying on the sensor mat) identified and enhanced by the haptic combination of visual display, and both auscultation and palpation detected vibrations, coughing from the subject, sneezing from the subject, vocalizing by the subject (talking, crying, etc.), and other miscellaneous diagnostic sounds.

[0071] In some embodiments, the sounds and vibrations generated continuously internally by the gastrointestinal system include reflux sound / vibration (plus cough). The sounds and vibrations may include borborygmi, isolated gut sounds, rhythmic gut sounds of peristalsis, and high-pitched "tinkling" sounds of bowel paralysis. Other miscellaneous gastrointestinal sounds and vibrations may also be included in the sounds and vibrations.

[0072] In some embodiments, the sounds and vibrations generated continuously internally by the environmental / local context may include snoring of a bed partner or other sleeper in the room, local talking, audio playing, television, and the like, external noise such as trucks, trains, aircraft (for example, helicopters) inducing arousal, dogs barking, and miscellaneous external sounds that may induce arousal. In some embodiments, these environmental and / or local context sounds may be simultaneously captured by environmental and / or local room sensors.

[0073] In some embodiments, additional acoustic signals may be captured by third party devices in communication with the system 100 and / or the signal acquisition module 106 through the network 114. For example, third party devices such as oximeters and electrocardiogram (ECG) monitors may be integrated with the system through, for example, APIs.

[0074] Sensors 122 may be configured to receive data in the form of at least one acoustic signal. In some embodiments, the sensors 122 are multimodal sensors. The sensors 122 may be “non-contact sensors”, meaning that they are not affixed to a subject. The subject may therefore move freely in relation to the sensors 122. In some embodiments, the sensors 122 may include vibration sensors, acoustic sensor and / or vibroacoustic sensors. Sensors 122 may be disposed proximate to a subject lying on a surface, such as a chair, mat or bed, or a combination of these. In some embodiments, the one or more sensors 122 are distributed in an array configuration. In some embodiments, two or more sensors may be affixed to the surface to capture environmental sounds, for example, within a room. In some embodiments, thesensors 122 may include, but are not limited to, thermal sensors, moisture sensors, and / or ultrasound sensors.

[0075] While embodiments described herein utilise vibration sensors, acoustic sensors and / or vibroacoustic sensors, other types of sensors may be used as an alternative or in addition to these sensors. For example, electroencephalogram (EEG), ECG, electromyography (EMG) or electrooculogram (EOG) sensors may be used. In some embodiments, the one or more sensors 122 may include one or more microphones, which may be used in conjunction with other types of sensors to sense acoustic signals relating to the subject, and / or to identify signals not relating to the subject. For example, signals not relating to a subject undergoing sleep monitoring may include snoring generated by a bed partner, sibling or parent in a child's room.

[0076] In some embodiments, the sensors may be disposed in a monitoring apparatus 124. The monitoring apparatus may include, for example, a mattress, adapted to receive a subject in a recumbent position. In some embodiments, the sensors may be disposed within the material of the mattress and / or may be positioned within respective recesses in the material of the mattress. The top surface of the sensors 122 may be located substantially flush with or projecting from a top surface of the mattress such that any one or more of the sensors 122 may press against the recumbent subject at any point in time. Nonetheless, a sheet or other covering may be positioned over the sensors 122 and / or mattress, between the sensors and the subject. In some embodiments, the sensors 122 are located at a position that is roughly one third of the length of the mattress from a head end of the mattress. The position of sensors 122 may correspond to the position at which the subject’s chest is located.

[0077] The signal acquisition module 106 is configured to transmit the acoustic signal obtained by the one or more sensors 122 to a processing module 108.

[0078] The processing module 108 is configured to determine signal data of the at least one acoustic signal obtained from at least one sensor 122. The processing module 108 is configured to determine signal data of the at least one acoustic signal obtained from the sensors 122. The processing module may determine the signal data by digitising the electrical signal. Alternatively, the processing module may receive the digitised signal data from the signal acquisition module 106, an external system (not shown) or a data store 120.

[0079] In some embodiments, the processing module 108 may be configured to analyse the signal data to identify one or more events in the signal data. An "event" may refer to any occurrence within the acoustic signal that can be characterised. An event may encompass a range of occurrences and / or data, including but not limited to, unusual occurrences in the signal that deviate from a predetermined or expected baseline, and / or normal or expected occurrences in the signal that are of particular interest for monitoring or analysis purposes. In some embodiments, an event may refer to an anomaly within the signal data, for example, a temporal anomaly in the signal. In some embodiments, events may be classified as unusual and / or anomalous events within the signal data, or may be classified as normal and / or expected events within the signal data. In some embodiments, the event may refer to pathological signals within the acoustic signal. For example, pathological signals may include anomalous events such as a snore, wheeze, crackle, extra heart sounds, murmurs, and the like. In some embodiments, the one or more events may be identified by the processing module 108 by identifying at least one of a spike in data values in the signal data and a drop in data values in the signal data. In some embodiments, the processing module may be configured to classify the one or more identified events based on at least one of frequency, amplitude, and duration. In some embodiments, identifying the one or more events comprises applying a machine learning model to the signal data to detect deviations from a predicted pattern in the signal data. In some embodiments, identifying the one or more events comprises applying a machine learning model to further refine the one or more events based on resemblant patterns in the signal data

[0080] In some embodiments, the processing module 108 is configured to classify the identified events in the signal data. Classification of events may be performed by machine learning (ML) models and / or by traditional classification algorithms. In some embodiments, event types may be identified by using straightforward algorithms for signal processing. These straightforward algorithms may comprise a “sliding window” that processes overtime series data for one or more input signals. When certain criteria are met, the start of the event can be determined. Similarly, when other criteria are met, the end of the active event is determined, meaning that the event may be identified and stored for subsequent processing. For example, a body movement event may be directly identified by determining whether the signal received from a vibroacoustic sensor exceeds a predetermined threshold.

[0081] In one example, an acoustic signal may be processed to determine a movement event. In this example, a sliding window of 1 second duration steps 0.1 seconds with every iteration.For each step, the root mean square (RMS) is calculated. If the RMS exceeds some proportion or level above the historical RMS (that is, over some preceding time window, e.g. 30 seconds) then the start of the event is determined. After some number of steps, the RMS may drop below some relative threshold (somewhere between the baseline and peak) whereupon the end of the event is determined, and the event is identified and stored. In some embodiments, additional processing may occur, for example, a check may be performed to ensure that the movement event is at least n seconds long to eliminate artifacts.

[0082] In some embodiments, more complex events may be identified using straightforward algorithms. The algorithms used to identify more complex events may utilise more complex logic than a threshold check over a sliding window. Additionally, more complex events may be determined using “lower level” events as building blocks. For example, large body movement events may be identified first in processing an acoustic signal. Then, the movement events may be inverted to find regions of quiescence in the acoustic signal to narrow the search for other events. This can substantially improve the speed at which the acoustic signal is processed for event identification.

[0083] In some embodiments, event types can be identified with ML. For example, an apnea or snoring can be identified by analysing the acoustic signal for specific patterns and features that are characteristic of these conditions, such as irregular breathing sounds, pauses in breathing, or the presence of snoring vibrations within a particular frequency.

[0084] The classification of events using ML may comprise identifying “candidate” events using traditional algorithms before applying a relevant ML model to classifies the candidate events. For example, a breath rate (BR) signal and heart rate (HR) signal may be generated form the acoustic signal using traditional algorithms. These BR and HR signals may then be used in one or more subsequent traditional algorithms to identify candidate events for potential heart or breath pathology (such as arrhythmia). The ML model may then be applied to refine the candidate events into identified pathology events.

[0085] One or more ML models may be used for each pathology event type. In some embodiments, each ML model uses a combination of input signals for a time window related to the event. In some embodiments, the ML model may comprise a binary classifier. For example, the binary classifier used to determine a “snore” event may categorise the acoustic signal data into “snore” events and “not snore” events. In some embodiments, the ML modelmay comprise a categorical classifier. For example the categorical classifier may classify the acoustic signal data into central apnea events, central hypopnea events, and / or a “neither” event.

[0086] In some embodiments, the processing module may be configured to identify the determined signal data of the at least one acoustic signal as baseline data (also referred to herein as normal data). In some embodiments, the processing module 108 may be configured to analyse the signal data to identify and / or categorise baseline data, that is, data which appears to be substantially free of any anomalous events or significant events. Baseline data may refer to one or more acoustic signals that are consistent with expected patterns and / or do not exhibit irregularities or deviations that could indicate a pathology event or other anomalous occurrence. In some embodiments, baseline data may refer to signal data that includes background signal data or vital sign signal data. In some embodiments, baseline data may include signal data that has substantially no pathology, or no identified pathology, such as normal breathing sounds or a normal heart rate. In some embodiments, baseline data or normal data may refer to signal data the provides pathology within in an expected range. For example, normal data may include heart rates within a predefined range, where the predefined range may be relevant to a subject. In some embodiments, baseline data may include signal data which provides a positive indication of normal function, including, but not limited to, normal lung sounds and / or heart sounds.

[0087] For a healthy patient, baseline data, which may be characterised by the absence of pathology, can be established through the monitoring of acoustic signals relevant to the subject, such as vital signs and breathing sounds. This baseline data provides a reference point that represents normal physiological functioning. By continuously collecting and analysing this data, a subject’s vital signs can be monitored, through obtaining and processing the acoustic signals, and determining whether the vital signs remain within the expected range. Signal data that includes baseline data may establish what constitutes normal functioning for an individual subject, and may assist in the early detection of any deviations from the baseline data. For example, if a patient’s breathing sounds deviate from the baseline data, it could indicate the onset of a respiratory condition, prompting further investigation. Therefore, processing and providing access to baseline data within the acoustic signal may be advantageous for facilitating timely diagnosis and / or intervention.

[0088] The processing module 108 may be further configured to partition the signal data into a plurality of intervals, thereby segmenting the signal data. Partitioning the signal data into intervals comprises dividing the continuous time series data of the signal data. The digital signal data includes a plurality of data points, with each data point timestamped to record the exact time of capture by the sensors 122. The partition may be based on a predetermined fixed time duration, such as five minute intervals, 20 minute intervals, one hour intervals and the like. In some embodiments, partitioning the signal data into a plurality of intervals may be based on a predetermined time duration and / or length of the acoustic signal. Partitioning the data into a plurality of intervals may include defining a length of each interval. For example, defining an interval length of 20 minutes means that each interval will represent a 20-minute period of continuous time-series data corresponding to the acoustic signal. The first interval will cover data received by sensors 122 from 00:00 to 00:20, the second from 00:20 to 00:40, and so on. The signal data is then segmented into the intervals accordingly.

[0089] The processing module 108 may be configured to group the data into bins or buckets based on the intervals. The processing module 108 may be configured to generate, for each interval in the plurality of intervals, a bin associated with the interval. In some embodiments, processing the data points of the signal data within the interval may include reducing the volume of the data, representing the data and / or simplifying the data while persevering characteristics for further analysis. Processing the data points of the signal data within the interval may include, but is not limited to, down sampling, filtering, random sampling, averaging, summarising, aggregating, quantizing, compressing, condensing, extracting subsets and / or decimating the data. Processing the data points of the signal data within the interval may include determining a statistical representation of the data points. In some embodiments, extracting a subset of data points for each interval may comprise calculating a statistical measure, the statistical measure comprising at least one of mean, median, mode, and standard deviation of the data points within the interval.

[0090] In some embodiments, the bin may be a decimation bin. Decimation bins may be generated by decimating the signal data within a particular interval, and grouping the decimated data into a bin. The decimation bins reduce the size of the data in the interval while preserving the characteristics of the data within the interval. Decimation bins may be generated by processing data points of the signal data within the interval to obtain a single value that represents the data points within the bin. The value of the decimation bin may be obtained by determining a statistical representation of the data points in the bin. Anyappropriate statistical representation of the data that can be performed over the whole acoustic signal may be used to process the data points of the signal data within the interval to generate the decimation bin. For example, if a median heart rate can be determined for the entire acoustic signal received over a period of time (e.g. a whole night), then a median heart rate can also be determined for a portion or time slice of the acoustic signal (e.g. five minutes) to generate a five minute decimation bin. That is, the same statistical calculations that can be applied over the whole time period over which the acoustic signal is recorded can be applied to each interval to generate the contents of each decimation bin corresponding to the interval.

[0091] Processing the data points of the signal data within the interval to obtain a statistical representation may include calculating a mean or average, median, range (including but not limited to, full, centile, decile and / or quartile), mode, count, sum, ratio, standard deviation, variance, quantiles, skew, kurtosis, percentiles, interquartile range (IQR), coefficient of variation (CV), correlation coefficient, autocorrelation, z-scores, and / or entropy. In some embodiments, processing the data points of the signal data within the interval comprises combining the chosen statistical representation with the appropriate unit or context. Units and / or context may include, but are not limited to count, rate, amplitude (e.g. dB), percentage (of bin time), duration, frequency, and the like.

[0092] In some embodiments, the decimation bin may comprise a representative value of the signal data and the signal data itself. In some embodiments, the decimation bin may comprise a representative value and a link to the signal data within the interval from which it was generated.

[0093] In some embodiments, the processing module 108 may be configured to derive a plurality of context signals from the at least one audio signal. The plurality of context signals may include at least one primary context signal, and at least one secondary context signal.The primary context signal may refer to any signal derived from, or related to, the at least one acoustic signal. The primary context signal may relate to the all or part of the acoustic signal associated with a particular context. In some embodiments, the primary context signal may refer to a signal associated with a specific pathology, and in some embodiments a pathology to be reviewed and / or monitored. For example, in some embodiments, the primary context signal may be associated with a particular context including, but is not limited to, one of a heart rate, apnea, lung crackle, breath rate, heart rate, snore rate, snore severity, obstruction severity, sleep cycle, and crackle severity and / or any other physiological or pathologicalcontext. In some embodiments, the primary context signal may relate to a particular vital sign or biosignal. In some embodiments, the primary context signal may include at least one of: breath sounds, heart sounds, snoring, obstructions, wheezing, coughing, and crackles. In some embodiments, the primary context signal may include any sound or vibration which is generated internally by the heart, breathing systems, and / or gastrointestinal systems.

[0094] The secondary context signal may refer to any signal derived from, or related to, the at least one acoustic signal. The secondary context signal may relate to all or part of the acoustic signal associated with a particular context. The secondary context signal may refer to any signal which is useful for understanding or providing context to the primary signal. For example, the secondary signal may include baseline data. The at least one secondary signal may be attached, associated or linked to the primary context signal. The at least one secondary signal may be the same signal regardless of different primary context signals. For example, the secondary signal may include breathing movement signal whether the primary context signal is a heart rate signal, snoring signal or apnea signal. In some embodiments, the secondary context signal may vary depending on the particular primary context signal being monitored. That is, in some embodiments, the secondary context signal may be determined on the basis of the primary context signal. In some embodiments, the processing module 108 is configured to generate a plurality of primary context signals and a plurality of secondary context signals from the at least one audio signal. In some embodiments, the secondary context signal may relate to breathing movement and / or cross-body movement to provide additional context to the primary context signal.

[0095] In some embodiments, at least one primary context signal may be a secondary context signal for another primary signal. Similarly, in some embodiments, at least one secondary context signal may also be a primary context signal which is associated with another secondary context signal.

[0096] In some embodiments, the at least one primary context signal generated by the processing module may be a composite signal. A composite signal may refer to a signal comprising one or more channels of the at least one acoustic signal. In some embodiments, generating a composite signal may include optimising the audibility of each source signal of the at least one acoustic signal, applying bandpass filtering to the at least one acoustic signal and / or mixing together one or more channels of the acoustic signal. For example, mixing together one or more channels of the acoustic signal that are related to different types ofsensors, such as mixing channels from room audio sensors with channels from vibroacoustic sensors. In some embodiments, the primary context signal may include, but is not limited to the following composite signals: Composite Steth Audio (CSA) which may comprise all channels related to vibroacoustic sensors, filtered to exclude heart sounds, Composite Heart Audio (CHA) which may comprise all channels related to vibroacoustic sensors, filtered to include only heart sounds, Composite Room Audio (CRA) which may include all room channels related to sensors within a room, Composite Core Audio (CCA) which may comprise CSA plus CRA (that is, all sound sources without heart), and / or Composite Full Audio (CFA) which may comprise CCA plus CHA (that is, all sound sources including enhanced heart).

[0097] In some embodiments, the secondary context signal generated by the processing module may be a composite signal. For example, where the secondary context signals include movement signals, such as breathing movement and / or cross body movement, the source movement channels may be combined together and enhanced to produce the composite signal. The secondary context signal may comprise a breath movement composite signal and / or a gross movement composite signal.

[0098] In some embodiments, the processing module may be configured to separate and / or derive channels from the at least one acoustic signal using one or more filters. The separation of channels may include identifying specific frequency ranges associated with each type of sound. For example, snoring may be characterised by frequencies within the range of about 30-300 Hz, breathing sounds may be characterised by frequencies within the range of about 300 and 2000 Hz. In another example, body movements produce sounds across the whole frequency range as they have a wideband effect (e.g. “white noise”). The strongest components of these movements may fall in the subsonic range, that is, below 5 Hz, which may be separated out from the full signal acquired at the sensor in hardware. Such an example may be used to create the at least one secondary context signal related to movement. In some embodiments, the processing module may be configured to apply bandpass filters tailored to a specific frequency range to isolate the components from the original acoustic signal. For example, a bandpass filter for snoring may be utilised to allow frequencies between 30 and 300 Hz to pass through while attenuating frequencies outside this range.

[0099] The filters may be applied to the acoustic signal to extract the relevant frequency signal. Each filtered signal may then be assigned to a separate channel. For example, thesignal filtered for snoring frequencies may be directed to a first channel, the signal filtered for breathing frequencies may be directed to a second channel, and the signal filtered for body movement frequencies may be directed to a third channel. In some embodiments, one or more channels may be mixed together to generate a composite signal.

[0100] The system further includes a user interface module 110 in communication with the processing module 108. The user interface module 110 is configured to generate a user interface, configure a user interface, and / or generate components to be rendered on or by a user interface. The user interface module 110 is configured to retrieve and / or access the decimation bins from the processing module 108 and, for each decimation bin, the user interface module 110 is configured to generate a responsive component on a user interface corresponding to the decimation bin. In some embodiments, the user interface module 110 is configured to generate responsive components in real time as bins are generated.

[0101] The system 100 further includes an audio access module 112 configured to access, for each responsive component, the signal data within an interval associated with the decimation bin that corresponds to the responsive component. That is, each responsive component is associated with a decimation bin, and the audio access module is configured to provide access to the signal data associated with the decimation bin. The audio access module 112 thereby enables access to the signal data of the acoustic signal by providing access to the relevant signal data interval for each responsive component.

[0102] Upon interaction with any one of the responsive components on the user interface, the audio access module 112 is configured to retrieve at least part of the signal data of the acoustic signal. In some embodiments, the audio access module 112 may be in communication with at least one extraction module (not shown) configured to process and / or extract a target data segment of the signal data in the interval. In some embodiments, the at least one extraction module may form part of the processing module 108 of system 100. In some embodiments, the extraction module may also be referred to as a “positioner”. The positioner may be configured to position a first time point and a second time point within the interval to extract and return at least part of the signal data of the acoustic signal from the interval between the first and second time points.

[0103] In some embodiments, responsive to interaction with any one of the responsive components on the user interface, the positioner may receive a request to retrieve at least partof the signal data within in an interval. The request may be sent by the audio access module 112. The request may contain an identifier of the particular interval associated with the bin that corresponds to the relevant responsive component. In some embodiments, the request may contain the signal data of the particular interval associated with the bin that corresponds to the relevant responsive component. Upon receiving the request, the positioner is configured to determine whether there are any events within the interval of signal data. Determining whether there are any events may include retrieving information about the events identified from the processing module, and / or a data store storing event information for the acoustic signal and determining their location. Responsive to identifying that there are one or more events within the interval, the positioner identifies at least one event within the interval. For example, the positioner may be configured to retrieve a list, set or annotation of one or more events associated with the interval from the processing module. Where it is determined that a plurality of events exist within the interval, the positioner may be configured to determine a primary event, which is an event that corresponds to the context of the signal data within the interval. For example, the primary event may be determined based on the primary context signal.

[0104] For example, if a primary context signal in the form of an apnea context signal is represented by the responsive components, and a responsive component is selected on the user interface in which there is also an obstructive apnea present, there may also be an obstructed breathing event (e.g. a snore) present within the signal data of the interval corresponding to that responsive component as well. That is, there may also be an event within the interval which corresponds to a different context signal. Selecting the responsive component corresponding to the interval in the apnea context signal will extract the target data segment that encompasses the primary event, the apnea, for that bin. This target data segment may incidentally encompass the obstructed breathing event at the end of the apnea event (which may be a “breakthrough snore”). If the same bin is selected for the obstruction severity context signal, the target data segment extracted for the obstruction severity context may encompass the same region of signal data covered by the target data segment for the apnea context, or the positioner may extract and return a target data segment for the obstruction severity context which focuses more tightly about the snore itself, that is the primary event of the obstruction severity context.

[0105] In some embodiments, the positioner is further configured to determine a first time point before the at least one event, and a second time point after the at least one event withinthe interval. In some embodiments, the first time point and the second time point align with the start point and the end point of the event determined by the processing module when the event is first identified. As such, the extraction module may determine the first time point and the second time point from information related to the event identified by processing module.

[0106] In some embodiments, the extraction module may be further configured to extract the signal data between the first time point and the second time point to define a target data segment and provide the target data segment to the audio access module 112 as the at least part of the signal data of the acoustic signal.

[0107] The at least part of the signal data may therefore comprise the target data segment. The size of the target data segment may vary. For example, the target data segment may be time-elastic or time-varying, in that each target data segment may comprise varying lengths of time depending on the signal data within the interval, and the context of the signal. For example, the target data segment may comprise a 2 minute apnea event, or a 30 second breakthrough snore event. The target data segment retrieved may be retrieved based on the one or more events identified by the processing module 108. For example, the signal data to be retrieved may be a part of the signal data that corresponds to one or more events, such as anomalous occurrences within the interval, significant occurrences or representative occurrences (such as average signals) within the interval. By retrieving a target data segment, a region of interest within the signal data may be accessed, retrieved and / or interrogated, without having to access the entirety of the interval. In some embodiments, the positioner may be configured to pre -calculate the target data segment for each bin for each primary context signal.

[0108] In some embodiments, where there is no identified event in the interval of signal data associated with a generated bin, then the interval of signal data may be determined to comprise baseline data. In such circumstances, the entire interval may be retrieved, or a sample of the interval representative of the baseline data may be retrieved.

[0109] In some embodiments, where there is no identified event in the interval of signal data associated with a generated bin, then the positioner may be configured to return a default data segment. The default may include returning a predefined initial time period, a predefined middle time period or a predefined end time period. For example, the first 30 seconds of the signal data, the middle 30 seconds of the signal data, and so on. In some embodiments, wherethere is no identified event in the interval of signal data associated with a generated bin, then the positioner may be configured to retrieve a target data segment which corresponds to the representative value of the generated bin. For example, where the representative value of a bin indicates an average heart rate, the target data segment that is extracted may be a section of signal data which represents the average heart rate of the bin. In this way, baseline data or normal data may also be retrieved and accessed through the system in addition to signal data that indicates events or anomalies.

[0110] In some embodiments, the system may include one or more positioners corresponding to one or more primary context signals. In some embodiments, where the system includes a plurality of positioners, one or more positioners may be configured differently to return a target data segment based on the primary context signal to which it corresponds. In some embodiments, the target data segment retrieved may be based on pathology context. In some embodiments, multiple primary context signals may default to the same target data segment where there is no identified event in the bin for one or more of the multiple primary context signals.

[0111] In one example, for obstruction contexts (e.g. snoring) there may be two primary context signals, the first relating to obstruction rate and the second relating to obstruction severity. The obstruction rate context signal may utilise an obstruction rate positioner configured to identify the period where the obstruction rate most accurately reflects the average rate for the bin (that is, such that the target data segment includes a “run” of snoring). The obstruction severity context signal may utilise an obstruction severity positioner that identifies the period where the obstruction amplitude or loudness is closest to the overall loudness of the bin (that is, such that the target data segment includes the loudest event, even if it is 1 or 2 snores). In some embodiments, both the obstruction rate positioner and the obstruction severity positions are configured to default to the same snippet if there is no obstruction present in the bin.

[0112] In some embodiments, upon interaction with any one of the responsive components on the user interface, the audio access module 112 may be configured to retrieve all signal data of the acoustic signal within the interval corresponding to the responsive component.

[0113] Referring back to system 100 of Figure 1 , after retrieving the at least part of the signal data corresponding to the relevant interval, the retrieved signal data is converted to anaudio signal. In some embodiments, where the audio access module 112 retrieves a target data segment, the audio access module 112 is configured to convert the retrieved target data segment into an audio signal. The audio access module 112 is then configured to output the audio signal.

[0114] The audio access module 112 may be further configured to transmit the audio signal to a playback component integrated within a user interface. The playback component may be part of a software application, a web-based platform, and / or an embedded system interface. The playback component may include visual elements such as buttons, sliders, and displays that allow users to interact with and control the playback of the audio signal. The playback component on the user interface may be configured to receive the audio signal from the audio access module 112 and render it for playback. In some embodiments, this may involve converting the audio signal into a format suitable for playback through the interface’s integrated playback component. The audio player may be configured to support various audio formats. In some embodiments, the playback component may be in communication with a control module, and / or may be integrated with a control module, to provide control features such as play, pause, stop, volume control. The audio signal may be transmitted in a format compatible with the playback component, minimising latency.

[0115] In some embodiments, the playback component may be configured to provide a visual representation of the audio signal. For example, a visual representation may include displaying a graphical representation of the audio signal, providing a visual context to the audio playback. In some embodiments, the playback component may provide a waveform representation that displays the amplitude of the audio signal over time, allowing users to see the structure and intensity of the sound. In some embodiments, the playback component may provide a spectrum analyser to break down the audio signal into its frequency components, showing the distribution of different frequencies and their respective amplitudes. The playback component on the user interface may be configured to interface with other software or hardware components. For example, it may be linked to a database for storing and retrieving audio files, or it may interface with external speakers or headphones for enhanced audio output.

[0116] In some embodiments, the audio access module 112 may be configured to output the audio signal to a speaker. In some embodiments, the audio access module 112 may be configured to output the audio signal to database. In some embodiments, the audio signal maybe output via a wired or wireless connections. For example, the audio access module may output the audio signal via Bluetooth to a connected speaker, or to speakers that form part of a computing device in communication with system 100 having audio access module 112.

[0117] In some embodiments, the audio access module 112 may be configured to output the audio signal to a haptic device. In some embodiments, the audio access module 112 is configured to output the audio signal to a haptic device. The haptic device may include actuators that convert the audio signal into vibrations, providing tactile feedback to the user. The haptic device may be configured to receive the output audio signal and convert the audio signal into mechanical vibrations. The audio access module 112 may modulate the audio signal to match the frequency and amplitude requirements of the haptic device, ensuring accurate and responsive haptic feedback. The haptic device may enable a clinician to palpate the audio signal, providing an additional mechanism by which to review the audio signal. This can be beneficial in identifying low frequency sounds which cannot necessarily be heard but could be felt. For example, some heart murmurs, and some lung or chest pleural rubs could be identified using a haptic device to feel the low frequency vibrations.

[0118] In some embodiments, the audio access module 112 may be configured to output the audio signal to multiple devices simultaneously. For example, the audio signal may be transmitted to both a speaker and a haptic device, allowing for synchronized audio and tactile experiences. In some embodiments, the audio access module 1112 may be configured to synchronize the playback of the audio signal with one or more external devices. In some embodiments, the audio signal may be transmitted to a playback component, configured to visually display the audio signal, and a speaker configured to auditorily play the audio signal and provide haptic feedback. The audio access module 112 may include a signal processing unit that manages the distribution of the audio signal, ensuring that each connected device and / or component receives the appropriate signal format and strength of the output audio signal.

[0119] The visual, auditory, and / or haptic representations of the output audio signal in pathology contexts may significantly enhance analysis and diagnosis. Visual representations such as waveforms and spectrograms may provide insights into complex auditory data and aid in the detection of significant events such as apneas or arrhythmias. Auditory analysis may enhance diagnostic precision by allowing clinicians to directly hear identified events, such as identifying irregularities in breathing or heart sounds. By providing an option for hapticfeedback through vibrations or tactile sensations, the system provides another aspect in which to analyse the audio signal to identify events in the signal data which may not be visually or auditorily apparent, for example, helping identify conditions like heart murmurs.

[0120] Efficiently accessing both normal baseline data and anomalous data through the systems and methods described herein provides advantages in medical diagnostics. By comparing both normal data along with anomalous data overtime, clinicians can swiftly identify deviations that may indicate underlying health issues. For example, it may be quickly identified when a subject develops a wheeze by comparing their normal historically recorded breathing sounds to recorded anomalous breathing sounds. Additionally, efficient access to normal and / or baseline data enables clinicians to determine when a patient is healthy and / or shows no indications of underlying health issues, thereby avoiding unnecessary interventions or further invasive tests.

[0121] The systems and methods described herein may improve diagnostic accuracy and efficiency by enabling clinicians to compare patient signals against established norms and quickly identify deviations. Specific segments of the acoustic signal may be isolated and extracted to provide visual, auditory and haptic representations to efficiently identify events in the signal data and streamline the diagnostic process, reducing human error in manually reviewing long and continuous acoustic signals. The multi-sensory output of audio signals may improve the accuracy of diagnoses and augment the clinician’s ability to make informed decisions quickly and efficiently.

[0122] The feedback methods and extracted data may be integrated into electronic health records and may also be used to support longitudinal studies and ongoing monitoring. Clinicians can track changes in audio signals over time, identifying trends that indicate the progression or improvement of conditions.

[0123] Figure IB is a process flow diagram of a method 150 for processing acoustic signals, according to some embodiments. In some embodiments, one or more process blocks of Figure IB may be performed by a system, such as system 100 as described herein. Method 150 may include determining signal data from at least one acoustic signal obtained from at least one sensor (at 155). For example, processor 102 of system 100 may execute signal acquisition module 106 to determine signal data from at least one acoustic signal obtained from at least one sensor, as described above. At 160, the method may include an optional process ofidentifying one or more events in the signal data of the acoustic signal. For example, processor 102 of system 100 may execute processing module 108 to identify one or more events in the signal data of the acoustic signal. However, it will be appreciated that in some cases, the method may not perform 160, for example, where the signal data of the acoustic signal includes only baseline data, and no events or pathology are identified. In such embodiments, the method 150 proceeds directly from 155 to 165. In some embodiments, method 150 may include an additional process (not shown) of determining whether there are any identified events in the signal data of the acoustic signal and, if not, proceeding to 165. The signal data may then be partitioned, at 165, into a plurality of intervals. For example, processor 102 of system 100 may execute processing module 108 to partition the signal data into a plurality of intervals, as described above.

[0124] The method 150 then generates, for each interval in the plurality of intervals, a bin associated with signal data in the interval by processing the data points within the interval at 170. For example, processor 102 of system 100 may execute processing module 108 to generate, for each interval in the plurality of intervals, a bin associated with signal data in the interval by processing the data points within the interval, as described above. For each bin, a responsive component on the user interface may be generated which corresponds to the bin, where each responsive component has access to the signal data of the acoustic signal associated with the bin corresponding to the responsive component at 175. For example, processor 102 of system 100 may execute user interface module to generate, for each bin, a responsive component on the user interface corresponding to the bin, as described above. Upon interaction with the responsive component, at 180, at least part of the signal data of the acoustic signal is retrieved. For example, processor 102 of system 100 may execute audio access module 112 to retrieve at least part of the signal data of the acoustic signal, as described above. The retrieved signal data is then converted to an audio signal at 185 and the audio signal is output at 190. For example, processor 102 of system 100 executing audio access module may convert the retrieved signal data to an audio signal and output the audio signal, as described above.

[0125] Although Figure IB shows example blocks of method 150, in some implementations, method 150 may include additional blocks, fewer blocks, different blocks, or differently arranged blocks than those depicted in Figure IB. Additionally, or alternatively, two or more of the blocks of method 150 may be performed in parallel.

[0126] In some embodiments, there is provided a user interface for accessing the signal data of at least one acoustic signal. Figure 2 is a schematic diagram of a user interface 200, according to some embodiments. The user interface 200 may be generated by the user interface module 110 of system 100. The user interface 200 may comprise a plurality of responsive components 202 arranged on the user interface 200. The responsive components 202 correspond to bins generated from the interval of signal data. That is, each responsive component 202 corresponds to a bin associated with an interval of signal data from an acoustic signal received from at least one sensor. In some embodiments, each responsive component 202 may be bound to the signal data in the interval associated with the bin to provide access to the signal data of the acoustic signal. In some embodiments, each responsive component may be bound to a representative value of the bin and may enable access to the signal data of the acoustic signal through a link between the bin and the signal data.

[0127] In some embodiments, each responsive component 202 may include an attached handler to determine an interaction with the responsive component. The handler may be configured to determine when an interaction with a responsive component has occurred. Upon determining such an interaction, the handler may be configured to retrieves at least a portion of the signal data of the acoustic signal bound to the responsive component.

[0128] The responsive components 202 may be of a fixed width and may comprise a graphical element representative of the generated bin. The plurality of responsive components 202 on user interface 200 are arranged sequentially in an array, such that each component provides part of a graphical representation of the at least one acoustic signal. The container components 206 may arrange responsive components on the interface in a sequential array that corresponds to the position of the corresponding intervals within the signal data. The array of responsive components are disposed within a container component 204, configured to hold each of the responsive components 202 associated with the at least one acoustic signal. In some embodiments, the container component 204 is configured to hold each of the responsive components 202 which correspond to a particular context signal. In some embodiments, the user interface 200 may include at least one container component. In some embodiments, the user interface 200 may include a plurality of container components. The user interface 200 includes a plurality of container components 206, each representing a different context. The container components 206 may be referred to as “explorers” or “signal explorers”. In a pathology context, the container components 206 may be referred to as“pathology explorers”. The container components 206 are configured such that they each represent a particular primary context signal, that is a signal within a particular context, and the container components 206 are made up of responsive components 202, such as is shown in container component 204 generated from signal data corresponding to the particular primary context signal. Whilst only container component 204 is shown having responsive components 202 in Figure 2, it will be appreciated that the container components 208, 210, and 212 shown on user interface 200 may also contain responsive components 202. Additionally, it will be appreciated that more or less container components may be rendered on user interface 200. Each container component 204, 208, 210, 212 may provide a visual element, such as a graph, where the responsive components 202 enable interrogation of data in the graph at particular time intervals. The container components 206 may be configured to adaptively fit the width of the screen on which the user interface is displayed. The plurality of container components 206 may be arranged such that each responsive component 202 corresponding to a particular time interval in each container component 204, 208, 210, 212 are vertically aligned. In some embodiments, the container component contains the responsive components to provide an integrated graphical representation of the at least one acoustic signal.

[0129] The pathology explorers may be interchanged within the user interface to display different pathology contexts. Since each pathology explorer is a container component that holds responsive components representing data for a specific pathology, the container component displayed on the user interface may be swapped out with different container components related to another context. In some embodiments, the responsive components within a container component may be swapped out with an array of responsive components related to another context signal. For example, the container components 204, 208, 210, and 212 on user interface 200 may display an apnea context, obstruction context, breathing context, and snoring context respectively. The container component 212 relating to a snoring context signal may be swapped out with a container component which relates to a heart rate context signal to provide access to signal data associated with the heart rate context signal. In some embodiments, the explorers may be dynamically swapped out, allowing users to switch between different pathology views, for example, relevant to a type of monitoring and / or analysis. In some embodiments, container components may be individually interchanged based on a selection. In some embodiments, container components may be automatically interchanged in accordance with a particular template view, for example, relating to a particular clinical context. This interchangeability of the container components allows theuser interface to remain flexible and adaptable, whilst still providing access to the relevant signal data to provide a comprehensive view of different pathological conditions.

[0130] The user interface 200 further includes a playback component 214. The playback component 214 is integrated within the user interface and is configured to receive an audio signal converted from at least part of the signal data of the acoustic signal. In some embodiments, the playback component 214 may comprise a signal display component 216 and a one or more control components 218. The signal display component 216 may be configured to provide a visual representation of an audio signal on the user interface based on retrieved signal data. The control components 218 are configured to receive input and control the playback of the audio signal based on the received input.

[0131] Figure 3 is a schematic diagram of the user interface 200, on which interaction with a responsive component 203 has occurred, according to some embodiments. The user interface 200 includes a plurality of responsive components 202, and an individual responsive component 203 has been selected, for example, by user input to a touch screen on a computing device. Selecting responsive component 203 sends a request to retrieve the relevant signal data associated with the decimation bin that corresponds to responsive component 203. As shown on user interface 200, the retrieved signal data has been converted to an audio signal and displayed on the playback component 214 of the user interface. The playback component 214 provides a visual representation of the audio signal within the signal display component 216. The signal display component 216 displays a plurality of signals 220 corresponding to the signal data. For example, the signal display component 216 displays a primary context signal 222, and two secondary signals 224 for the signal data associated with the decimation bin to which the responsive component 203 corresponds. Interaction with the responsive component 203 provides fast and efficient access to the associated signal data.

[0132] Figure 4 is a schematic diagram of user interface 200 showing control of the audio signal playback through the user interface, according to some embodiments. The user interface 200, includes a plurality of responsive components 202. The user interface 200 shows a configuration in which an individual responsive component 203 has been interacted with, for example, by user input to a touch screen on a computing device. As a result, the audio signal has been provided to the playback component 214. The audio signal is visually represented on the signal display component 216 of the playback component 214. The signal display component 216 includes a progress bar 226, visually indicating the position of theaudio signal playback. The control components 218 include a plurality of control buttons 228, configured to receive input and adjust and / or control the playback of the audio signal. In this configuration, the audio playback and the visual representation of the audio signal are synchronised. The audio signal may be provided visually and / or audibly. In some embodiments, the user interface may be synchronised with a haptic feedback device, such that the audio signal is provided to the haptic feedback device such that haptic feedback is synchronised with the visual representation and / or audio playback of the audio signal.

[0133] Some embodiments of the systems, methods and user interfaces described herein may provide technical advantages in user interface technology. Dividing the user interface into reusable, self-contained components representing intervals of the signal data enhances modularity and maintainability of the user interface. Each component can be individually created, allowing flexibility to process all or part of an acoustic signal. Additionally, individual responsive components allow the intervals of acoustic signal data to be interrogated independently. The modular approach facilitates easier updates to the user interface, as well as scalability, as individual responsive components, or entire container components, can be modified, interchanged or replaced without affecting the entire system or the speed of the user interface. Additionally, using the responsive components to graphically represent the bins of signal data corresponding to intervals enables the interface to effectively load based on a condensed, or decimated, subset of the data, providing faster initial load times.

[0134] Some embodiments of the systems and user interfaces may provide advantages in asynchronous data fetching. By asynchronously retrieving data only when needed, but providing access to all data, this enables the user interface to remain responsive while waiting for data to be retrieved, with the user interface operating smoothly and more efficiently through interactions. By asynchronously fetching the signal data relating to in interval, this mitigates issues of freezing or become unresponsive during data retrieval. Additionally, when the asynchronous fetch segments the interval data or retrieves only part of the interval signal data, this provides further advantages in mitigating unresponsiveness of the user interface. Additionally, asynchronous data fetching may improve rendering and loading speeds of the user interface. By fetching data in the background, the user interface can quickly display initial content, such as the graphical representation of bins, and progressively load additional data, such as signal data relating to the acoustic signal on an as-needed basis, reducing delays and latency.

[0135] When a responsive component is selected, initially returning only the most relevant part of the signal data can optimise performance of the user interface. This strategy limits the amount of data the user interface needs to load at any given time, reducing load times and bandwidth usage. All data remains accessible, if required, but by prioritising the most relevant information to be returned first, the systems and user interfaces are more efficient. The use of responsive components being bound to individual intervals of signal data improves the speed and responsiveness of the user interface and enables users to quickly find and interact with the relevant data by retrieving part of the signal data in the interval that is considered most relevant.

[0136] Figures 5 A to 5 D illustrate various examples of how decimation bins may be represented by graphical elements on a responsive component. These examples demonstrate the combination of different data visualisation methods to create information-rich responsive components. The examples provided are not exhaustive, and it will be appreciated that other visual representations may be provided to represent the decimation bin corresponding to a responsive component.

[0137] Figure 5A illustrates a responsive component having a graphical representation in the form of a bar / line plot graphical representation 500, which is a combination of a bar plot 502 and an overlaying line plot 504. The bar / line plot graphical representation 500 may be used to represent movement percentage and movement rate. A plurality of the bar / line plot graphical representations 506 shows how the graphical representation may be stitched together over a plurality of responsive components. Figure 5B illustrates a responsive component having a graphical representation in the form of a superimposed bar plot 508, which is a combination of a first bar plot 510 and a second bar plot 512. The superimposed bar plot 508 may be used to depict all apneas alongside obstructive apneas. A plurality of the superimposed bar plot representations 514 shows how the graphical representation may be stitched together over a plurality of responsive components.

[0138] Figure 5C illustrates a responsive component having a graphical representation in the form of a line plot with a range 516, which is a line plot 518 and a range plot 520. The line plot with a range 516 may be used to show heart rate and heart rate decile range. A plurality of the line plots with range graphical representations 522 shows how the graphical representation may be stitched together over a plurality of responsive components. Figure 5D illustrates a responsive component having a graphical representation in the form of acombined line plot 524, which includes a first line plot 526 and a second line plot 528. The combined line plot graphical representation 524 may be used to track both snore rate and breath rate. A plurality of the combined line plot graphical representations 530 shows how the graphical representation may be stitched together over a plurality of responsive components.

[0139] In some embodiments, the audio access module 112 of system 100 may be in communication with a system 600 for processing target segment data, as described herein, and configured to retrieve an audio segment from the system 600.

[0140] Figure 6 is a block diagram of a system 600 for processing a target audio segment, according to some embodiments. The system 600 comprises one or more processor(s) 602 and / or memory 604. The processor(s) 602 may have substantially similar features and / or functionality to processor(s) 102. The memory 604 may have substantially similar features and / or functionality to memory 104.

[0141] The memory 604 is configured to store program code accessible by the processor(s) 602. The program code comprises executable program code modules. In other words, memory is configured to store executable code modules which, when executed by the processors, cause the system to perform certain functionalities. For example, memory 604 may comprise at least one extraction module 606 and an audio segment output module 608. In some embodiments, memory 604 stores instructions (such as program code) which when executed by the processor(s) 602 causes the system 600 to perform methods for processing a target audio segment as described herein.

[0142] In some embodiments, the system 600 may be implemented as a distributed system comprising multiple server systems configured to communicate over a network to provide the functionality of the system 600. For example, one or more of the program code(s) (for example modules or engines) may be deployed on one or more disparate or remote servers, which may cooperate to provide the functionality of the system 600 described. In some embodiments, the system 600 may be in communication with a network 614, and include a network interface 616 to facilitate communication with additional components, including a user interface 610, and one or more data stores 612. The network interface 616 may have substantially similar features and / or functionality to network interface 116. Network 614 may have substantially similar features and / or functionality to network 114. The one or more datastores 612 may have substantially similar features and / or functionality to the one or more data stores 120.

[0143] The user interface 610 may form part of or be local to the system 600, or may be remote from and accessible to the system 600, for example, through the network 614. The user interface 610 may be graphical or text based. In some embodiments, the user interface 610 may comprise a display configured to present data and receive input from a user, including but not limited to touch input, keyboard input, peripheral input (such as through a mouse) or voice commands. The user interface 610 may further include components to facilitate user interaction with the system 600. In some embodiments, the components render visual elements, such as icons, menus, and windows. In some embodiments, the user interface may comprise an annotation component configured to annotate a visual representation of the audio signal. In some embodiments, the user interface 610 may be substantially similar to example user interfaces 200 or 800 as described herein.

[0144] The user interface 610 may be accessed by one or more computing devices (not shown), including but not limited to desktop computers, laptops, tablets, and smartphones. These devices may connect to the system 600 through wired or wireless networks.

[0145] In some embodiments, the user interface 610 may be accessed via a web browser, allowing for platform -independent interaction with the system 600. In some embodiments, the user interface 610 may be accessed through an application installed on the one or more computing devices. In some embodiments, the user interface 610 may be designed to adapt to different screen sizes and resolutions.

[0146] The system may include an extraction module 606 in communication with the user interface 610. The at least one extraction module 606 may be configured to, responsive to determining an interaction has occurred with a responsive component of the plurality of responsive components on the interface, determine an interval of signal data associated with the responsive component. The at least one extraction module 606 may be configured to determine a first time point within the interval, and a second time point within the interval, and extract the signal data between the first time point and the second time point to define a target data segment. In some embodiments, the first time point may be determined based on a predefined buffer prior to a start of at least one identified event. The predefined buffer may include baseline signal data or no anomalous activity. In some embodiments, the second timepoint may be determined based on a predefined buffer after an end of at least one identified event. The predefined buffer may include baseline signal data or no anomalous activity. In some embodiments, the extraction module may be configured to store the target data segment in a data store. In some embodiments, the extraction module may be further configured to apply a compression algorithm to the target data segment to reduce a size of the target data segment before converting the target data segment into an audio segment. In some embodiments, the extraction module may be configured to generate metadata for the target data segment, wherein the metadata includes at least the first time point and the second time point.

[0147] In some embodiments, a plurality of extraction modules may form part of the system 600, each of the plurality of extraction modules in communication with one or more responsive components or container components of the user interface 610. The plurality of extraction modules may be configured to cover different methods of extracting the target data segment, for example, based on different methods of processing the signal data, depending upon the context of the signal being analysed or extracted, or based on the container in which the responsive components are held.

[0148] In some embodiments, the system 600 may further comprise an audio segment output module 608. The audio segment output module 608 may be configured to receive the target data segment from the at least one extraction module 606. The audio segment output module 608 may be configured to convert the target data segment into an audio segment, and output the audio segment. In some embodiments, the audio segment output module may be configured to convert the target data segment into an audio segment by converting the target data segment to an analog signal to create an audio signal. In some embodiments, the audio segment output module may be configured to synchronize the playback of the audio segment with one or more external devices. In some embodiments, the audio segment output module may be configured to generate a visual alert on the user interface when playback of the audio segment reaches the at least one identified event.

[0149] In some embodiments, features of system 100 may be configured to perform the functions of system 600. For example, the processing module 108 may include at least one extraction module configured to perform the functions of extraction module 606. Audio access module 112 may be configured to retrieve a target data segment from the processing module 108 and perform the functions of audio segment output module 608. In someembodiments, system 100 may be in communication with system 600. For example, the processing module 108 may be in communication with the at least one extraction module 606, and the audio access module 112 may be in communication with the audio segment output module 608.

[0150] Figure 7 is a process flow diagram of a method 700 for processing a target audio segment, according to some embodiments. In some embodiments, one or more process blocks of Figure 7 may be performed by a system, such as system 100 or system 600 as described herein. Method 700 may include providing a user interface with a plurality of responsive components at 710. At 720, upon interaction with a responsive component of the plurality of responsive components, an interval of signal data bound with the responsive component is determined. For example, processor 602 of system 600 may execute extraction module 606 to determine an interval of signal data bound with the responsive component. At 730, at least one event within the relevant interval is identified. For example, processor 602 of system 600 may execute extraction module 606 to identify at least one event within the relevant interval. However, it will be appreciated that in some cases, the method may not perform 730, for example, where the interval of signal data includes only baseline data, and no events or pathology are identified. In such embodiments, the method 700 proceeds directly from 720 to 740. In some embodiments, method 700 may include an additional process (not shown) of determining whether there are any identified events in the interval of signal data and, if not, proceeding to 740. At 740, a first time point before the at least one event is determined, and a second time point after the at least one event is determined. For example, processor 602 of system 600 may execute extraction module 606 to determine a first time point before the at least one event, and a second time point after the at least one event, as described above.

[0151] Method 700 then includes, at 750, extracting the signal data between the first time point and the second time point to define a target data segment. For example, processor 602 of system 600 may execute extraction module 606 extract the signal data between the first time point and the second time point to define a target data segment. At 760, the target audio segment is provided to an audio output module. For example, processor 602 of system 600 may execute extraction module 606 to provide the target audio segment to an audio output module, as described above. At 770, the target data segment is converted into an audio segment and, at 780, the audio segment is output. For example, processor 602 of system 600 may execute audio segment output module 608 to convert the target data segment into an audio segment and output the audio segment.

[0152] Although Figure 700 shows example blocks of method 700, in some implementations, method 700 may include additional blocks, fewer blocks, different blocks, or differently arranged blocks than those depicted in Figure 7. Additionally, or alternatively, two or more of the blocks of method 700 may be performed in parallel.

[0153] In some embodiments, the systems 100, 600 and user interfaces 200, 610 described herein may be used in a clinical setting. In such a clinical setting, the user interface may be utilised to interrogate data from acoustic signals obtained during sleep monitoring.

[0154] Figure 8 is an example user interface 800 from a clinical setting, according to some embodiments. The user interface 800 comprises a plurality of responsive components, such as responsive component 802. Each responsive component corresponds to a decimation bin associated with a five minute interval of signal data from an acoustic signal received from one or more vibration sensors during sleep monitoring of a subject. The responsive components on the user interface 800 represent fixed intervals of the continuous time series data in the form of an acoustic signal obtained during the sleep monitoring. Each responsive component on the interface is bound to the signal data within the interval, thereby collectively providing access to the entire acoustic signal.

[0155] The responsive components comprise a graphical element, which visually represents the value of the decimation bin. When the graphical elements corresponding to a primary context signal are combined together, the responsive components form a visual display of the entire primary context signal recorded during sleep monitoring. The responsive components relating to a primary context signal are arranged and held within a pathology explorers (or container components). The user interface 800 as shows four pathology explorers 804, 806, 808, and 810, each comprising a plurality of responsive components having graphical elements that create a fully integrated graphical representation of the primary context signal over a period of time. The pathology explorer enables exploration and / or interrogation of the acoustic signal and / or different channels of the acoustic signal, and provides access to the signal data from each interval. On user interface 800, pathology explorer 804 shows a movement rate context signal, pathology explorer 806 shows an obstruction rate context signal, pathology explorer 808 shows an obstruction severity context signal, and pathology explorer 810 shows a heart rate context signal.

[0156] Figure 9 is an image of the example user interface 800 showing the process of interchanging a pathology explorer, according to some embodiments. In this example, a clinician may want to swap the pathology explorer 804 relating to the movement rate context signal, to a pathology explorer related to a different context signal for diagnostic and / or analysis purposes. A menu component 812 accessible through the pathology explorer 804 provides a list of other available primary context signals which may be displayed on a pathology explorer. The selection of menu item “Apnea” 814 triggers a handler to retrieve the primary context signal relating to Apnea, and render the interface to include an Apnea pathology explorer (not shown in Figure 9).

[0157] Figure 10 is an image of the example user interface 800 showing the Apnea pathology explorer 816, according to some embodiments. As can be seen, the pathology explorer 804 has been interchanged with new pathology explorer 816. The pathology explorer 816 includes a plurality of responsive components having graphical elements that provide an integrated graphical representation of the Apnea context signal over a period of time. The pathology explorer 816, when interchanged, is configured such that each of the responsive components corresponding to five minute intervals of signal data are vertically aligned with the corresponding responsive components in the pathology explorers 806, 808 and 810.

[0158] Figure 11 is an image of the example user interface 800 showing interaction with a responsive component in the Apnea pathology explorer 816, according to some embodiments. A responsive component 818 is selected within the Apnea pathology explorer 816. The responsive component is highlighted to show its selection. A handler attached to the responsive component, detects the selection and / or interaction with the responsive component 818, which may be a click, tap or other input received on the user interface. On detecting an interaction, the handler retrieves a part of the signal data relating to the Apnea context signal during that five minute interval of signal data. The part of the signal data is converted to an audio signal which is then provided to playback component 820. Playback component 820 provides a visual representation of the apnea context signal 822 and may be configured to simultaneously provide audio playback. In addition, playback component 820 also provides a visual representation of two secondary context signals, a breathing movement or breathing cycle signal 824, and a movement signal 826, provided below the apnea primary context signal 822.

[0159] The playback component 820 allows clinicians to listen to the acoustic signals, providing an auditory dimension to analysis to assist with diagnosing conditions for example, sleep apnea. The combination of visual and auditory data analysis provided through the user interface may enhance the clinician’s ability to effectively diagnose and monitor sleep-related conditions.

[0160] Figure 12 is an image of the example user interface 800 showing interaction with a responsive component in the obstruction rate pathology explorer 806, according to some embodiments. A responsive component 828 is selected within the Obstruction Rate pathology explorer 806. The responsive component 828 is highlighted to show its selection. Responsive component 828 is located directly beneath responsive component 818, and therefore covers the same five minute interval, but on a different primary context signal. A handler attached to the responsive component 828, detects the selection and / or interaction with the responsive component 828, which may be a click, tap or other input received on the user interface. On detecting an interaction, the handler retrieves a part of the signal data relating to the Obstruction Rate context signal during that five minute interval of signal data. The retrieved part of the signal data is converted to an audio signal which is then provided to playback component 820. Playback component 820 provides a visual representation of the obstruction rate context signal 830 and may be configured to simultaneously provide audio playback. In addition, playback component 820 also provides a visual representation of the same two secondary context signals 824 and 826 as shown for the apnea context signal in Figure 10. However, secondary context signals 824 and 826 appear differently in Figure 11 as the retrieved part of the signal data from the five minute interval of the obstruction rate context signal 830 is shorter than the length of the apnea context signal 822 during the same five minutes. The obstruction rate context signal 830 is shown as being approximately 20 seconds long, to cover the obstruction event. In contrast, the apnea event shown from the apnea context signal 822 corresponding to the same five minute interval is approximately 60 seconds long. This contrast shows how the retrieved part of the signal data can vary depending on the primary context signal and the events occurring during the five minute interval.

[0161] Figure 13 is an image of the example user interface 800 showing interaction with a responsive component in the heart rate pathology explorer 810, according to some embodiments. A responsive component 832 is selected within the heart rate pathology explorer 810. The responsive component 832 is highlighted to show its selection. A handleratached to the responsive component 832 detects the selection and / or interaction with the responsive component 832, which may be a click, tap or other input received on the user interface. On detecting an interaction, the handler retrieves a part of the signal data relating to the heart rate context signal during that five minute interval of signal data. The retrieved part of the signal data is converted to an audio signal which is then provided to playback component 820. Playback component 820 provides a visual representation of the heart rate context signal 834 and may be configured to simultaneously provide audio playback. In addition, playback component 820 also provides a visual representation of the two secondary context signals 824 and 826 corresponding to the selected five minute interval.

[0162] The configuration of the user interface 800, and the ability to access all intervals of signal data across the entire acoustic signal enables clinicians to visually analyse the signal data over time, quickly identify paterns and anomalies, and quickly retrieve signal data relevant to a particular pathology context.

[0163] The systems, methods and user interfaces described herein enables a clinician to perform an asynchronous physical examination, either locally or remotely, by allowing the clinician to look, listen, and feel (or palpate) the key elements of a medical physical examination. As the system captures auscultation sounds using vibroacoustic sensors in a mat on which the subject lies, processes these signals, and presents them to the clinician's senses through the user interface. The clinician can review the entire recording or specific durations, observe the context of other events and surroundings, listen to the sounds generated by the heart and breathing, and palpate for any accompanying vibrations. This advantageously allows the clinician to conduct a virtual physical examination and assess the core vital signs of the subject at any moment during the recording of the signal using the systems and methods disclosed herein.

[0164] Depending on the context and use case, some embodiments of the present disclosure may be viewed from different perspectives: the first as systems, methods and devices that provide continuous auscultatory data for review by a clinician as the primary source of value, where the summary data and overall metrics (e.g., average heart rate, wheezes per hour, etc.) are of secondary importance; and the second as systems, methods, and devices that provide observation metrics and data for patient condition as a primary source of value, where the ability to review key examples of interest is of secondary importance.

[0165] The value of embodiments of the disclosure are both correct, and neither is to the exclusion of the other. This demonstrates that the coexistence of both underlying signal data and top-level metrics is beneficial in supporting an effective decision-making process in clinical environments, regardless of which particular perspective takes priority.

[0166] In some embodiments, the systems and methods described herein employ auscultatory methods, which involve the auditory monitoring of internal bodily sounds. Unlike conventional systems that predominantly utilise oscillometric or electrocardiogram (ECG) techniques, the systems and methods described herein capture and represent data from continuous signals and discontinuous events in a consistent manner, thereby enhancing the user experience and providing more efficient access to the signal data. By leveraging auscultatory signals, the systems and methods described herein provide a unique and comprehensive perspective on patient metrics and summary data, offering a holistic view of the patient's condition from the signal data that diverges from traditional monitoring systems.

[0167] Understanding the thought process behind how a clinician arrives at their diagnostic conclusions is advantageous in building the novel approaches to interpreting and representing data, as described herein. This reveals the practical reality that while clinicians depend upon the source signals to confirm their diagnosis, they are fundamentally interested in how these signals deviate from the expected. Put another way, the signals are a tool for determining when homeostasis is disrupted. These moments of homeostatic disruption are clinically significant events. Upon arriving at this conclusion, the hierarchy of importance becomes clear: events should be at the forefront, with the underlying source signals available to both support the detected event and to provide a broader context for that which may not be immediately apparent from the single event alone. For example, seeing a snore event, but then looking and listening to it in context might reveal the cessation of breathing prior to the event and therefore reveal an obstructive apnea.

[0168] In some embodiments disclosed herein, a systems and methods have been developed to universally represent data from continuous signals and discontinuous events. Discontinuous events, such as snoring and apnea, and continuous signals, which include source signals like stethoscope data and movement, as well as derived signals for vital signs such as heart rate and breathing rate, are captured and represented in a way that is consistent.

[0169] For derived signals such as heart rate and breath rate, these are often represented as a continuous line on a chart. However, they are still ultimately the product of a series of discrete events, such as a series of heartbeats or a series of inspiratory and expiratory cycles. From here, the ability to partition the study into decimated bins is made significantly easier, as is the ability to process the relevant statistics.

[0170] For source signals such as movement, stethoscope data, and even imported source signals such as SpO2 and blood pressure, these signals are typically used to identify actual events that are themselves discontinuous. The extraction of movement and stethoscope events is well covered, but to extend for imported source signals: SpO2 is used to identify desaturation events, while blood pressure is used to determine the maximum and minimum (i.e., systolic and diastolic).

[0171] By reducing all clinically relevant information into events and extending their review into the continuous source, representing complex data in a way that is consistent and accessible may be realised through embodiments of the present disclosure.

[0172] It will be appreciated by persons skilled in the art that numerous variations and / or modifications may be made to the above-described embodiments, without departing from the broad general scope of the present disclosure. The present embodiments are, therefore, to be considered in all respects as illustrative and not restrictive.

Claims

CLAIMS:

1. A system for processing acoustic signals, the system comprising: at least one sensor configured to obtain at least one acoustic signal over a period of time; a processing module, configured to: determine signal data of the at least one acoustic signal obtained from the at least one sensor; partition the signal data into a plurality of intervals; generate, for each interval in the plurality of intervals, a bin associated with the interval by processing data points of the signal data within the interval; a user interface module, configured to generate, for each bin, a responsive component on a user interface corresponding to the bin; an audio access module configured to: access, for each responsive component, the signal data within the interval associated with the bin that corresponds to the responsive component; wherein upon interaction with the responsive component, the audio access module is configured to: retrieve at least part of the signal data of the acoustic signal convert the retrieved signal data to an audio signal; and output the audio signal.

2. The system according to claim 1, wherein the processing module is further configured to identify one or more events in the signal data of the acoustic signal.

3. The system according to claim 2, wherein the processing module is further configured to classify the one or more identified events based on at least one of frequency, amplitude, and duration.

4. The system according to claim 2 or 3, wherein the one or more events is determined based on at least one of a spike in data values in the signal data and a drop in data values in the signal data.

5. The system according to any one of claims 2 to 4, wherein identifying the one or more events comprises applying a machine learning model to further refine the one or more events based on resemblant patterns in the signal data.

6. The system according to any one of claims 1 to 5, wherein the audio access module includes a playback component integrated with the user interface, and wherein the audio access module is configured to output the audio signal to the playback component.

7. The system according to any one of claims 1 to 6, wherein the audio access module is configured to provide a visual representation of the audio signal on the user interface based on the retrieved signal data.

8. The system according to any one of claims 1 to 7, wherein the audio access module includes a control module configured to control playback of the audio signal, the control module having one or more control components integrated with the user interface.

9. The system according to any one of claims 1 to 8, wherein the system further comprises a haptic feedback device, configured to receive the output audio signal, and convert the audio signal into mechanical vibrations.

10. The system according to any one of claims 1 to 9, wherein the at least one sensor is a vibration sensor.

11. The system according to any one of claims 1 to 10, wherein the acoustic signal includes at least one biosignal.

12. The system according to any one of claims 1 to 11, wherein the acoustic signal includes at least one primary context signal.

13. The system according to claim 12, wherein the primary context signal includes at least one of: breath sounds, heart sounds, snoring, obstructions, wheezing, coughing, and crackles.

14. The system according to claim 12 or claim 13, wherein the user interface module is configured to generate the bin for each interval based on the at least one primary context signal.

15. The system according to any one of claims 12 to 14, wherein the processing module is configured to classify one or more events in the signal data based on the primary context signal.

16. The system according to any one of claims 1 to 15, wherein the acoustic signal includes at least one secondary signal.

17. The system according to claim 16, wherein the at least one secondary signal comprises a breathing movement signal and / or a movement signal.

18. The system according to any one of claims 1 to 17, wherein the processing module is configured to filter the acoustic signal into one or more frequency ranges of interest to create a plurality of signals.

19. The system according to any one of claims 1 to 18, wherein partitioning the signal data into a plurality of intervals is based on a predetermined time duration and / or length of the acoustic signal.

20. The system according to any one of claims 1 to 19, wherein the bin is a decimation bin.

21. The system according to any one of claims 1 to 20, wherein processing the data points within the interval includes extracting a subset of data points in the interval.

22. The system according to any one of claims 1 to 21, wherein processing the data points within the interval includes aggregating and / or condensing the data points in the interval.

23. The system according to claim 21 wherein extracting the subset of data points for each interval comprises calculating a statistical measure, the statistical measure comprising at least one of count, sum, ratio, mean, median, mode, quantile, and standard deviation of the data points within the interval.

24. The system according to claim 20, wherein the decimation bin provides a representation of the signal data within the interval.

25. The system according to any one of claims 1 to 24, wherein the responsive component comprises a graphical element representative of the generated bin.

26. The system according to any one of claims 1 to 25, wherein the user interface module is configured to generate responsive components in real time as bins are generated.

27. The system according to any one of claims 1 to 26, wherein the user interface module is configured to generate a container component to arrange responsive components on the interface in a sequential array corresponding to a position of the corresponding intervals within the signal data.

28. The system according to claim 27 when dependent on any one of claims 12 to 15, wherein the user interface module is configured to generate, for a plurality of primary context signals, a container component corresponding to each primary context signal, the container component including an array of responsive components corresponding to intervals of a primary context signal.

29. The system according to claim 27 when dependent on any one of claims 12 to 15, or claim 28, wherein the responsive components within a container component enable interrogation of the signal data corresponding to the container component.

30. The system according to any one of claims 1 to 29, wherein the retrieved signal data includes baseline data.

31. A method for processing acoustic signals, the method comprising: determining signal data from at least one acoustic signal obtained from at least one sensor; partitioning the signal data into a plurality of intervals; generating, for each interval in the plurality of intervals, a bin associated with signal data in the interval by processing data points within the interval; generating, for each bin, a responsive component on a user interface corresponding to the bin; upon interaction with the responsive component, retrieving at least part of the signal data of the acoustic signal; converting the retrieved signal data to an audio signal; and outputting the audio signal.

32. A user interface, comprising: a plurality of responsive components arranged on the user interface, each of the plurality of responsive components including an attached handler; wherein each responsive component corresponds to a bin associated with an interval of signal data of at least one acoustic signal received from at least one sensor; andwherein each responsive component is bound to interval data associated with the bin to provide access to the signal data; at least one container component configured to hold a plurality of the responsive components associated with the at least one acoustic signal, the container component arranging the responsive components on the user interface to provide an integrated graphical representation of the at least one acoustic signal; at least one handler attached to each responsive component, configured to, upon determining an interaction with a responsive component has occurred, retrieve at least part of the signal data of the acoustic signal bound to the responsive component; and a playback component integrated within the user interface, the playback component configured to receive an audio signal converted from the at least part of the signal data of the acoustic signal.

33. A system for processing a target audio segment, the system comprising: a user interface having a plurality of responsive components; at least one extraction module, configured to: upon interaction with a responsive component of the plurality of responsive components, determine an interval of signal data associated with the responsive component; determine a first time point and a second time point; and extract the signal data between the first time point and the second time point to define a target data segment; an audio segment output module, configured to: convert the target data segment into an audio segment; and output the audio segment.

34. The system according to claim 33, wherein the extraction module is further configured to: identify at least one event within the interval of signal data; and determine the first time point as a time point before the at least one event, and the second time point as a time point after the at least one event.

35. The system according to claim 34, wherein the at least one event is determined based on at least one of a spike in data values in the signal data and a drop in data values in the signal data.

36. The system according to claim 34 or claim 35, wherein the first time point is determined based on a predefined buffer prior to a start of the at least one identified event in which there is no anomalous activity.

37. The system according to any one of claims 34 to 36, wherein the second time point is determined based on a predefined buffer after an end of the at least one identified event in which there is no anomalous activity.

38. The system according to any one of claims 34 to 37, wherein the audio segment output module is further configured to generate a visual alert on the user interface when playback of the audio segment reaches the at least one identified event.

39. The system according to any one of claims 33 to 38, wherein the first time point and the second time point are determined based on a time interval.

40. The system according to any one of claims 33 to 39, wherein the audio segment output module includes a playback component integrated with the user interface, and wherein the audio segment output module is configured to initiate playback of the audio segment through the playback component.

41. The system according to any one of claims 33 to 40, wherein the audio segment output module is configured to provide a visual representation of the audio segment on the user interface based on the retrieved signal data.

42. The system according to any one of claims 33 to 41, wherein the audio segment output module includes a control module configured to control the playback component, the control module having one or more control components integrated with the user interface.

43. The system according to any one of claims 33 to 42, wherein the audio segment output module is further configured to synchronise the playback of the audio segment with a visual representation of the signal data.

44. The system according to any one of claims 33 to 43, wherein the audio segment output module is further configured to synchronise the playback of the audio segment with one or more external devices.

45. The system according to any one of claim 33 to 44, wherein the system further comprises a haptic feedback device, configured to receive the audio segment and convert the audio segment into mechanical vibrations.

46. The system according to any one of claims 33 to 45, wherein each of the plurality of responsive components represents a bin associated with the interval of signal data.

47. The system according to any one of claims 33 to 46, wherein the signal data is data from an acoustic signal.

48. The system according to claim 47, wherein the acoustic signal includes at least one primary context signal.

49. The system according to claim 48, wherein the primary context signal includes at least one of: breath sounds, heart sounds, snoring, obstructions, wheezing, coughing, and crackles.

50. The system according to claim 48 or claim 49, wherein the user interface includes a responsive component corresponding to each interval of the signal data of the at least one primary context signal.

51. The system according to any one of claims 33 to 50, wherein the acoustic signal includes at least one secondary signal.

52. The system according to claim 51, wherein the at least one secondary signal comprises a breathing movement signal and / or a movement signal.

53. The system according to any one of claim 33 to 52, wherein the audio segment output module is configured to convert the target data segment into an audio segment by converting the target data segment to an analog signal to create an audio signal.

54. The system according to any one of claims 33 to 53, wherein the extraction module is further configured to store the target data segment in a data store.

55. The system according to any one of claims 33 to 54, wherein the extraction module is further configured to apply a compression algorithm to the target data segment to reduce a size of the target data segment before converting the target data segment into an audio segment.

56. The system according to any one of claims 33 to 55, wherein the extraction module is further configured to generate metadata for the target data segment, wherein the metadata includes at least the first time point and the second time point.

57. The system according to any one of claims 33 to 56, wherein the user interface further comprises an annotation component configured to annotate the visual representation of the audio signal.

58. A method for processing a target audio segment, the method comprising: providing a user interface with a plurality of responsive components; upon interaction with a responsive component of the plurality of responsive components, determining an interval of signal data bound with the responsive component; determining a first time point and a second time point; and extracting the signal data between the first time point and the second time point to define a target data segment; converting the target data segment into an audio segment; and outputting the audio segment.

59. A user interface, comprising: a plurality of responsive components arranged on the user interface, each of the plurality of responsive components including an attached handler; wherein each responsive component corresponds to a bin associated with an interval of signal data of at least one acoustic signal received from at least one sensor; and wherein each responsive component is bound to the interval data associated with the bin to provide access to the signal data; at least one container component configured to hold a plurality of the responsive components associated with the at least one acoustic signal, the container component arranging the responsive components on the user interface to provide an integrated graphical representation of the at least one acoustic signal; and at least one handler attached to each responsive component, the handler configured to, upon determining an interaction with a responsive component has occurred, retrieve an audio segment; wherein the audio segment comprises extracted signal data between a first time point and a second time point within the interval.

Images