Computer implementation method, computer program, and computer-readable storage medium for acquiring respiratory-related sounds from audio recordings.

Abstract

A method (100) for acquiring respiration-related sounds (160) (RRS) originating from a target patient is disclosed, the method comprising the following steps: acquiring input audio recordings (110, 111) of the target patient's sleep environment, acquiring a respiratory trace (150) of the target patient's breathing, identifying (120) RRSs (130) in the input audio recordings, and selecting (140) RRSs (160) originating from the target patient from the RRSs based on the respiratory trace. The selecting step further comprises determining a first subset and / or a second subset of RRSs having a high and / or low probability of originating from the target patient, training a classifier based on the first and / or second subset to select RRSs originating from the target patient, and selecting the RRSs (160) originating from the target patient using the trained classifier.

Description

Technical Field

[0001] The present invention particularly relates to a method for acquiring respiratory related sounds (RRS) generated from a target patient.

Background Art

[0002] One of the elements to be studied in the field of sleep analysis is respiratory related sounds (RRS). RRS are short audio fragments of the sounds emitted from a patient during sleep analysis, such as snoring sounds, sighing sounds, heavy breathing sounds, groaning sounds, etc. Furthermore, further analysis of such sounds can be used to diagnose sleep disorders such as sleep apnea. Additionally, it may be desirable to count the duration of each RRS, the frequency of RRS, and the total number of RRS, and analyze various aspects of RRS.

[0003] RRS and related metrics can be obtained from the audio recording of a patient during sleep.

[0004] One method for obtaining such an audio recording is to attach a recording microphone to the patient's face and bring it as close as possible to the patient's nose or mouth. The advantage of this method is that, by design, external sounds and noise are reduced. However, the presence of such a microphone may have an adverse effect on the patient's sleep, and as a result, the detected RRS may not accurately reflect the patient's natural sleep.

[0005] Alternatively, an audio recording device, such as a digital audio recording device like a mobile phone, or a dedicated audio recording device, can be further placed near the target patient. By doing so, the patient can obtain a more natural sleep without being disturbed even if there is a microphone or some other device on or near the face. However, the drawback in this case is that if the patient is not sleeping alone in the room, the RRS of another person may be recorded in the audio recording.

[0006] U.S. Patent Application Publication 2020 / 261687 discloses a solution for dynamically masking audible breathing noise determined to be generated by one or more sleep partners. In one embodiment, the subject's sleep is protected by detecting audible breathing noise in the sleep environment, determining that the audible breathing noise is not generated by the subject, and mitigating the perception of audible breathing noise determined to be originating from another subject, such as a bed partner or pet. Dynamic masking reduces the subject's exposure to unnecessary sounds and reduces the likelihood that the masking sounds will disturb the subject's sleep. [Prior art documents] [Patent Documents]

[0007] [Patent Document 1] U.S. Patent Application Publication No. 2020 / 261687 [Overview of the project] [Problems that the invention aims to solve]

[0008] Therefore, an object of the present invention is to solve, or at least mitigate, one or more of the problems described above. In particular, this disclosure aims to provide a method for identifying RRS in a subject patient in a relatively comfortable manner without disturbing the patient's natural sleep. [Means for solving the problem]

[0009] To this end, according to a first aspect, a computer implementation method for acquiring respiratory-related sounds (RRS) generated from a subject patient is provided, the method comprising the steps of: acquiring an input audio recording of the subject patient's sleep environment; acquiring a respiratory trace of the subject patient's respiration characterizing the patient's respiration during the period of the audio recording; identifying RRS in the input audio recording; and selecting RRS generated from the subject patient from the RRS based on the respiratory trace, the selection step comprising: determining a first subset and / or a second subset of RRS having a high and / or low probability of having originated from the subject patient; training a classifier based on the first subset and / or the second subset to select RRS generated from the subject patient; and selecting RRS generated from the subject patient by the trained classifier.

[0010] The input audio recording covers the sleep environment of the subject patient, meaning it may include RRS from other people or animals, and other ambient sounds, in addition to the subject patient's RRS. Therefore, the input audio recording contains RRS from multiple subjects. Subsequently, in the selection step, all or some of these are selected. To distinguish RRS originating from the subject patient from other sounds, RRS sounds are selected based on a respiratory trace (i.e., a representation of the subject patient's respiration as a function of time covering the duration of the input audio recording). Since RRS originating from the subject patient is related to the subject patient's respiration, there is a relationship between these RRS and respiration. As a result, RRS originating from the subject patient can be distinguished from other sounds in the input audio recording.

[0011] As a result, a set of sounds free from other sounds that could negatively affect the analysis can be obtained, enabling accurate sleep analysis. Furthermore, because other sounds are excluded, there is no need to record audio very close to the patient's mouth or chest. In other words, the microphone does not suppress RRS from the patient, nor does the microphone itself generate unwanted RRS.

[0012] For the first subset, only RRSs whose probability of occurrence from the target patients exceeds a certain threshold (e.g., a probability higher than 90%) can be selected. This ensures a low output error. Furthermore, selecting RRSs with a high probability is generally easier, meaning that the required computational power and / or memory capacity is small.

[0013] For the second subset, only RRSs whose probability of occurrence from the target patients falls below a certain threshold (e.g., having a probability of less than 10%) can be selected. This second subset can then be further discarded from the results.

[0014] The results obtained according to the first and / or second subset can be further refined by a trained classifier by adding other RRSs that were not assigned to the first and / or second subset. To achieve this, the classifier is first trained using one or both of the subsets to classify RRSs as either belonging to the target patient or not. In other words, the first and / or second subset is used as labeled data. Then, the trained classifier is used to further classify the other RRSs and select more RRSs that originated from the target patient.

[0015] Respiratory traces can be further obtained by techniques available in the art, for example, by deriving traces from signals obtained by polysomnography, electrocardiography, electromyography, or photoplethysmography (PPG).

[0016] One step is RRS identification. According to one embodiment, this step further includes determining respiratory and non-respiratory sounds, and discarding non-respiratory sounds.

[0017] In other words, sounds unrelated to respiration are first discarded from the audio recording, resulting in a subset of RRS sounds that are not necessarily originating solely from the target patient. Then, based on the respiratory trace, RRS sounds originating from the target patient are selected from this subset.

[0018] According to one embodiment, the identification step includes determining a set of sounds, wherein the sounds in the set originate from the same sound source, and the selection step further includes selecting RRS originating from the patient in question from the set of sounds, based on a respiratory trace.

[0019] In other words, the sounds are initially divided into sets or clusters according to their origin. At this point, it is not yet known which sets originated from the patient in question. By referring to the respiratory trace, the RRS of a given set can be associated with the patient. Optionally, identification and discarding of non-RRS sounds can be performed before or after determining the sets.

[0020] Clustering sounds into sets according to their respective sound sources can be done, for example, by a trained classifier.

[0021] Optionally, classifier training can be performed only if there are too many unclassified RRSs, i.e., if there are still many identified RRSs that do not have a high or low probability of originating from the target patient. In such cases, performing a more computationally intensive classification operation may be useful.

[0022] According to one embodiment, the step of determining a first subset includes the steps of determining an audio timestamp associated with the RRS from an input audio recording and a respiratory timestamp associated with the RRS from a respiratory trace, and determining a first subset based on the audio timestamp and the respiratory timestamp.

[0023] In other words, the voice timestamp indicates the occurrence of each RRS in the input voice recording, and the respiration timestamp indicates the occurrence of each respiration cycle of the target patient. Since the RRS of the target patient is related to the patient's respiration, selection can be performed based on these obtained timestamps. For this purpose, the timestamps can be characterized by any detectable temporal feature such as, for example, start, local maximum, or local minimum. In this way, the selection operation is reduced to first identifying the temporal features and then performing operations on these temporal features.

[0024] One operation can be to determine the time difference between the voice timestamp and each respiration timestamp. Since the RRS of the target patient is related to respiration, the time differences associated with the patient are quite constant, while the time differences associated with other sound sources are more randomly distributed.

[0025] Next, by obtaining a histogram of the time differences, those with a high probability of belonging to the target patient are relatively more present at the peak of the histogram, and those with a low probability are at the tail of the histogram.

[0026] According to a second aspect, a controller is disclosed that includes at least one processor and at least one memory including computer program code, the at least one memory and the computer program code being configured to cause the at least one processor to execute the method according to the first aspect.

[0027] According to a third aspect, a computer program product is disclosed that includes computer-executable instructions for performing the method according to the first aspect when the program is executed on a computer.

[0028] According to a fourth aspect, a computer-readable storage medium is disclosed that includes the computer program product according to the third aspect.

Brief Description of the Drawings

[0029] [Figure 1] The following steps are shown, performed according to an exemplary embodiment, for selecting respiratory-related sounds from an audio recording generated by a patient. [Figure 2] The following steps are shown, performed according to an exemplary embodiment, for selecting a respiratory sound originating from a patient from multiple respiratory sounds and respiratory traces. [Figure 3] The steps performed according to an exemplary embodiment are shown for an expanded set of selected respiratory-related sounds generated from a patient. [Figure 4] The following steps are shown, performed according to an exemplary embodiment, for selecting respiratory-related sounds from an audio recording generated by a patient. [Figure 5] The diagram shows an exemplary plot of an audio recording containing the required respiratory-related sounds, and a plot of a respiratory trace containing respiratory-related timestamps and respiratory-related sound timestamps. [Figure 6] Another exemplary plot of an audio recording with the requested respiratory-related sounds is shown, along with a plot of a respiratory trace with respiratory-related timestamps and respiratory-related sound timestamps. [Figure 7A] This shows a histogram of the time-lag occurrence when all RRS cases originate from the target patient. [Figure 7B] This shows a histogram of delayed occurrence when no RRS (Rapid Routing Syndrome) was observed in the target patient. [Figure 7C] This shows a histogram of the time-delayed occurrence of RRS when it originates from multiple different audio sources. [Figure 8] This shows a computing system suitable for performing various steps according to an exemplary embodiment. [Modes for carrying out the invention]

[0030] Figure 1 shows several different steps of a computer implementation method 100 for identifying respiratory-related sounds 160 (RRS) generated from an input voice recording 110, i.e., a patient under monitoring. RRS corresponds to audible events generated by breathing during sleep. Such RRS may correspond to, for example, snoring, sighing, heavy breathing, moaning, or sounds generated during apnea events. RRS occurs within the respiratory cycle, for example, during inspiration, expiration, or both. Therefore, a patient who snores will generate a series of RRS over a specific time interval, e.g., a few seconds, a few minutes, or a few hours. Having a trace of RRS generated from a patient under monitoring is valuable in sleep analysis because it can reveal or explain various types of health conditions.

[0031] This method begins by obtaining an audio track 110 or audio recording 110 in which an RRS 160 generated by the patient is identified or selected. The audio track is recorded within audible distance from the patient, i.e., within the patient's sleeping environment. The recording can be done, for example, by placing an audio recording device next to the patient's bed or somewhere in the patient's bedroom. An exemplary example of such an audio recording is further shown in plot 111, where the amplitude 112 of the recorded audio signal is presented as a function of time.

[0032] In step 120 of method 100, different RRSs 131-134 are identified from the audio recording 110. These identified RRSs may be associated with one specific type of RRS, e.g., snoring only, or with several or all possible RRSs. Identification of RRSs eliminates other sounds or noises (e.g., sounds from outside the room) from further steps. RRSs can be identified, for example, by indicating their start time, end time, and / or duration, which allows for the unique identification of the RRS within the audio recording 110.

[0033] RRS identification can be performed, for example, by performing one or more of the following steps: a) For example, the sound envelope of signal 112 can be determined by calculating the analysis signal of signal 112, by calculating the moving average of signal 112, for example, the root mean square (RMS) value of signal 112, or by calculating the peak of signal 112. b) Determine the thresholds that characterize the active sound segments. This can be done, for example, by calculating local signal energy values ​​and establishing lower percentile values ​​of local signal energy to define baseline thresholds. c) Calculate when the sound envelope exceeds this threshold. d) Label all episodes where the envelope exceeds the threshold as active segments. e) Combine or delete active segments according to a set of decision rules, for example, to avoid impossibly large or small active segments. f) The active segment thus obtained is characterized by calculating a set of features, including Mel-frequency cepstral coefficients (MFCCs), signal output within a specific frequency range, temporal features such as the mean and standard deviation of the signal, features that characterize the entropy of the signal, and features that characterize the formant and pitch. g) For example, RRS segments are identified from active segments by a pre-trained classifier that classifies all active segments as either RRS or non-RRS, thereby obtaining a set of RRS segments originating from one or more audio sources.

[0034] The identified RRS 130 do not necessarily all originate from the patient under observation. For example, some may originate from another person sleeping next to or in the same room as the patient. Also, some RRS may originate from an animal, such as a dog sleeping in the same room. Therefore, in the subsequent selection step 140, a subset 160 of the RRS 130 is selected as originating from the patient under observation. To make this selection, a respiratory trace 150 from the patient is used to select the subset 160. Such a respiratory trace characterizes the patient's respiration during the period of the audio recording 110. Plot 151 shows such a trace of the patient as a function of time. In this case, the rising edge can correspond to inspiration, the falling edge to expiration, or vice versa. The respiratory trace may also correspond to discrete timestamps characterizing different respiratory cycles. An observable temporal relationship exists between the trace 150 and the RRS originating from the patient, but other RRS do not exhibit such a temporal relationship. Based on this, RRS 160 generated by the patient is selected as the output of step 140.

[0035] Respiratory traces can be obtained directly or indirectly from measurements of the patient. For example, traces can be derived from signals obtained by polysomnography, electrocardiography, electromyography, photoplethysmography (PPG), or accelerometer.

[0036] According to one embodiment, the selection 140 of RRS 160 can be performed by the steps 200 shown in Figure 2. First, in steps 201 and 202, the timestamp 203 of RRS 130 and the timestamp 204 of the respiratory trace 150 are identified. With respect to RRS 130, the RRS timestamp 203 can characterize the start of the RRS, the end of the RRS, or any predetermined time criterion within the occurrence of the RRS. In the respiratory trace 150, the respiratory timestamp 204 identifies the start, end, or any predetermined time criterion within a respiratory cycle, e.g., either an inspiratory or expiratory respiratory cycle. Next, in step 205, the difference 206 between timestamps 203 and 204 is determined, i.e., for each RRS timestamp 203, the time difference with a neighboring respiratory timestamp 204 (e.g., the next or previous respiratory timestamp) is determined. As a result, a sequence of time differences 206 is obtained, each time difference associated with its respective RRS. In the next step 207, a histogram 208 is constructed from these time differences 206. Histogram 208 represents the occurrence of a particular time lag or time lag interval. In such a histogram 208, time lags with a high occurrence rate indicate a strong temporal correlation between the relevant RRS and the respiratory trace, and therefore have a high probability of originating from the patient. Similarly, time lags with a low occurrence rate indicate little temporal correlation between the relevant RRS and the respiratory trace, and therefore have a low probability of originating from the patient. Thus, RRS 212 with an occurrence rate higher than a certain first threshold are selected as having a high probability of originating from the patient and added to the selection of patient RRS 160. Further RRS 210 with an occurrence rate lower than a certain second threshold can be selected as having a low probability of originating from the patient. The remaining RRS 211 are left unassigned. The unassigned RRS 211 can still be used to further extend the set of patient RRS 160, as will be further described in this embodiment with reference to Figures 3 and 4.

[0037] Another method for selecting patient RRS 160 is to calculate the coherence of one or more RRS 130 with the respiratory trace 150, i.e., the degree of synchronization between the voice signal and respiratory signal of one or more RRSs within the same time interval. In this case, one or more RRSs with high coherence are considered to have a high probability of originating from the patient, and one or more RRSs with low coherence are considered to have a low probability of originating from the patient, thereby obtaining a similar set of RRS 210, 211, and 212 in this case as well. Then, similar to the method in Figure 2, the RRS 212 with a high probability is selected as originating from the patient.

[0038] The selection of RRS from patients based on probability, for example, the selection by the steps in Figure 2, can be further extended depending on the results. For example, a significant number of RRS 211 may still be unassigned, i.e., their probability of originating from a patient is neither low nor high. In such cases, step 300, shown in Figure 3, can be performed. In the first step 301, an initial selection 302 is made by selecting RRS with high and / or low probabilities, for example by performing step 200 as described with reference to Figure 2. Next, in step 303, further RRS are identified as originating from patients based on sets of RRS with high and / or low probabilities, e.g., sets 210 and 212. Based on these sets, some of the unassigned RRS are further assigned as either originating from a patient or not. This step 303 can be performed in various ways. According to the first example, step 303 includes training a classifier for classifying RRS according to whether or not they originated from a patient. In training, RRS with high and / or low probabilities are used as labeled training data. Next, the trained classifier is used to add the unassigned RRS (e.g., RRS 211) to selection 160. According to the second example, unsupervised clustering is used to select unassigned RRS from a high-probability set or a low-probability set that have similar feature content with similar temporal coherence to the RRS. Then, the unassigned RRS clustered by the high-probability set are added to selection 160.

[0039] Figures 5, 6, and 7 further illustrate step 200. Figure 5 shows a first plot including the audio recording 510 and the identified RRS 511 obtained, for example, by step 120 in Figure 1. Figure 5 further shows a second plot including a respiratory trace 520. In the respiratory trace 520, the start of RRS 511 is indicated by a circle 521, representing the RRS timestamp 524. In the respiratory trace 520, the periodic minimum of the trace is indicated by an × mark 522, representing the respiratory-related timestamp 525. In this case, the time difference 526 is represented by the space between the dashed line representing the RRS timestamp and the preceding or following dotted line representing the RR timestamp. All RRS 511 shown in Figure 5 originated from the patient. Therefore, there is a strong temporal relationship between the RR timestamp 525 and the RRS timestamp 524, which can be observed by a nearly constant time difference 526. Next, Figure 7A shows a histogram of the time difference derived from RRS that occurred only from the patients shown in Figure 5.

[0040] Similar to Figure 5, Figure 6 shows a first plot including the audio recording 610 and the identified RRS 611 obtained, for example, by step 120 in Figure 1. Figure 6 further shows a second plot including a respiratory trace 620. In the respiratory trace 620, the start of RRS 611 is indicated by a circle 621, representing the RRS timestamp 624. In the respiratory trace 620, the periodic minimum of the trace is indicated by an × mark 622, representing the respiratory-related timestamp 625. In this case, the time difference 626 is represented by the space between the dashed line representing the RRS timestamp and the nearest dotted line representing the RR timestamp. The RRS 611 shown in Figure 6 did not originate from the patient. Therefore, there is a weak temporal relationship between the RR timestamp 625 and the RRS timestamp 624, which can be observed by the large, fluctuating time difference 626. Next, Figure 7B shows a histogram 720 of time differences derived from RRSs that originated only from the patient shown in Figure 6.

[0041] Next, Figure 7C shows histogram 730 based on the time difference from both Figures 5 and 6, i.e., the combination of histograms 710 and 720. Thus, the data in histogram 730 may correspond to the histogram data 208 of method 200. As explained with reference to step 209 in Figure 2, a first threshold 731 can be defined to select RRS with a high probability 735, and a second threshold 732 can be defined to select RRS with a low probability 733, 737. The remaining RRS are left unassigned, as indicated by regions 734, 736.

[0042] According to one embodiment, a further clustering step can be performed in the method 100 shown in Figure 1. This will be further explained with reference to the method in Figure 4. In a first step 420 which may correspond to step 120, RRS 430 are identified from the input audio recording 410. Then, an additional clustering step 470 is performed. In this step 470, when RRS have a high probability of belonging to the same audio source, those RRS are grouped into a cluster.

[0043] The clustering method 470 first identifies a set of features that characterize the RRS, such as temporal features like Mel-frequency cepstrum coefficients (MFCCs), signal output within a specific frequency range, signal mean and standard deviation, features characterizing the RRS entropy, and features characterizing the formant and pitch. In addition to or supplementing this, RRS occurring in temporally repeating patterns can be identified, thereby obtaining different chains of RRS. Next, the RRS are clustered into several different valid sound sources based on their relevance to the temporal chain and / or based on the similarity between the derived different features. Feature-based clustering can be performed by clustering algorithms such as K-means clustering or Gaussian Mixture Model (GMM) clustering. Clustering based on the obtained temporal chain can be performed, for example, by identifying repetitive RRS patterns with specific time intervals between occurrences. After clustering, some RRS may still remain unassigned, i.e., have a high probability of not belonging to a particular sound source. In such cases, a further supervised clustering step can be performed. In this case, the classifier is trained to classify RRS into clusters by using already clustered RRS as labeled training data. The classifier can be a support vector machine (SVM) or a neural network.

[0044] Next, the clusters 471 of RRS thus obtained are used as input to a further selection step 440, in which clusters with a high and / or low probability of originating from a patient are identified. The clusters with a high probability are then selected as output 160. Step 440 can be performed in the same way as step 140 or step 200, but based on clusters of RRS instead of individual RRS. Furthermore, an additional step 403 can be performed, adding clusters of RRS that have not yet been assigned to output 160 in the same way as step 303, but based on clusters of RRS instead of individual RRS.

[0045] The steps according to the embodiments described above can be performed by any suitable computing circuit, such as a mobile phone, tablet, desktop computer, laptop, and local or remote server. The steps according to the embodiments described above may be performed in the same device as the voice recording device. For this reason, voice recording may be performed by, for example, a mobile phone, tablet, desktop computer, or laptop. Alternatively, the steps according to the embodiments described above may be performed by a suitable circuit located away from the patient's environment. In such a case, the voice recording may be provided to the circuit via a communication network such as the Internet or a private network.

[0046] Figure 8 shows a preferred computing system 800 with circuitry enabling the execution of the steps according to the described embodiment. The computing system 800 may generally be formed as a preferred general-purpose computer and comprises a bus 810, a processor 802, local memory 804, one or more optional input interfaces 814, one or more optional output interfaces 816, a communication interface 812, a storage element interface 806, and one or more storage elements 808. The bus 810 may comprise one or more conductors enabling communication between components of the computing system 800. The processor 802 may include any type of conventional processor or microprocessor that interprets and executes programming instructions. The local memory 804 may include random access memory (RAM) or another type of dynamic storage device that stores information and instructions executed by the processor 802, and / or read-only memory (ROM) or another type of static storage device that stores static information and instructions used by the processor 802. The input interface 814 may include one or more conventional mechanisms that enable an operator or user to input information into the computing device 800, such as a keyboard 820, a mouse 830, a pen, a voice recognition and / or biometric authentication mechanism, or a camera. The output interface 816 may include one or more conventional mechanisms that output information to an operator or user, such as a display 840. The communication interface 812 may include any transceiver, such as one or more Ethernet interfaces, that enables the computing system 800 to communicate with other devices and / or systems, such as other computing devices 881, 882, 883. The communication interface 812 of the computing system 800 may be connected to such other computing systems via a local area network (LAN) or wide area network (WAN), for example, via the Internet.The storage element interface 806 may include a storage interface such as a Serial Advanced Technology Attachment (SATA) interface or a Small Computer System Interface (SCSI) for connecting the bus 810 to one or more storage elements 808, one or more local disks such as SATA disk drives, and may control the reading and writing of data to and from these storage elements 808. Although the storage elements 808 are described above as local disks, any other suitable computer-readable media can generally be used, such as removable magnetic disks, optical storage media such as CDs or DVDs, ROM disks, solid-state drives, or flash memory cards.

[0047] As used in this application, the term “circuit” may mean one or more or all of the following: (a) Hardware-only circuit implementations, such as implementations using only analog and / or digital circuits. (b) Combinations of hardware circuits and software, for example, (applicable): (i) combinations of analog and / or digital hardware circuits and software / firmware, and (ii) a portion of the hardware processor equipped with software (including a digital signal processor, software, and memory that work together to enable a device such as a mobile phone or server to perform various functions), and (c) Hardware circuits and / or processors, such as a microprocessor or part of a microprocessor, that require software (e.g., firmware) to operate but may not be present when the software is not required to operate. The definition of "circuit" in this application applies to all uses of this term in this application, including any claim. Further examples include, as used in this application, the term "circuit" also covers hardware circuitry only, or processors (or more processors) only, or a portion of hardware circuitry or processors and / or their associated software and / or firmware implementations. The term "circuit" also covers, for example, a baseband integrated circuit or processor integrated circuit for a mobile terminal, or a similar integrated circuit in a server, cellular network device, or other computing or network device, where applicable to a particular claim element.

[0048] While the present invention has been described with reference to specific embodiments, it will be apparent to those skilled in the art that the present invention is not limited to the details of the exemplary embodiments described above, and that the present invention can be embodied with various changes and modifications without departing from its scope. Therefore, these embodiments are considered illustrative and non-limiting in all respects, and the scope of the present invention is indicated not by the foregoing description but by the appended claims, and therefore all changes that fall within the meaning and equivalence of the claims are intended to be incorporated into the present invention. In other words, all changes, modifications, or equivalents that fall within the scope of fundamental principles and whose essential attributes are asserted in this patent application are intended to be covered. Furthermore, readers of this patent application will understand that the terms “comprising” or “comprise” do not exclude other elements or steps, and the terms “a” or “an” do not exclude plural, and that a single element, such as a computer system, processor, or another integrated unit, may realize the functions of several means enumerated in the claims. No reference numeral in the claims should be construed as limiting the respective claims relating thereto. Terms such as “first,” “second,” “third,” “a,” “b,” and “c,” when used in the specification or claims, are introduced to distinguish similar elements or steps and do not necessarily describe a sequence or chronological order. Similarly, terms such as “upper,” “lower,” “top,” and “bottom” are introduced for descriptive purposes and do not necessarily indicate a relative position. It should be understood that the terms used in this manner are interchangeable under appropriate circumstances, and that embodiments of the invention can operate in other sequences or orientations different from those described or illustrated above. Furthermore, in order to maintain the disclosures made at the time of filing this application, the descriptions of claims 1 to 12 at the time of filing this application are added below. (Claim 1) A computer implementation method (100, 400) for acquiring respiratory-related sounds (160, 511) (RRS) generated from a target patient, wherein the method comprises the following steps, namely: The steps include obtaining input audio recordings (110, 111, 410, 510, 610) of the sleep environment of the aforementioned patient, A step of obtaining respiratory traces (150, 450, 520, 620) of the subject patient's respiration, which characterize the subject patient's respiration during the period of the input voice recording, The step of identifying RRS (130, 430, 511, 611) in the input audio recording (120, 420, 470), The step of selecting (140, 200, 300, 440, 403) the RRS (160) generated from the subject patient from the RRS based on the respiratory trace, The aforementioned selection step is, The steps include (209) determining a first subset (212, 735) and / or a second subset of RRS that have a high probability and / or low probability of having occurred from the aforementioned patient, The steps include training a classifier based on the first subset and / or the second subset to select RRS generated from the aforementioned target patients (303, 403), The steps include selecting the RRS(160) generated from the target patient using the trained classifier, Methods that include... (Claim 2) The method according to claim 1, wherein the identifying step includes the steps of determining (120, 420) respiratory-related sounds and non-respiratory-related sounds, and discarding the non-respiratory-related sounds. (Claim 3) The method according to claim 1 or 2, wherein the identifying step includes the step of determining (470) a set of sounds (471) wherein the sounds in the set originate from the same sound source, and the selecting step further includes the step of selecting (440, 403) an RRS (160) originating from the subject patient from the set of sounds based on the respiratory trace. (Claim 4) The method according to any one of claims 1 to 3, wherein the selection step further comprises the step of discarding the second subset from the RRS. (Claim 5) The method according to any one of claims 1 to 4, wherein the selection step includes performing the training depending on the amount of RRS(211, 734, 736) not assigned to the first subset and the second subset. (Claim 6) The method according to any one of claims 1 to 5, wherein the step of determining the first subset includes the steps of determining (201, 202) voice timestamps (203, 521, 621) associated with the RRS from the input voice recording (130) and breath timestamps (204, 522, 622) associated with the RRS from the breath trace (150), and the steps of determining the first subset based on the voice timestamps and the breath timestamps (205, 207, 209). (Claim 7) The method according to claim 6, wherein the step of determining the first subset further includes the step of determining the time difference (206, 526, 625) between the voice timestamp and the respective breath timestamps. (Claim 8) The method according to claim 7, wherein the step of obtaining the first subset further comprises the steps of obtaining (207) a histogram of the time difference (730) and identifying the first subset (212) from the histogram (209). (Claim 9) The method according to any one of claims 1 to 8, wherein the respiratory trace is derived from signals obtained by a polysomnograph, electrocardiograph, electromyograph, or photoplethysmography (PPG). (Claim 10) A controller (800) comprising at least one processor and at least one memory containing computer program code, wherein the at least one memory and the computer program code are configured to cause the controller to perform the method according to any one of claims 1 to 9 using the at least one processor. (Claim 11) A computer program product that, when the program is executed on a computer, includes a computer-executable instruction for performing the method described in any one of claims 1 to 9. (Claim 12) A computer-readable storage medium comprising a computer-executable instruction for performing the method described in any one of claims 1 to 9 when the program is executed on a computer.

Claims

1. A computer implementation method for acquiring respiratory sounds generated from a target patient, The steps include: obtaining an input audio recording of the sleep environment of the aforementioned patient, A step of obtaining a respiratory trace of the subject patient's respiration, which characterizes the subject patient's respiration during the period of the input voice recording, The step of identifying respiratory-related sounds, i.e., RRS, in the input audio recording, Based on the respiratory trace, the step of selecting the RRS that originated from the target patient from the RRS. Includes, The aforementioned selection step is, The steps include determining a first subset and / or a second subset of RRS that have a high probability and / or low probability of originating from the aforementioned patient, A step of training a classifier based on the first subset and / or the second subset to select RRSs that have occurred from the subject patients, The steps include selecting the RRS generated from the target patient using the trained classifier, and Methods that include...

2. The method according to claim 1, wherein the identification step includes the steps of determining respiratory-related sounds and non-respiratory-related sounds, and discarding the non-respiratory-related sounds.

3. The method according to claim 1 or 2, wherein the identifying step includes determining a set of sounds, the sounds in the set originating from the same sound source, and the selecting step further includes selecting RRS originating from the subject patient from the set of sounds based on the respiratory trace.

4. The method according to any one of claims 1 to 3, wherein the selection step further comprises the step of discarding the second subset from the RRS.

5. The method according to any one of claims 1 to 4, wherein the selection step includes performing the training depending on the amount of RRS not allocated to the first subset and the second subset.

6. The method according to any one of claims 1 to 5, wherein the step of determining the first subset includes the steps of determining an audio timestamp associated with the RRS from the input audio recording and a respiratory timestamp associated with the RRS from the respiratory trace, and determining the first subset based on the audio timestamp and the respiratory timestamp.

7. The method according to claim 6, wherein the step of determining the first subset further includes the step of determining the time difference between the audio timestamp and each breath timestamp.

8. The method according to claim 7, wherein the step of obtaining the first subset further comprises the steps of obtaining a histogram of the time difference and identifying the first subset from the histogram.

9. The method according to any one of claims 1 to 8, wherein the respiratory trace is derived from signals obtained by a polysomnograph, electrocardiograph, electromyograph, or photoplethysmography (PPG).

10. The method according to any one of claims 1 to 9, wherein the respiratory trace is obtained directly or indirectly from measurements of the patient.

11. A controller comprising at least one processor and at least one memory containing computer program code, the controller configured to use the at least one processor, the at least one memory and the computer program code to perform the method according to any one of claims 1 to 10.

12. A computer program comprising a computer executable instruction for performing the method described in any one of claims 1 to 10 when the computer program is executed on a computer.

13. A computer-readable storage medium comprising a computer-executable instruction for performing the method according to any one of claims 1 to 10 when a program including the method is executed on a computer.

Images