Method and system to emulate heart sounds

Abstract

Disclosed is a method for creating audio file (650) to emulate heart sounds. the method comprising: obtaining heart rate sound sample(s) (302, 304, 402, 650A, 650B, 650C, 650D, 650E, 650F) and relative timing information (652) of heart rate sound sample(s), wherein heart rate sound sample(s) is associated with respective first phase of first phase(s) of first breathing cycle signal (306); measuring second breathing cycle signal (202, 664) and heart rate signal (204); associating, using relative timing information and measured second breathing cycle signal (202, 664), heart rate sound sample(s) to respective second phase of second phase(s) of second breathing cycle signal, wherein respective second phase corresponds to respective first phase of first breathing cycle signal; and merging heart rate sound sample(s) based on association to create audio file, wherein timing of heart rate sound sample(s) is aligned with heart rate signal. Disclosed also is a system (500) thereof.

Description

[0001] METHOD AND SYSTEM TO EMULATE HEART SOUNDS

[0002] TECHNICAL FIELD

[0003] Generally, the present disclosure relates to childcare. Particularly, the present disclosure relates to a method and a system for creating an audio file to emulate heart sounds.

[0004] BACKGROUND

[0005] Generally, rhythms originating from heartbeats and breathing patterns play a crucial role in soothing infants, especially when such rhythms originate from a person familiar to the infant. Further, such rhythms can be captured using various means including sounds using stethoscope, movement using inertial measurement units or piezoelectric sensors and electrical activity via electrocardiograms. The recorded signals, such as the sound signals, can be played to the infant after recording. Moreover, utilization of vital signals for generating calming movements, such as in devices like kangaroo care equipment, represents an extension of the aforesaid principles into real-world applications.

[0006] However, acclimatization of a newborn infant to haptic and auditory physiological signals commences in utero and continues after birth during close skin-to-skin contact, such as, with mother of the infant. Further, despite providing such physiological signals during breast feeding or holding the infant to provide nourishment and closeness, there persists a gap in providing regular closeness and security to the infant. For example, during night-time sleep or when the infant is not in direct contact with the mother, a comforting presence associated with heartbeat and breathing patterns of the mother are absent for the infant.

[0007] Conventionally, existing solutions comprising devices offering artificial kangaroo therapy attempt to cover the aforementioned gap. However, such devices fail to adequately replicate a multisensory experience of closeness and security derived from physiological signals derived by the newborn infant from a familiar person, such as the mother, a father or another trusted individual when such a person is unavailable to provide necessary care for the infant.

[0008] In light of the above discussion, there exists an urgent need for solutions that accurately replicate and convey the physiological signals associated closeness and security provided by a familiar person, especially for newborn infants.

[0009] SUMMARY

[0010] An aim of the present disclosure is to provide a method and a system for creating and utilizing an audio file to emulate heart sounds as defined in appended independent claims to which reference is made to. The method and system provide an emulated auditory and haptic experience that mimics natural rhythms of human heartbeats and respiration cycles, enhancing sense of closeness and security, such as, for newborn infants that are away from their parents. The emulated auditory and haptic experience enhances emotional and physical well-being of such infants by providing a simulated sense of proximity and familiarity, thereby promoting relaxation and security for the infants. Advantageous features are set out in the appended dependent claims.

[0011] BRIEF DESCRIPTION OF DRAWINGS

[0012] The summary above, as well as the following detailed description of illustrative embodiments, is better understood when read in conjunction with the appended drawings. For the purpose of illustrating the present disclosure, exemplary constructions of the disclosure are shown in the drawings. However, the present disclosure is not limited to specific methods and instrumentalities disclosed herein. Moreover, those skilled in the art will understand that the drawings are not to scale. Wherever possible, like elements have been indicated by identical numbers. Embodiments of the present disclosure will now be described, by way of example only, with reference to the following diagrams wherein:

[0013] FIG. 1 is an illustration of a flowchart of a method for creating an audio file to emulate heart sounds, in accordance with an embodiment of the present disclosure;

[0014] FIG. 2 is an illustration of an amplitude vs time graph depicting breathing cycle signal and heart rate signal, in accordance with an embodiment of the present disclosure;

[0015] FIG. 3 is an illustration of an amplitude vs time graph depicting heart rate sound samples, in accordance with an embodiment of the present disclosure;

[0016] FIG. 4 is an illustration of an amplitude vs time graph depicting heart rate sound samples, in accordance with an embodiment of the present disclosure;

[0017] FIG. 5 is an illustration of a system for emulating heart sounds, in accordance with an embodiment of the present disclosure

[0018] FIG. 6 is an illustration on steps related to creating an audio file to emulate heart sounds and

[0019] FIG. 7 is an illustration of example of shifting sounds samples.

[0020] In the accompanying drawings, an underlined number is employed to represent an item over which the underlined number is positioned or an item to which the underlined number is adjacent. A non-underlined number relates to an item identified by a line linking the non-underlined number to the item. When a number is non-underlined and accompanied by an associated arrow, the non-underlined number is used to identify a general item at which the arrow is pointing. DETAILED DESCRIPTION OF EMBODIMENTS

[0021] In one aspect, the present disclosure provides a method for creating an audio file to emulate heart sounds, the method comprising:

[0022] - obtaining a set of heart rate sound samples and a relative timing information of each of the heart rate sound samples of the set in respect to a first phase of a first breathing cycle signal;

[0023] - measuring a second breathing cycle signal and a heart rate signal;

[0024] - associating, using the relative timing information and the measured second breathing cycle signal, each of the heart rate sound samples of the set to a second phase of the second breathing cycle signal that corresponds to a respective first phase of the first breathing cycle signal; and

[0025] - merging the heart rate sound samples based on the association to create the audio file, wherein timing of each of the heart rate sound samples is aligned with the heart rate signal.

[0026] The term "heart sound" as used throughout the present disclosure refers auditory sensations associated with heartbeats, cardiac muscle movement, or sounds corresponding to opening and closing sounds of valves in the heart. The sounds (i.e., heart sounds) can be perceived by a device, such as a stethoscope. The heart sounds vary as function of breathing since, during the breathing cycle lungs compress the heart thus affecting audio environment of the heart. Such heart sounds enable to create a "feel-scape" for the newborn child of being held, nursed or cradled by the parent (or for example, guardians), such as, in arms of the parent when the child is held against chest of the parent or when the child is nursed in lap of the parent. In this regard, throughout the present disclosure, the term "feel-scape" refers to the sensory experience artificially created to simulate the feeling of being held, nursed, or cradled by a familiar person, for a child. For example, the feel-scape may be an audio file comprising recordings of a set of heart sound samples or a haptic sensation replicating the soft movement of the chest (up-and- down movement) observed during the breathing cycle (e.g., a sleeping pad vibrating softly to imitate chest movement during breathing). Consequently, recording, processing and subsequently, replaying the heart sounds enables the newborn infant to experience being hold, being close, being in proximity or in contact, nursed or cradled by the parents to another trusted individual, such as a caretaker, a babysitter, a relative of the child and the like.

[0027] The method comprises obtaining the set of heart rate sound samples and the relative timing information of each of the heart rate sound samples of the set in respect to the first phase of the first breathing cycle signal. Herein, the method comprises obtaining a set of heart rate sound samples and a relative timing information of each of the heart rate sound samples of the set, wherein each of the heart rate sound samples is associated with a respective first phase of a plurality of first phases of a first breathing cycle signal. The term "heart rate sound sample" as used throughout the present disclosure refers to heartbeat data recorded from a person. The heart rate sound samples can be recorded from a person or those can be obtained via computer simulated sound samples. As an example, the heart rate sound samples can be recorded from a parent of the child, for example, by using a stethoscope. As another example the heart rate sample can be recorded with stethoscope from an arbitrary person. In this regard, the set of heart rate sound samples are recorded in order to capture the unique auditory characteristics of the person's heartbeat (such as the "lub-dub" sounds produced by the heart's valves during contraction and relaxation, respectively) thereby allowing creation of personalized audio files to provide a comforting feel-scape which can aid in soothing the child. Thus, the step of obtaining may be from a preexisting data or information, such as the set of heart rate sound samples recorded previously and stored in a data repository. Alternatively, the step of obtaining may be fethching directly measuring or recording set of heart rate sound samples from a sensor or microphone associated with a system for creating an audio file to emulate heart sounds. Beneficially, the obtaining from the sensor or microphone allows real-time or near- real-time creation of the audio file. Moreover, use of different sensors enables obtaining the raw data more reliable and enhances accuracy of the method. However, and obtaining the pre-recorded data enables analysing the heart rate signal trend over a period of time.

[0028] The term "breathing cycle signal" as used throughout the present disclosure refers to respiratory information recorded from a first or second person to determine, for example, a number of breaths taken by the person during each minute and their relative timings. Such respiratory information is recorded as movement of chest of the person, for example, by placing a smartphone on the chest of the person and allowing a gyroscope or accelerometer of the smartphone to determine the movement of the chest of the person during each inhalation phase and each exhalation phase. Thus, the term "breathing cycle signal" comprises digital or analogue representation of the respiration cycle. In this regard, the first breathing cycle signal refers to respiration information (represented as digital or analogue representation) collected from the first person. The second breathing cycle signal refers to respiration information (represented as digital or analogue representation) collected from the second person. It may be appreciated that the first and second persons may refer to two different individuals, (such as parents of a child, guardians of the child, caregivers, babysitters, or any other person). The technical effect is collection of diverse data or diverse heart sound samples for further processing and creating an audio file that can precisely replicate required feel-scape to soothe the child. In one example, the smartphone is placed under neck of the person (namely, the first person and / or the second person) to determine the movement of the chest during respiration, which is further represented as breathing cycle signal. Optionally, the smartphone obtains rotational movement data of the chest of the person around a longitudinal axis of the smartphone, such as, from top of the smartphone to bottom of the smartphone when the smartphone is placed parallel to a shoulder line of the person. In another example, the smartphone is placed on upper chest of the person when the person is lying in a recliner, a sofa, a bed or the like, to measure chest movements with the smartphone. In such an example, a 180 to 600 second period of acceleration data and / or gyroscope data is recorded from the person using the smartphone. Optionally, the obtained set of heart rate sound samples is processed, for example, by employing a low-pass filter on the gyroscope data acquired using the smartphone, suppressing frequencies above 1 Hz and calculating a ratio of 1 to 3 second average and 15 to 30 second average from a y-component of the low-pass filtered gyroscope data. Subsequently, the calculated ratio is employed to determine local maxima points whose prominence exceeds a value of 0.2 to 0.5. In a further example, a wearable device comprising sensors (such as, a gyroscope, accelerometer and the like) is mounted or worn around chest or abdomen of the person to measure expansion and contraction during of the chest or abdomen of the person during the inhalation and exhalation phases. In another example, a microphone or acoustic sensors are placed around the throat or airpipe of the person to determine parameters associated with the respiration cycle. The technical advantage is ease of accurately recoding of heart sound samples and breathing cycle signal.

[0029] In may be appreciated that alignment of each of the set of heart rate sound samples with the breathing cycle signal, ensures that the subsequently created audio file mimics natural physiological rhythms (i.e., the heart sounds) thereby enhancing sense of realism. Moreover, by incorporating breathing patterns, it is possible to create the audio file that provides a more immersive and calming experience for the child. Furthermore, use of common devices like smartphones, to measure breathing cycle signals, makes it easy to obtain such respiratory information in a non-clinical (or non-healthcare) settings, such as at home, at daycare or any such establishment.

[0030] The term "phase of the breathing cycle signal" as used throughout the present disclosure refers to a specific part of a duration between an inhalation performed by the person and a subsequent exhalation performed by the person. In other words, the phase of breathing cycle is defined as a time duration from start of inhalation to end of exhalation. Thus, the phase of the respiration is counted between two successive inhalations (or, optionally but less preferably, two successive exhalations). In one example, the phase of the breathing cycle is determined in terms of number of heartbeats occurring between two successive inhalations. In such an example, when six heartbeats are present between an inhalation and a subsequent inhalation, the breathing cycle is split into six phases. Thereafter, a phase of the breathing signal would refer to any one of sextiles corresponding to the six heartbeats of the person. Optionally, the phase of the breathing cycle signal also refers to a proximity to a first inhalation or a second inhalation between which the heartbeats are located. In this regard, it may be appreciated that each breathing cycle signal (namely, the first breathing cycle signal and the second breathing cycle signal) comprises a plurality of phases (for clarity, the first breathing cycle signal comprises a plurality of first phases, and the second breathing cycle signal comprises a plurality of second phases). Therefore, the first breathing cycle signal is split into a plurality of first phases, and the relative timing information of each heart rate sound sample is associated with one of the plurality of first phases. Similarly, the second breathing cycle signal is split into a plurality of second phases, and the relative timing information of each heart rate sound sample is associated with one of the plurality of second phases. The term "relative timing information" as used throughout the present disclosure relates to temporal positioning of a heart rate sound sample relative to a specific phase in the breathing cycle signal. It may be appreciated that the relative timing information can also be referred to as the duration of the said heart rate sound sample. In this regard, the relative timing information is used to corelate the heart rate sound sample to specific phase in the breathing cycle signal. The technical effect is accurate emulation of heart sounds. The respiratory information is obtained simultaneously during measurement of the set of heart rate sound samples such that timing of the respiratory information is relatable with timing of the heartbeat data of the person. Optionally, a prerecorded audio file comprising heart rate sound samples that is not associated with the same person from whom the first breathing cycle signal is obtained, is employed to obtain the respiratory information. Preferably, the set of heart rate sound samples are recorded with a stethoscope and added to the obtained respiratory information (comprising data associated with movement of chest) such that the set of heart rate sound samples are recorded from the same person than the first breathing cycle is measured. Further, the heart rate sound samples are tagged (such as, indexed or associated) with the first phase of the breathing cycle signal to determine the phase of the respiration cycle during which the heart rate sound sample originated. For example, when six heart rate sound samples originated between two consecutive inhalations of a person, a heart rate sound sample is tagged as belonging to a second phase of six total phases (corresponding to the six heart rate sound samples). In another example, when eight heart rate sound samples originated between two consecutive inhalations of a person, a heart rate sound sample is tagged as belonging to a seventh phase of the eight total phases. Such tagging of the heart rate sound samples is important because the heart rate sound samples experience changes according to respiration owing to lungs of the person "compressing" the heart of the person when the lungs are full and allowing the heart to "expand" when the lungs are empty. In general, respiration modulates heart sound both with regards of amplitude and frequency content, and the process described here provides that these respiratory related modulations will be adequately transferred into the resulting audio file. In this context, it may be appreciated that each heart rate sound sample is associated with respective second phase (from amongst the plurality of second phases of the second breathing cycle signal) and is used to synchronize the heart sounds and breathing rhythms. A technical advantage is the reproduction of heart rate sound samples recorded under different conditions. For example, heart sounds recorded while a parent is sleeping can be synchronized with corresponding respiratory data to create a comforting and familiar auditory environment for the child. Another advantage is the incorporation of relative timing information, ensuring that the audio file (containing the heart rate sound samples) remains synchronized with the natural breathing rhythm of a parent, caregiver, or guardian. This synchronization enhances realism, making the auditory experience more immersive and reassuring for the child.

[0031] Moreover, the relative timing information of each of the heart rate sound samples of the set is obtained in respect to the first phase of the first breathing cycle signal. For example, during operation, five different first breathing cycle signals are obtained such that the successive first breathing cycle signals last for 3.94 seconds, 3.78 seconds, 3.96 seconds, 5.13 seconds and 4.25 seconds. It will be appreciated that the heartbeat would correspond to different locations of the first breathing cycle signal. Also, sound characteristics such as type, amplitude and duration of the heartbeat represented as a digital or analogue representation (or "heart rate signal", described herein later) would change depending on the phase of the first breathing cycle signal owing to compression or expansion of the heart. For example, the first breathing cycle signal corresponds to a respiration cycle of 5.13 seconds. Further, the heart rate sound samples of the person are determined using an accelerometer associated with a smartphone placed on the upper chest of the person because a stethoscope is not be normally available in non-healthcare settings. Further, a first derivative of an accelerometer signal obtained from the accelerometer along with a low-pass filter are employed to determine a duration of the heartbeat intervals and a location of the heartbeats with respect to phases of the first breathing cycle signal (such as, in terms of sextiles or 360° divided into 6 equal segments of 60° each). In such an example, the heartbeats fall into sextiles 1, 2, 3, 4, 5 and 1 based on locations thereof in the first breathing cycle signal (or first heartbeat encompasses 0-60°, second heartbeat encompasses 60- 120°, third heartbeat encompasses 120-180°, fourth heartbeat encompasses 180-240°, fifth heartbeat encompasses 240-300° and sixth heartbeat encompasses 300-360°). The sixth heartbeat is associated with a contraction that occurs soon after beginning of a subsequent respiration (such as, inhalation) and is therefore considered to fall within sextile 1 of a corresponding breathing cycle signal. Thereafter, a number of heartbeats based on the duration of the heart rate sound samples (such as, 6 heartbeats when the heart rate sound samples are split into sextiles) are selected from the set of heart rate sound samples so that a beginning of a first heartbeat corresponds to a peak point of the respiration cycle (such that the peak point corresponds to inhalation) and an ending of a last heartbeat corresponds a successive peak point of the respiration cycle (such that the peak point corresponds to exhalation).

[0032] Further, the method comprises measuring a second breathing cycle signal and a heart rate signal. The term "heart rate signal" as used throughout the present disclosure relates to data associated with number of heartbeats of the person per unit time, inter beat intervals (intervals between two consecutive heart beats) and frequency of heart beats (beats per minute "bpm"). It may be appreciated that the "set of heart rate sound samples" and the "heart rate signal are distinct concepts that form the input and output for further analysis, such as detecting arrhythmias or calculating heart rate variability (HRV). The "set of heart rate sound samples" is defined as discrete, individual sound data points captured at specific intervals (e.g., raw data from a sensor or a microphone), at a high frequency. The "heart rate signal" is defined as a processed or continuous signal derived from the set of sound samples, representing the heart rate over time, such as by employing signal processing techniques. In this regard, the frequency of heartbeats is determined and subsequently analysed using, for example, band-pass filtering, derivation and threshold detection. For example, the measurement of the heart rate signal is performed by employing bandpass filtering and peak value detection. The measurement of the second breathing cycle signal is performed using gyroscope data obtained from the smartphone. The measurement of the second breathing cycle signal and the heart rate signal enables determination of the respiration cycles and the heartbeats of the person in a way that those can be synchronized and associated with each other's. This ensures that the emulated heart sounds are personalized and reflective of actual physiological states of the person. For example, the measured second breathing cycle signal enables personalization of the emulated heart sounds (or feel-scape) for the child based on personal characteristics of the person from whom the information is measured, such that the personal characteristics include but is not limited to, age, size, chest size, heart pumping power, number of heartbeats of the person during one respiration cycle at rest, heart rate, heart rate variation, breathing frequency, breathing frequency variation and breath-induced modulation in how strongly the heartbeat is felt and heard in different phases of the respiration cycle. Consequently, the second breathing cycle signal is measured to correspond to at least one respiration cycle and the heart rate signal is measured during the said at least one respiration cycle. In preferred embodiment number of respiration cycles is more than one for example 10, 20, or more respiration cycles. During said time of 10, 20 etc respiration cycles, the related heart related signal is also measured. This way it is feasible to obtain reliably the personal characteristics of the person.

[0033] Optionally, the heart rate signal is measured with respect to the breathing cycle signal. This way heart rate signal can be associated with the breathing cycle. Breathing cycle can refer to duration between inhalation or beginning of the respiration to exhalation or ending of the respiration for the person. The second breathing cycle signal is measured, for example, similarly to obtaining the first breathing cycle signal, such as, by placing the smartphone under the neck or on the upper chest of the person to determine the movement of the chest. Optionally, an orientation of the smartphone with respect to the shoulder line of the person is altered to obtain the second breathing cycle signal. For example, the smartphone is placed under the neck or on the upper chest of the person such that the smartphone is located parallel to a head-to- toe line of the person. It will be appreciated that the emotional state of the person during the measurement of the second breathing cycle signal and / or the heart rate signal will impact the feel-scape generated for the child (explained in detail herein later). Optionally, the method comprises providing feedback to the person subsequently to measuring the second breathing cycle signal and heart rate signal. Such feedback is associated with, for example, an emotional state of the person, thus allowing the person to adjust the emotional state such as by becoming more relaxed or calmer before reattempting to measure the second breathing cycle signal and / or the heart rate signal.

[0034] Further, the heart rate signal is measured as the ratio between a duration of the respiration cycle and the number of heartbeats comprised within the duration. For example, when the heart rate of the person is 72 beats per minute and a number of respirations is 12 breaths per minute, the heart rate signal for the person corresponds to 6 heartbeats per respiration cycle. Generally, the heart rate for different people at rest is in a range of 3 to 9 heartbeats per respiration cycle. In one example, false detections of heart beats are ruled out using median filtering. This way one can rule out false detections that may arise due to movement artifacts or extra beats. Optionally, such determination of the heartbeats further comprises ensuring that z-direction acceleration (i.e., zero-mean) remains below 500 milli-g (where "g" refers to a standard earth gravity). For example, the person will look for a suitable position to rest and a suitable location to place the smartphone on the upper chest or under the neck prior to commencement of the measurement. Therefore, when the measurement of heart rate signal is carried out at a value above 500 milli-g of the z-direction acceleration, then both the person and the smartphone will move, providing erroneous data. Therefore, the heart rate signal is measured when the z-direction acceleration is of a value below 500 milli-g. Optionally, the measurement is automatically commenced only after such z-direction acceleration has calmed down or stopped completely. Consequently, a duration comprising movement soon after commencement of the measurement is rejected, such as, based on a number of acceleration peaks measured in the z-direction that exceed 300 milli-g when a long-term average of a same acceleration component has been subtracted from the acceleration component along the z-direction. In an example, the person from whom the heart rate signal is measured, moved during the first 30 seconds before settling down. In such an example, acceleration in the z-direction is subtracted for the first 30 seconds. Optionally, a notification is provided to the person who is still, for example, for 15-40 seconds, if the heart rate signal and / or breathing cycle signal cannot be reliably measured. For example, the notification is provided to reattempt the measurement of the heart rate signal if less than a pre-determined portion of the heartbeats (such as, less than 60-80%) is measured to be reliable, the person fails to reach a relaxed or restful state within a provided duration (such as, for 30 seconds or 1 minute) or if no two respiration cycles are reliably identified within a first 30 seconds such that the respiration cycles are associated with a same size as those within subsequent 2-3 seconds. It will be appreciated that such unreliability of the measured heart rate signal is detected based on, for example, relaxation or restfulness of the person during the measurement determined using heart rate variation. Optionally, the measurement of the heartbeats is started only when the movement of the chest of the person has not exceeded a value of 300 milli-g for five seconds. It will be appreciated that the median criterion is selected to be lower if the person is over 50 years old or if the heart rate is over 80 bpm. Optionally, the median criterion gradually changes with age and / or the heart rate of the person increases. More optionally, if an unreliable heart rate is determined, the heart rate signal is rejected. The technical effects of incorporating z-direction acceleration monitoring and the application of a median criterion during heart rate signal measurement include enhanced measurement accuracy, significant reduction in false detections and errors caused by motion artifacts, and improved adaptability to the individual physiological characteristics of the person.

[0035] Moreover, the method comprises associating, using the relative timing information and the measured second breathing cycle signal, each of the heart rate sound samples of the set to a second phase of the second breathing cycle signal that corresponds to a respective first phase of the first breathing cycle signal. Herein, the method comprises associating, using the relative timing information and the measured second breathing cycle signal, each of the heart rate sound samples of the set to a respective second phase of a plurality of second phases of the second breathing cycle signal, wherein the respective second phase corresponds to the respective first phase of the first breathing cycle signal. Herein, the step of "associating" involves linking or mapping individual heart rate sound samples to specific phases of the second breathing cycle signal, to establish a relationship between each heart rate sound sample and corresponding phase of the breathing cycle. Moreover, the step of associating ensures organizing or categorizing the heart rate sound samples based on their timing and phase alignment with the breathing cycle signals. Such associating enables to create a simulation of the heartbeats with respect to the respiration cycles of the person. The created audio file enables the child to sense the parent or another trusted individual in an auditory (and optionally, haptic) manner even when the parent or individual is not physically present near the child, allowing to calm or soothe the child whenever needed. The association further enables to locate the obtained heart rate sound samples of the set based on the personal characteristics of the person, thereby, enabling to reliably emulate the respiration and heartbeat patterns for the child. For example, the association of the heart rate sound samples to the second phase of the second breathing cycle signal enables to create or update different feel-scape as and when need, such as, shortly after birth when the child becomes familiar with lower heart rate of the mother as compared to heart rate of the mother during pregnancy because resting heart rate of the mother drops after giving birth compared to the resting heart rate during the pregnancy. Similarly, the heart rate sound samples are associated with the respiration and heartbeat patterns of the father of the child to maximize feelings of familiarity. Subsequently, the association is updated by generating new feel-scape associated with a new embrace or a hug in various emotional states from time to time. For example, the association of the heart rate sound samples to the second phase of the second breathing cycle signal enables to use personal characteristics of the person such as heart rate variation and breathing frequency during different emotional states such as relaxation, excitement or overwhelm, thereby, enabling to personalize the interaction between the person and the child.

[0036] The method comprises merging the heart rate sound samples based on the association to create the audio file, wherein timing of each of the heart rate sound samples is aligned with the heart rate signal. In this regard, "merging" involves combining the associated heart rate sound samples into a single audio file, thereby integrating the individual samples into a cohesive output in order to create a unified audio file where the timing of each heart rate sound sample is aligned with the heart rate signal, ensuring a synchronized and continuous representation. The merging of the heart rate sound samples is performed such that original dynamics of heartbeats of the person with whom the second breathing cycle signal is associated are preserved. It will be appreciated that abrupt movements arising from coughing or heart rhythm disturbances (or arrythmia) do not lead to transient changes in the heart rate sound samples. Optionally, such changes are removed prior to merging the heart rate sound samples to ensure absence of discontinuities in modulation of respiration and strong non-physiological impulses in the heartbeat timing information. Further, breathing-induced changes in amplitude and waveform of the merged heart rate sound samples are retained in the created audio file to ensure that changes in the respiration cycle are preserved. Consequently, the created audio file only comprises respiration and heartbeat information of the person in a relaxed state and does not comprise any changes associated with the abrupt movements of the person.

[0037] The audio file is created by merging the heart rate sound samples based on the association of the heart rate sound samples with the second phase of the second breathing cycle signal. It may be appreciated that each of the set of heart rate sound samples is corelated with the measured heart rate signal thereby ensuring that the audio file created accurately replicates the feel-scape of the person to soothe the child. The created audio file comprises the heart rate sound samples that are reliably aligned with the heart rate signal, thus, ensuring that when the audio file is played back, the audio file emulates the heart sounds of the person according to the personal characteristics thereof. Consequently, playing back of the audio file enables to generate the feel-scape for the child of being held, nursed or cradled by another trusted individual such as a caretaker, a babysitter, a relative of the child and the like, to have a required impact on the child. Indeed, this way a set of heart rate sound samples which are obtained (or recorded) from a first person can be used to create an audio file which emulates heart sounds of a second person. In this emulated audio file, the heart rate sound samples of the first person are synchronized to breathing cycle (and heart rate signal) of the second person. In this audio file heart rate sound samples from the first person are played back essentially during same phase of breathing cycles. In practical terms a first audio sample of the set of heart rate audio samples is recorded from the first person at a start of inhaling. The first audio sample is associated with same phase (i.e., start of the inhaling) of the second person. This way heart rate sound samples are played back in a way that breathing cycle of the second person is taken in consideration. Technical benefit of this method is that we do not need to record audio samples of the second person heart sounds. Recording of the audio samples requires special equipment (such as high-quality stethoscope) which might not be available for the second person. Indeed, we generate audio file which provides natural variations of the heart sounds as those are function of chest movements / haptics.

[0038] Optionally, the created audio file enables adjustment of various components associated with auditory sensation (such as, below or above audible frequencies) and optionally, haptic sensation, of the feel-scape that the child is subjected thereto. Such adjustment is performed manually by the trusted individual or automatically based on the required impact to be had on the child based on personal characteristics of the person (such as, regularity of movement, heartbeat, respiration and the like), personal characteristics of the child (such as, age of the child, developmental phase, mood of the child and the like), behaviour of the child (for example, based on movement of the child detected using ballistographic sensors disposed under mattress or cradle of the child) or a desired impact for the child (such as, relaxation, activation, soothing and the like). For example, the audio file is played back to the child according to one of multiple operating modes including a relaxation mode, an activation mode and the like. In another example, the adjustment is automatically performed based on a quality of a support structure (such as, mattress or cradle) that is child is lying in, biological signals measured from the child and the like.

[0039] In one embodiment, obtaining the set of heart rate sound samples and the relative timing information comprises:

[0040] - recording heart rate sound samples during measuring the first breathing cycle signal;

[0041] - splitting the recorded heart rate sounds to obtain the heart rate sound samples; and

[0042] - creating the relative timing information for each of the heart rate sound samples in respect to the first breathing cycle signal by allocating the heart rate sound samples to a respective first phase of the first breathing cycle signal during which said heart rate sound sample was recorded.

[0043] The heart rate sound samples are recorded simultaneously with the measurement of the first breathing cycle signal. The heart rate sound samples and related first breathing cycle signal are recorded from a first person in one embodiment. For example, the smartphone comprising the gyroscope and / or accelerometer is placed under the neck or on the upper chest of the person to measure the first breathing cycle signal. Simultaneously, the stethoscope is employed to record the heart rate sound samples of the person. Subsequently, the recorded heart rate sounds are split to obtain individual heart rate sound samples, such that each individual heart rate sound sample is associated with a heartbeat. Thereafter, the relative timing information for each heartbeat sound sample is created, such as, by using a beginning of each heartbeat to determine the sextile of the respiration cycle to which the heartbeat corresponds. Such a determination is performed by splitting the respiration cycle into six equal durations and subsequently checking where the beginning of the heartbeat falls. Optionally, a representative example of the respiration cycle is selected. Moreover, the heartbeats that correspond to sextiles 1 and 6 are ensured to be longer on average (or removed median) than the heartbeats corresponding to sextiles 3 and 4 owing to respiratory sinus arrythmia and because different people will arrange the smartphone in different directions on the upper chest or below the neck (for example, even if the longitudinal direction of the smartphone is parallel to the shoulder line, the left-right and top-bottom edges of the smartphone will be oriented along different directions). However, if the heartbeats corresponding to the sextiles 1 and 6 are not determined to be longer, the phases of the breathing cycle signals are redefined by replacing sextile 4 with sextile 1, sextile 5 with sextile 2, sextile 6 with sextile 3, sextile 1 with sextile 4, sextile 5 with sextile 5 and sextile 3 with sextile 6. Such a replacement enables to ensure that the first sextile corresponds to the inhalation phase subsequent to exhalation of a preceding respiration cycle (i.e., when the heartbeat is longest). Optionally, a duration is selected from the first breathing cycle signal that extends from valley to valley (for example, by multiplying the signal by -1 and repeating the aforesaid process). Optionally, unreliable information of heartbeats is removed from the obtained set of heart rate sound samples, thereby, ensuring that the first breathing cycle signal does not comprise abrupt changes therein. For example, the method comprises detecting heart rate sound samples that are unreliable. Further, when such an unreliable heart rate sound sample is detected, the unreliable heart rate sound sample is removed (or rejected) from the set of heart rate sound samples. As an example, if more than one heartbeat within a breathing cycle is detected to be unreliable, the entire breathing cycle might be removed. If only one heartbeat is detected as unreliable, we could remove only that one unreliable heartbeat cycle since one deleted beat does not necessarily spoil the breathing cycle "experience", or delete the entire breathing cycle even then e.g. if there are fewer than 5 heart beats in that particular breathing cycle.

[0044] In another embodiment, each of the heart rate sound samples comprise a sound related to an opening sound (SI) of heart valve and a sound related to a closing sound (S2) of the heart valve. The sound related to the opening sound (SI) and the sound related to the closing sound (S2) of the heartbeat are separated by a duration that varies from heartbeat to heartbeat. Notably, the sound related to the opening of the heart valve is the 'lub' sound, while the closing sound is the 'dub' sound. In one example, the set of heartbeat sound samples comprises 6 heartbeats and one respiration cycle associated with a duration of 6 seconds. Further, the amplitudes of the heartbeats are scaled between -1 to 1. Further, each of SI and S2 last for approximately 100 milliseconds and the duration between the SI and S2 is between 100-300 milliseconds depending on the heart rate. Typically, higher heart rate is associated with a shorter duration between SI and S2. The technical effect of capturing the opening and closing sounds is accurate replication of heart sounds of the person.

[0045] In yet another embodiment, each of the first and the second breathing cycle signals is associated with six phases. For example, each of the first breathing cycle signal and the second breathing cycle signal comprises six phases, such as, 360° divided into equally long 60° phases. The splitting of the first breathing cycle signal and the second breathing cycle signal into the six phases enable reliable determination of location of the heartbeat relative to a portion of the respiration cycle (such as, inhalation or exhalation). Optionally, the first breathing cycle signal and the second breathing cycle signal is split into any number of phases. For example, the first breathing cycle signal and the second breathing cycle signal is split into 360 phases of 1° each, 36 phases of 10° each, 9 phases of 40° each, 12 phases of 30°, 6 phases of 60° each and the like. Alternatively, number of phases can be 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14 ,15, 16, 17, 18, 19, 20 or higher depending on needed resolution. Benefit of 6 phases is that this way we get highest probability of having one heartbeat per each phase, thus making association and using the relative timing when association the heart rate sound samples easier. The technical advantage of splitting the first and second breathing cycle signals into six phases (e.g., dividing the 360° breathing cycle into equally long 60° phases) lies in the precise synchronization and accurate representation of the relationship between heartbeats and the breathing cycle.

[0046] Moreover, heartbeats are not uniformly distributed across the breathing cycle, and therefore, relative timing and characteristics (e.g., amplitude and frequency) of heart sound samples also vary depending on whether the person is inhaling or exhaling. By associating each heart sound sample with a specific phase (namely, first phase and second phase), such variations can be accurately captured. Another technical effect is that any change in heart sounds caused by breathing (e.g., compression of the heart during inhalation and expansion during exhalation) can be accurately captured and reproduced.

[0047] In an embodiment, during merging, separating in time domain, sounds related to the opening and closing from each other's depending on a heart rate determined from the heart rate signal, wherein the separation is a function of the heart rate. In other words, during merging, sounds related to the opening sound and closing sound are separated in time domain from each other depending on a heart rate determined from the heart rate signal, wherein the separation in time domain is a function of the heart rate. The sounds associated with the opening sound (SI) of the heart valve are separated from the sounds associated with the closing sound (S2) of the heart valve based on the heart rate determined from the heart rate signal. The extent of separation in time domain is a function of the heart rate. For example, the set of heartbeat sound samples comprises 6 heartbeats and one respiration cycle associated with a duration of 6 seconds. Moreover, each of SI and S2 last for approximately 120 milliseconds. The SI and S2 are separated from each other such that SI and S2 are separated by a quieter period of 100-300 milliseconds depending on the heart rate.

[0048] In one embodiment, the obtained set of heart rate sound samples and the first breathing cycle signal are from a first person and the measured second breathing cycle signal and the heart rate signal are from a second person and wherein the second person is different from the first person. The measured second breathing cycle signal and the heart rate signal associated with the second person are matched to the obtained set of heart rate sound samples and the first breathing cycle signal associated with the first person, such as, using the relative timing information and / or information of the second phase of the second breathing cycle signal and the first phase of the first breathing cycle signal. It will be appreciated that when the created audio file is played back to a newborn child to emulate the heart sounds, the heart rate sound samples are obtained from a person unknown to the child (such as, using an online marketplace). However, since the heart rate sounds are associated with the second breathing cycle (of the second person) the created audio file will emulate mothers (or whoever has been providing the second breathing cycle) heart rate sound environment to a child in a surprisingly satisfactory manner. Indeed, although, the child will have become accustomed to the heartbeats and respiration cycles of the mother of the child during a gestation period. Consequently, even if the heart rate sound samples obtained from the one or more unknown persons are used as such, the heart rate sound samples would be unsuitable to be played back to the child that is accustomed to the timing of heartbeats and respiration cycles of the mother because a mutual connection between the child and the mother would be different from that between the child and the unknown person. Consequently, when the created audio file is employed to emulate the heart sound to the child with said association to breathing cycles of the second person (mother for example) the created audio file provides added connective (such as, soothing, relaxing and / or trusting) value to the child. This technical effect is due to the association of each recorded heart rate audio sample (such as a single beat) to a first breathing cycle (of same person from which the audio sample was recorded). This is needed as the heart rate audio changes during breathing cycle. In practical terms when person inhales it compresses heart slightly affecting its audio "output". When person out hales the heart can beat a bit more freely and audio is changed accordingly. In other words, heart sounds sound different depending on phase of breathing. Recorded audio samples for each breathing cycle are associated / timed with breathing cycle of second person (mother) thus allowing to emulate right type of audio for heart sound, i.e., when mother is inhaling corresponding audio samples (recorded at time of inhaling from the first person) are played back. Respectively when, the mother is out hailing corresponding audio samples (recorded at time of out hailing from the first person) are played back.

[0049] In yet another embodiment, the second person is one of a mother, a father, a surrogate of a child and the method further comprises playing the created audio file to the child to emulate the heart sounds of the second person. In one example, the second person is the mother of the child, and the created audio file is played to the child when the child is laid into a cradle to enable calming and soothing to be provided to the child when the mother is unavailable to hold (such as, hug), nurse or cradle the child in her arms or lap. In another example, a babysitter of the child plays the audio file to the child to emulate the heart sounds of the mother or the father of the child.

[0050] In an embodiment, during the merging of the heart rate sound samples, an audio noise is added between the heart rate sound samples if a heart rate obtained from the heart rate signal is lower than a rate of the heart rate sound samples, to obtain a continuous audio file. Optionally, one or more heart rate sound samples are shorter than a previous or a subsequent heart rate sound sample, thereby, potentially leading to formation of a discontinuous audio file. For example, when the obtained heart rate sound samples comprise 6 heart rate sound samples, a heart rate sound sample 4 of the 6 heart rate sound samples are shorter than the other heart rate sound samples. In such an example, the audio noise is added after the heart rate sound sample 4 to extend the heart rate sound sample 4 to a length corresponding to the other heart rate sound samples. Further, a length of the audio noise added to the heart rate sound sample 4 is equal to a difference between the length of another heart rate sound sample (such as, heart rate sound sample 3 or heart rate sound sample 5) and the heart rate sound sample 4. For example, when the 6 heart rate sound samples are recorded, each of the heart rate sound samples 1, 2, 3, 5 and 6 are associated with a length of 1000 milliseconds. However, the heart rate sound sample 4 is associated with a length of 980 milliseconds. In such an example, an audio noise of length 20 milliseconds is added to the heart rate sound sample 4. Consequently, when the 6 heart rate sound samples are merged to create the audio file, a continuous audio file is created. Optionally, the audio noise is recorded during obtaining the heart rate sound samples. Typically, heart rate sound samples are between 800-1000 milliseconds (corresponding to heart rates of 75 to 60 bpm (beats per minute)).

[0051] In another embodiment, an audio noise is added between the opening sound (SI) and the closing sound (S2) if the separation is smaller than in the heart rate sound samples. Herein, the term "separation" refers to the separation in time domain, as explained above. In other words, if the separation in time domain is larger in the heart rate sound samples than in recorded sound sample then opening sound (SI) and closing sound (S2) are separated further away (in time domain) from each other's. Created "empty gap" is filled with audio noise. I.e., the audio noise is added in this example after opening sound (SI) and before closing sound (S2). The addition of the audio noise between the opening sound (SI) and the closing sound (S2) enables to ensure that the opening sound (SI) and the closing sound (S2) are adequately separated from each other. Consequently, the closing sound (S2) is accurately heard after the end of the opening sound (SI) and the opening sound (SI) of the subsequent heartbeat is heard after the end of the closing sound (S2) of the previous heartbeat.

[0052] In yet another embodiment, fading functions are applied when merging the opening (SI) and closing sounds (S2), heart rate sound samples and / or the audio noise. The application of the fading functions enable to ensure smooth transitions between the opening sound (SI) and closing sound (S2) of individual heartbeats, closing sound (S2) and opening sound (SI) of adjacent heartbeats as well as the audio noise that is added between the opening sounds (SI) and closing sounds (S2).

[0053] The present disclosure also relates to a system for emulating heart sounds. Various embodiments and variants disclosed above, with respect to the aforementioned method, apply mutatis mutandis to the system. T1

[0054] In another aspect, the present disclosure provides a system for emulating heart sounds, the system comprising

[0055] - a first apparatus for measuring a second breathing cycle signal and a heart rate signal from a second person;

[0056] - a processing system configured to: store a set of heart rate sound samples and a relative timing information of each of the heart rate sound samples of the set in respect to a first breathing cycle signal; obtain, from the first apparatus, the measured second breathing cycle signal and the heart rate signal; create from the stored set of heart rate sound samples and the relative timing information, an audio file to emulate the heart rate sounds of the second person; and provide the created audio file to a second apparatus; and

[0057] - the second apparatus configured to: receive the created audio file; and play the audio file to emulate the heart sounds.

[0058] The term "first apparatus" as used throughout the present disclosure refers to an electronic or electromechanical device that enables a person to determine characteristics associated with the respiration cycle of the person, such as, chest movement of the person during inhalation and exhalation. For example, the first apparatus comprises a gyroscope, an accelerometer, an inertial measurement unit and the like that enables to determine the chest movement of the person during inhalation and exhalation. The first apparatus further enables to determine the heart rate signal of the second person using for example accelerometer of the first apparatus (or, heart rate monitoring sensor, such as photoplethysmography (PPG), of the first apparatus). For example, the first apparatus is selected from a heart rate monitor, a fitness tracker, a chest-band device, smart phone with an accelerometer or gyroscope and the like that enables the person to determine the heart rate signal. It will be appreciated that a newborn child is able to sense when affection is provided to the newborn child by a close person, such as, by being held, nursed or cradled in arms or lap of the close person. Further, such a sensory experience of the newborn child only partially possessing sensory capabilities such as visual or auditory identification, sense of speech and the like, comprises of the newborn child sensing the heartbeat and breathing patterns of the close person. The first apparatus enables to capture such a sensory experience (or feel-scape) for the newborn child by enabling creation of the audio file that emulates the heart sound of the second person to be played back to the newborn child as and when needed (such as, before sleep time).

[0059] The term "processing system" as used throughout the present disclosure refers to an electronic component capable of storing information (such as the set of heart rate sound samples and the relative timing information of each of the set of heart rate sound samples in respect to a first breathing cycle signal (as measured from a first person), receiving raw data measured by the first apparatus (such as the second breathing signal and / or the heart rate signal from the second person), analysing the stored information and the received raw data to create an audio file to emulate the heart sounds (of the second person) and subsequently, transmitting the created audio file to another apparatus (such as the second apparatus). Optionally, the processing system is capable of storing data or information (such as the set of heart rate sound samples and the relative timing information of each of the set of heart rate sound samples). In one example, the processing system is implemented as a cloud processor. In such an example, the processing system comprises a cloud database that is operatively connected / coupled with the cloud processor, such that the cloud database is capable of storing the information.

[0060] The term "second apparatus" as used throughout the present disclosure refers to an electronic or electromechanical device that enables to play back the audio file created using the first apparatus to emulate the heart sounds for the newborn child. The second apparatus is capable of outputting a combined signal, such as, by reproducing both auditory frequencies (such as, frequencies that can be sensed by hearing) and haptic frequencies (such as, frequencies that can be sensed by touch). The second apparatus generates the feel-scape of being held, nursed or cradled by the close person and can therefore be placed in immediate proximity of the newborn child, for example, under a mattress, a cradle and the like. The second apparatus typically comprises a speaker to output audio.

[0061] In one embodiment, when creating the audio file, the processing system is configured to: use the relative timing information and the measured second breathing cycle signal to associate each of the heart rate sound samples of the set to a second phase of the second breathing cycle signal that corresponds to a respective first phase of the first breathing cycle signal; and merge the heart rate sound samples based on the association to create the audio file, wherein timing of each of the heart rate sound samples is aligned with the heart rate signal.

[0062] Herein, the processing system is configured to use the set of heart rate sound samples, namely, corresponding to the second breathing cycle signal, and the relative timing information of each of the heart rate sound samples of the set, in respect to the first breathing cycle signal. Subsequently, the audio file to emulate the heart rate sounds of the second person is created.

[0063] In another embodiment, the first apparatus is a smartphone and wherein the second breathing cycle signal and the heart rate signal are measured with at least one of: an accelerometer, an inertial measurement unit and / or a microphone of the smartphone. Such an implementation of the first apparatus as the smartphone enables convenient measurement of the second breathing cycle signal because most modern smartphones comprise a motion sensor such as a gyroscope, an accelerometer and / or an inertial measurement unit capable of measuring the movement associated with the chest of the person. Consequently, when the first apparatus is implemented as the smartphone, the smartphone is further capable of measuring the heart rate signal using the microphone of the smartphone. It will be appreciated that when the set of heart rate signals are recorded using a microphone of a smartphone, the microphone will record noise associated with pumping action of the heart (such as, sound of blood flow and opening and closing sounds of heart valve). Further, the microphone will record noise associated rustling of clothing or erratic movements of the smartphone on body of the person. Optionally, the smartphone comprises one or more motion sensors capable of recording acceleration signals (such as, within a range of 1-30 Hz) that can be haptically sensed, for example, a 100 Hz sampling frequency. However, such acceleration signals lack audible frequencies (such as, frequencies above 25 Hz). In practice frequencies below 25Hz comprise haptics of the second person (i.e., movement data). Since this haptic data is below 25Hz and on the other hand the set of heart rate sounds samples (from a first person) are in different frequency domain their merging as single set can be done without interference.

[0064] In yet another embodiment, the second apparatus is a speaker arrangement of at least one of: a smart pillow, a mattress or a cradle. The speaker arrangement of the smart pillow, the mattress or the cradle enables to safely play the audio file in close proximity to the newborn child, thereby, enabling the newborn child to acquire auditory sensations associated with the feel-scape provided by the heart sounds. Optionally, the smart pillow, the mattress or the cradle comprises an actuator that is capable of producing frequencies for additionally providing haptic sensations to the newborn child.

[0065] In an embodiment, the set of heart rate sound samples is recorded with a stethoscope from a first person. Such recording of the heart rate sound samples using the stethoscope enables reliable detection and recording of the heart rate sound samples such that the created audio file enables accurate emulation of the heart sounds of the first person. Optionally, the first person is a mother, a father or a surrogate of a child (such as a grandparent, a legally appointed guardian and the like). It will be appreciated that the heart rate sound samples can be accurately recorded using the stethoscope. However, the stethoscope is ordinarily unavailable outside of healthcare settings (such as, in homes, daycare, or any such establishment). Therefore, in a preferred embodiment the set of heart rate audio samples are recorded by a person different from using the system since a precise audio recording requires a special set up. Optionally, the stethoscope may be implemented as a digital stethoscope. In such case, the stethoscope is communicably coupled to the processing system, to transfer the recorded the heart rate sound samples thereto.

[0066] In an embodiment, the processing system is configured to store heart rate sound samples recorded with a stethoscope from a first person, wherein the second person is different from the first person. In this regard, the heart rate sound samples recorded are stored in a data repository communicably coupled to the processing system. Such stored data can be accessed remotely at the non-healthcare setting such as home, daycare, or any such establishment, thereby enabling replay of created audio file. It may be appreciated that the heart rate sound samples recorded for the second person different from the first person can also be recorded and stored in the data repository. Notably, recording devices such as a smartphone, can be used to record the set of heart sound samples and breathing cycle signals (i.e., the second breathing cycle signal) from the second person at the non-clinical setting.

[0067] As a summary a system and method for emulating heart sounds is provided. As a pre step before using the system or method a set of heart rate audio samples are recorded (or synthetised with audio synthetiser) from a first person. At the time of recording a first breathing cycle is measured also. Each audio sample is associated with relative timing information related to the breathing cycle.

[0068] When using system (or method) a first apparatus is used to measure a second breathing cycle signal and a heart rate signal from a second person. Example apparatus for this is a smart phone. The smart phone is placed on chest of the second person (mother of a child for example) and gyroscope and / or accelerometer of the smart phone measures signals related to the breathing and heart rate.

[0069] Stored (in pre step) the set of heart rate sound samples and a relative timing information of each of the heart rate sound samples of the set in respect to a first breathing cycle signal are used to create an audio file to emulate heart rate sounds of the second person. This is done by using the relative timing information of the stored heart rate sound samples by aligning those with the second breathing cycle signal phases and timing those with the heart rate beat of the second person.

[0070] Created audio file is provided for play back purposes to a second apparatus which can be child's bed. The second apparatus plays the audio to emulate the heart sounds of the second person. This provides technical effect of using a first person's heart sounds as starting point to make an audio file which resembles heart sounds of the second person without recording audio samples from the second person. This reduces complexity of the setup since it is challenging to obtain good, noise free audio recording of heart sounds without special equipment. In a one example the method for creating the audio file to emulate heart sounds is used to create the audio file. Thereafter the system for emulating heart sounds is used to play back the created audio file. In said example heart rate sound samples from a first person are used to create emulation of heart rate sounds of the second person.

[0071] DETAILED DESCRIPTION OF DRAWINGS

[0072] Referring to FIG. 1, there is shown an illustration of a flowchart of a method 100 for creating an audio file to emulate heart sounds, in accordance with an embodiment of the present disclosure. At a step 102, a set of heart rate sound samples and a relative timing information of each of the heart rate sound samples of the set are obtained in respect to a first phase of a first breathing cycle signal. At a step 104, a second breathing cycle signal and a heart rate signal are measured. At a step 106, using the relative timing information and the measured second breathing cycle signal, each of the heart rate sound samples of the set is associated to a second phase of the second breathing cycle signal that corresponds to a respective first phase of the first breathing cycle signal. At a step 108, the heart rate sound samples are merged based on the association to create the audio file, wherein timing of each of the heart rate sound samples is aligned with the heart rate signal. The first phase of the first breathing signal can be for example 0-60 degrees. In this scenario the second phase of the second breathing signal, is also 0-60 degrees (i.e., it corresponds to the respective first phase).

[0073] Referring to FIG. 2, there is shown an illustration of an amplitude (in arbitrary units) vs time graph 200 depicting a second breathing cycle signal 202 and heart rate signal 204, in accordance with an embodiment of the present disclosure. As shown, the graph 200 illustrates the breathing cycle signal 202 obtained over one respiration cycle spanning 0-360°. Further, the graph 200 illustrates the obtained heart rate signal 204 corresponding to the second breathing cycle signal 202. Six different phases (0-60, 60-120, 120-180, 180-240, 240-300 and 300-360 degrees) of the second breathing cycle are indicated in the figure. As an example, one can see that there is a single heartbeat taking place in phase 60-120 degrees.

[0074] Referring to FIG. 3, there is shown an illustration of an amplitude (in arbitrary units) vs time graph 300 depicting a set of heart rate sound samples 302, 304, in accordance with an embodiment of the present disclosure. As shown, each heart rate sound sample 302, 304 comprises a QRS wave followed by a T wave. Further, the set of heart rate sound samples 302, 304 are obtained with respect to a first breathing cycle signal 306. Moreover, an audio noise 308 can be seen between the heart rate sound samples 302, 304.

[0075] Referring to FIG. 4, there is shown an illustration of an amplitude vs time graph 400 depicting heart rate sound samples 402, in accordance with an embodiment of the present disclosure. As shown, wherein fading functions 404, 406 are applied when merging the heart rate sound samples 402.

[0076] Referring to FIG. 5, there is shown an illustration of a system 500 for emulating heart sounds, in accordance with an embodiment of the present disclosure. The system 500 comprises a first apparatus 502. As shown, the first apparatus 502 is implemented as a smartphone associated with a second person 504. The system 500 further comprises a processing system 506. The processing system 506 is implemented as a cloud server that comprises database and processing unit. The system 500 comprises a second apparatus 508. The second apparatus 508 is disposed with a baby stroller or cradle 510. FIG. 6 is an illustration of a second person 622 measuring her breathing cycle related signal 664 and heart rate signal with a first apparatus 624 or 622. Each breathing cycle 660 is divided to phases 662. An enlargement of created audio file 650 is presented in the figure. The audio file comprises a set of audio samples 650A, 650B, 650C, 650D, 650E and 650F. The samples are associated with respecting phase of the second breathing cycle signal. The audio samples are aligned with the heart rate signal from the second person. The audio samples in the figure correspond to timeline 652. An enlargement of one audio sample 650E is illustrated. The audio sample comprises opening sound SI of heart valve and a closing sound S2 of heart valve separated with time delta t. If original audio sample timing requires the first and second part might be separated more than in original audio file by adding some white noise between the sample parts. Alternatively, to the white noise, a noise from recorded heart rate samples can be used. The noise can be noise between opening SI and closing S2 or noise outside of closing and opening sounds.

[0077] Fig. 7 is an illustration of example of shifting sound samples. When shifting or tuning the sound signal from the first signal so that it matches in length with the heartbeat interval in question (856 ms in this case) in the second recording (such as the accelerometer recording from the chest), and where the interval between the opening sound (SI) and the closing sound (S2) is adjusted for the heartbeat interval. Sound 1 (solid line in the middle graph): the timing of the peak SI is matched with the peak of the accelerometer signal. Sound 2: the same signal as Sound 1 but the timing is shifted so that the timing of the peak S2 is matched with the expected delay after the peak of SI according to the heartbeat interval (IBI) in question: Delay = 0.16*IBI+0.17 s. To put it simply, if the original sound interval in the heart sound (that was selected according to the breathing phase) was shorter than the desired delay, Sound 2 will be shifted forward in time, and vice versa. Modifications to embodiments of the present disclosure described in the foregoing are possible without departing from the scope of the present disclosure as defined by the accompanying claims. Expressions such as "including", "comprising", "incorporating", "have", "is" used to describe and claim the present disclosure are intended to be construed in a nonexclusive manner, namely allowing for items, components or elements not explicitly described also to be present. Reference to the singular is also to be construed to relate to the plural.

Claims

CLAIMS1. A method for creating an audio file (650) to emulate heart sounds, the method comprising- obtaining a set of heart rate sound samples (302, 304, 402, 650A, 650B,650C, 650D, 650E, 650F) and a relative timing information (652) of each of the heart rate sound samples of the set in respect to a first phase of a first breathing cycle signal (306);- measuring a second breathing cycle signal (202, 664) and a heart rate signal (204);- associating, using the relative timing information and the measured second breathing cycle signal (202, 664), each of the heart rate sound samples of the set to a second phase of the second breathing cycle signal that corresponds to a respective first phase of the first breathing cycle signal; and- merging the heart rate sound samples based on the association to create the audio file, wherein timing of each of the heart rate sound samples is aligned with the heart rate signal.

2. A method according to claim 1, wherein obtaining the set of heart rate sound samples (302, 304, 402) and the relative timing information comprises:- recording heart rate sound samples (302, 304, 402) during measuring the first breathing cycle signal (202, 306);- splitting the recorded heart rate sounds (302, 304, 402) to obtain the heart rate sound samples (302, 304, 402); and- creating the relative timing information for each of the heart rate sound samples (302, 304, 402) in respect to the first breathing cycle signal(202, 306) by allocating the heart rate sound samples (302, 304, 402) to a respective first phase of the first breathing cycle signal (202, 306) during which said heart rate sound sample (302, 304, 402) was recorded.

3. A method according to any of the preceding claims 1 or 2, wherein each of the heart rate sound samples (302, 304, 402) comprise: a sound related to an opening sound (SI) of heart valve and a sound related to a closing sound (S2) of the heart valve.

4. A method according to any of the preceding claims, wherein each of the first and the second breathing cycle signals (202, 306, 202, 306) is associated with six phases.

5. A method according to any of the preceding claims 3 or 4, wherein during merging, separating in time domain, sounds related to the opening sound (SI) and closing sound (S2) from each other's depending on a heart rate determined from the heart rate signal (204), wherein the separation is a function of the heart rate.

6. A method according to any of the preceding claims, wherein the obtained set of heart rate sound samples (302, 304, 402) and the first breathing cycle signal (202, 306) are from a first person and the measured second breathing cycle signal (202, 306) and the heart rate signal (204) are from a second person (504) and wherein the second person (504) is different from the first person.

7. A method according to claim 6, wherein the second person (504) is one of: a mother, a father, a surrogate of a child and the method further comprises playing the created audio file to the child to emulate the heart sounds of the second person (504).

8. A method according to any of the preceding claims, wherein, during the merging of the heart rate sound samples (302, 304, 402), an audio noise (308) is added between the heart rate sound samples (302, 304,402) if a heart rate obtained from the heart rate signal (204) is lower than a rate of the heart rate sound samples (302, 304, 402), to obtain a continuous audio file.

9. A method according to any of the preceding claims 5-8, wherein an audio noise is added between the opening sound (SI) and the closing sound (S2) if the separation is larger than in the heart rate sound samples (302, 304, 402).

10. A method according to claim9, wherein fading functions (404, 406) are applied when merging the opening and closing sounds, heart rate sound samples (302, 304, 402) and / or the audio noise.

11. A system (500) for emulating heart sounds, the system comprising- a first apparatus (502, 624, 622) for measuring a second breathing cycle signal (202, 664) and a heart rate signal (204) from a second person (604, 504);- a processing system (506) configured to: store a set of heart rate sound samples (302, 304, 402, 650A, 650B, 650C, 650D, 650E and 650E), and a relative timing information of each of the heart rate sound samples of the set in respect to a first breathing cycle signal (202, 306); obtain, from the first apparatus, the measured second breathing cycle signal (202, 306) and the heart rate signal (204); create from the stored set of heart rate sound samples (302, 304, 402) and the relative timing information, an audio file (650) to emulate the heart rate sounds of the second person (504); and provide the created audio file to a second apparatus (508); andthe second apparatus (508) configured to: receive the created audio file; and play the audio file to emulate the heart sounds.

12. A system (500) according to claim 11, wherein, when creating the audio file, the processing system (506) is configured to: use the relative timing information and the measured second breathing cycle signal (202, 306) to associate each of the heart rate sound samples (302, 304, 402) of the set to a second phase of the second breathing cycle signal (202, 306) that corresponds to a respective first phase of the first breathing cycle signal (202, 306); and merge the heart rate sound samples (302, 304, 402) based on the association to create the audio file, wherein timing of each of the heart rate sound samples (302, 304, 402) is aligned with the heart rate signal (204).

13. A system (500) according to any of the preceding claims 11 or 12, wherein the first apparatus (502) is a smartphone and wherein the second breathing cycle signal (202, 306) and the heart rate signal (204) are measured with at least one of: an accelerometer, an inertial measurement unit and / or a microphone of the smartphone.

14. A system (500) according to any of the preceding claims 11-13, wherein the second apparatus (508) is a speaker arrangement of at least one of: a smart pillow, a mattress, a cradle (510).

15. A system (500) according to any of the preceding claims 11-14, wherein the set of heart rate sound samples (302, 304, 402) is recorded with a stethoscope from a first person.

Images