An audio playing method, an electronic device and a storage medium

Abstract

The application provides an audio playing method, an electronic device and a storage medium, and relates to the technical field of terminals. The method is applied to an electronic device configured with a breathing training task. The method comprises the following steps: in response to a triggering operation, the electronic device runs the breathing training task and outputs a task instruction to guide a user to perform breathing training; the electronic device matches, based on an acquired heart rate value of the user, a pre-stored corresponding relationship between heart rate values and heart sound audios, and plays a heart sound audio matched with the heart rate value of the user; then, after the electronic device acquires a changed heart rate value of the user in the breathing training task, the electronic device matches, based on the changed heart rate value, the corresponding relationship and plays a heart sound audio matched with the changed heart rate value. In this way, the electronic device can play a heart sound audio matched with the heart rate value of the user, so that the user can perceive the training effect of the breathing training performed by the user, real-time feedback of the breathing training is realized, and the user experience is effectively improved.

Description

Technical Field

[0001] This application relates to the field of terminal technology, and in particular to an audio playback method, electronic device and storage medium. Background Technology

[0002] With the rapid development of society, people are increasingly aware of the importance of breathing. Breathing training can adjust breathing and guide users into a fixed breathing pattern, so breathing training has become a common training method.

[0003] In related technologies, electronic devices such as mobile phones and smartwatches can play voice prompts or display images to guide users in breathing exercises, allowing users to adjust their breathing by following the voice or images. However, during the process of guiding users in breathing exercises, electronic devices generally play fixed instructions or display fixed and changing guidance images, lacking a feedback process on the effectiveness of breathing exercises, which can easily affect the user experience. Summary of the Invention

[0004] To address the aforementioned issues, this application provides an audio playback method, an electronic device, and a storage medium, aiming to resolve the problem of a lack of feedback on the effectiveness of breathing training during the training process and effectively improve the user experience.

[0005] Firstly, this application provides an audio playback method applicable to electronic devices, such as mobile phones, tablets, or wearable devices equipped with breathing training tasks. For example, when applied to a mobile phone, the phone may include an application (e.g., a sports and health application) equipped with a breathing training task; when applied to a wearable device such as a smartwatch, the smartwatch may be equipped with a breathing training task.

[0006] In this audio playback method, a user can trigger a breathing training task configured on the electronic device. In response to the user's trigger, the electronic device runs the breathing training task and can output task instructions to guide the user in breathing training. For example, it can output voice or changing images as task instructions to guide the user. When the electronic device is a mobile phone, it can acquire the user's heart rate value collected by a wearable device. When the electronic device is a wearable device, it can acquire the user's heart rate value collected by itself. Based on the user's heart rate value, the electronic device matches it against a pre-stored correspondence between heart rate values ​​and heart sound audio, and plays the matched heart sound audio corresponding to the user's heart rate value. The pre-stored heart rate value is positively correlated with the heart sound frequency of the corresponding heart sound audio; that is, the pre-stored... The higher the heart rate value, the higher the heart sound frequency of the corresponding heart sound audio. The lower the pre-stored heart rate value, the lower the heart sound frequency of the corresponding heart sound audio. For example, the heart sound frequency of the heart sound audio corresponding to a heart rate value of 80 is higher than that of the heart sound audio corresponding to a heart rate value of 78. In this breathing training task, when the user's heart rate value changes, the electronic device can obtain the user's changed heart rate value. Based on the changed heart rate value, it matches the pre-stored correspondence between heart rate values ​​and heart sound audio and plays the matched heart sound audio corresponding to the changed heart rate value. For example, when the user's heart rate value is 80, the electronic device plays the heart sound audio corresponding to heart rate value 80; when the user's changed heart rate value is 79, the electronic device plays the heart sound audio corresponding to heart rate value 79.

[0007] In this way, when an electronic device is running a breathing training task, it can not only guide the user to perform breathing training, but also play matching heart sound audio based on the user's heart rate value in the breathing training task, so that the user can perceive the training effect when performing breathing training, give the user real-time feedback, and thus improve the user experience.

[0008] In one possible implementation, the heart sound audio played by the electronic device is generated based on the user's actual heart sound. That is, in the pre-stored correspondence between heart rate values ​​and heart sound audio, all the stored heart sound audio is generated based on the user's actual heart sound. In this way, the heart sound audio played to the user is more closely matched to the user's actual heart sound, achieving personalized customization and further enhancing the user experience.

[0009] In one possible implementation, the aforementioned heart sound audio can be obtained through the following steps: The electronic device first acquires the actual heart sounds corresponding to the user at N heart rate values. For example, the electronic device can record the user's actual heart sounds or acquire them using tools such as an electronic stethoscope. Here, N is a positive integer; for example, N can be 4, 8, etc. The electronic device then performs curve fitting based on the correspondence between the N heart rate values ​​and the N actual heart sounds to obtain M heart sound audios corresponding one-to-one with M heart rate values. Here, M is an integer greater than N; for example, when N is 4, M can be 15, 20, etc. Thus, considering the variable changes in the user's heart rate in practical applications, using curve fitting to generate heart sound audios corresponding to a wider range of different heart rate values ​​based on the recorded actual heart sounds facilitates playing matching heart sound audios to the user and avoids acquiring more actual heart sounds multiple times, improving the user experience.

[0010] In one possible implementation, the steps for obtaining the M heart sound audios corresponding one-to-one with the M heart rate values ​​may include: the electronic device first analyzes the acquired N actual heart sounds to obtain the loudness values ​​and spectrograms corresponding to these N actual heart sounds; then, the electronic device performs curve fitting based on the correspondence between the N loudness values ​​and the N heart rate values ​​to obtain a first curve, the first curve including the correspondence between the M loudness values ​​and the M heart rate values; then, the electronic device performs curve fitting based on the correspondence between the N spectrograms and the N heart rate values ​​to obtain M spectrograms corresponding one-to-one with the M heart rate values; subsequently, for each of the M heart rate values, the electronic device generates the heart sound audio corresponding to that heart rate value based on the loudness value corresponding to that heart rate value in the first curve and the spectrogram corresponding to that heart rate value, thus obtaining the M heart sound audios corresponding one-to-one with the M heart rate values. For example, for the first heart rate value out of M heart rate values, the heart sound audio corresponding to the first heart rate value can be generated based on the loudness value corresponding to the first heart rate value in the first curve and the spectrum corresponding to the first heart rate value. Similarly, the heart sound audio corresponding to each of the M heart rate values ​​can be obtained. In this way, by analyzing the basic characteristics of actual heart sounds, namely loudness value and spectrum, more realistic heart sound audio can be generated, further improving the user experience.

[0011] In one possible implementation, the aforementioned spectrogram includes a correspondence between multiple frequencies and multiple amplitudes. The M spectrograms corresponding one-to-one with the M heart rate values ​​can be obtained through the following steps: First, for each of the multiple frequencies, the electronic device performs curve fitting based on the correspondence between the N amplitude values ​​and N heart rate values ​​at that frequency from the N spectrograms, obtaining a second curve corresponding to that frequency and including the correspondence between the M amplitude values ​​and the M heart rate values. This yields multiple second curves corresponding one-to-one with the multiple frequencies. Then, for each of the M heart rate values, the multiple frequencies are used as multiple horizontal axes, and the multiple amplitude values ​​of the multiple second curves at that heart rate value are used as multiple vertical axes corresponding one-to-one with the multiple horizontal axes, obtaining the spectrogram corresponding to that heart rate value. This yields spectrograms corresponding to each of the M heart rate values. Thus, by using curve fitting, a spectrogram that more closely resembles the actual heart sound is generated, which is beneficial for generating more realistic heart sound audio and thus improving the user experience.

[0012] In one possible implementation, the audio playback method may further include: if the electronic device determines that the user's heart rate remains unchanged within a first preset time period, it can determine a target heart sound audio with a lower heart rate frequency based on the heart sound audio that matches the user's heart rate from a pre-stored correspondence between heart rate values ​​and heart sound audio. For example, if the user's heart rate remains at 80 seconds for 10 seconds, the heart sound audio corresponding to a heart rate of 79 or 78 can be determined from the correspondence as the target heart sound audio. Subsequently, the electronic device no longer plays the heart sound audio that matches the user's heart rate, but instead plays the target heart sound audio with a lower heart rate frequency. In this way, the user perceives the heart sound audio corresponding to a heart rate lower than their own, further enhancing the guidance effect of breathing training.

[0013] In one possible implementation, the audio playback method may further include: if the electronic device determines that S consecutive heart rate values ​​of the user are all less than a preset heart rate value, indicating that the user may be about to fall asleep or has already fallen asleep, then it can stop outputting task instructions and stop playing the heart sound audio. S is an integer greater than or equal to 2; for example, S can be 20. If the preset heart rate value is 50, then if the electronic device determines that 20 consecutive heart rate values ​​of the user are all less than 50, it can stop outputting task instructions and stop playing the heart sound audio. This ensures a quiet environment when the user may be about to fall asleep or has already fallen asleep, improving the user experience, and also saves the power and resources of the electronic device.

[0014] In one possible implementation, the audio playback method may further include: if the electronic device determines that the user's heart rate value remains below a preset heart rate value for a second preset time period, it can stop outputting task instructions and stop playing the heart sound audio. For example, the second preset time period can be one minute, and the preset heart rate value is 55. If the electronic device determines that the user's heart rate value is consistently below 55 for one minute, it can stop outputting task instructions and stop playing the heart sound audio. This provides a quiet environment for the user, improving the user experience, and also saves the electronic device's power and resources.

[0015] In a second aspect, this application provides an electronic device including a memory and a processor; the memory stores computer program code, which includes computer instructions; one or more processors invoke the computer instructions to cause the electronic device to perform the audio playback method described in the first aspect.

[0016] Thirdly, this application provides a computer-readable storage medium storing a computer program that, when executed by a processor, implements the audio playback method of the first aspect described above. Attached Figure Description

[0017] Figure 1a This is a schematic diagram of an audio playback method in related technologies;

[0018] Figure 1b This is a schematic diagram of another audio playback method in related technologies;

[0019] Figure 2a This is a schematic diagram illustrating an application scenario during the preparation stage, as provided in an embodiment of this application.

[0020] Figure 2b This is a schematic diagram illustrating another application scenario for the preparation stage provided in this application embodiment;

[0021] Figure 3 A user interface diagram provided for an embodiment of this application;

[0022] Figure 4 This is a schematic diagram illustrating an application scenario during the training phase, provided in an embodiment of this application.

[0023] Figure 5 A flowchart illustrating an audio playback method provided in an embodiment of this application;

[0024] Figure 6 A flowchart illustrating another audio playback method provided in an embodiment of this application;

[0025] Figure 7 This is a schematic diagram of the composition of an electronic device provided in an embodiment of this application. Detailed Implementation

[0026] To ensure clarity and conciseness in the description of the following embodiments, the terminology used in the embodiments of this application will first be explained. It should be understood that this explanation is for the purpose of better understanding the embodiments of this application and does not necessarily constitute a limitation on the embodiments of this application.

[0027] Heart rate: refers to the number of times the human heart beats per minute. In some embodiments, the unit of heart rate is beats per minute, which will not be repeated in the following embodiments.

[0028] Heart sounds: These are the sounds produced by mechanical wave phenomena caused by myocardial contraction, closure of heart valves, and blood impacting the walls of the ventricles and aortas; in other words, the sound of the heart beating. In this embodiment, actual heart sounds refer to the recorded user heart sounds, while digital heart sounds refer to the generated heart sounds (also known as heart sound audio).

[0029] The following section compares and explains the technical advantages of the audio playback method, electronic device, and storage medium provided in this application, in conjunction with relevant technologies. For ease of understanding, an example scenario is used for illustration.

[0030] In related technologies, taking a mobile phone as an example, the phone can provide users with breathing training functions. After the user triggers the breathing training function, the phone can play instructions to guide the user in breathing. For example, such as... Figure 1a As shown, the phone can issue instructions such as "inhale" - "stop" - "exhale" to guide the user to first inhale, then hold their breath, and finally exhale, repeating the breathing exercise multiple times.

[0031] Taking a smartwatch as an example, a smartwatch can also provide users with breathing training functions. After the user triggers the breathing training function, the smartwatch can display animated images to guide the user in breathing exercises as the animated images change. For example, such as... Figure 1b As shown, when the animated image (also known as an image that changes size) becomes larger, the user inhales; when the animated image becomes smaller, the user exhales, and this process is repeated multiple times.

[0032] Both of these methods, if consistently followed by the user, can alleviate tension and promote relaxation. For example, following an electronic device for breathing exercises before sleep can reduce sympathetic nerve tension and increase vagal nerve tension, thus promoting sleep. Alternatively, following an electronic device for breathing exercises when feeling tense or agitated can help control emotions and restore breathing rhythm. However, the instructions or animated images displayed on electronic devices are often too simplistic and lack feedback on the effectiveness of the breathing exercises, which can negatively impact the user experience.

[0033] Therefore, to solve the above problems, this application provides an audio playback method. In this method, an electronic device is configured with a breathing training task. The user can trigger the breathing training task. In response to the user's triggering operation, the electronic device runs the breathing training task and can play tasks such as "inhale"-"stop"-"exhale" voice prompts or display images with changing sizes to guide the user to follow the changes in voice or images for breathing training. The electronic device also matches the user's heart rate value with a pre-stored correspondence between heart rate values ​​and heart sound audio, and plays heart sound audio that matches the user's heart rate value. In the pre-stored correspondence between heart rate values ​​and heart sound audio, the pre-stored heart rate value and the corresponding heart sound audio's heart rate frequency are positively correlated, that is, the higher the heart rate value, the higher the heart sound audio's heart rate frequency. In the breathing training task, if the user's heart rate value changes, the electronic device can obtain the user's changed heart rate value and then match it with the pre-stored correspondence between heart rate values ​​and heart sound audio, playing heart sound audio that matches the changed heart rate value. In this way, when the electronic device outputs task instructions to guide the user to perform breathing training, it also plays heart sound audio corresponding to the user's current heart rate value. The heart sound audio allows the user to perceive the results of their breathing training, indicating that the electronic device can provide real-time feedback on the training effect of the user's breathing training, thereby effectively improving the user experience.

[0034] To enable those skilled in the art to more clearly understand the solution of this application, the following is combined with... Figures 2a-4 This application describes the application scenarios of the audio playback method provided in the embodiments of this application. It should be noted that there are many ways for users to perform breathing training through electronic devices; the following are only some examples.

[0035] In some embodiments, the audio playback method provided in this application may include a preparation phase and a training phase. In the preparation phase, the user's heart sounds (also known as actual heart sounds) at different heart rates can be recorded, and more digital heart sounds corresponding to different heart rates can be generated based on this. In the training phase, the electronic device may include a health and fitness application that provides a breathing training function (also known as a breathing training task). After the user triggers the breathing training function, the health and fitness application can run the breathing training task to guide the user in breathing training. While guiding the user in breathing training, the electronic device can collect the user's heart rate in real time and play digital heart sounds corresponding to the user's real-time heart rate.

[0036] Next, let's combine Figures 2a-3 This section introduces application scenarios in the preparation stage of audio playback methods.

[0037] In some embodiments, users can record their heart sounds using an electronic device with a microphone. Taking a mobile phone as an example, the phone can provide breathing training functions. When a user uses this function for the first time, the phone may prompt the user to place the phone's microphone close to their heart to record the sound of their heartbeat. Figure 2a As shown, users can hold their phone's microphone close to their heart to record.

[0038] In other embodiments, users can record their own heart sounds using an electronic stethoscope with communication capabilities. The electronic stethoscope has highly sensitive auscultation capabilities, accurately capturing and amplifying the sounds of the heart and its beating.

[0039] In one possible implementation, taking a mobile phone as an example, the electronic stethoscope can have the function of wirelessly communicating with the mobile phone. For example, as... Figure 2b As shown, the electronic stethoscope can communicate with a mobile phone via Bluetooth. After establishing a Bluetooth connection, the user can place the stethoscope on their heart and press the start button to listen to the heart sounds. The recorded heart sounds can then be transmitted to the mobile phone.

[0040] In another possible implementation, using a mobile phone as an example, the electronic stethoscope can also have the function of wired communication with the mobile phone. For example, the electronic stethoscope can be wired to the mobile phone via a data cable to transmit the recorded heart sounds to the mobile phone.

[0041] In some embodiments, the mobile phone may include a fitness and health application, such as Figure 3 As shown, after the mobile phone launches the sports and health application, it displays the application's running interface 310, which includes multiple controls such as a breathing training function control 311. After the user clicks the breathing training function control 311, the mobile phone responds to the user's click operation and displays the breathing training function interface 320. Since this is the user's first time using this function, the breathing training function interface can display prompts to the user to record their own heart sounds.

[0042] It should be noted that the above example of providing breathing training functions through a sports and health app is just an example. It can also be a system app on the phone or other third-party apps downloaded to the phone. This application does not limit it in this regard.

[0043] In one possible implementation, the phone could prompt the user to record their heart sounds at excessively fast, normal, and excessively slow heart rates, and could recommend specific recording scenarios. For example, such as... Figure 3As shown, the phone can display prompts such as "Please hold your phone's microphone close to your heart for 5 seconds in the following or similar scenarios to record the sound of your heartbeat..."

[0044] It should be noted that the above recording time of 5 seconds is only an example, and the recording time can also be other values ​​such as 8 seconds or 10 seconds. This application does not limit this.

[0045] For example, a mobile phone can prompt a user to record their heart sounds at a higher heart rate after strenuous exercise such as running. As another example, the phone can prompt a user to record their heart sounds at a normal heart rate after prolonged sitting at an office or after a walk. Yet another example, the phone can prompt a user to record their heartbeat when they first wake up in the morning, obtaining a heart sound at a lower heart rate.

[0046] It should be understood that the user's heart rate during breathing training is uncertain, and the recorded heart sounds may not contain any heart sounds corresponding to the user's heart rate during breathing training. Therefore, in some embodiments, more digital heart sounds at different heart rates can be generated based on the recorded heart sounds.

[0047] In one possible implementation, the mobile phone can perform loudness and frequency analysis on the recorded heart sounds, and then generate digital heart sounds based on the loudness and frequency analysis results.

[0048] For example, suppose a user records heart sound 1 corresponding to a heart rate of 120 after running and heart sound 2 corresponding to a heart rate of 118 after running; a user records heart sound 3 corresponding to a heart rate of 80 after walking and heart sound 4 corresponding to a heart rate of 75 after sitting quietly; and a user records heart sound 5 corresponding to a heart rate of 54 after waking up.

[0049] The mobile phone can first analyze the loudness of heart sounds 1 to 5, and then fit the loudness values ​​1 to 5 corresponding to heart sounds 1 to 5 respectively to obtain more fitting curves of loudness values ​​under different heart rates. For example, it can include the loudness values ​​of heart sounds corresponding to each heart rate from 50 to 130.

[0050] Then, the mobile phone can analyze the frequencies of heart sounds 1-5 to obtain their corresponding spectrum diagrams, and then, based on the spectrum diagrams 1-5 respectively, perform a spectrum analysis. Figure 5 By performing fitting, more spectrograms of heart sounds at different heart rates can be obtained, such as spectrograms of heart sounds corresponding to each heart rate from 50 to 130.

[0051] The phone can then generate digital heart sounds corresponding to each heart rate from 50 to 130 based on the loudness value of the heart sounds corresponding to each heart rate from 50 to 130, as well as the spectrogram of the heart sounds corresponding to each heart rate from 50 to 130.

[0052] Next, combine Figures 3-4 This section introduces application scenarios during the training phase of audio playback methods.

[0053] In some embodiments, such as Figure 3 As shown, based on the above example, a mobile phone's health and fitness app can provide a breathing training function. The user can click the breathing training function control 311 on the running interface 310. The phone responds to the user's click and displays the breathing training function interface 320. Since the phone has already generated digital heart sounds, the breathing training function interface 320 can display a start control 321. The user clicks the start control 321, triggering the phone's breathing training function. The health and fitness app can then run a breathing training task to guide the user in breathing exercises.

[0054] In some embodiments, a user may wear an electronic device capable of measuring heart rate in real time, also known as a wearable device, such as a smartwatch or smart bracelet. The electronic device for measuring heart rate in real time can establish a communication connection with an electronic device storing digital heart sounds, for example, via Bluetooth; this application does not limit this to a specific method.

[0055] For example, such as Figure 4 As shown, taking a smartwatch as an electronic device for real-time heart rate measurement and a mobile phone as an electronic device storing digital heart sounds as an example, the user wears the smartwatch on their wrist, and the mobile phone and smartwatch have established a Bluetooth connection. After the user triggers the breathing training function provided by the mobile phone, the mobile phone can display continuously changing animated images to guide the user in breathing exercises. At the same time, the smartwatch measures the user's real-time heart rate and transmits it to the mobile phone. Based on the real-time heart rate, the mobile phone can determine the digital heart sound corresponding to the real-time heart rate from multiple stored digital heart sounds and play it. In other words, the mobile phone can play the digital heart sound at the corresponding heart rate in real time while playing the guidance, allowing the user to perceive the effect of the breathing training in real time and improving the user experience.

[0056] For example, let's take a smartwatch as the electronic device that measures heart rate in real time and a mobile phone as the electronic device that stores digital heart sounds. The user wears the smartwatch on their wrist, and the phone and smartwatch are connected via Bluetooth. After the user triggers the breathing training function provided by the phone, the smartwatch can display constantly changing animated images to guide the user in breathing exercises. Simultaneously, the smartwatch measures the user's real-time heart rate and transmits it to the phone. Based on the real-time heart rate, the phone can determine the corresponding digital heart sound from a list of stored digital heart sounds and transmit that sound to the smartwatch for playback. In other words, the smartwatch can display animated images while simultaneously playing the corresponding digital heart sound at the user's heart rate in real time, allowing the user to perceive the effect of the breathing training in real time.

[0057] It should be noted that the above-mentioned forms of breathing training through electronic devices are only examples. Other forms of guiding users to conduct breathing training can also be used. While guiding users to conduct breathing training, digital heart sounds can be played to the users based on their heart rate. This application does not limit this.

[0058] Next, based on the application scenarios introduced above, let's take a mobile phone as an example of an electronic device that records, generates, and stores digital heart sounds, and a smartwatch as an electronic device that measures heart rate in real time. Figure 5 and Figure 6 The audio playback method provided in the embodiments of this application will be described in detail.

[0059] This audio playback method may include a preparation phase and a training phase.

[0060] like Figure 5 As shown, the preparation phase of an audio playback method may include the following steps:

[0061] S501: Mobile phone recording of user's heart sounds at different heart rates.

[0062] In some embodiments, a user can hold the phone close to their heart with the microphone to record the sound of their heartbeat and obtain a heart sound. (See also...) Figure 2a As shown.

[0063] It should be understood that when users perform breathing exercises before falling asleep, if they experience difficulty falling asleep due to tension, anxiety, or excitement, the excitability of the sympathetic nervous system will significantly increase, and the heart rate will also rise accordingly. Conversely, when users are about to fall asleep or have already fallen asleep, the excitability of the sympathetic nervous system will decrease, while the excitability of the vagus nerve will gradually increase, and the heart rate will slow down accordingly. When users perform breathing exercises while emotionally agitated or tense, the instability of their emotions will activate the sympathetic nervous system, causing their heart rate to increase, while it will gradually slow down as their emotions calm down. These scenarios indicate that the range of heart rate fluctuations during breathing exercises can be quite wide. Therefore, it is necessary to record more heart sounds corresponding to different heart rates to facilitate playback to users during breathing exercises.

[0064] However, users cannot know their own heart rate, and therefore may not be able to record heart sounds corresponding to a wide range of different heart rates. Since heart rate has different characteristics in different scenarios, the phone could prompt users to record heart sounds in different scenarios. Recording heart sounds corresponding to different heart rate characteristics would help generate digital heart sounds corresponding to a wider range of different heart rates. (See also...) Figure 3 As shown, the phone can display prompts on the breathing training function interface 320.

[0065] In some embodiments, when a user exercises or experiences emotional excitement or tension, their heart rate increases, and the phone can record the heart sounds corresponding to the higher heart rate. For example, it can record the heart sounds corresponding to a heart rate of 125 and a heart rate of 130, etc.

[0066] In some embodiments, when a user is engaged in gentle activities such as sitting or walking, their heart rate is stable, and the phone can record heart sounds corresponding to a normal heart rate. For example, it can record heart sounds corresponding to a heart rate of 80, heart sounds corresponding to a heart rate of 84, etc.

[0067] In some embodiments, typically after waking up in the morning, the excitability of the sympathetic nervous system begins to increase, while the excitability of the vagus nerve begins to decrease, and the heart rate gradually increases. Therefore, in the scenario where a user has just woken up, the heart rate is relatively slow, and the mobile phone can record heart sounds corresponding to lower heart rates. For example, it can record heart sounds corresponding to a heart rate of 54, heart sounds corresponding to a heart rate of 56, etc.

[0068] It should be understood that the longer a mobile phone continuously records heart sounds, the more heart sounds corresponding to different heart rates can be obtained. Based on a rich amount of heart sound data, it is beneficial to generate accurate and realistic digital heart sounds. However, considering users' high demand for time, in order to save users' time, a time threshold for recording can be prompted to the user.

[0069] However, the rate of heart rate change varies in different scenarios. For example, when a user is resting after exercise, the heart rate will decrease rapidly at first, and then gradually decrease at a slower pace. In this case, it is possible to record heart sounds corresponding to different heart rates in a short period of time. For instance, a mobile phone can record for 10 seconds to obtain the heart sounds corresponding to each heart rate between 130 and 120.

[0070] For example, when a user is engaged in gentle activities such as sitting or walking, the heart rate changes are not obvious. In this case, a longer recording time can be recorded. For example, the phone can record for 30 seconds to obtain the heart sounds corresponding to each heart rate between 75 and 80.

[0071] Understandably, the phone will need to play heart sounds based on the user's real-time heart rate. Therefore, during the recording of heart sounds, the phone also needs to obtain the current heart rate to determine the correspondence between the recorded heart sounds and the heart rate.

[0072] In some embodiments, during recording, the mobile phone can calculate the user's heart rate based on the recorded heart sounds, thereby determining the correspondence between heart sounds and heart rates. For example, the mobile phone can first record two consecutive heart sounds and calculate the time interval (in seconds) between these two consecutive heart sounds. The mobile phone then divides 60 seconds (corresponding to 1 minute) by the time interval to obtain the user's current heart rate. Subsequently, the mobile phone can determine that this heart rate corresponds to the first of the two consecutive heart sounds.

[0073] For example, after a user runs, the phone can record heart sounds 1 and 2. Based on the recording time of heart sound 1 and heart sound 2, the time interval between heart sounds 1 and 2 can be calculated. Assuming the time interval is 0.46 seconds, based on the ratio of 60 seconds to 0.46 seconds, the user's heart rate can be calculated to be approximately 130. The phone can then determine that the heart rate of 130 corresponds to heart sound 1, meaning that when the user's heart rate is 130, the sound of the user's heartbeat is heart sound 1.

[0074] In some embodiments, N heart sounds of a user at N different heart rates (also known as heart rate values) can be recorded, where N is a positive integer, such as 3, 5 or 8, and this application does not limit this.

[0075] S502: Heart sounds recorded and stored on the mobile phone.

[0076] In some embodiments, after the heart sounds are recorded by the mobile phone, the heart sounds and the correspondence between heart sounds and heart rates can be stored in a heart sound database for easy use when generating digital heart sounds later.

[0077] For example, the mobile phone can store the recorded heart sounds in a database on the device side, that is, in a local file on the mobile phone.

[0078] As another example, the mobile phone can also store the recorded heart sounds in a cloud-based database, that is, in a cloud server. This application does not limit this.

[0079] It should be understood that the recorded heart sounds may contain noise due to issues such as the recording environment, accidental touches, and device connection problems. To generate higher quality digital heart sounds, audio enhancement or noise reduction can be performed on the recorded heart sounds.

[0080] Audio enhancement refers to extracting useful heart sounds from a noisy background when heart sounds are interfered with or even drowned out by various kinds of noise. In some embodiments, the heart sounds can be enhanced using traditional signal processing methods such as spectral subtraction or over-subtraction, or deep learning methods, before being stored in a heart sound database. This application does not limit this approach.

[0081] Audio noise reduction refers to processing various external sounds, especially noise, to obtain a pure heart sound. In some embodiments, noise reduction can be performed using linear filters or trained neural networks before storing the heart sound in a heart sound database; this application does not limit this approach.

[0082] S503: The mobile phone analyzes the heart sounds and obtains the loudness value and spectrum of the heart sounds.

[0083] Loudness and frequency are fundamental characteristics of sounds such as heart sounds. Loudness refers to the intensity of a heart sound, which can also be understood as the perceived volume of the heart sound by a user, measured in decibels (dB). Frequency refers to the speed at which the heart sound vibrates; the higher the frequency of the heart sound, the faster it vibrates, which can also be understood as the higher the perceived heart sound by a user; the lower the frequency of the heart sound, the slower it vibrates, which can also be understood as the lower the perceived heart sound by a user.

[0084] In some embodiments, the mobile phone can analyze the heart sounds stored in the heart sound database to obtain the loudness value and the spectrum of the heart sounds.

[0085] For example, suppose the heart sound database stores five heart sounds: heart sound 1 corresponding to a heart rate of 130, heart sound 2 corresponding to a heart rate of 125, heart sound 3 corresponding to a heart rate of 80, heart sound 4 corresponding to a heart rate of 75, and heart sound 5 corresponding to a heart rate of 54. Then the mobile phone can first analyze heart sounds 1 through 5 to obtain their respective loudness values ​​1 through 5.

[0086] It should be understood that the frequencies of sounds such as heart sounds are constantly changing and quite complex. Even a single sound may contain many different frequency bands.

[0087] In some embodiments, the frequency domain information of the heart sound can be obtained by performing a Fourier transform on the heart sound using an analog-to-digital converter, and then a corresponding spectrum can be plotted based on the frequency domain information. The horizontal axis represents the frequency (Hz), and the vertical axis represents the amplitude (dB) of the heart sound at the corresponding frequency.

[0088] For example, based on the heart sounds 1 to 5 in the above example, a spectrum diagram corresponding to each heart sound can be drawn, and the frequency characteristics of the heart sounds can be represented by the spectrum diagram.

[0089] S504: The mobile phone generates loudness values ​​corresponding to multiple heart rates within a preset heart rate range based on the loudness value of the heart sound.

[0090] Based on the above description, taking a user's breathing training before falling asleep as an example, when a user is unable to fall asleep, their heart rate may be higher, and when the user is about to fall asleep, their heart rate may be lower. This indicates that during the process of falling asleep, the heart rate may gradually decrease from a higher value to a lower value. Therefore, it is necessary to obtain loudness values ​​corresponding to multiple heart rates within a heart rate range. For example, the preset heart rate range can be 50-120 or 40-130, and this application does not limit it to this range.

[0091] In some embodiments, the heart rate corresponding to the heart sound can be used as x, and the loudness value corresponding to the heart sound can be used as y. An approximate functional relationship between y and x can be found, and then the loudness values ​​corresponding to multiple heart rates within a preset range can be simulated.

[0092] In one possible implementation, a curve fitting method can be used to fit the heart rate and loudness value corresponding to the heart sound onto a curve (also known as the first curve), and then solve the equation of the fitted curve to obtain the loudness values ​​corresponding to multiple heart rates within a preset range.

[0093] For example, curve fitting can be performed using methods such as least squares, spline, and maximum likelihood estimation. For instance, based on the above example, if the preset heart rate range is 50-120, and the heart sound database includes heart sounds 1-5, then the following set of data can be obtained: (130, loudness value 1), (125, loudness value 2), (80, loudness value), (75, loudness value), and (54, loudness value). This data can be distributed on a coordinate axis with heart rate on the x-axis and loudness values ​​of the heart sound intensity on the y-axis. Then, based on any of the above curve fitting methods, a loudness curve for the heart sounds can be fitted using the data on the coordinate axis. Based on this loudness curve, the loudness values ​​corresponding to multiple heart rates within the 50-120 heart rate range can be obtained.

[0094] In some embodiments, M loudness values ​​corresponding to different heart rates can be generated, where M is an integer greater than N. For example, when N is 5, M can be 20.

[0095] S505: The mobile phone generates a spectrum of multiple heart rates within a preset heart rate range based on the heart sound spectrum.

[0096] Based on the above introduction, the spectrum of heart sounds includes the amplitude corresponding to multiple frequencies.

[0097] In some embodiments, M spectrum diagrams corresponding to different heart rates can be generated, where M is an integer greater than N. For example, when N is 8, M can be 30.

[0098] In some embodiments, the heart rate corresponding to the heart sound can be used as the abscissa, and the amplitude corresponding to the heart sound at the same frequency can be used as the ordinate. By using a curve fitting method, the different amplitudes corresponding to different heart rates at that frequency can be fitted onto a curve to obtain the corresponding amplitude curve (also known as the second curve). By analogy, the amplitude curves corresponding to the heart sound at different frequencies can be obtained.

[0099] Then, using a frequency as the abscissa and the amplitude corresponding to a heart rate in the amplitude curve corresponding to that frequency as the ordinate, a coordinate point is obtained; then using another frequency as the abscissa and the amplitude corresponding to the same heart rate in the amplitude curve corresponding to that frequency as the ordinate, another coordinate point is obtained, and so on, to obtain multiple coordinate points. Then, a curve is plotted based on multiple coordinate points to obtain the spectrum diagram corresponding to that heart rate.

[0100] Subsequently, based on other heart rates within the preset heart rate range, the above steps are executed sequentially to generate spectrum diagrams corresponding to multiple heart rates.

[0101] For example, assuming the preset heart rate range is 50-120, the heart sounds 1-5 mentioned above correspond to the spectrum diagram 1-spectrum. Figure 5 Furthermore, each of these five spectrograms can include the amplitudes corresponding to 40Hz-400Hz. Taking 40Hz as an example, from spectrogram 1, we can determine the amplitude 1 corresponding to heart sound 1 at 40Hz; from spectrogram 2, we can determine the amplitude 2 corresponding to heart sound 2 at 40Hz; and so on. We can determine the amplitudes corresponding to heart sounds 1-5 at a frequency of 40Hz as amplitude 1-amplitude 5.

[0102] Plotting heart rate on the x-axis and amplitude on the y-axis yields the data set (130, amplitude 1), (125, amplitude 2), (80, amplitude 3), (75, amplitude 4), and (54, amplitude 5), which can then be distributed across the coordinate axes. Subsequently, using any of the aforementioned curve fitting methods, an amplitude curve for the heart sound at 40 Hz can be fitted based on the data on the coordinate axes. This amplitude curve can then be used to obtain the amplitudes corresponding to multiple heart rates within the range of 50-120 Hz. Similarly, amplitude curves for each frequency of the heart sound within the range of 40 Hz to 400 Hz can be obtained.

[0103] Based on these amplitude curves, taking a heart rate of 50 as an example, the amplitude corresponding to each frequency within the range of 40Hz-400Hz can be obtained from multiple amplitude curves. Then, with frequency as the horizontal axis and amplitude as the vertical axis, we can obtain a set of data (40Hz, amplitude 6), (41Hz, amplitude 7), (42Hz, amplitude 8), (43Hz, amplitude 9), ..., (400Hz, amplitude 366). Distributing this data on the coordinate axis, we can draw the spectrum corresponding to a heart rate of 50. Similarly, we can draw the spectrum corresponding to each heart rate within the range of 50-120Hz.

[0104] S506: The mobile phone generates digital heart sounds corresponding to multiple heart rates within a preset heart rate range based on the loudness values ​​corresponding to multiple heart rates within the preset heart rate range, as well as the spectrum diagrams corresponding to multiple heart rates within the preset heart rate range.

[0105] In some embodiments, assuming that multiple heart rates within a preset heart rate range include heart rate A and heart rate B, a digital heart sound corresponding to heart rate A can be generated based on the loudness value and the spectrum corresponding to heart rate A; and a digital heart sound corresponding to heart rate B can be generated based on the loudness value and the spectrum corresponding to heart rate B.

[0106] In some embodiments, based on the loudness value and spectrogram corresponding to the same heart rate, a digital heart sound corresponding to that heart rate can be generated using an open-source audio generation tool. For example, the audio generator Sinegen can be used to generate digital heart sounds corresponding to multiple heart rates within a preset heart rate range.

[0107] In some embodiments, M digital heart sounds can be generated that correspond one-to-one with M heart rates, where M is an integer greater than N. For example, when N is 12, M can be 50.

[0108] Furthermore, in some embodiments, the sound generation model can be trained using spectrograms. For example, spectrograms corresponding to multiple heart sounds can be collected as training data, the heart sounds corresponding to the spectrograms can be used as training labels, the spectrograms can be input into the model to be trained to generate predicted heart sounds, and the parameters of the model to be trained can be adjusted based on the predicted heart sounds, the real heart sounds corresponding to the spectrograms, and a preset loss function until the training cutoff condition is met, thus completing the training and obtaining the sound generation model.

[0109] Subsequently, the spectrograms corresponding to multiple heart rates within the preset heart rate range can be input into the sound generation model. Then, based on the loudness values ​​corresponding to multiple heart rates within the preset heart rate range, the loudness of the sound output by the sound generation model can be adjusted to obtain the digital heart sounds corresponding to multiple heart rates within the preset heart rate range.

[0110] For example, by inputting the spectrogram corresponding to a heart rate of 50 into a sound generation model, and then adjusting the loudness of the sound output by the sound generation model based on the loudness value corresponding to a heart rate of 50, a digital heart sound corresponding to a heart rate of 50 can be obtained.

[0111] In some embodiments, the model to be trained may be a recurrent neural network (RNN), a transformer network, or a combination of both, and this application does not limit this.

[0112] In some embodiments, before training the model to be trained, the spectrogram can be preprocessed with image enhancement, image normalization, etc., to obtain a spectrogram of better quality. Then, the preprocessed spectrogram is used as training data to train the model to be trained. This application does not limit this.

[0113] like Figure 6 As shown, in the preparation stage of the audio playback method, the mobile phone can record the user's heart sounds at different heart rates, store the recorded heart sounds, analyze the loudness and frequency of the heart sounds, and finally generate digital heart sounds corresponding to different heart rates based on the analysis results.

[0114] like Figure 5 As shown, the training phase of the audio playback method may include the following steps:

[0115] S507: The mobile phone receives the user's trigger operation for the breathing training function.

[0116] In some embodiments, such as Figure 3As shown, the mobile phone's health and fitness application can provide a breathing training function. Users can trigger the mobile phone's breathing training function by clicking the breathing training function control 311 on the running interface 310 and then clicking the start control 321, and the mobile phone will run the configured breathing training task.

[0117] S508: In response to the user's trigger action, the phone plays breathing training instructions.

[0118] In some embodiments, in response to a user's triggering of the breathing training function, the phone can play instructions such as "inhale" - "stop" - "exhale" to guide the user through the breathing exercises. (See also...) Figure 1a As shown.

[0119] Furthermore, in some embodiments, the mobile phone can also respond to a user's triggering of the breathing training function by displaying animated images to guide the user in breathing exercises as the animated images change. For example, the mobile phone can display images such as... Figure 1b The animated images displayed on the smartwatch can also be other animated images, and this application does not limit this.

[0120] S509: The mobile phone receives real-time heart rate data transmitted from the smartwatch.

[0121] In some embodiments, the user wears the smartwatch on their wrist, and a communication connection is established between the smartwatch and the smartphone. The smartwatch includes a photoplethysmography (PPG) sensor, whose LED light source on the back emits light of a specific wavelength (usually green light) to collect the PPG signal. The user's real-time heart rate can then be calculated based on the collected PPG signal. The smartwatch transmits the user's real-time heart rate to the smartphone (this can also be described as the smartphone acquiring the user's heart rate value).

[0122] For example, mobile phones and smartwatches can be like Figure 4 The connection shown is via Bluetooth to achieve wireless communication. Alternatively, the phone and smartwatch can also connect to the same Wi-Fi network to achieve wireless communication. Furthermore, the phone and smartwatch can also connect via a data cable to achieve wired communication. This application does not limit the scope of the application.

[0123] S510: The mobile phone determines the digital heart sound corresponding to the real-time heart rate from multiple digital heart sounds and plays it.

[0124] In some embodiments, based on the above description, the mobile phone has generated digital heart sounds corresponding to multiple heart rates within a preset heart rate range. The mobile phone then determines and plays the corresponding digital heart sound based on the real-time heart rate transmitted from the smartwatch. This can also be understood as matching the corresponding digital heart sound from a pre-stored correspondence between multiple heart rates and multiple digital heart sounds. For example, assuming a user's heart rate is 100, the mobile phone can play the digital heart sound corresponding to heart rate 100. As the user's heart rate gradually decreases from 100 to 90, the mobile phone can sequentially play the digital heart sounds corresponding to each heart rate from 100 to 90.

[0125] It should be noted that this application does not limit the execution order of S508 and S509-S510. S508 and S509-S510 can be executed simultaneously after the user triggers the breathing training function provided by the mobile phone. In other words, the instructions and digital heart sounds are played at the same time.

[0126] Furthermore, it should be understood that the higher a user's heart rate, the higher the frequency of their heart sounds. Therefore, in some embodiments, to enhance the guidance effect of breathing training and enable the user to lower their heart rate more quickly, when the user's heart rate remains constant within a preset time period (also known as the first preset time period), a digital heart sound with a lower frequency of heart sounds (also known as the target heart sound audio) corresponding to a lower real-time heart rate can be played to the user. For example, if the phone obtains that the user's heart rate is 100 and remains unchanged for 20 seconds, a digital heart sound corresponding to a heart rate of 99 or 98 can be played to the user.

[0127] like Figure 6 As shown, during the training phase of the audio playback method, the smartwatch can collect PPG signals in real time and calculate the user's real-time heart rate based on this. Then, while the phone plays instructions to guide the user in breathing training, it can also play digital heart sounds based on the user's real-time heart rate.

[0128] Furthermore, in some embodiments, the mobile phone receives the real-time heart rate transmitted from the smartwatch. If it determines that the user's real-time heart rate is below a heart rate threshold (also known as a preset heart rate value) within a preset time period (also known as a second preset time period), then it can stop playing the instructions and digital heart sounds (also known as stopping the output of task instructions and stopping the playback of heart sound audio), that is, stop the breathing training function. For example, the heart rate threshold can be 55 or 52, and the preset time period can be 40 seconds or 1 minute; this application does not limit this.

[0129] In other embodiments, the mobile phone receives real-time heart rate data transmitted from the smartwatch. If it determines that multiple consecutive real-time heart rates received from the user (also referred to as S consecutive heart rate values) are all below a heart rate threshold (also referred to as a preset heart rate value), the breathing training function can be stopped, and the playback of instructions and digital heart sounds can be stopped. For example, the breathing training function can be stopped when 10 or 20 consecutive real-time heart rates are all below the heart rate threshold; this application does not limit this to a specific case.

[0130] Thus, in scenarios where users perform breathing training before falling asleep, the audio playback method provided in this application can play digital heart sounds that change with the user's real-time heart rate while playing instructions or images, providing feedback to the user on the effectiveness of the breathing training. This can improve the user experience, encourage users to continue the breathing training, and further promote the user's ability to fall asleep and improve the user's sleep quality.

[0131] Furthermore, in some embodiments, the multiple digital heart sounds stored in the mobile phone may also be generated in advance based on the actual heart sounds of most users, and this application does not limit this.

[0132] In addition, in some embodiments, when a user is in a situation where they need to perform breathing training, such as when they are in a state of emotional tension or excitement, the mobile phone can also execute the audio playback method provided in this application embodiment, and play digital heart sounds based on the user's heart rate value while guiding the user to perform breathing training. This application does not limit the use scenario.

[0133] Next, we will introduce the components of electronic devices.

[0134] It should be noted that the mobile phones and wearable devices with communication connections in the above embodiments are merely examples. In some embodiments, the electronic devices that generate digital heart sounds based on recorded heart sounds can also be tablets, in-vehicle devices, augmented reality (AR) / virtual reality (VR) devices, laptops, ultra-mobile personal computers (UMPCs), netbooks, personal digital assistants (PDAs), etc. In some embodiments, the electronic devices that store digital heart sounds, guide users in breathing exercises, and play digital heart sounds can also be wearable devices; this application does not limit this.

[0135] This application does not impose any special restrictions on the number or specific form of the above-mentioned electronic devices, as long as they can implement the audio playback method provided in the embodiments of this application.

[0136] like Figure 7As shown, the electronic device 700 may include a processor 710, internal memory 720, a universal serial bus (USB) interface 730, antenna 1, antenna 2, mobile communication module 740, wireless communication module 750, audio module 760, sensor module 770, and display screen 780. The audio module 760 includes a speaker 760A, a microphone 760B, and a headphone jack 760C, etc. The sensor module 770 may include a PPG sensor 770A and a bone conduction sensor 770B, etc.

[0137] It is understood that the structure illustrated in this embodiment does not constitute a specific limitation on the electronic device. In other embodiments, the electronic device may include more or fewer components than illustrated, or combine some components, or split some components, or have different component arrangements. The illustrated components may be implemented in hardware, software, or a combination of software and hardware.

[0138] The processor 710 may include one or more processing units, such as an application processor (AP), a graphics processing unit (GPU), an image signal processor (ISP), or a digital signal processor (DSP). These different processing units may be independent devices or integrated into one or more processors.

[0139] The processor 710 can also be equipped with memory for storing instructions and data.

[0140] The internal memory 720 can be used to store computer executable program code, which includes instructions. The processor 710 executes various functional applications and data processing of the electronic device 700 by running the instructions stored in the internal memory 720.

[0141] In some embodiments, the internal memory 720 stores instructions for performing an audio playback method. The processor 710 can implement the audio playback method provided in this application embodiment by executing the instructions stored in the internal memory 720.

[0142] The USB interface 730 is a USB standard-compliant interface, specifically a Mini USB interface, a Micro USB interface, a USB Type-C interface, etc. In some embodiments, it can be used for data transfer between the electronic device 700 and peripheral devices. For example, when the electronic device is a mobile phone, the USB interface 730 can be used to connect an electronic stethoscope to receive heart sounds recorded by the stethoscope. As another example, when the electronic device is a mobile phone, the USB interface 730 can be used to connect a wearable device to receive heart rate data collected by the wearable device.

[0143] The wireless communication function of electronic device 700 can be implemented through antenna 1, antenna 2, mobile communication module 740, and wireless communication module 750. Mobile communication module 740 can provide wireless communication solutions for electronic device 700, including 2G / 3G / 4G / 5G. Wireless communication module 750 can provide wireless communication solutions for electronic device 700, including wireless local area networks (WLAN) (such as wireless fidelity (Wi-Fi) networks), Bluetooth (BT), global navigation satellite system (GNSS), frequency modulation (FM), near field communication (NFC), and infrared (IR) technologies.

[0144] In some embodiments, antenna 1 of electronic device 700 is coupled to mobile communication module 740, and antenna 2 is coupled to wireless communication module 750, enabling electronic device 700 to communicate with networks and other devices via wireless communication technology. For example, when electronic device 700 is a mobile phone, it can communicate with wearable devices and electronic stethoscopes via wireless communication technology.

[0145] The electronic device 700 implements its display function through a GPU, a display screen 780, and an application processor. The display screen 780 is used to display images, videos, etc. In some embodiments, the display screen 780 can display animated images to guide the user in breathing exercises.

[0146] Electronic device 700 can implement audio functions through audio module 760, speaker 760A, microphone 760B, headphone jack 760C, and application processor. For example, it can play instructions to guide users in breathing exercises and play digital heart sounds.

[0147] The audio module 760 is used to convert digital audio information into analog audio signals for output, and also to convert analog audio input into digital audio signals. The audio module 760 can also be used for encoding and decoding audio signals. In some embodiments, the audio module 760 may be located in the processor 710, or some functional modules of the audio module 760 may be located in the processor 710.

[0148] The speaker 760A, also known as a "loudspeaker," is used to convert audio electrical signals into sound signals. In some embodiments, the electronic device 700 can play instructions and digital heart sounds through the speaker 760A.

[0149] Microphone 760B, also known as a "microphone" or "voice transducer," is used to convert sound signals into electrical signals. In some embodiments, electronic device 700 may be equipped with one or more microphones 760B, which, in addition to acquiring sound signals, can also perform noise reduction, identify sound sources, and perform directional recording, etc. For example, microphone 760B can record a user's heartbeat.

[0150] The headphone jack 760C is used to connect wired headphones. The headphone jack 760C can be a USB 730 interface or a 3.5mm Open Mobile Terminal Platform (OMTP) standard interface, or a CTIA (Cellular Telecommunications Industry Association of the USA) standard interface. In some embodiments, a user can wear a wired headphone connected to an electronic device, which then plays instructions and digital heartbeats to the user through the wired headphones.

[0151] The PPG sensor 770A can detect the differences in the intensity of reflected light after being absorbed by human blood and tissues, record the changes in blood vessel volume during the cardiac cycle, and calculate the heart rate from the obtained pulse waveform.

[0152] The bone conduction sensor 770B can acquire vibration signals. In some embodiments, the bone conduction sensor 770B can acquire vibration signals from vibrating bone segments in the human vocal cords. The bone conduction sensor 770B can also contact the human pulse to receive blood pressure signals. The application processor can analyze heart rate information based on the blood pressure signals acquired by the bone conduction sensor 770B to implement heart rate detection functionality.

[0153] This application also provides a computer-readable storage medium storing a computer program, which, when executed by a computer, can implement one or more steps of any of the above-described audio playback methods.

[0154] Computer-readable storage media can be non-transitory computer-readable storage media, such as ROM, random access memory (RAM), CD-ROM, magnetic tape, floppy disk, and optical data storage devices.

[0155] Another embodiment of this application provides a computer program product containing instructions. When the computer program product is executed by a computer, it can implement one or more steps of any of the above-described audio playback methods.

[0156] The electronic device, computer-readable storage medium, and computer program product provided in this embodiment are all used to execute the corresponding audio playback method provided above. Therefore, the beneficial effects they can achieve can be referred to the beneficial effects in the corresponding audio playback method provided above, and will not be repeated here.

[0157] The terms "first," "second," and "third," etc., used in this application specification, claims, and drawings are used to distinguish different objects, not to limit a specific order.

[0158] In the embodiments of this application, the terms "exemplary," "for example," or "for instance" are used to indicate examples, illustrations, or explanations. Any embodiment or design described as "exemplary," "for example," or "for instance" in the embodiments of this application should not be construed as being more preferred or advantageous than other embodiments or designs. Specifically, the use of terms such as "exemplary," "for example," or "for instance" is intended to present the relevant concepts in a specific manner.

[0159] The above-described embodiments are only used to illustrate the technical solutions of this application, and are not intended to limit them. Although this application has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features. Such modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of this application.

Claims

1. An audio playback method, characterized in that, Applied to an electronic device configured with a breathing training task, the method includes: In response to the breathing training task being triggered, a task instruction is output to guide the user in performing breathing training. Based on the acquired heart rate value of the user, a match is made in the pre-stored correspondence between heart rate values ​​and heart sound audio, and the heart sound audio that matches the user's heart rate value is played; the pre-stored heart rate value and the heart sound frequency of the corresponding heart sound audio are positively correlated; the heart sound audio is generated based on the user's actual heart sounds; the heart sound audio is used to indicate the training effect of breathing training; In response to obtaining the user's changed heart rate value in the breathing training task, the system matches the changed heart rate value in the corresponding relationship and plays a heart sound audio that matches the changed heart rate value. The steps for obtaining the heart sound audio include: Obtain the actual heart sounds corresponding to the user at N heart rate values; where N is a positive integer. Based on the correspondence between the N heart rate values ​​and the N actual heart sounds, curve fitting is performed to obtain M heart sound audio values ​​that correspond one-to-one with the M heart rate values; where M is an integer greater than N.

2. The method according to claim 1, characterized in that, The curve fitting based on the correspondence between the N heart rate values ​​and the N actual heart sounds yields M heart sound audio frequencies that correspond one-to-one with the M heart rate values, including: Analyze the N actual heart sounds to obtain the loudness values ​​and spectrum diagrams corresponding to the N actual heart sounds; Curve fitting is performed based on the correspondence between N loudness values ​​and N heart rate values ​​to obtain a first curve including the correspondence between M loudness values ​​and M heart rate values; Curve fitting is performed based on the correspondence between N spectrograms and the N heart rate values ​​to obtain M spectrograms that correspond one-to-one with the M heart rate values. For each of the M heart rate values, a heart sound audio corresponding to that heart rate value is generated based on the loudness value corresponding to that heart rate value in the first curve and the spectrum corresponding to that heart rate value.

3. The method according to claim 2, characterized in that, The spectrum diagram includes the correspondence between multiple frequencies and multiple amplitudes; the curve fitting based on the correspondence between the N spectrum diagrams and the N heart rate values ​​yields M spectrum diagrams that correspond one-to-one with the M heart rate values, including: For each of the plurality of frequencies, curve fitting is performed based on the correspondence between the N amplitude values ​​and N heart rate values ​​of the N spectrum diagrams at that frequency to obtain a second curve corresponding to that frequency and including the correspondence between M amplitude values ​​and the M heart rate values; For each of the M heart rate values, the multiple frequencies are used as multiple horizontal axes, and the multiple amplitude values ​​of the multiple second curves at that heart rate value are used as multiple vertical axes corresponding one-to-one with the multiple horizontal axes, to obtain the spectrum diagram corresponding to that heart rate value.

4. The method according to any one of claims 1-3, characterized in that, The method further includes: It is determined that the user's heart rate value remains unchanged within a first preset time period, and a target heart sound audio is determined from the correspondence; the heart sound frequency of the target heart sound audio is lower than the heart sound frequency of the heart sound audio that matches the user's heart rate value; Play the target heart sound audio.

5. The method according to any one of claims 1-3, characterized in that, The method further includes: If it is determined that S consecutive heart rate values ​​of the user are all less than the preset heart rate value, the task instruction will be stopped and the heart sound audio will be stopped; S is an integer greater than or equal to 2.

6. The method according to any one of claims 1-3, characterized in that, The method further includes: If the user's heart rate is determined to be consistently lower than a preset heart rate value within a second preset time period, the task instruction will be stopped and the heart sound audio will be stopped.

7. An electronic device, characterized in that, Including memory and processor; The memory is coupled to the processor and is used to store computer program code, the computer program code including computer instructions, wherein one or more of the processors invoke the computer instructions to cause the electronic device to perform the audio playback method as described in any one of claims 1-6.

8. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores a computer program that, when executed by a processor, implements the audio playback method as described in any one of claims 1-6.

Images