Audio control method and device, equipment and storage medium

Abstract

The invention provides an audio control method and device, equipment and a storage medium, and relates to the technical field of computers, in particular to the technical fields of data processing, human-computer interaction, audio control and the like. According to the specific implementation scheme, in the audio playing process, a first audio file is collected; determining a first breathing waveform by using the first audio file; determining a first period of the first breathing waveform and a second period of a pre-stored sleep breathing waveform; sampling the first breathing waveform in N first periods to obtain a first sampling result; sampling the sleep respiration waveform in N second periods to obtain a second sampling result; n is a positive integer; and comparing the first sampling result with the second sampling result, and controlling the audio according to the comparison result. According to the invention, automatic control of audio playing can be realized.

Description

Technical Field

[0001] The present disclosure relates to the field of computer technology, and in particular to technical fields such as data processing, human-computer interaction, and audio control. Background Art

[0002] In recent years, with the rapid development of smart terminal technology, many users play audio before going to bed to help them sleep. However, this phenomenon is accompanied by some challenges: the user may have fallen asleep while the audio is still playing, causing unnecessary noise interference; on the other hand, if the user sets the audio off time in advance but does not fall asleep before the audio is turned off, they need to continue playing the audio manually. These situations will affect the user's sleep quality and user experience. Therefore, how to achieve automatic control of audio playback is a technical problem that needs to be solved. Summary of the invention

[0003] The present disclosure provides an audio control method, apparatus, device and storage medium.

[0004] According to one aspect of the present disclosure, there is provided an audio control method, comprising:

[0005] During audio playback, collecting a first audio file; determining a first breathing waveform using the first audio file;

[0006] Determine a first cycle of a first respiratory waveform and a second cycle of a pre-stored sleep respiratory waveform;

[0007] In N first cycles, the first breathing waveform is sampled to obtain a first sampling result; and in N second cycles, the sleep breathing waveform is sampled to obtain a second sampling result; N is a positive integer;

[0008] The first sampling result is compared with the second sampling result, and the audio is controlled according to the comparison result.

[0009] According to another aspect of the present disclosure, there is provided an audio control device, comprising:

[0010] A waveform determination module, used to collect a first audio file during audio playback; and determine a first breathing waveform using the first audio file;

[0011] A cycle determination module, used to determine a first cycle of a first respiratory waveform and a second cycle of a pre-stored sleep respiratory waveform;

[0012] The sampling module is used to sample the first breathing waveform within N first cycles to obtain a first sampling result; and to sample the sleep breathing waveform within N second cycles to obtain a second sampling result; N is a positive integer;

[0013] The control module is used to compare the first sampling result with the second sampling result, and control the audio according to the comparison result.

[0014] According to another aspect of the present disclosure, there is provided an electronic device, comprising:

[0015] at least one processor; and

[0016] a memory communicatively connected to the at least one processor; wherein,

[0017] The memory stores instructions that can be executed by the at least one processor, and the instructions are executed by the at least one processor to enable the at least one processor to execute any method in the embodiments of the present disclosure.

[0018] According to another aspect of the present disclosure, a non-transitory computer-readable storage medium storing computer instructions is provided, wherein the computer instructions are used to enable the computer to execute any method according to the embodiments of the present disclosure.

[0019] According to another aspect of the present disclosure, a computer program product is provided, including a computer program, which implements any method according to the embodiments of the present disclosure when executed by a processor.

[0020] The present invention determines whether the user is in a sleeping state by comparing the similarity between the user's breathing waveform and a pre-saved breathing waveform, and then controls the state of audio playback according to the user's sleeping state, thereby realizing automatic control of audio playback.

[0021] It should be understood that the content described in this section is not intended to identify the key or important features of the embodiments of the present disclosure, nor is it intended to limit the scope of the present disclosure. Other features of the present disclosure will become easily understood through the following description. BRIEF DESCRIPTION OF THE DRAWINGS

[0022] The accompanying drawings are used to better understand the present solution and do not constitute a limitation of the present disclosure.

[0023] Figure 1 is a flow chart of an implementation of an audio control method according to an embodiment of the present disclosure;

[0024] Figure 2 It is a schematic diagram of the configuration of the audio acquisition equipment;

[0025] Figure 3 This is a schematic diagram of the user audio collection process of an embodiment of the present application;

[0026] Figure 4 is a flowchart of an audio control method according to an embodiment of the present application;

[0027] Figure 5is a structural schematic diagram of an audio control device 500 according to an embodiment of the present disclosure;

[0028] Figure 6 is a structural schematic diagram of an audio control device 600 according to an embodiment of the present disclosure;

[0029] Figure 7 A schematic block diagram of an example electronic device 700 that may be used to implement embodiments of the present disclosure is shown. DETAILED DESCRIPTION

[0030] The following is a description of exemplary embodiments of the present disclosure in conjunction with the accompanying drawings, including various details of the embodiments of the present disclosure to facilitate understanding, which should be considered as merely exemplary. Therefore, it should be recognized by those of ordinary skill in the art that various changes and modifications may be made to the embodiments described herein without departing from the scope of the present disclosure. Similarly, for the sake of clarity and conciseness, the description of well-known functions and structures is omitted in the following description.

[0031] The “and / or” in the embodiments of the present disclosure indicates that three relationships may exist. For example, A and / or B may indicate the following three situations: A exists alone, A and B exist at the same time, and B exists alone. The term “at least one” herein indicates any combination of at least two of any one or more of a plurality of types. For example, at least one of A, B, and C may indicate any one or more elements selected from the set consisting of A, B, and C. The terms “first” and “second” herein refer to and distinguish between multiple similar technical terms, and do not mean to limit the order or to limit the meaning to only two. For example, the first feature and the second feature refer to two types / two features. The first feature may be one or more, and the second feature may also be one or more.

[0032] With the rapid development of smart terminal technology, many users play audio before going to bed to help them sleep. How to stop the audio being played by the device when people are asleep is a problem that needs to be solved.

[0033] Currently, the main methods for automatically stopping audio playback are:

[0034] (1) By setting the audio off time in the audio playback device or software, the audio playback device or software stops playing the audio when the audio off time is reached;

[0035] (2) Use wearable devices to monitor the user's biological state parameters, and determine whether the user is in a sleeping state based on the collected biological state parameters, thereby controlling the audio playback state.

[0036] If audio playback is stopped based on the set shutdown time, the audio may stop playing but the user is not asleep and needs to continue to play the audio manually, which disturbs the user's rest. Or, if audio playback is controlled based on a wearable device, the wearable device will often cause discomfort to the user when worn for a long time, and may even cause skin pressure and pain, affecting the user's overall experience.

[0037] Therefore, currently available audio control methods are neither efficient nor convenient, nor are they sufficiently user-friendly. The present disclosure provides an audio control method, Figure 1 This is a flow chart for implementing an audio control method according to an embodiment of the present disclosure. The method uses the audio acquisition module in the audio playback device to collect user audio, extract the user's breathing audio from the user audio and determine the user's breathing waveform; compare the user's breathing waveform with the pre-saved breathing waveform of the human body in a sleeping state, judge the user's physiological state according to the comparison result, and finally control the audio according to the user's physiological state. For example, if the user is awake, the audio continues to play; if the user is asleep, the audio playback is automatically stopped. Among them, the audio playback device may include a mobile terminal, and the user audio includes the user's breathing sound and the sound generated by the user's environment.

[0038] like Figure 1 As shown, the method may include:

[0039] S110. During audio playback, collect a first audio file; and determine a first breathing waveform using the first audio file;

[0040] S120, determining a first cycle of a first respiratory waveform and a second cycle of a pre-stored sleep respiratory waveform;

[0041] S130, sampling the first breathing waveform in N first cycles to obtain a first sampling result; and sampling the sleep breathing waveform in N second cycles to obtain a second sampling result; N is a positive integer;

[0042] S140: Compare the first sampling result with the second sampling result, and control the audio according to the comparison result.

[0043] In some examples, by adjusting the configuration of the audio acquisition device, the recording effect of the first audio file can be effectively controlled and optimized. Figure 2 This is a schematic diagram of the audio acquisition device configuration, such as Figure 2 As shown, in Figure 1 The process shown above also includes:

[0044] S210, determining an audio collection device;

[0045] S220: Set at least one of the sound collection frequency and the sound channel of the audio collection device.

[0046] Before recording user audio, you need to select a suitable audio acquisition device based on the application scenario or user needs to ensure that the selected audio acquisition device can accurately capture and restore the sound.

[0047] In addition, by setting the sound collection frequency of the audio collection device, the sampling accuracy of the audio signal can be precisely controlled, which directly affects the clarity and restoration of the recorded audio. For example, a higher sampling frequency can capture more sound details and reduce audio distortion and aliasing, but it will also increase the storage requirements of the audio file. Therefore, it is necessary to set the sound collection frequency according to the specific needs of the recording and the target audio quality and other indicators.

[0048] On the other hand, the channel setting determines the directionality and stereoscopic sense of the sound collected during audio recording. For example, mono recording only records the sound in one direction, which is suitable for scenes where the stereo effect is not required or where storage space needs to be saved; while dual-channel or even multi-channel recording can simulate a more realistic recording environment, allowing the audience to feel the directionality and spatial sense of the sound. Therefore, when setting the channel, it is also necessary to consider the purpose and scene of the recording to ensure that the recorded audio file can meet the expected sound quality and effect requirements.

[0049] By determining and setting parameters of the audio acquisition device, it is possible to ensure that high-quality user audio is captured, providing a basis for subsequent audio control.

[0050] Figure 3 It is a schematic diagram of the user audio collection process of an embodiment of the present application.

[0051] like Figure 3 As shown, the audio acquisition process includes the following steps:

[0052] S301. The audio playback software applies for microphone permission from the audio playback device to collect the user's audio.

[0053] S302, judging the playing state of the audio. If the audio is in the playing state, executing step S303; if the audio is in the stopped playing state, executing step S306.

[0054] S303: When the audio is in a playing state, the microphone of the audio playing device is used to collect the user's audio.

[0055] S304: Analyze the collected user audio, and determine the user's physiological state according to the user audio analysis result.

[0056] S305: Control the playing status of the audio based on the biological status of the user.

[0057] For example, when audio is playing, if it is determined that the user is awake by analyzing the collected user audio, the audio playback is maintained, and the microphone of the audio playback device continues to collect the user's audio, looping the audio control process; if it is determined that the user is asleep by analyzing the collected user audio, the audio playback is stopped and step S306 is executed.

[0058] S306: If the audio is in a stopped state, the audio playback software automatically turns off the microphone permission of the audio playback device to ensure that the user's personal privacy is safely protected.

[0059] S307, determine whether the user audio has been collected. If no user audio has been collected, end the user audio collection process; if user audio has been collected, execute step S308.

[0060] S308: Delete the collected user audio and end the user audio collection process.

[0061] Figure 4 1 is a flow chart of an audio control method according to an embodiment of the present application. Figure 4 As shown, it includes the following processes:

[0062] S401: Collect user audio and determine a first breathing waveform.

[0063] S402 , normalizing the first respiratory waveform and the pre-stored respiratory waveform of a human body in a sleeping state (referred to as the sleep respiratory waveform), sampling the two normalized waveforms, and determining sampling pairs and the number of sampling pairs.

[0064] For example, the first breathing waveform after normalization is sampled. The number of samples in each cycle is M, and the M sampling points are recorded as A1, A2, ...Am; the sleep breathing waveform after normalization is sampled, the number of samples in each cycle is M, and the M sampling points are recorded as B1, B2, ...Bm; A1, A2, ...Am and B1, B2, ...Bm are combined into M sampling pairs, namely (A1, B1), (A2, B2), ...(Am, Bm).

[0065] S403, calculating the amplitude difference between each pair of sampling points. Continuing with the above example, the difference between the sampling points in each sampling pair is calculated and the absolute value is taken to obtain M amplitude differences, namely C1, C2, ...Cm.

[0066] S404: Determine the number of similar sampling pairs according to the set amplitude difference threshold. Specifically, compare the amplitude difference of each pair of sampling points with the set amplitude difference threshold. If the amplitude difference of a pair of sampling points is less than the amplitude difference threshold, the pair of sampling points is considered to be a similar sampling pair.

[0067] For each sampling pair, the ratio of the amplitude difference C1, C2, ...Cm to the corresponding sampling points A1, A2, ...Am (or B1, B2, ...Bm) is calculated to obtain the difference between the two sampling points in each sampling pair. For example, the two sampling pairs are (0.1, 0.2) and (0.8, 0.9), and the amplitude difference of the two sampling pairs is 0.1, but the difference between the two sampling points in each sampling pair is 50% and 11.1%, respectively. Obviously, the difference between the two sampling points in the sampling pair (0.8, 0.9) is smaller, and the sampling pair is more similar.

[0068] The degree of difference is compared with the set amplitude difference threshold. If the degree of difference between two sampling points in a sampling pair is less than or equal to the amplitude difference threshold, the sampling pair is considered to be a similar sampling pair; if the degree of difference between two sampling points in a sampling pair is greater than the amplitude difference threshold, the sampling pair is considered to be dissimilar.

[0069] S405: Calculate the ratio of similar sampling pairs to the total sampling pairs.

[0070] S406: Determine whether the ratio of the similar sampling pairs to the total sampling pairs is greater than or equal to the first threshold. If it is greater than or equal to the first threshold, execute step S409; if it is less than the first threshold, execute step S407. The first threshold reflects the similarity between the first respiratory waveform and the sleep respiratory waveform, and is the key to determining whether the user is in a sleeping state. The threshold can be set according to the user's age, gender, health status and sleeping environment.

[0071] For example, for young and healthy users, in a relatively quiet sleeping environment, since the breathing waveform is relatively stable and regular, and the proportion of similar sampling pairs is large, the first threshold can be set relatively strictly; while for older users with respiratory diseases or other health problems, the collection of breathing waveforms may be subject to more interference, and the proportion of similar sampling pairs is small, then the setting of the first threshold should be relatively relaxed.

[0072] S407: Delete the collected user audio.

[0073] S408, delete all breathing waveforms, wait for a period of time, and then execute S401 again.

[0074] S409, determine whether the waveform comparison time reaches the preset time. If it does not reach the preset time, execute step S410; if it has reached the preset time, execute step S412.

[0075] S410 . Sample the user audio again, determine a second breathing waveform, and perform normalization processing on the second breathing waveform.

[0076] S411 , sampling the normalized first respiratory waveform and the normalized second respiratory waveform, determining the sampling logarithm, and returning to execute S403 .

[0077] S412: Stop audio playback.

[0078] S413: Delete the collected user audio.

[0079] In the embodiment of the present application, the first breathing waveform and the second breathing waveform are two breathing waveforms extracted from two audio segments recorded by the same user.

[0080] By adopting the audio control method proposed in the embodiment of the present disclosure, it is possible to automatically stop the currently playing audio when the user is in a sleeping state, which helps to improve the user's sleeping experience.

[0081] When collecting the actual situation of user audio files, it is necessary to realize that the user's environment may be complex and changeable, which will inevitably introduce additional sound elements into the recorded audio files. Specifically, the user may be in an environment with various background noises, such as family conversations, outdoor noises, or the operation of household appliances, etc. These external noises will leave obvious traces when recording audio.

[0082] In addition, if the user audio file is recorded when the user is asleep, the user himself may produce some non-respiratory sounds, the most common and significant of which is snoring, which usually occurs with changes in breathing rhythm, has a large volume and unique audio characteristics, and is easy to stand out in the audio. Moreover, the user may also talk in his sleep, that is, he makes speech sounds unconsciously, and these sounds will also be mixed into the recorded user audio file, increasing its complexity. Therefore, it is necessary to perform sound separation and waveform extraction operations on the above-mentioned user audio file. Here, taking the use of the first audio file to determine the first breathing waveform as an example, a method for extracting the user breathing waveform is introduced, and this method is also applicable to other situations of extracting the user breathing waveform.

[0083] In some implementations, the process of determining the first breathing waveform using the first audio file includes:

[0084] Dividing the first audio file into audio frames to obtain a first audio frame sequence;

[0085] Determine the frequency spectrum distribution corresponding to each audio frame in the first audio frame sequence, and extract the first breathing audio frame sequence from the first audio frame sequence according to the frequency spectrum distribution corresponding to the pre-stored breathing audio;

[0086] Determine a first breathing audio according to the first breathing audio frame sequence;

[0087] A first breathing waveform is determined according to the first breathing audio.

[0088] In the process of processing the first audio file to extract and determine the first breathing waveform, a key step is to subdivide the entire audio file into smaller units, namely audio frames. This process is called audio frame division, and its purpose is to facilitate the subsequent analysis and processing of the audio signal. Specifically, when dividing the first audio file into audio frames, it is first necessary to determine a suitable frame length, that is, the audio sample time contained in each audio frame, such as 20ms. According to the frame length, the audio processing tool can start from the starting position of the first audio file and intercept the audio data segment by segment to form a series of audio frames. These audio frames are arranged in the order of the original audio file to form a first audio frame sequence.

[0089] When determining the spectrum distribution corresponding to each audio frame in the first audio frame sequence, it is necessary to perform spectrum analysis on each audio frame. This usually involves converting the audio frame from the time domain to the frequency domain and performing spectrum analysis of the audio signal. The most commonly used technical means is Fast Fourier Transform (FFT), which can reveal the intensity and phase information of different frequency components in the audio frame, thereby achieving accurate analysis of different frequency components in the audio frame. After determining the spectrum distribution of each audio frame, the spectrum distribution corresponding to the pre-saved breathing audio can be used as a reference template and compared with the spectrum distribution of each audio frame in the first audio sequence. Through the comparison results, the audio frames whose spectrum distribution in the first audio file is most consistent with the pre-saved breathing audio spectrum distribution can be extracted to obtain the first breathing audio frame sequence.

[0090] The first breathing audio can be further determined based on the extracted first breathing audio frame sequence. This process can rearrange the filtered breathing audio frames according to their time sequence in the original audio file to form a continuous audio stream. In this way, the first breathing audio fully presents the characteristics of the sound changing over time during the breathing process, including information such as the periodicity, depth, and rate of breathing.

[0091] According to the first breathing audio, the first breathing waveform is determined, and this process can be implemented using a neural network model. Among them, the neural network model is pre-trained. During the training process, a large amount of breathing audio data is input into the neural network model, and the waveform corresponding to each breathing audio is provided as a training target, so as to conduct in-depth training and optimization of the model until the neural network model reaches a stable state during the training process and completes the model training. The first breathing audio is input into the trained and optimized neural network model, and the neural network model finally generates time series data corresponding to the first breathing audio by identifying the characteristics of the first breathing audio, such as pitch, loudness, and timbre, that is, the first breathing waveform, that is, the breathing waveform of the user in the current state.

[0092] Using this method, the user's breathing waveform can be accurately extracted from audio with noise, thereby providing strong support for the subsequent accurate judgment of the user's physiological state and control of the audio playback state.

[0093] In some implementations, sampling the first breathing waveform within N first cycles to obtain a first sampling result; and sampling the sleep breathing waveform within N second cycles to obtain a second sampling result, including:

[0094] Normalizing the first breathing waveforms in the N first cycles and the sleep breathing waveforms in the N second cycles respectively to obtain a normalized first breathing waveform and a normalized sleep breathing waveform;

[0095] From the normalized first breathing waveform, M first sampling points are determined, and the M first sampling points constitute a first sampling result; and from the normalized sleep breathing waveform, M second sampling points are determined, and the M second sampling points constitute a second sampling result; M is a positive integer.

[0096] Normalizing the two waveforms before waveform sampling helps to eliminate the dimension and scale differences between different waveforms, thereby improving the comparability of the sampled data and the accuracy of subsequent processing.

[0097] In some embodiments, comparing the first sampling result with the second sampling result includes:

[0098] Grouping M first sampling points and M second sampling points into M sampling pairs, each sampling pair including a first sampling point and a second sampling point, and a sampling position of the first sampling point in each sampling pair is the same as a sampling position of the second sampling point;

[0099] The two sampling points in each sampling pair are compared respectively, and the similarity between the first sampling result and the second sampling result is determined according to the comparison results of the plurality of sampling pairs.

[0100] In the embodiment of the present application, the periods of the two waveforms used for comparison may be different, so directly sampling at fixed time intervals may not be sufficient to fully reflect the relative dynamics between the waveforms. In order to more accurately analyze the relationship between the two waveforms, a waveform equal-division sampling method can be used, that is, each waveform is divided into M intervals within a complete cycle, and the end point (or starting point, the two waveforms must remain consistent) of each interval is used as a sampling point. Alternatively, a non-equal-division sampling method can be used, for example, a more intensive sampling is performed in the middle part of a cycle, and a more sparse sampling is performed at the beginning and end of a cycle. Using the above waveform sampling method, for different waveforms with different periods, it is possible to ensure that synchronous sampling is performed at the same position of the two waveforms being compared, thereby facilitating subsequent comparison and analysis.

[0101] In some implementations, comparing two sampling points in each sampling pair respectively, and determining the similarity between the first sampling result and the second sampling result according to the comparison results of the plurality of sampling pairs, comprises:

[0102] Calculate the amplitude difference between the two sampling points in each sampling pair respectively;

[0103] When the amplitude difference is less than or equal to the amplitude difference threshold, determining the sampling pair as a similar sampling pair;

[0104] Calculate the ratio of the number of similar sampling pairs to M;

[0105] Based on the ratio, the degree of similarity between the first sampling result and the second sampling result is determined.

[0106] By adopting the above steps, the differences between the sampling pairs can be accurately quantified, and similar sampling pairs in the two waveforms can be determined based on the preset amplitude difference threshold; further, by calculating the ratio of the number of similar sampling pairs to the total number of sampling pairs M, an intuitive indicator can be obtained, which accurately reflects the similarity between the first sampling result and the second sampling result, and further reflects the similarity between the first breathing waveform and the sleep breathing waveform, which is helpful for subsequent audio control decisions.

[0107] In some embodiments, the normalization process includes:

[0108] The amplitude of the waveform is divided by the maximum amplitude of the waveform to obtain a normalized waveform.

[0109] In the embodiment of the present application, the core idea of ​​normalization is to adjust the amplitude values ​​of different waveforms to a uniform range, usually by scaling all the amplitude values ​​of the waveforms to between 0 and 1. This process can be achieved by calculating each amplitude value of the waveform divided by the maximum amplitude value of the waveform.

[0110] Specifically, for a given arbitrary waveform, we first need to traverse all waveform amplitude values ​​within its definition domain and find the maximum amplitude value. Next, we calculate the ratio of all amplitude values ​​in the waveform to the maximum amplitude value. Through this calculation, all amplitude values ​​in the original waveform are scaled proportionally, so that the maximum amplitude of the normalized waveform is 1, and the minimum amplitude value is 0 (for a completely non-negative waveform) or a negative number (for a waveform containing negative amplitudes).

[0111] The normalized waveform retains the shape characteristics of the original waveform, such as period, frequency, and phase relationship, and removes the difference in amplitude magnitude, so that different waveforms can be compared on the same scale, thereby improving the accuracy of comparison between waveforms.

[0112] In some implementations, controlling the audio according to the comparison result includes:

[0113] When the similarity between the first sampling result and the second sampling result is greater than or equal to a first threshold, the audio playback is stopped.

[0114] In some implementations, controlling the audio according to the comparison result includes:

[0115] When the degree of similarity between the first sampling result and the second sampling result is greater than or equal to the first threshold, collecting a second audio file, and determining a second breathing waveform using the second audio file;

[0116] Determine a third period of the second respiratory waveform, and sample the second respiratory waveform within N third periods to obtain a third sampling result;

[0117] When the similarity between the first sampling result and the third sampling result is greater than or equal to the first threshold, stop playing the audio.

[0118] In the embodiment of the present application, the premise for collecting the second audio file and determining the second breathing waveform is that the similarity between the first sampling result and the second sampling result is greater than or equal to the first threshold, which means that the first breathing waveform is consistent with the sleep breathing waveform. However, given that the period of the initial comparison is relatively short, this alone is not enough to determine that the user is in a sleeping state.

[0119] Therefore, in order to enhance the accuracy and reliability of user status judgment, the user audio is collected again, and the user's breathing waveform (i.e., the second breathing waveform) is extracted therefrom. Subsequently, the second breathing waveform is sampled and compared with the first breathing waveform within a period of time. For specific sampling methods, refer to S410-S411, and for specific comparison methods, refer to S403-S406. When the similarity between the first breathing waveform and the second breathing waveform is less than the first threshold, it is considered that the user is in a wakeful state, and S407 is executed; when the similarity between the first breathing waveform and the second breathing waveform is equal to or greater than the first threshold, and after the two waveform comparisons are completed, the waveform comparison time reaches the preset time requirement, then it is confirmed that the user is in a sleeping state, and S412 is executed. Through this more detailed comparison process, it is possible to more accurately verify and confirm whether the user is currently in a sleeping state, thereby accurately completing the control of the audio.

[0120] The present disclosure also provides an audio control device. Figure 5 is a structural diagram of an audio control device 500 according to an embodiment of the present disclosure, comprising: a waveform determination module 510, a period determination module 520, a sampling module 530 and a control module 540: wherein,

[0121] The waveform determination module 510 is used to collect a first audio file during audio playback; and determine a first breathing waveform using the first audio file;

[0122] A cycle determination module 520, for determining a first cycle of a first respiratory waveform and a second cycle of a pre-stored sleep respiratory waveform;

[0123] The sampling module 530 is used to sample the first breathing waveform in N first cycles to obtain a first sampling result; and to sample the sleep breathing waveform in N second cycles to obtain a second sampling result; N is a positive integer;

[0124] The control module 540 is used to compare the first sampling result with the second sampling result, and control the audio according to the comparison result.

[0125] In some embodiments, the waveform determination module 510 is used to:

[0126] Dividing the first audio file into audio frames to obtain a first audio frame sequence;

[0127] Determine the frequency spectrum distribution corresponding to each audio frame in the first audio frame sequence, and extract a first breathing audio frame sequence from the first audio frame sequence according to the frequency spectrum distribution corresponding to each audio frame and the frequency spectrum distribution corresponding to the pre-stored breathing audio;

[0128] Determine a first breathing audio according to the first breathing audio frame sequence;

[0129] A first breathing waveform is determined according to the first breathing audio.

[0130] Figure 6 is a structural diagram of an audio control device 600 according to an embodiment of the present disclosure. In some implementations, the acquisition module 530 includes a normalization submodule 531 and a sampling point determination submodule 532: wherein,

[0131] A normalization submodule 531 is used to perform normalization processing on the first breathing waveforms in the N first cycles and the sleep breathing waveforms in the N second cycles, respectively, to obtain the normalized first breathing waveform and the normalized sleep breathing waveform;

[0132] The sampling point determination submodule 532 is used to determine M first sampling points from the first breathing waveform after normalization, and the M first sampling points constitute a first sampling result; and to determine M second sampling points from the sleep breathing waveform after normalization, and the M second sampling points constitute a second sampling result; M is a positive integer.

[0133] In some implementations, the control module 540 includes a sampling pair determination submodule 541 and a comparison submodule 542: wherein,

[0134] A sampling pair determination submodule 541 is used to form M sampling pairs from the M first sampling points and the M second sampling points, each sampling pair includes a first sampling point and a second sampling point, and the sampling position of the first sampling point in each sampling pair is the same as the sampling position of the second sampling point;

[0135] The comparison submodule 542 is used to compare the two sampling points in each sampling pair respectively, and determine the similarity between the first sampling result and the second sampling result according to the comparison results of the multiple sampling pairs.

[0136] In some embodiments, the comparison submodule 542 is used to:

[0137] Calculate the amplitude difference between the two sampling points in each sampling pair respectively;

[0138] When the amplitude difference is less than or equal to the amplitude difference threshold, determining the sampling pair as a similar sampling pair;

[0139] Calculate the ratio of the number of similar sampling pairs to M;

[0140] Based on the ratio, the degree of similarity between the first sampling result and the second sampling result is determined.

[0141] In some embodiments, the normalization process includes:

[0142] The amplitude of the waveform is divided by the maximum amplitude of the waveform to obtain a normalized waveform.

[0143] In some embodiments, the control module 540 is used to:

[0144] When the similarity between the first sampling result and the second sampling result is greater than or equal to a first threshold, the audio playback is stopped.

[0145] In some embodiments, the control module 540 is used to:

[0146] When the degree of similarity between the first sampling result and the second sampling result is greater than or equal to the first threshold, collecting a second audio file, and determining a second breathing waveform using the second audio file;

[0147] Determine a third period of the second respiratory waveform, and sample the second respiratory waveform within N third periods to obtain a third sampling result;

[0148] When the similarity between the first sampling result and the third sampling result is greater than or equal to the first threshold, stop playing the audio.

[0149] In some implementations, the audio control device 600 further includes: a device determination module 650, configured to determine an audio collection device;

[0150] The setting module 660 is used to set at least one of the sound collection frequency and the channel of the audio collection device.

[0151] For the description of specific functions and examples of each module and submodule of the device in the embodiment of the present disclosure, reference can be made to the relevant description of the corresponding steps in the above method embodiment, which will not be repeated here.

[0152] In the technical solution disclosed in the present invention, the acquisition, storage and application of personal information of users involved are in compliance with the provisions of relevant laws and regulations and do not violate public order and good morals.

[0153] According to an embodiment of the present disclosure, the present disclosure also provides an electronic device, a readable storage medium and a computer program product.

[0154] Figure 7 A schematic block diagram of an example electronic device 700 that can be used to implement an embodiment of the present disclosure is shown. The electronic device is intended to represent various forms of digital computers, such as laptop computers, desktop computers, workstations, personal digital assistants, servers, blade servers, mainframe computers, and other suitable computers. The electronic device can also represent various forms of mobile devices, such as personal digital assistants, cellular phones, smart phones, wearable devices, and other similar computing devices. The components shown herein, their connections and relationships, and their functions are merely examples and are not intended to limit the implementation of the present disclosure described and / or required herein.

[0155] like Figure 7 As shown, the device 700 includes a computing unit 701, which can perform various appropriate actions and processes according to a computer program stored in a read-only memory (ROM) 702 or a computer program loaded from a storage unit 708 into a random access memory (RAM) 703. In the RAM 703, various programs and data required for the operation of the device 700 can also be stored. The computing unit 701, the ROM 702, and the RAM 703 are connected to each other via a bus 704. An input / output (I / O) interface 705 is also connected to the bus 704.

[0156] A number of components in the device 700 are connected to the I / O interface 705, including: an input unit 706, such as a keyboard, a mouse, etc.; an output unit 707, such as various types of displays, speakers, etc.; a storage unit 708, such as a disk, an optical disk, etc.; and a communication unit 709, such as a network card, a modem, a wireless communication transceiver, etc. The communication unit 709 allows the device 700 to exchange data with other devices through a computer network such as the Internet and / or various telecommunication networks.

[0157] The computing unit 701 may be a variety of general and / or special processing components with processing and computing capabilities. Some examples of the computing unit 701 include, but are not limited to, a central processing unit (CPU), a graphics processing unit (GPU), various dedicated artificial intelligence (AI) computing chips, various computing units running machine learning model algorithms, digital signal processors (DSPs), and any appropriate processors, controllers, microcontrollers, etc. The computing unit 701 performs the various methods and processes described above, such as detection methods. For example, in some embodiments, the detection method may be implemented as a computer software program, which is tangibly contained in a machine-readable medium, such as a storage unit 708. In some embodiments, part or all of the computer program may be loaded and / or installed on the device 700 via ROM 702 and / or communication unit 709. When the computer program is loaded into RAM 703 and executed by the computing unit 701, one or more steps of the detection method described above may be performed. Alternatively, in other embodiments, the computing unit 701 may be configured to perform the detection method in any other appropriate manner (e.g., by means of firmware).

[0158] Various implementations of the systems and techniques described above herein can be implemented in digital electronic circuit systems, integrated circuit systems, field programmable gate arrays (FPGAs), application specific integrated circuits (ASICs), application specific standard products (ASSPs), systems on chips (SOCs), load programmable logic devices (CPLDs), computer hardware, firmware, software, and / or combinations thereof. These various implementations can include: being implemented in one or more computer programs that can be executed and / or interpreted on a programmable system including at least one programmable processor, which can be a special purpose or general purpose programmable processor that can receive data and instructions from a storage system, at least one input device, and at least one output device, and transmit data and instructions to the storage system, the at least one input device, and the at least one output device.

[0159] The program code for implementing the method of the present disclosure may be written in any combination of one or more programming languages. These program codes may be provided to a processor or controller of a general-purpose computer, a special-purpose computer, or other programmable data processing device, so that the program code, when executed by the processor or controller, enables the functions / operations specified in the flow chart and / or block diagram to be implemented. The program code may be executed entirely on the machine, partially on the machine, partially on the machine and partially on a remote machine as a stand-alone software package, or entirely on a remote machine or server.

[0160] In the context of the present disclosure, a machine-readable medium may be a tangible medium that may contain or store a program for use by or in conjunction with an instruction execution system, device, or equipment. A machine-readable medium may be a machine-readable signal medium or a machine-readable storage medium. A machine-readable medium may include, but is not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, device, or equipment, or any suitable combination of the foregoing. A more specific example of a machine-readable storage medium may include an electrical connection based on one or more lines, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing.

[0161] To provide interaction with a user, the systems and techniques described herein can be implemented on a computer having: a display device (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor) for displaying to the user; and a keyboard and pointing device (e.g., a mouse or trackball) through which the user can provide input to the computer. Other types of devices can also be used to provide interaction with the user; for example, the feedback provided to the user can be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user can be received in any form (including acoustic input, voice input, or tactile input).

[0162] The systems and techniques described herein may be implemented in a computing system that includes back-end components (e.g., as a data server), or a computing system that includes middleware components (e.g., an application server), or a computing system that includes front-end components (e.g., a user computer with a graphical user interface or a web browser through which a user can interact with implementations of the systems and techniques described herein), or a computing system that includes any combination of such back-end components, middleware components, or front-end components. The components of the system may be interconnected by any form or medium of digital data communication (e.g., a communication network). Examples of communication networks include: a local area network (LAN), a wide area network (WAN), and the Internet.

[0163] A computer system may include a client and a server. The client and the server are generally remote from each other and usually interact through a communication network. The relationship of client and server is generated by computer programs running on respective computers and having a client-server relationship with each other. The server may be a cloud server, a server of a distributed system, or a server combined with a blockchain.

[0164] It should be understood that the various forms of processes shown above can be used to reorder, add or delete steps. For example, the steps recorded in this disclosure can be executed in parallel, sequentially or in different orders, as long as the desired results of the technical solutions disclosed in this disclosure can be achieved, and this document does not limit this.

[0165] The above specific implementations do not constitute a limitation on the protection scope of the present disclosure. It should be understood by those skilled in the art that various modifications, combinations, sub-combinations and substitutions can be made according to design requirements and other factors. Any modification, equivalent substitution and improvement made within the principles of the present disclosure shall be included in the protection scope of the present disclosure.

Claims

1. An audio control method, comprising: During audio playback, collecting a first audio file; Determining a first breathing waveform using the first audio file; Determining a first cycle of the first respiratory waveform and a second cycle of a pre-stored sleep respiratory waveform; In N of the first periods, sampling the first breathing waveform to obtain a first sampling result; and sampling the sleep breathing waveform within N of the second periods to obtain a second sampling result; wherein N is a positive integer; The first sampling result is compared with the second sampling result, and the audio is controlled according to the comparison result.

2. The method according to claim 1, wherein: The determining a first breathing waveform by using the first audio file includes: Dividing the first audio file into audio frames to obtain a first audio frame sequence; Determine a frequency spectrum distribution corresponding to each audio frame in the first audio frame sequence, and extract a first breathing audio frame sequence from the first audio frame sequence according to the frequency spectrum distribution corresponding to each audio frame and the frequency spectrum distribution corresponding to the pre-stored breathing audio; Determine a first breathing audio according to the first breathing audio frame sequence; The first breathing waveform is determined according to the first breathing audio.

3. The method according to claim 1 or 2, wherein: The first breathing waveform is sampled within N of the first periods to obtain a first sampling result; And within N of the second periods, the sleep breathing waveform is sampled to obtain a second sampling result, including: Normalizing the first breathing waveforms in the N first cycles and the sleep breathing waveforms in the N second cycles respectively to obtain a normalized first breathing waveform and a normalized sleep breathing waveform; From the normalized first breathing waveform, M first sampling points are determined, and the M first sampling points constitute the first sampling result; and from the normalized sleep breathing waveform, M second sampling points are determined, and the M second sampling points constitute the second sampling result; M is a positive integer.

4. The method according to claim 3, wherein: The comparing the first sampling result with the second sampling result includes: The M first sampling points and the M second sampling points are grouped into M sampling pairs, each of the sampling pairs includes a first sampling point and a second sampling point, and the sampling position of the first sampling point in each sampling pair is the same as the sampling position of the second sampling point; The two sampling points in each of the sampling pairs are compared respectively, and the similarity between the first sampling result and the second sampling result is determined according to the comparison results of the plurality of sampling pairs.

5. The method according to claim 4, wherein: The step of respectively comparing the two sampling points in each of the sampling pairs and determining the similarity between the first sampling result and the second sampling result according to the comparison results of the plurality of sampling pairs includes: respectively calculating the amplitude difference between two sampling points in each sampling pair; When the amplitude difference is less than or equal to the amplitude difference threshold, determining that the sampling pair is a similar sampling pair; Calculate the ratio of the number of similar sampling pairs to M; Based on the ratio, a degree of similarity between the first sampling result and the second sampling result is determined.

6. The method according to any one of claims 3 to 5, wherein: The normalization process includes: The amplitude of a waveform is divided by the maximum amplitude of the waveform to obtain a normalized waveform.

7. The method according to any one of claims 1 to 6, wherein: The controlling the audio according to the comparison result includes: When the similarity between the first sampling result and the second sampling result is greater than or equal to a first threshold, stop playing the audio.

8. The method according to any one of claims 1 to 6, wherein: The controlling the audio according to the comparison result includes: When the degree of similarity between the first sampling result and the second sampling result is greater than or equal to a first threshold, collecting a second audio file, and determining a second breathing waveform using the second audio file; Determine a third period of the second breathing waveform, and sample the second breathing waveform within N third periods to obtain a third sampling result; When the degree of similarity between the first sampling result and the third sampling result is greater than or equal to a first threshold, stop playing the audio.

9. The method according to any one of claims 1 to 8, before collecting the first audio file, further comprising: Determine the audio acquisition device; At least one of the sound collection frequency and the sound channel of the audio collection device is set.

10. An audio control device, comprising: A waveform determination module, used for collecting a first audio file during audio playback; Determining a first breathing waveform using the first audio file; A cycle determination module, used to determine a first cycle of the first respiratory waveform and a second cycle of a pre-stored sleep respiratory waveform; A sampling module, used for sampling the first breathing waveform within N first periods to obtain a first sampling result; and sampling the sleep breathing waveform within N of the second periods to obtain a second sampling result; Said N is a positive integer; A control module is used to compare the first sampling result with the second sampling result, and control the audio according to the comparison result.

11. The device according to claim 10, wherein: The sampling module comprises: A normalization submodule, used for normalizing the first breathing waveforms in the N first cycles and the sleep breathing waveforms in the N second cycles, respectively, to obtain a normalized first breathing waveform and a normalized sleep breathing waveform; The sampling point determination submodule is used to determine M first sampling points from the first respiratory waveform after normalization, and the M first sampling points constitute the first sampling result; and determine M second sampling points from the sleep respiratory waveform after normalization, and the M second sampling points constitute the second sampling result; M is a positive integer.

12. The device according to claim 10, wherein: The control module comprises: a sampling pair determination submodule, configured to form M sampling pairs from the M first sampling points and the M second sampling points, each of the sampling pairs comprising a first sampling point and a second sampling point, and a sampling position of the first sampling point in each of the sampling pairs is the same as a sampling position of the second sampling point; The comparison submodule is used to compare the two sampling points in each of the sampling pairs respectively, and determine the similarity between the first sampling result and the second sampling result according to the comparison results of multiple sampling pairs.

13. An electronic device comprising: at least one processor; as well as a memory communicatively connected to the at least one processor; wherein, The memory stores instructions that can be executed by the at least one processor, and the instructions are executed by the at least one processor to enable the at least one processor to perform the method according to any one of claims 1 to 9.

14. A non-transitory computer-readable storage medium storing computer instructions, wherein: The computer instructions are used to cause the computer to execute the method according to any one of claims 1-9.

15. A computer program product, comprising a computer program, which, when executed by a processor, implements the method according to any one of claims 1 to 9.

Images