Snore detection method and model training method, and related devices

Abstract

The application discloses a snoring sound detection method and device, a snoring sound detection model training method and device. The snoring sound detection method comprises the following steps: acquiring an audio signal in an environment; pre-processing the audio signal to obtain a to-be-detected signal; obtaining frequency spectrum information of the to-be-detected signal; inputting the frequency spectrum information of the to-be-detected signal into a snoring sound detection model to obtain a detection result of the audio signal, wherein the detection result is used for representing whether the audio signal comprises a snoring sound signal or not. Technical support is provided for realizing low-power snoring sound detection.

Description

TECHNICAL FIELD

[0001] The present application relates to the field of signal processing, in particular to a snoring sound detection method and device, and a snoring sound detection model training method and device. BACKGROUND

[0002] Since most of the snoring sound signals have high energy mainly concentrated in the 50-2000HZ bandwidth range, the snoring sound detection device based on the audio signal needs to have a high sampling rate and high computing power to successfully detect the snoring sound signal in the audio signal. The high sampling rate and high computing power will inevitably lead to high power consumption, which will reduce the endurance. Therefore, how to realize low-power snoring sound detection has become a technical problem to be solved. SUMMARY

[0003] The present application provides a snoring sound detection method and device, and a snoring sound detection model training method and device, to at least solve the above technical problems in the prior art.

[0004] The present application provides a snoring sound detection method, which comprises:

[0005] obtaining an audio signal in an environment;

[0006] preprocessing the audio signal to obtain a to-be-detected signal;

[0007] obtaining the frequency spectrum information of the to-be-detected signal;

[0008] inputting the frequency spectrum information of the to-be-detected signal into a snoring sound detection model to obtain a detection result of the audio signal, wherein the detection result is used to represent whether the audio signal includes a snoring sound signal or not.

[0009] In the above scheme, the preprocessing of the audio signal to obtain the to-be-detected signal comprises:

[0010] filtering the audio signal to obtain a filtered signal;

[0011] detecting the filtered signal by using an endpoint detection algorithm to obtain the to-be-detected signal.

[0012] In the above scheme, the detecting the filtered signal by using the endpoint detection algorithm to obtain the to-be-detected signal comprises:

[0013] using the endpoint detection algorithm to frame the filtered signal to obtain each frame signal of the filtered signal;

[0014] calculating the decibel number of each frame signal, and determining the to-be-detected signal from each frame signal of the filtered signal according to the decibel number of each frame signal.

[0015] In the foregoing solution, the obtaining of the frequency spectrum information of the to-be-detected signal comprises:

[0016] transforming each frame of signal of the to-be-detected signal from time domain to frequency domain to obtain a frequency spectrum of each frame of signal;

[0017] obtaining a spectrogram of the to-be-detected signal according to the frequency spectrum of each frame of signal;

[0018] obtaining the frequency spectrum information of the to-be-detected signal based on the spectrogram.

[0019] In the foregoing solution, the obtaining of the frequency spectrum information of the to-be-detected signal based on the spectrogram comprises:

[0020] performing frequency spectrum mapping on the spectrogram to obtain a mel frequency spectrum;

[0021] taking the mel frequency spectrum as the frequency spectrum information of the to-be-detected signal.

[0022] In the foregoing solution, before the inputting of the frequency spectrum information of the to-be-detected signal into the snoring sound detection model to obtain the detection result of the audio signal, the method further comprises:

[0023] obtaining an audio sample signal in an environment and a sample label, the sample label being used to represent that the audio sample signal is an audio signal including snoring sound or an audio signal not including snoring sound;

[0024] preprocessing the audio sample signal to obtain a to-be-detected sample signal;

[0025] obtaining frequency spectrum information of the to-be-detected sample signal;

[0026] taking the frequency spectrum information of the to-be-detected sample signal as an input of a to-be-trained model and taking the sample label as an output of the to-be-trained model, and training the to-be-trained model to obtain the snoring sound detection model.

[0027] The application provides a training method of a snoring sound detection model, and the method comprises:

[0028] obtaining an audio sample signal in an environment and a sample label, the sample label being used to represent that the audio sample signal is an audio signal including snoring sound or an audio signal not including snoring sound;

[0029] preprocessing the audio sample signal to obtain a to-be-detected sample signal;

[0030] obtaining frequency spectrum information of the to-be-detected sample signal;

[0031] The spectrum information of the to-be-detected sample signal is taken as the input of the to-be-trained model, the sample label is taken as the output of the to-be-trained model, the to-be-trained model is trained, and a snoring sound detection model is obtained, which is used to detect whether an audio signal in an environment includes a snoring sound signal.

[0032] In the foregoing solution, the pre-processing of the audio sample signal to obtain the to-be-detected sample signal comprises:

[0033] The audio sample signal is filtered to obtain a filtered sample signal.

[0034] An endpoint detection algorithm is used to detect the filtered sample signal to obtain the to-be-detected sample signal.

[0035] The present application provides a snoring sound detection device, comprising:

[0036] A first obtaining unit is configured to obtain an audio signal in an environment.

[0037] A first pre-processing unit is configured to pre-process the audio signal to obtain a to-be-detected signal.

[0038] A second obtaining unit is configured to obtain spectrum information of the to-be-detected signal.

[0039] A detection unit is configured to input the spectrum information of the to-be-detected signal into a snoring sound detection model to obtain a detection result of the audio signal, wherein the detection result is used to represent whether the audio signal includes a snoring sound signal.

[0040] The present application provides a snoring sound detection model training device, comprising:

[0041] A first obtaining module is configured to obtain an audio sample signal in an environment and a sample label, wherein the sample label is used to represent whether the audio sample signal is an audio signal including snoring sound or an audio signal not including snoring sound.

[0042] A first pre-processing module is configured to pre-process the audio sample signal to obtain a to-be-detected sample signal.

[0043] A second obtaining module is configured to obtain spectrum information of the to-be-detected sample signal.

[0044] A training module is configured to take the spectrum information of the to-be-detected sample signal as the input of a to-be-trained model, take the sample label as the output of the to-be-trained model, train the to-be-trained model, and obtain a snoring sound detection model, which is used to detect whether an audio signal in an environment includes a snoring sound signal.

[0045] The application provides a snoring sound detection method and device, a snoring sound detection model training method and device, wherein the snoring sound detection method comprises the following steps: acquiring an audio signal in an environment; pre-processing the audio signal to obtain a to-be-detected signal; obtaining frequency spectrum information of the to-be-detected signal; inputting the frequency spectrum information of the to-be-detected signal into a snoring sound detection model to obtain a detection result of the audio signal, wherein the detection result is used to represent whether the audio signal includes a snoring sound signal or not. The detection result can be obtained by inputting the processing result into the snoring sound detection model through pre-processing and spectrum calculation of the audio signal, so that the process is not complicated, the data calculation amount is effectively reduced, technical support is provided for realizing low-power snoring sound detection, and the endurance is improved.

[0046] It should be understood that the content described in this part is not intended to identify key or important features of the embodiments of the application, nor is it used to limit the scope of the application. Other features of the application will become apparent from the following description. BRIEF DESCRIPTION OF DRAWINGS

[0047] The above and other objects, features and advantages of the exemplary embodiments of the present application will be more apparent from the following detailed description read in conjunction with the accompanying drawings, in which:

[0048] In the drawings, identical or corresponding reference numerals indicate identical or corresponding parts.

[0049] Figure 1 An implementation process schematic diagram of a snoring sound detection model training method of an embodiment of the application is shown;

[0050] Figure 2 A training process schematic diagram of a to-be-trained model of an embodiment of the application is shown;

[0051] Figure 3 An implementation process schematic diagram of a snoring sound detection method of an embodiment of the application is shown Figure 1 ;

[0052] Figure 4 An implementation process schematic diagram of a snoring sound detection method of an embodiment of the application is shown Figure 2 ;

[0053] Figure 5 A component structure schematic diagram of a snoring sound detection device of an embodiment of the application is shown;

[0054] Figure 6 A component structure schematic diagram of a snoring sound detection model training device of an embodiment of the application is shown. DETAILED DESCRIPTION

[0055] In order to make the purposes, characteristics and advantages of the present application more obvious and easy to understand, the technical solutions in the embodiments of the present application will be clearly and completely described below with reference to the drawings in the embodiments of the present application. Obviously, the described embodiments are only a part of the embodiments of the present application, but not all the embodiments. Based on the embodiments in the present application, all other embodiments obtained by those skilled in the art without creative labor are within the scope of protection of the present application.

[0056] In practical applications, there are mainly three methods for snoring detection of audio signals: the first method is based on traditional signal processing, such as a double-threshold method of time-frequency characteristics and an autocorrelation function method. The second method is based on machine learning, such as a support vector machine method and an artificial neural network method. The third method is based on deep learning, such as a convolutional neural network and a long short-term memory network.

[0057] Among the three methods, the traditional signal processing method is simple in design and easy to implement, but the separation of snoring is not ideal in a low signal-to-noise ratio. The machine learning method usually has a relatively high accuracy, but has the disadvantages of relying on prior knowledge, complex operation and large amount of calculation. The deep learning has automatic feature engineering and does not need to rely on too much prior knowledge, but has high energy consumption.

[0058] As can be seen, considering that most of the energy of snoring signals is mainly concentrated in a bandwidth of 50-2000HZ, the snoring detection based on audio signals at least sets a sampling rate of 5 times the highest frequency (2000HZ), which requires a high sampling rate. The range of a digital audio signal usually uses 16-bit quantization bits, and the data amount of 1 second is 31.25KB, which requires strong computing power. High sampling rate and high computing power will inevitably lead to high power consumption, which will reduce the endurance. The technical solution of the present application can reduce the amount of data calculation and realize low-power snoring detection, thereby improving the endurance of the snoring detection device.

[0059] The technical solution of the embodiment of the present application relates to a scheme for training a snoring detection model and a scheme for snoring detection of an acquired audio signal by using the trained snoring detection model.

[0060] The embodiment of the present application provides a training method of a snoring detection model, as shown in Figure 1 The method comprises the following steps:

[0061] S101: acquiring an audio sample signal in an environment and a sample label, the sample label being used to represent whether the audio sample signal is an audio signal including snoring or an audio signal not including snoring;

[0062] In this step, the snoring sound detection model training device collects audio signals in the environment as audio sample signals, which include snoring sound and do not include snoring sound. The audio sample signals can be multiple. Among these audio sample signals, they are mainly divided into two categories, one is the audio signal including snoring sound, and the other is the audio signal not including snoring sound.

[0063] By training the snoring sound detection model through audio signals including snoring sound and audio signals not including snoring sound, the accuracy of the snoring sound detection model when detecting snoring sound can be ensured.

[0064] It can be understood that the audio sample signals in the embodiments of the present application are all fixed-length audio signals.

[0065] S102: Preprocess the audio sample signal to obtain a to-be-detected sample signal;

[0066] In this step, since the audio sample signal is an audio signal including all possible environmental sounds, by preprocessing the audio sample signal, an audio sample signal that can be snoring sound, i.e., a to-be-detected sample signal, is obtained. The possible environmental sounds include but are not limited to snoring sound, noise, and other environmental sounds other than snoring sound and noise.

[0067] The preprocessing of the audio sample signal to obtain the to-be-detected sample signal includes: filtering the audio sample signal to obtain a filtered sample signal; and using an endpoint detection algorithm to detect the filtered sample signal to obtain the to-be-detected sample signal.

[0068] Further, the endpoint detection algorithm is used to detect the filtered sample signal to obtain the to-be-detected sample signal, which includes:

[0069] The endpoint detection algorithm is used to frame the filtered sample signal to obtain each frame signal of the filtered sample signal; the decibel number of each frame signal is calculated, and the to-be-detected sample signal is determined from the frame signals of the filtered sample signal according to the decibel number of each frame signal.

[0070] It can be understood that since the energy of most snoring sound signals is mainly concentrated in the 50-2000HZ bandwidth range, the audio sample signal can be subjected to 50-2000Hz second-order Butterworth band-pass filtering processing, so as to suppress other environmental sounds and noise in the audio sample signal except the main snoring sound bandwidth, and obtain the filtered sample signal.

[0071] The time-domain endpoint detection method is used to frame the filtered sample signal by a signal window with a length of 25 milliseconds and a step length of 10 milliseconds, to obtain each frame signal of the filtered sample signal, and to calculate the decibel number of each frame signal. If the decibel number of a frame signal is higher than or equal to a preset first threshold value, for example, 45 decibels, it is considered that the frame signal may include snoring sound, and the subsequent operation is continued. If the decibel number of each frame signal in M (M is a positive integer greater than or equal to 2, for example, M = 5) consecutive frame signals is less than 40 decibels, it is considered that each frame signal in the M consecutive frame signals does not include snoring sound, and the set of frame signals including snoring sound signals is regarded as a to-be-detected sample signal.

[0072] In this application, the sideband pass filtering and the framing operations are performed on the audio sample signal. Such consecutive operations can speed up the processing flow of the audio sample signal and shorten the processing period.

[0073] In the foregoing scheme, by using the endpoint detection method, specifically the time-domain endpoint detection method, the computational duty cycle of the snoring sound detection algorithm can be reduced, thereby reducing the power consumption of the device and improving the endurance time of the snoring sound detection device.

[0074] In this application, the sideband pass filtering is used to filter part of the noise other than the snoring sound signal, and the endpoint detection algorithm is used to detect the signal that may include snoring sound. Through two processes, the audio sample signal that may include snoring sound, i.e., the to-be-detected sample signal, can be obtained. Compared with the prior art in which the audio signal collected by containing all environmental sounds is directly input into the to-be-trained model for training, the frequency spectrum information of the to-be-detected sample signal obtained after two pre-processings such as filtering and endpoint detection is input into the to-be-trained model for training in this application, which greatly reduces the data calculation amount, and thus the endurance capability of the snoring sound detection device can be improved.

[0075] S103: Obtain the frequency spectrum information of the to-be-detected sample signal;

[0076] In this step, the frequency spectrum information of the to-be-detected sample signal includes the mel spectrum of the to-be-detected sample signal.

[0077] The frequency spectrum information of the to-be-detected sample signal is obtained by:

[0078] The frequency spectrum of each frame signal of the to-be-detected sample signal is obtained by transforming each frame signal of the to-be-detected sample signal from the time domain to the frequency domain;

[0079] The spectrogram of the to-be-detected sample signal is obtained according to the frequency spectrum of each frame signal.

[0080] The frequency spectrum information of the to-be-detected sample signal is obtained based on the spectrogram.

[0081] Further, based on the spectrogram, the spectral information of the sample signal to be detected is obtained, including: performing spectral mapping on the spectrogram to obtain the Mel spectrum; and using the Mel spectrum as the spectral information of the sample signal to be detected.

[0082] The sample signal to be detected is determined through preprocessing. Each frame of the sample signal is then subjected to a short-time Fourier transform (SFT) with a length of 25 milliseconds, a step size of 10 milliseconds, and a periodic Hann time window to obtain the spectrum of each frame. The spectra of each frame are then stacked along a certain dimension (e.g., the time dimension) to obtain a two-dimensional image containing the sound data of the sample signal to be detected, i.e., the spectrogram of the sample signal. The dimensions of the two-dimensional image refer to both the time and frequency dimensions. The spectrogram is then mapped to 64 metronome (mel) bands covering the 50-2000 Hz range to calculate the mel spectrogram. The mel spectrogram is then framed into examples with 50% overlap and a length of 0.96 seconds. Each example covers 64 mel bands, for a total of 96 frames, each frame being 10 milliseconds long. The mel spectrum of each example is the spectral information of the sample signal to be detected.

[0083] Since sound signals are one-dimensional time-domain signals, it is difficult to intuitively see the frequency variation pattern. Therefore, time-domain signals are usually converted into frequency-domain signals. Compared to directly inputting the sample signal to be detected into the model to be trained, using the Mel spectrum of the sample signal to be detected as input to the model to be trained improves the accuracy of model training.

[0084] S104: The spectral information of the sample signal to be detected is used as the input of the model to be trained, and the sample label is used as the output of the model to be trained. The model to be trained is trained to obtain a snoring detection model. The snoring detection model is used to detect whether the audio signal in the environment includes a snoring signal.

[0085] In this step, the model to be trained includes two types of outputs: audio signals that include snoring or audio signals that do not include snoring.

[0086] like Figure 2 As shown, the Mel spectrogram (Mel spectrum) obtained by processing the target domain (the sample signal to be detected) obtained from the preprocessing of the audio sample signal is input into the training model to train the model. The training model has a total of 86 layers, with the 84th layer containing two output classes: audio signals including snoring or audio signals excluding snoring. In this application, the snoring detection model can be obtained by training only the parameters of the 84th layer.

[0087] The to-be-trained model in the present application can be obtained by structural migration from a pre-trained model YAMNet model. Specifically, the YAMNet model is a pre-trained deep network and is a classic speech classification model. The YAMNet model can predict 521 audio event categories based on an AudioSet corpus. The YAMNet model has a total of 86 layers, among which the 84th layer contains 521 output classes, the 85th layer is a softmax layer, and the 86th layer is an output layer. The YAMNet model is structurally fine-tuned, all layers except the last three layers are retained, and the 521 output classes contained in the 84th layer are replaced by only two output classes, thereby obtaining the to-be-trained model in the present application.

[0088] Using the YAMNet model for transfer learning can overcome the difficulty of collecting and labeling positive and negative audio sample signals, thereby solving the problem that the model generalization performance is not high due to small data volume, and quickly realizing a snoring sound detection model with high accuracy. At the same time, for the to-be-trained model, only training the parameters of the 84th layer can further reduce the computational complexity of the model while obtaining high model performance and high detection accuracy. Further guarantee low-power snoring sound detection.

[0089] In an optional embodiment, the MYAMNet-TL model is used as a model for snoring sound detection, and 1000 sample points collected and labeled are divided into a training set and a test set in a ratio of 8:2, wherein the number of audio samples in the training set and the test set is 800 and 200, respectively. The training set is used to train the model, and the test set is used to test the performance of the model. The MYAMNet-TL model is a model obtained by training only the 84th layer parameters through transfer learning, i.e., a snoring sound detection model. The MYAMNet model is a model obtained by retraining all parameters after transfer learning. The YAMNet model is a pre-trained model. The test results of the three models on the test set are represented by performance indicators as shown in Table 1. The performance indicators include accuracy, recall, specificity, and Matthew correlation coefficient. The calculation formula of the accuracy (Ac for short) is: The calculation formula of the recall (Re for short) is: The calculation formula of the specificity (Sp for short) is: The calculation formula of the Matthew correlation coefficient (Mcc) is: where TP, TN, FP and FN represent true positive, true negative, false positive and false negative instances, respectively. Specifically, TP represents instances of detecting an audio signal originally including snoring sound signals as an audio signal including snoring sound signals. TN represents instances of detecting an audio signal originally not including snoring sound signals as an audio signal not including snoring sound signals. FP represents instances of erroneously detecting an audio signal originally not including snoring sound signals as an audio signal including snoring sound signals. FN represents instances of erroneously detecting an audio signal originally including snoring sound signals as an audio signal not including snoring sound signals. The Matthew correlation coefficient MCC has a value ranging from -1 to 1, and MCC = -1 represents the worst possible prediction, and a value of 1 represents the best possible prediction scheme. In addition, MCC = 0 represents a random prediction.

[0090] Table 1

[0091]

[0092]

[0093] For the model, the higher the accuracy (Ac for short) is, the stronger the model performance is. The higher the recall (Re for short) is, the stronger the model performance is. The better the specificity (Sp for short) is, the stronger the model performance is. The higher the Matthew correlation coefficient (Mcc) is, the stronger the model performance is.

[0094] It can be seen that, compared with the pre-trained YAMNet model, the MYAMNet model after transfer learning and retraining all parameters and the MYAMNet-TL model after transfer learning and training only the 84th layer parameters have obvious performance improvement on the test set for real-time snoring sound detection, and the accuracy, recall, specificity and Matthew correlation coefficient of the MYAMNet-TL model are higher than those of the other two models. That is, the model performance of the MYAMNet-TL model is the best.

[0095] Steps S101-S104 obtain the to-be-detected sample signal by pre-processing the audio sample signal, obtain the spectrum information of the to-be-detected sample signal, input the spectrum information into the to-be-trained model for training to obtain the snoring sound detection model. The calculation amount of model training is reduced, the performance of the model is improved, and technical support is provided for realizing low-power snoring sound detection.

[0096] The above is the training scheme of the snoring sound detection model of the present application. The specific application of the snoring sound detection model obtained by training can be referred to the relevant description below.

[0097] The present application also provides a snoring sound detection method, as shown in Figure 3 The method comprises the following steps:

[0098] S301: Obtain an audio signal in an environment.

[0099] In this step, the sound signal in the environment collected by the snoring sound detection device is taken as the audio signal. The collected audio signal can be an audio signal including snoring sound or an audio signal not including snoring sound.

[0100] It can be understood that the audio signal in the embodiment of the present application is a fixed-length sound signal.

[0101] S302: Preprocess the audio signal to obtain a to-be-detected signal.

[0102] In this step, since the audio signal is a sound signal including all environmental sounds, by preprocessing the audio signal, an audio signal that can be snoring sound, i.e., a to-be-detected signal, is obtained.

[0103] By preprocessing, other signals and noises in the audio signal are excluded, and then an audio signal that can be snoring sound is obtained. The frequency spectrum information of the audio signal that can be snoring sound is taken as the input of the snoring sound detection model. Compared with directly inputting the collected audio signal containing all environmental sounds into the snoring sound detection model to obtain the output result, the data calculation amount is greatly reduced, and the endurance of the snoring sound detection device can be improved.

[0104] S303: Obtain the frequency spectrum information of the to-be-detected signal.

[0105] In this step, the frequency spectrum information of the to-be-detected signal includes the mel-frequency spectrum of the to-be-detected signal.

[0106] Since the sound signal is a one-dimensional time-domain signal, it is difficult to intuitively see the frequency variation rule, so the time-domain signal is usually converted into a frequency-domain signal. Compared with directly inputting the to-be-detected signal into the snoring sound detection model to obtain the output result, taking the mel-frequency spectrum of the to-be-detected signal as the input of the snoring sound detection model improves the accuracy of the output result of the snoring sound detection model.

[0107] S304: Input the frequency spectrum information of the to-be-detected signal into the snoring sound detection model to obtain a detection result of the audio signal, the detection result being used to represent whether the audio signal includes snoring sound or not.

[0108] By inputting the frequency spectrum information of the to-be-detected signal into the trained snoring sound detection model, it can be known whether the audio signal includes snoring sound or not.

[0109] The present application obtains the to-be-detected signal by preprocessing the obtained audio signal, obtains the frequency spectrum information of the to-be-detected signal, and inputs the frequency spectrum information into the snoring sound detection model to obtain the detection result. The calculation amount of the snoring sound detection model is reduced, the performance of the snoring sound detection model is improved, and technical support is provided for realizing low-power snoring sound detection.

[0110] In an optional solution, as shown in Figure 4 The pre-processing of the audio signal to obtain the to-be-detected signal comprises:

[0111] S302':filtering the audio signal to obtain a filtered signal;

[0112] S302':filtering the audio signal to obtain a filtered signal;

[0113] It can be understood that, since the energy of most snoring signals is mainly concentrated in the 50-2000HZ bandwidth range, the audio signal is subjected to the 50-2000Hz second-order Butterworth band-pass filtering processing, so as to suppress other environmental sound and noise in the audio signal except the main snoring bandwidth, and obtain the filtered signal.

[0114] Further, the endpoint detection algorithm is adopted to detect the filtered signal to obtain the to-be-detected signal, which comprises:

[0115] The filtered signal is framed by using the endpoint detection algorithm to obtain each frame signal of the filtered signal;

[0116] The decibel number of each frame signal is calculated, and the to-be-detected signal is determined from the frame signals of the filtered signal according to the decibel number of each frame signal.

[0117] The filtered signal is framed by using the time-domain endpoint detection method through a signal window with a length of 25 milliseconds and a step length of 10 milliseconds to obtain each frame signal of the filtered signal, and the decibel number of each frame signal is calculated. If the decibel number of a frame signal is higher than or equal to a preset first threshold value, such as 45 decibels, it is considered that the frame signal may include snoring, and the subsequent operation is continued. If the decibel numbers of M (M is a positive integer greater than or equal to 2, such as M = 5) consecutive frame signals are all less than 40 decibels, it is considered that each frame signal in the M consecutive frame signals does not include snoring, and the set of frame signals including snoring signals is regarded as the to-be-detected signal.

[0118] In the present application, the edge band-pass filtering and the framing operations are performed on the audio signal. Such consecutive operations can speed up the processing flow of the audio signal and shorten the processing period.

[0119] In the foregoing solution, by using the endpoint detection method, specifically the time-domain endpoint detection method, the computation duty cycle of the snoring detection algorithm can be reduced, thereby reducing the power consumption of the device and prolonging the endurance time of the snoring detection device.

[0120] The application filters part of the noise signal except snoring sound signal through band-pass filtering, and detects the signal possibly including snoring sound through endpoint detection algorithm. Through twice processing, the audio signal possibly including snoring sound, i.e. the to-be-detected signal, can be obtained. Compared with the prior art of directly inputting the collected audio signal including all environmental sound into the snoring sound detection model for detection, the application inputs the frequency spectrum information of the to-be-detected signal obtained through twice preprocessing such as filtering and endpoint detection into the snoring sound detection model for detection, greatly reduces the data calculation amount, and further improves the endurance of the snoring sound detection device.

[0121] In an optional solution, the obtaining of the frequency spectrum information of the to-be-detected signal comprises:

[0122] The time-domain-to-frequency-domain transformation is performed on each frame signal of the to-be-detected signal to obtain the frequency spectrum of each frame signal;

[0123] The spectrogram of the to-be-detected signal is obtained according to the frequency spectrum of each frame signal;

[0124] The frequency spectrum information of the to-be-detected signal is obtained based on the spectrogram.

[0125] Further, the obtaining of the frequency spectrum information of the to-be-detected signal based on the spectrogram comprises: performing frequency spectrum mapping on the spectrogram to obtain a mel spectrum; and taking the mel spectrum as the frequency spectrum information of the to-be-detected signal.

[0126] The frequency spectrum of each frame signal of the to-be-detected signal is obtained through the short-time Fourier transform with a length of 25 milliseconds, a step of 10 milliseconds, and a periodic Hann time window, the frequency spectrum of each frame signal is stacked along a certain dimension (such as the time dimension) to obtain a two-dimensional graph including the sound data of the to-be-detected signal, i.e. the spectrogram of the to-be-detected signal. The dimension of the two-dimensional graph refers to two dimensions of time dimension and frequency dimension. The mel spectrogram is calculated by mapping the spectrogram to 64 mel bins covering the range of 50-2000 Hz. The mel spectrogram is framed into an example with 50% overlap and a length of 0.96 seconds. Each example covers 64 mel bands, and there are a total of 96 frames, each frame being 10 milliseconds. The mel spectrum of each example is the frequency spectrum information of the to-be-detected signal.

[0127] In the application, the spectrogram of the to-be-detected signal is obtained by framing and frequency spectrum changing of the filtered signal, and the frequency spectrum information of the to-be-detected signal is obtained by converting the spectrogram into a mel spectrum, which is simple and easy to implement in engineering, and provides an information basis for realizing low-power snoring sound detection.

[0128] In an optional solution, before the spectrum information of the to-be-detected signal is input into the snoring sound detection model to obtain a detection result of the audio signal, the method further includes:

[0129] obtaining an audio sample signal in an environment and a sample label, the sample label being used to represent whether the audio sample signal is an audio signal including snoring sound or an audio signal not including snoring sound;

[0130] preprocessing the audio sample signal to obtain a to-be-detected sample signal;

[0131] obtaining spectrum information of the to-be-detected sample signal;

[0132] inputting the spectrum information of the to-be-detected sample signal into a to-be-trained model as an input of the to-be-trained model, inputting the sample label into the to-be-trained model as an output of the to-be-trained model, and training the to-be-trained model to obtain a snoring sound detection model.

[0133] That is, before the snoring sound detection model is applied to detect snoring sound, the snoring sound detection model needs to be trained to obtain a snoring sound detection model with good stability and high performance. The snoring sound detection model with good stability and high performance can realize accurate identification of snoring sound detection. For specific training process, please refer to the foregoing related description, and no further description is given.

[0134] The foregoing solution is a solution for detecting snoring sound by using a snoring sound detection model. This process can be regarded as an application solution for detecting snoring sound by using a trained model to detect snoring sound of an obtained audio signal. The obtained audio signal is preprocessed to obtain a to-be-detected signal, the spectrum information of the to-be-detected signal is obtained, and the spectrum information is input into the snoring sound detection model to obtain a detection result. This reduces the calculation amount of the snoring sound detection model and improves the performance of the snoring sound detection model, thereby providing technical support for realizing low-power snoring sound detection.

[0135] The embodiment of the present application provides a snoring sound detection device, as shown in Figure 5 The device includes:

[0136] A first obtaining unit 501 is configured to obtain an audio signal in an environment.

[0137] A first preprocessing unit 502 is configured to preprocess the audio signal to obtain a to-be-detected signal.

[0138] A second obtaining unit 503 is configured to obtain spectrum information of the to-be-detected signal.

[0139] A detection unit 504 is configured to input the spectrum information of the to-be-detected signal into a snoring sound detection model to obtain a detection result of the audio signal, the detection result being used to represent whether the audio signal includes snoring sound or not.

[0140] In an optional implementation, the first preprocessing unit 502 is configured to perform filtering processing on the audio signal to obtain a filtered signal, and perform endpoint detection on the filtered signal to obtain the to-be-detected signal.

[0141] In an optional implementation, the first preprocessing unit 502 is configured to perform frame division on the filtered signal by using an endpoint detection algorithm to obtain each frame signal of the filtered signal, calculate a decibel number of each frame signal, and determine the to-be-detected signal from the frame signals of the filtered signal according to the decibel number of each frame signal.

[0142] In an optional implementation, the second acquisition unit 503 is configured to perform time-domain-to-frequency-domain conversion on each frame signal of the to-be-detected signal to obtain a frequency spectrum of each frame signal, obtain a spectrogram of the to-be-detected signal according to the frequency spectrum of each frame signal, and obtain the frequency spectrum information of the to-be-detected signal based on the spectrogram.

[0143] In an optional implementation, the second acquisition unit 503 is configured to perform frequency spectrum mapping on the spectrogram to obtain a mel frequency spectrum, and take the mel frequency spectrum as the frequency spectrum information of the to-be-detected signal.

[0144] In an optional implementation, the apparatus further includes:

[0145] The first training unit is configured to acquire an audio sample signal in an environment and a sample label, the sample label being used to represent that the audio sample signal is an audio signal including snoring sound or an audio signal not including snoring sound, perform preprocessing on the audio sample signal to obtain a to-be-detected sample signal, obtain frequency spectrum information of the to-be-detected sample signal, take the frequency spectrum information of the to-be-detected sample signal as an input of a to-be-trained model, take the sample label as an output of the to-be-trained model, and train the to-be-trained model to obtain the snoring sound detection model.

[0146] An embodiment of the present application provides a snoring sound detection model training apparatus, as shown in Figure 6 The apparatus includes:

[0147] The first acquisition module 601 is configured to acquire an audio sample signal in an environment and a sample label, the sample label being used to represent that the audio sample signal is an audio signal including snoring sound or an audio signal not including snoring sound.

[0148] The first preprocessing module 602 is configured to perform preprocessing on the audio sample signal to obtain a to-be-detected sample signal.

[0149] The second acquisition module 603 is configured to obtain frequency spectrum information of the to-be-detected sample signal.

[0150] The training module 604 is configured to take the frequency spectrum information of the sample signal to be detected as input of a model to be trained, take a sample label as output of the model to be trained, train the model to be trained, and obtain a snoring sound detection model for detecting whether an audio signal in an environment includes a snoring sound signal.

[0151] In an optional implementation, the first preprocessing module 602 is configured to perform filtering on the audio sample signal to obtain a filtered sample signal, and perform endpoint detection on the filtered sample signal to obtain the sample signal to be detected.

[0152] In an optional implementation, the first preprocessing module 602 is configured to perform framing on the filtered sample signal by using an endpoint detection algorithm to obtain each frame of the filtered sample signal, calculate a decibel value of each frame, and determine the sample signal to be detected from the frames of the filtered sample signal according to the decibel value of each frame.

[0153] In an optional implementation, the second obtaining module 603 is configured to perform time-domain to frequency-domain conversion on each frame of the sample signal to be detected to obtain a frequency spectrum of each frame, obtain a spectrogram of the sample signal to be detected according to the frequency spectrum of each frame, and obtain the frequency spectrum information of the sample signal to be detected based on the spectrogram.

[0154] In an optional implementation, the second obtaining module 603 is configured to perform frequency spectrum mapping on the spectrogram to obtain a mel frequency spectrum, and take the mel frequency spectrum as the frequency spectrum information of the sample signal to be detected.

[0155] It should be noted that the snoring sound detection device and the snoring sound detection model training device of the embodiments of the present application have similar principles to the foregoing snoring sound detection method and snoring sound detection model training method, and therefore the implementation process and implementation principles and advantages of the snoring sound detection device and the snoring sound detection model training device can be understood with reference to the foregoing descriptions of the implementation process and implementation principles and advantages of the method, and repeated descriptions are omitted.

[0156] In addition, the terms "first" and "second" are used only for descriptive purposes, and cannot be understood as indicating or implying relative importance or implicitly indicating the number of the indicated technical features. Therefore, the features defined as "first" and "second" can explicitly or implicitly include at least one of the features. In the description of the present application, the meaning of "a plurality of" is two or more, unless otherwise specifically limited.

[0157] The above merely provides the specific implementation of the present application, but the protection scope of the present application is not limited to this. Any person skilled in the art can easily think of the changes or replacements within the technical range disclosed by the present application, which should be covered in the protection scope of the present application. Therefore, the protection scope of the present application should be subject to the protection scope of the claims.

Claims

1. A snoring sound detection method characterized by, The method comprises: obtaining an audio signal in an environment; filtering the audio signal to obtain a filtered signal; using an endpoint detection algorithm to frame the filtered signal to obtain each frame signal of the filtered signal; calculating the decibel number of each frame signal, and determining a to-be-detected signal from each frame signal of the filtered signal according to the decibel number of each frame signal; obtaining spectral information of the to-be-detected signal; inputting the spectral information of the to-be-detected signal into a snoring sound detection model to obtain a detection result of the audio signal, wherein the detection result is used to represent whether the audio signal includes a snoring sound signal or not; wherein, before the spectral information of the to-be-detected signal is input into the snoring sound detection model to obtain the detection result of the audio signal, the method further comprises: obtaining an audio sample signal in an environment and a sample label, wherein the sample label is used to represent whether the audio sample signal is an audio signal including snoring sound or an audio signal not including snoring sound; preprocessing the audio sample signal to obtain a to-be-detected sample signal; obtaining spectral information of the to-be-detected sample signal; inputting the spectral information of the to-be-detected sample signal into a to-be-trained model as an input of the to-be-trained model, inputting the sample label into the to-be-trained model as an output of the to-be-trained model, and training parameters of the 84th layer of the to-be-trained model to obtain a snoring sound detection model, wherein the to-be-trained model is obtained by replacing 521 output classes of the 84th layer of a pre-trained YAMNet model with 2 output classes.

2. The method of claim 1, wherein, The method comprises: transforming each frame signal of the to-be-detected signal from time domain to frequency domain to obtain a spectrum of each frame signal; obtaining a spectrogram of the to-be-detected signal according to the spectrum of each frame signal; obtaining spectral information of the to-be-detected signal based on the spectrogram.

3. The method of claim 2, wherein, The method comprises: obtaining an audio sample signal in an environment and a sample label, wherein the sample label is used to represent whether the audio sample signal is an audio signal including snoring sound or an audio signal not including snoring sound; filtering the audio sample signal to obtain a filtered sample signal; 4.A method for training a snoring sound detection model, characterized in that, using an endpoint detection algorithm to frame the filtered sample signal to obtain each frame signal of the filtered sample signal; calculating the decibel number of each frame signal, and determining a to-be-detected sample signal from each frame signal of the filtered sample signal according to the decibel number of each frame signal; obtaining spectral information of the to-be-detected sample signal; inputting the spectral information of the to-be-detected sample signal into a to-be-trained model as an input of the to-be-trained model, inputting the sample label into the to-be-trained model as an output of the to-be-trained model, and training parameters of the 84th layer of the to-be-trained model to obtain a snoring sound detection model, wherein the snoring sound detection model is used to detect whether an audio signal in an environment includes a snoring sound signal or not; wherein the to-be-trained model is obtained by replacing 521 output classes of the 84th layer of a pre-trained YAMNet model with 2 output classes. The device comprises: a first obtaining unit configured to obtain an audio signal in an environment; ​ 5. A snoring sound detection device, characterized by ​ ​ The first preprocessing unit is configured to filter the audio signal to obtain a filtered signal, frame the filtered signal by using an endpoint detection algorithm to obtain each frame signal of the filtered signal, calculate a decibel number of each frame signal, and determine a to-be-detected signal from the frame signals of the filtered signal according to the decibel number of each frame signal. The second acquisition unit is configured to obtain spectral information of the to-be-detected signal. The detection unit is configured to input the spectral information of the to-be-detected signal into a snoring sound detection model to obtain a detection result of the audio signal, where the detection result is used to represent whether the audio signal includes a snoring sound signal or not. The first training unit is configured to obtain an audio sample signal in an environment and a sample label, where the sample label is used to represent whether the audio sample signal is an audio signal including snoring sound or an audio signal not including snoring sound, preprocess the audio sample signal to obtain a to-be-detected sample signal, obtain spectral information of the to-be-detected sample signal, use the spectral information of the to-be-detected sample signal as an input of a to-be-trained model, use the sample label as an output of the to-be-trained model, train a parameter of the 84th layer of the to-be-trained model to obtain a snoring sound detection model, and obtain the snoring sound detection model by replacing 521 output classes of the 84th layer of a pre-trained YAMNet model with 2 output classes. 6.A device for training a snoring sound detection model, comprising: The device comprises: The first acquisition module is configured to obtain an audio sample signal in an environment and a sample label, where the sample label is used to represent whether the audio sample signal is an audio signal including snoring sound or an audio signal not including snoring sound. The first preprocessing module is configured to filter the audio sample signal to obtain a filtered sample signal, frame the filtered sample signal by using an endpoint detection algorithm to obtain each frame signal of the filtered sample signal, calculate a decibel number of each frame signal, and determine a to-be-detected sample signal from the frame signals of the filtered sample signal according to the decibel number of each frame signal. The second acquisition module is configured to obtain spectral information of the to-be-detected sample signal. The training module is configured to use the spectral information of the to-be-detected sample signal as an input of a to-be-trained model, use the sample label as an output of the to-be-trained model, train a parameter of the 84th layer of the to-be-trained model to obtain a snoring sound detection model, and use the snoring sound detection model to detect whether an audio signal in an environment includes a snoring sound signal, where the snoring sound detection model is obtained by replacing 521 output classes of the 84th layer of a pre-trained YAMNet model with 2 output classes.

Images