Sleep breathing disorder recognition system based on deep learning

Abstract

The invention discloses a sleep breathing disorder recognition system based on deep learning. The sleep breathing disorder recognition system comprises an audio signal feature extraction module, a feature processing module, a time weighted attention mechanism module and a bidirectional long-short term memory network module which are connected in sequence. The audio signal feature extraction module extracts multi-dimensional audio features; the feature processing module processes features through convolution and a full connection layer; the time weighted attention mechanism module highlights key moment information; the Bi-LSTM module models a forward and backward time sequence dependency relationship and classifies the forward and backward time sequence dependency relationship through a full connection layer and a Softmax layer. The equipment comprises a snore input module, a sound detection module, an embedded module and a data display module, can monitor snore signals in real time, displays detection results through a graphical interface after processing, realizes efficient and accurate identification of various sleep breathing disorders, overcomes the problems of incomplete feature extraction, low modeling efficiency and the like in the prior art, and is suitable for popularization and application. And the accuracy and robustness of the audio signal classification task are improved.

Description

Technical Field

[0001] The present invention belongs to the field of audio signal processing and deep learning, and specifically relates to a sleep breathing disorder recognition system based on deep learning. Background Art

[0002] With the widespread application of audio signal processing in speech recognition, sound event detection, music analysis and other fields, how to efficiently and accurately extract useful features from raw audio signals has always been an important research issue in this field. Traditional audio feature extraction methods such as Mel Frequency Cepstral Coefficients (MFCC) and Short Time Energy (STE) have been widely used in speech and audio signal processing. However, these traditional methods usually cannot fully capture the temporal relationship and complex spectral information in audio signals, so their classification performance is limited in some complex tasks.

[0003] At present, with the rapid development of deep learning, neural network structures such as convolutional neural networks (CNN) and recurrent neural networks (RNN) are widely used in audio signal processing and classification tasks. In particular, the long short-term memory network (LSTM) can effectively capture the long-term and short-term dependencies in time series data, providing a better solution for the time series modeling of audio signals. In addition, network models based on the attention mechanism have been proven to be able to highlight the important time steps in the processing signal, further improving the performance of the classification model.

[0004] However, in the existing technology, how to ensure the accuracy of audio signal feature extraction while combining effective time series modeling and classification strategies is still an urgent problem to be solved. Existing feature extraction methods and time series modeling technologies often have problems such as insufficient feature extraction and low modeling efficiency, which limits their application effect in complex audio signal classification tasks. Summary of the invention

[0005] In order to solve the shortcomings of the prior art, achieve efficient extraction of multi-dimensional features of audio signals, and improve the accuracy and robustness of audio signal classification tasks, the present invention adopts the following technical solutions:

[0006] A sleep breathing disorder recognition system based on deep learning includes a sound detection module, wherein the sound detection module includes an audio signal feature extraction module, a feature processing module, a time-weighted attention mechanism module, and a bidirectional long short-term memory network module connected in sequence;

[0007] The audio signal feature extraction module collects the sleep breathing audio signal and extracts the audio features;

[0008] The feature processing module extracts multiple time series features from the audio features, including spectrum features and time domain features. This processing method effectively improves the separability and representation ability of the features, laying the foundation for subsequent classification tasks;

[0009] The time-weighted attention mechanism module weights the temporal features of the audio signal and highlights the information at key moments in the audio signal to enhance the response sensitivity of the model at key moments, thereby improving the overall classification accuracy.

[0010] The bidirectional long short-term memory network module captures the dependency between previous and next moments in the audio signal by modeling the forward and backward directions of the time series data, generates a hidden state representation containing context information, and adjusts the network parameters according to the training results of the classification task; obtains the classification result by forward propagation, and completes the network training by the back propagation algorithm; the module combines the global information of the time series to provide a more accurate time series feature representation for the classification task; the classification layer uses the fully connected layer and the Softmax layer to classify the features, outputs the category probability of the audio signal, and optimizes the model by the cross entropy loss function to ensure that the performance is continuously improved during the training process and has good generalization ability.

[0011] Furthermore, in the audio signal feature extraction module, the audio features are pre-emphasized and framed. Pre-emphasis is used to compensate for the attenuation of lower frequencies in the speech signal and emphasize high-frequency information. Framing is used to perform short-time feature analysis on the audio signal. Each frame is windowed to reduce spectrum leakage, and the time domain signal is converted into a frequency domain signal to further reduce the impact of spectrum leakage. A power spectrum is generated based on the frequency domain conversion of each frame signal. The key features of the audio signal are extracted based on the power spectrum and the filter group output energy applied to the power spectrum.

[0012] Furthermore, in the pre-emphasis and framing, the pre-emphasis formula is:

[0013] y(t)=x(t)-αx(t-1)

[0014] Where t represents the time, x(t) represents the audio signal at the tth time, y(t) represents the signal after pre-emphasis processing, and α represents the filter coefficient for adjusting the degree of low-frequency suppression. Through this processing, the attenuation of the lower frequencies in the speech signal is compensated, the high-frequency information is emphasized, and the spectral resolution of the signal is improved.

[0015] In order to perform short-time feature analysis on an audio signal, the present invention divides the audio signal into a plurality of continuous short-time frames, and the framing process is implemented in the following manner:

[0016] x[n]→x m [n]=x[n+mH],0≤n <N

[0017] Where x[n] represents the original discrete audio signal, n represents the index of the sampling point in the frame, and x m [n] represents the mth frame signal after framing, m represents the frame index, H represents the frame shift (set to 50% of the frame length), and N represents the number of sample points per frame;

[0018] In order to reduce spectrum leakage, a weighted Kaiser window is applied to each frame signal, and its window function is defined as:

[0019]

[0020] Where I0 represents the zero-order modified Bessel function, β represents the window function parameter, which is used to control the main lobe width and side lobe suppression, and n represents the index of the sampling point in the frame;

[0021] After windowing, the time domain signal is converted into a frequency domain signal using the fast Fourier transform (FFT) to reduce the impact of spectrum leakage. The calculation formula is:

[0022]

[0023] Among them, x ′ [n] represents the normalized signal, k represents the frequency variable, X(k) represents the frequency domain function after fast Fourier transform, and n is the index variable, which represents the nth sampling point in the current frame;

[0024] The power spectrum is obtained by converting each frame of signal into frequency domain and summing the squares of the frequency domain signal:

[0025] P(k)=|X(k)| 2 .

[0026] Furthermore, the filter adopts a Mel filter, the key features include Mel frequency cepstral coefficients, and the Mel frequency cepstral coefficients are extracted by discrete cosine transform of the energy of the logarithmic filter bank;

[0027] The frequency response of the Mel filter is:

[0028]

[0029] Among them, f m-1 ,f m ,f m+1 They represent the left and right boundary frequencies and the center frequency of the Mel filter respectively, k represents the frequency index, and m represents the number of the Mel filter;

[0030] The output energy of each Mel filter is calculated by the following formula:

[0031]

[0032] Among them, P(k) represents the power spectrum of the signal, H m (k) represents the frequency response of the Mel filter.

[0033] Furthermore, the key feature includes filter bank energy, and the energy of the filter bank is directly output as another audio feature to obtain a filter bank energy feature;

[0034] Take the logarithm of the output energy of the filter bank:

[0035]

[0036] By performing discrete cosine transform on the log filter bank energy, the cepstral coefficients c are extracted. n , the formula is as follows:

[0037]

[0038] Wherein, M represents the total number of filters, which is used to normalize the frequency range, and the first N' cepstral coefficients are taken as the Mel-frequency cepstral coefficients.

[0039] Furthermore, the key features include short-time energy. By extracting the short-time energy of each frame, the instantaneous intensity change of the signal is captured. At the same time, in order to improve the stability of the signal characteristics and reduce the influence of external noise on the short-time energy, the short-time energy is normalized to obtain the normalized short-time energy.

[0040] The short-time energy calculation formula for each frame signal is:

[0041]

[0042] In order to improve the stability of signal characteristics and reduce the influence of external noise on short-time energy, the present invention performs normalization processing on short-time energy:

[0043]

[0044] Among them, max(E) represents the maximum value obtained in the short-time energy sequence of the entire signal.

[0045] Further, the key features include linear prediction coefficients, which are extracted using the Levinson-Durbin algorithm by calculating the autocorrelation of the audio signal;

[0046] The correlation uses the following correlation function:

[0047]

[0048] Among them, p represents the LPC order, which determines the complexity and fitting ability of the model, and here it is 10th order;

[0049] The linear prediction coefficient extraction formula is as follows:

[0050]

[0051] Among them, a k Represents the k-th order linear prediction coefficient, j is an integer index from 1 to p, indicating that the j-th order linear prediction coefficient is currently being solved.

[0052] Furthermore, the feature processing module extracts spectral features from the audio features through a convolution operation, extracts time domain features through a fully connected layer, and converts them into feature vectors suitable for classification tasks; the output of the convolution layer is downsampled through a pooling operation; the fully connected layer performs feature mapping to compress and map the extracted time domain features and output time series features;

[0053] The dimensions of the output features are:

[0054]

[0055] Among them, K represents the convolution kernel size, s represents the pooling step size, and F input Represents the input dimension.

[0056] Furthermore, the time-weighted attention mechanism module highlights the impact of key moments in the audio signal according to the feature weight of each time step of the time series data, so as to improve the response sensitivity of the model at the key moment, and finally obtains a context vector containing time step information; the specific execution process is as follows:

[0057] 1) Weight calculation: For each time step t, calculate the attention weight e t :

[0058] e t =tanh(W h ·h t +b h )

[0059] Among them, W h represents the weight matrix, h t represents the hidden state of the current time step, b h represents bias, tanh(·) represents the hyperbolic tangent activation function;

[0060] 2) Normalized weight: In order to make the sum of the weights of all time steps equal to 1, calculate the normalized attention weight α t :

[0061]

[0062] Among them, e t represents the attention score at each time step, αt represents the normalized attention weight;

[0063] 3) Weighted feature output: The features of the time step are weighted by the attention weights, and the context vector is finally obtained:

[0064]

[0065] Among them, c represents the context vector, which contains the information of important time steps and is used for subsequent classification tasks.

[0066] Furthermore, the bidirectional long short-term memory network module generates a hidden state containing context information for each time step, and at each time step, the forward and backward hidden states are spliced ​​into a joint hidden state, and the output of the bidirectional long short-term memory network is compressed and nonlinearly mapped through a fully connected layer to generate features required for classification, and the features output by the fully connected layer are converted into predicted probabilities for each category, and the error between the predicted probability and the true label is optimized through a cross entropy loss function; the specific execution process is as follows:

[0067] 1) The forward LSTM network starts from the first time step t=1 of the time series and processes backward step by step to update the hidden state and memory status The update formula is as follows:

[0068]

[0069] in, represents the output gate activation value at the current time step t, Represents the memory state of the current time step t, which is used to store long-term information before and after time step t;

[0070] The backward long short-term memory network starts from the last time step t = T of the time series, and gradually processes forward to update the hidden state and memory status The calculation formula of the backward LSTM network is similar to that of the forward LSTM network, except that the processing order is reversed:

[0071]

[0072] 2) At each time step t, the hidden states of the forward LSTM network and the backward LSTM network are concatenated into a joint hidden state h t :

[0073]

[0074] Finally, the output shape O of the bidirectional long short-term memory network is expressed as:

[0075] O=B×T×(2·H)

[0076] Where B represents the batch size, T represents the length of the sequence, and H represents the hidden state dimension of the unidirectional long short-term memory network;

[0077] The output of the bidirectional long short-term memory network is compressed and nonlinearly mapped through the fully connected layer to generate the feature z required for classification:

[0078] z=σ(W z ·c+b z )

[0079] in, represents the weight matrix, represents the context vector, b z ∈R d represents the bias, σ represents the activation function; after being mapped by the fully connected layer, the dimension of feature z becomes R d ;

[0080] 3) The feature z output by the fully connected layer is converted into the predicted probability p(y=i|z) of each category through the Softmax layer:

[0081]

[0082] Where C represents the number of categories (snoring, hypopnea, airflow limitation, pause, a total of 4 categories), p(y=i|z) represents the predicted probability that the input feature z belongs to category i;

[0083] 4) The classification model uses the cross entropy loss function to optimize the error between the predicted probability and the true label:

[0084]

[0085] Among them, N represents the batch size, C represents the number of categories, and y n,i represents the true label (one-hot encoding) of the nth sample in category i, p n,i Represents the predicted probability of the nth sample in category i.

[0086] The advantages and beneficial effects of the present invention are:

[0087] The deep learning-based sleep breathing disorder recognition system of the present invention comprises an audio signal feature extraction module, a feature processing module, a time-weighted attention mechanism module, and a bidirectional long short-term memory network (Bi-LSTM) module which are connected in sequence. The present invention combines a temporal modeling method based on a convolutional neural network and a bidirectional LSTM to efficiently extract the multi-dimensional features of the audio signal while improving the accuracy and robustness of the audio signal classification task, thereby realizing the recognition of four types of sleep breathing disorders, namely snoring, hypopnea, airflow limitation, and pause. BRIEF DESCRIPTION OF THE DRAWINGS

[0088] Figure 1 It is a schematic diagram of the structure of the system in an embodiment of the present invention.

[0089] Figure 2 It is a process diagram of system execution in an embodiment of the present invention. DETAILED DESCRIPTION

[0090] The specific implementation of the present invention is described in detail below in conjunction with the accompanying drawings. It should be understood that the specific implementation described here is only used to illustrate and explain the present invention, and is not used to limit the present invention.

[0091] like Figure 1 , Figure 2 As shown in the figure, the sleep breathing disorder recognition system based on deep learning includes a snoring input module, a sound detection module, an RK3558 embedded Linux module and a data display module. The snoring input module is responsible for real-time monitoring and capturing snoring signals, and transmitting these signals to the RK3558 embedded Linux system; then, these signals will be processed by the sound detection module in turn, and finally, the data display module will display the processing results to the user in an intuitive way, and provide possible diagnosis suggestions for sleep breathing disorders based on the analysis results. The system ensures efficient processing and accurate analysis of snoring signals, providing users with a reliable sleep health monitoring tool. The system can record and identify the input snoring signals, and display the identification and classification results on the data display module, and present them to the user in the form of a graphical interface, so that users can intuitively understand sleep breathing disorders.

[0092] In the embodiment of the present invention, the snoring input module uses a professional snoring detection microphone to monitor and identify snoring signals in real time, and transmits and stores them in the RK3558 embedded Linux module; the sound detection module adopts the above-mentioned sleep breathing disorder disease identification and classification method; the RK3558 embedded Linux module provides a computing platform and operating system support for the module to ensure that the modules can work together efficiently; the data display module is used to display the snoring detection results to the user in the form of a graphical interface, so that the user can intuitively understand the sleep breathing disorder.

[0093] The sound detection module includes an audio signal feature extraction module, a feature processing module, a time-weighted attention mechanism module, and a bidirectional long short-term memory network (Bi-LSTM) module, which are connected in sequence. The modules work together to achieve efficient classification and identification of sleep breathing disorders.

[0094] The audio signal feature extraction module is responsible for preprocessing, frequency domain transformation and feature extraction of the input sleep audio signal to generate audio features suitable for identifying four types of sleep breathing disorders: snoring, hypopnea, airflow limitation and pause;

[0095] Specifically, the audio signal feature extraction module is used to extract multiple key features from the original audio signal, including Mel frequency cepstral coefficient (MFCC), filter bank energy (Fbanks), short time energy (STE) and linear prediction coefficient (LPC). MFCC can effectively represent the spectral characteristics of the audio signal, Fbanks reflects the frequency domain energy distribution of the audio signal, STE captures the instantaneous changes of the signal, and LPC is mainly used to describe the channel characteristics of the audio. The feature extraction process is as follows:

[0096] 1) Pre-emphasize and frame the audio signal. Pre-emphasize to compensate for the attenuation of lower frequencies in the speech signal and emphasize high-frequency information. Frame analysis is performed to analyze the short-term characteristics of the audio signal. Each frame is windowed to reduce spectrum leakage. The time domain signal is converted to a frequency domain signal to further reduce the impact of spectrum leakage. The power spectrum is generated based on the frequency domain conversion of each frame signal to calculate the periodogram estimate of the power spectrum.

[0097] A high-pass filter is used for pre-emphasis processing; the high-frequency components in the speech signal are compensated by subtracting from the previous sampling point. The pre-emphasis formula is:

[0098] y(t)=x(t)-αx(t-1)

[0099] Where t represents the time, x(t) represents the audio signal at the tth time, y(t) represents the signal after pre-emphasis processing, and α represents the filter coefficient for adjusting the degree of low-frequency suppression, with a typical value of 0.97. Through this processing, the attenuation of lower frequencies in the speech signal is compensated, the high-frequency information is emphasized, and the spectral resolution of the signal is improved.

[0100] In order to perform short-time feature analysis on an audio signal, the present invention divides the audio signal into a plurality of continuous short-time frames, and the framing process is implemented in the following manner:

[0101] x[n]→x m [n]=x[n+mH],0≤n <N

[0102] Where x[n] represents the original discrete audio signal, n represents the index of the sampling point in the frame, and x m [n] represents the mth frame signal after framing, m represents the frame index, H represents the frame shift (set to 50% of the frame length), and N represents the number of sample points per frame;

[0103] In order to reduce spectrum leakage, a weighted Kaiser window is applied to each frame signal, and its window function is defined as:

[0104]

[0105] Where I0 represents the zero-order modified Bessel function, β represents the window function parameter, which is used to control the main lobe width and side lobe suppression, and n represents the index of the sampling point in the frame;

[0106] After windowing, the time domain signal is converted into a frequency domain signal using the fast Fourier transform (FFT) to reduce the impact of spectrum leakage. The calculation formula is:

[0107]

[0108] Among them, x ′ [n] represents the normalized signal, k represents the frequency variable, X(k) represents the frequency domain function after fast Fourier transform, and n is the index variable, which represents the nth sampling point in the current frame.

[0109] The power spectrum is obtained by converting each frame of signal into frequency domain and summing the squares of the frequency domain signal:

[0110] P(k)=|X(k)| 2

[0111] 2) Apply the Mel filter bank to the power spectrum and find the energy of the filter bank;

[0112] The frequency response of the Mel filter is:

[0113]

[0114] Among them, f m-1 ,f m ,f m+1 They represent the left and right boundary frequencies and the center frequency of the Mel filter respectively, k represents the frequency index, and m represents the number of the Mel filter;

[0115] The output energy of each Mel filter is calculated by the following formula:

[0116]

[0117] Among them, P(k) represents the power spectrum of the signal, H m (k) represents the frequency response of the Mel filter.

[0118] 3) Extract the cepstral coefficients MFCC by discrete cosine transform of the logarithmic filter bank energy;

[0119] To compress the dynamic range, the output energy of the filter bank is logarithmic:

[0120]

[0121] The cepstrum coefficients are extracted by performing discrete cosine transform (DCT) on the energy of the logarithmic filter bank, as follows:

[0122]

[0123] Wherein, M represents the total number of filters, which is used to normalize the frequency range, and the first 13 cepstral coefficients are taken as Mel frequency cepstral coefficients (MFCC).

[0124] 4) Extract filter bank energy features Fbanks, which provide spectral information of the audio signal and can help distinguish different audio modes;

[0125] The energy of the filter bank is directly output as another audio feature, and the energy feature Fbanks is obtained as follows:

[0126] F m =E m ,1≤m≤M

[0127] 5) Extract the short-time energy STE of each frame to capture the instantaneous intensity change of the signal;

[0128] The short-time energy calculation formula for each frame signal is:

[0129]

[0130] In order to improve the stability of signal characteristics and reduce the influence of external noise on short-time energy, the present invention performs normalization processing on short-time energy:

[0131]

[0132] Among them, max(E) represents the maximum value obtained in the short-time energy sequence of the entire signal.

[0133] 6) By calculating the autocorrelation of the audio signal, the linear prediction coefficient LPC is extracted using the Levinson-Durbin algorithm;

[0134] The correlation uses the following correlation function:

[0135]

[0136] Among them, p represents the LPC order, which determines the complexity and fitting ability of the model, and here it is 10th order;

[0137] The linear prediction coefficient extraction formula is as follows:

[0138]

[0139] Among them, a k Represents the k-th order linear prediction coefficient, j is an integer index from 1 to p, indicating that the j-th order linear prediction coefficient is currently being solved.

[0140] The feature processing module further processes the extracted audio features, extracts spectral features through convolution operations, extracts time domain features through fully connected layers, and converts them into feature vectors suitable for classification tasks;

[0141] Specifically, the feature processing module extracts a variety of time series features from the audio signal, including spectral features (MFCC and Fbanks) and time domain features (STE and LPC). The spectral features are extracted through convolution operations, and the time domain features are extracted through a fully connected layer.

[0142] The convolutional network extracts effective information from the spectral features (MFCC and Fbanks) through convolution operations. The output of the convolutional layer is downsampled through pooling operations. Assuming that the convolution kernel size is K, the pooling step size is s, and the input dimension is F input , then the dimension of the output feature is:

[0143]

[0144] When extracting time domain features (short-time energy STE and linear prediction coefficient LPC), a fully connected layer is used for feature mapping. The fully connected layer is used to compress and map the extracted time domain features and output time series features.

[0145] The time-weighted attention mechanism module highlights the impact of key moments in the audio signal based on the feature weight of each time step of the time series data, so as to improve the response sensitivity of the model at key moments;

[0146] Specifically, the time-weighted attention mechanism module calculates the feature weights of each time step, thereby weighting the features to highlight the information of important moments. The specific execution process is as follows:

[0147] 1) Weight calculation: For each time step t, calculate the attention weight e t :

[0148] e t =tanh(W h ·h t +b h )

[0149] Among them, W h represents the weight matrix, h t represents the hidden state of the current time step, b h represents bias, tanh(·) represents the hyperbolic tangent activation function;

[0150] 2) Normalized weight: In order to make the sum of the weights of all time steps equal to 1, calculate the normalized attention weight α t :

[0151]

[0152] Among them, e t represents the attention score at each time step, α t represents the normalized attention weight;

[0153] 3) Weighted feature output: The features of the time step are weighted by the attention weights, and the context vector is finally obtained:

[0154]

[0155] Among them, c represents the context vector, which contains the information of important time steps and is used for subsequent classification tasks.

[0156] The bidirectional long short-term memory network (Bi-LSTM) module captures the dependency between previous and next moments in the audio signal by modeling the forward and backward directions of the time series data, and adjusts the network parameters according to the training results of the classification task; the classification results are obtained through forward propagation, and the network training is completed through the back-propagation algorithm.

[0157] Specifically, the Bi-LSTM module uses bidirectional LSTM to capture the dependencies between previous and next time steps, thereby generating a hidden state containing contextual information for each time step. The specific execution process is as follows:

[0158] 1) The forward LSTM starts from the first time step of the time series (t=1), processes backward step by step, and updates the hidden state and memory status The update formula is as follows:

[0159]

[0160] in, represents the output gate activation value at the current time step t, Represents the memory state of the current time step t, which is used to store long-term information before and after time step t;

[0161] The backward LSTM starts from the last time step of the time series (t = T), processes forward step by step, and updates the hidden state and memory status The calculation formula of backward LSTM is similar to that of forward LSTM, except that the processing order is reversed:

[0162]

[0163] 2) At each time step t, the hidden states of the forward LSTM and the backward LSTM are concatenated into a joint hidden state h t :

[0164]

[0165] Finally, the output shape O of the bidirectional LSTM is expressed as:

[0166] O=B×T×(2·H)

[0167] Among them, B represents the batch size, T represents the length of the sequence, and H represents the hidden state dimension of the unidirectional LSTM.

[0168] The output of the bidirectional LSTM is compressed and nonlinearly mapped through the fully connected layer to generate the feature z required for classification:

[0169] z=σ(W z ·c+b z )

[0170] in, represents the weight matrix, represents the context vector, b z ∈R d represents the bias, σ represents the activation function; after being mapped by the fully connected layer, the dimension of feature z becomes R d .

[0171] 3) The feature z output by the fully connected layer is converted into the predicted probability p(y=i|z) of each category through the Softmax layer:

[0172]

[0173] Where C represents the number of categories (snoring, hypopnea, airflow limitation, pause, a total of 4 categories), p(y=i|z) represents the predicted probability that the input feature z belongs to category i;

[0174] 4) The classification model uses the cross entropy loss function to optimize the error between the predicted probability and the true label:

[0175]

[0176] Among them, N represents the batch size, C represents the number of categories, and y n,i represents the true label (one-hot encoding) of the nth sample in category i, p n,i Represents the predicted probability of the nth sample in category i.

[0177] The present invention realizes efficient classification and recognition of complex sleep breathing disorders through the collaborative design of the above modules, achieving high accuracy and robustness.

[0178] The above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit the same. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that the technical solutions described in the aforementioned embodiments may still be modified, or some or all of the technical features thereof may be replaced by equivalents. However, these modifications or replacements do not deviate the essence of the corresponding technical solutions from the scope of the technical solutions of the embodiments of the present invention.

Claims

1. A sleep apnea recognition system based on deep learning, including a sound detection module, characterized in that: The sound detection module includes an audio signal feature extraction module, a feature processing module, a time-weighted attention mechanism module, and a bidirectional long short-term memory network module which are connected in sequence; The audio signal feature extraction module collects the sleep breathing audio signal and extracts the audio features; The feature processing module extracts multiple time series features from the audio features, including spectrum features and time domain features; The time-weighted attention mechanism module weights the temporal features of the audio signal to highlight the information at key moments in the audio signal; The bidirectional long short-term memory network module captures the dependency between previous and next moments in the audio signal by modeling the forward and backward directions of the time series data, generates a hidden state representation containing context information, and adjusts network parameters according to the training results of the classification task; obtains the classification results through forward propagation, and completes the network training through the back propagation algorithm.

2. The deep learning-based sleep apnea recognition system according to claim 1, characterized in that: In the audio signal feature extraction module, the audio features are pre-emphasized and framed, each frame is windowed, and then the time domain signal is converted into a frequency domain signal. A power spectrum is generated based on the frequency domain conversion of each frame signal, and the key features in the audio signal are extracted based on the power spectrum and the filter group output energy applied to the power spectrum.

3. The deep learning-based sleep apnea recognition system according to claim 2, characterized in that: In the pre-emphasis and framing, the pre-emphasis formula is: y(t)=x(t)-αx(t-1) Wherein, t represents the time, x(t) represents the audio signal at the tth time, y(t) represents the signal after pre-emphasis processing, and α represents the filter coefficient for adjusting the degree of low-frequency suppression; It is divided into multiple consecutive short time frames. The framing process is achieved in the following way: x[n]→x m [n]=x[n+mH],0≤n<N Where x[n] represents the original discrete audio signal, n represents the index of the sampling point in the frame, and x m [n] represents the mth frame signal after framing, m represents the frame index, H represents the frame shift, and N represents the number of sample points per frame; A weighted Caesar window is applied to each frame of signal, and its window function is defined as: Where I0 represents the zero-order modified Bessel function, β represents the window function parameter, which is used to control the main lobe width and side lobe suppression, and n represents the index of the sampling point in the frame; After windowing, the time domain signal is converted into a frequency domain signal using the fast Fourier transform (FFT), and the calculation formula is: Among them, x ′ [n] represents the normalized signal, k represents the frequency variable, X(k) represents the frequency domain function after fast Fourier transform, and n is the index variable, which represents the nth sampling point in the current frame; The power spectrum is obtained by converting each frame of signal into frequency domain and summing the squares of the frequency domain signal: P(k)=|X(k)| 2 。 4. The deep learning-based sleep apnea recognition system according to claim 2, characterized in that: The filter adopts a Mel filter, and the key features include Mel frequency cepstrum coefficients, and the Mel frequency cepstrum coefficients are extracted by discrete cosine transform of the energy of the logarithmic filter bank; The frequency response of the Mel filter is: Among them, f m-1 ,f m ,f m+1 They represent the left and right boundary frequencies and the center frequency of the Mel filter respectively, k represents the frequency index, and m represents the number of the Mel filter; The output energy of each Mel filter is calculated by the following formula: Among them, P(k) represents the power spectrum of the signal, H m (k) represents the frequency response of the Mel filter.

5. The deep learning-based sleep apnea recognition system according to claim 2, characterized in that: The key feature includes filter bank energy, and the energy of the direct output filter bank is used as another audio feature to obtain a filter bank energy feature; Take the logarithm of the output energy of the filter bank: By performing discrete cosine transform on the log filter bank energy, the cepstral coefficients c are extracted. n , the formula is as follows: Wherein, M represents the total number of filters, and the first N' cepstral coefficients are taken as the Mel-frequency cepstral coefficients.

6. The deep learning-based sleep apnea recognition system according to claim 2, characterized in that: The key features include short-time energy, which captures the instantaneous intensity change of the signal by extracting the short-time energy of each frame, and at the same time, normalizes the short-time energy to obtain the normalized short-time energy; The short-time energy calculation formula for each frame signal is: The short-time energy is normalized: Among them, max(E) represents the maximum value obtained in the short-time energy sequence of the entire signal.

7. The deep learning-based sleep apnea recognition system according to claim 2, characterized in that: The key features include linear prediction coefficients, which are extracted by calculating the autocorrelation of the audio signal; The correlation uses the following correlation function: Where p represents the LPC order; The linear prediction coefficient extraction formula is as follows: Among them, a k Represents the k-th order linear prediction coefficient, j is an integer index from 1 to p, indicating that the j-th order linear prediction coefficient is currently being solved.

8. The deep learning-based sleep apnea recognition system according to claim 1, characterized in that: The feature processing module extracts spectral features from audio features through convolution operations, extracts time domain features through a fully connected layer, and converts them into feature vectors suitable for classification tasks; the output of the convolution layer is downsampled through a pooling operation; the fully connected layer performs feature mapping to compress and map the extracted time domain features and outputs time series features; The dimensions of the output features are: Among them, K represents the convolution kernel size, s represents the pooling step size, and F input Represents the input dimension.

9. The sleep apnea recognition system based on deep learning according to claim 1, characterized in that: The time-weighted attention mechanism module highlights the influence of key moments in the audio signal according to the feature weight of each time step of the time series data, and finally obtains a context vector containing time step information; the specific execution process is as follows: 1) Weight calculation: For each time step t, calculate the attention weight e t : e t =tanh(W h ·h t +b h ) Among them, W h represents the weight matrix, h t represents the hidden state of the current time step, b h represents bias, tanh(·) represents the hyperbolic tangent activation function; 2) Normalized weight: In order to make the sum of the weights of all time steps equal to 1, calculate the normalized attention weight α t : Among them, e t represents the attention score at each time step, α t represents the normalized attention weight; 3) Weighted feature output: The features of the time step are weighted by the attention weights, and the context vector is finally obtained: Among them, c represents the context vector, which contains the information of important time steps and is used for subsequent classification tasks.

10. The deep learning-based sleep apnea recognition system according to claim 8, characterized in that: The bidirectional long short-term memory network module generates a hidden state containing context information for each time step. At each time step, the forward and backward hidden states are spliced ​​into a joint hidden state. The output of the bidirectional long short-term memory network is compressed and nonlinearly mapped through a fully connected layer to generate features required for classification. The features output by the fully connected layer are converted into predicted probabilities for each category, and the error between the predicted probability and the true label is optimized through a cross entropy loss function. The specific execution process is as follows: 1) The forward LSTM network starts from the first time step t=1 of the time series and processes backward step by step to update the hidden state and memory status The update formula is as follows: in, represents the output gate activation value at the current time step t, Represents the memory state of the current time step t, which is used to store long-term information before and after time step t; The backward long short-term memory network starts from the last time step t = T of the time series, and gradually processes forward to update the hidden state and memory status The calculation formula of the backward LSTM network is similar to that of the forward LSTM network, except that the processing order is reversed: 2) At each time step t, the hidden states of the forward LSTM network and the backward LSTM network are concatenated into a joint hidden state h t : Finally, the output shape O of the bidirectional long short-term memory network is expressed as: O=B×T×(2·H) Where B represents the batch size, T represents the length of the sequence, and H represents the hidden state dimension of the unidirectional long short-term memory network; The output of the bidirectional long short-term memory network is compressed and nonlinearly mapped through the fully connected layer to generate the feature z required for classification: z=σ(W z ·c+b z ) in, represents the weight matrix, represents the context vector, b z ∈R d represents the bias, σ represents the activation function; after being mapped by the fully connected layer, the dimension of feature z becomes R d ; 3) The feature z output by the fully connected layer is converted into the predicted probability p(y=i|z) of each category through the Softmax layer: Where C represents the number of categories, and p(y=i|z) represents the predicted probability that the input feature z belongs to category i; 4) The classification model uses the cross entropy loss function to optimize the error between the predicted probability and the true label: Among them, N represents the batch size, C represents the number of categories, and y n,i represents the true label of the nth sample in category i, p n,i Represents the predicted probability of the nth sample in category i.

Images