Deep learning based sleep disordered breathing identification system

Abstract

The application discloses a sleep breathing disorder identification system based on deep learning, comprising an audio signal feature extraction module, a feature processing module, a time-weighted attention mechanism module and a bidirectional long short-term memory network module connected in sequence. The audio signal feature extraction module extracts multi-dimensional audio features; the feature processing module processes the features through convolution and full connection layers; the time-weighted attention mechanism module highlights key moment information; the Bi-LSTM module models forward and backward time sequence dependency relationships and is classified through a full connection layer and a Softmax layer. The device comprises a snoring sound input module, a sound detection module, an embedded module and a data display module, can monitor snoring sound signals in real time, displays the detection results in a graphical interface after processing, realizes efficient and accurate identification of various sleep breathing disorders, overcomes the problems of incomplete feature extraction and low modeling efficiency in the prior art, and improves the accuracy and robustness of the audio signal classification task.

Description

TECHNICAL FIELD

[0001] The present application belongs to the field of audio signal processing and deep learning, and particularly relates to a sleep breathing disorder recognition system based on deep learning. BACKGROUND

[0002] With the wide application of audio signal processing in the fields of speech recognition, sound event detection, music analysis, etc., how to efficiently and accurately extract useful features from original audio signals has always been an important research problem in this field. Traditional audio feature extraction methods such as Mel frequency cepstral coefficient (MFCC) and short-time energy (STE) have been widely used in speech and audio signal processing. However, these traditional methods usually cannot fully capture the timing relationship and complex spectral information in audio signals, so their classification performance is limited in some complex tasks.

[0003] At present, with the rapid development of deep learning, neural network structures such as convolutional neural network (CNN) and recurrent neural network (RNN) are widely used in audio signal processing and classification tasks. In particular, long short-term memory network (LSTM) can effectively capture the long and short term dependencies in time series data, providing a better solution for the timing modeling of audio signals. In addition, network models based on attention mechanism have been proven to highlight important time steps in processing signals, further improving the performance of classification models.

[0004] However, in the prior art, how to ensure the accuracy of audio signal feature extraction while combining effective timing modeling and classification strategies remains a problem to be solved. Existing feature extraction methods and timing modeling techniques often have problems such as insufficient comprehensive feature extraction and low modeling efficiency, which limit their application effect in complex audio signal classification tasks. SUMMARY

[0005] To solve the problems of the prior art and achieve the purpose of efficiently extracting multi-dimensional features of audio signals while improving the accuracy and robustness of audio signal classification tasks, the present application adopts the following technical solutions:

[0006] The sleep breathing disorder recognition system based on deep learning comprises a sound detection module, which comprises an audio signal feature extraction module, a feature processing module, a time-weighted attention mechanism module and a bidirectional long short-term memory network module connected in sequence.

[0007] The audio signal feature extraction module collects sleep breathing audio signals and extracts audio features.

[0008] The feature processing module extracts multiple timing features from the audio features, including spectral features and time domain features, which effectively improves the separability and representation ability of the features, laying a foundation for subsequent classification tasks;

[0009] The time-weighted attention mechanism module weights the timing features of the audio signal, highlights the information at key moments in the audio signal, and improves the response sensitivity of the model at key moments, thereby improving the overall classification accuracy;

[0010] The bidirectional long short-term memory network module captures the dependency between the previous and subsequent moments in the audio signal by forward and backward modeling of the time series data, generates a hidden state representation containing context information, and adjusts the network parameters according to the training results of the classification task. The classification result is obtained by forward propagation, and the network training is completed by the back propagation algorithm. This module combines the global information of the time series to provide more accurate timing feature representation for the classification task. The classification layer uses a fully connected layer and a Softmax layer to classify the features and output the class probability of the audio signal. The model is optimized by the cross-entropy loss function to ensure that it continuously improves its performance and has good generalization ability during the training process.

[0011] Further, in the audio signal feature extraction module, the audio features are pre-emphasized and framed. During pre-emphasis, the attenuation of low frequencies in the speech signal is compensated for, and high-frequency information is emphasized. Framing is to analyze the short-time characteristics of the audio signal. Each frame is windowed to reduce spectral leakage. The time domain signal is then converted to a frequency domain signal to further reduce the impact of spectral leakage. The power spectrum is generated based on each frame of signal frequency domain conversion. The energy is output based on the power spectrum and the filter bank applied to the power spectrum. The key features of the audio signal are extracted.

[0012] Further, in the pre-emphasis and framing, the pre-emphasis formula is:

[0013] y(t) = x(t) - alpha * x(t-1)

[0014] Where t represents the time, x(t) represents the audio signal at the t time, y(t) represents the signal after pre-emphasis processing, and alpha represents the filter coefficient for adjusting the degree of low frequency suppression. Through this processing, the attenuation of low frequencies in the speech signal is compensated for, and the high-frequency information is emphasized, improving the spectral resolution of the signal.

[0015] In order to analyze the short-time characteristics of the audio signal, the present application divides it into multiple consecutive short-time frames. The framing process is achieved by the following method:

[0016] x[n]→x m [n] = x[n + mH], 0 ≤ n < N

[0017] where x[n] represents the original discrete audio signal, n represents the index of the sample point within the frame, x m [m] represents the m-th frame signal after framing, m represents the frame index, H represents the frame shift (set as 50% of the frame length), and N represents the number of sample points per frame;

[0018] In order to reduce spectral leakage, a Kaiser window is added to each frame signal, and the window function is defined as:

[0019]

[0020] where I0 represents the zero-order modified Bessel function, β represents the window function parameter for controlling the main lobe width and side lobe suppression, and n represents the index of the sample point within the frame;

[0021] After windowing, the time-domain signal is converted to the frequency-domain signal using the Fast Fourier Transform (FFT) to reduce the effect of spectral leakage, and the calculation formula is:

[0022]

[0023] where x ′ [n] represents the normalized signal, k represents the frequency variable, X(k) represents the frequency-domain function after the Fast Fourier Transform, n is the index variable, and represents the nth sample point within the current frame;

[0024] By performing frequency-domain conversion on each frame signal and summing the squares of the frequency-domain signals, the power spectrum is obtained:

[0025] P(k) = |X(k)| 2 .

[0026] Further, the filter uses a Mel filter, and the key features include Mel frequency cepstral coefficients, which are extracted by discrete cosine transform of the log filter bank energy;

[0027] The frequency response of the Mel filter is:

[0028]

[0029] where f m-1 ,f m ,f m+1 represent the left and right boundary frequencies and the center frequency of the Mel filter, respectively, k represents the frequency index, and m represents the number of Mel filters;

[0030] The output energy of each Mel filter is calculated by the following formula:

[0031]

[0032] Wherein, P(k) represents the power spectrum of the signal, H m (k) represents the frequency response of the Mel filter.

[0033] Further, the key features include filter bank energy, taking the energy of the direct output filter bank as another audio feature, to obtain filter bank energy features;

[0034] The output energy of the filter bank is logarithmically taken:

[0035]

[0036] The cepstral coefficients c n are extracted by performing discrete cosine transform on the logarithmic filter bank energy, and the formula is as follows:

[0037]

[0038] Wherein, M represents the total number of filters, used for normalizing the frequency range, and the first N' cepstral coefficients are taken as Mel frequency cepstral coefficients.

[0039] Further, the key features include short-time energy, by extracting the short-time energy of each frame, the instantaneous intensity change of the signal is captured, and at the same time, in order to improve the stability of the signal features, the influence of external noise on the short-time energy is reduced, and the short-time energy is normalized to obtain the normalized short-time energy.

[0040] The short-time energy calculation formula of each frame signal is:

[0041]

[0042] In order to improve the stability of the signal features and reduce the influence of external noise on the short-time energy, the short-time energy is normalized in the application:

[0043]

[0044] Wherein, max(E) represents the maximum value taken in the short-time energy sequence of the entire signal.

[0045] Further, the key features include linear prediction coefficients, by calculating the autocorrelation of the audio signal, the linear prediction coefficients are extracted using the Levinson-Durbin algorithm.

[0046] The correlation uses the following correlation function:

[0047]

[0048] Wherein, p represents the LPC order, which determines the complexity and fitting ability of the model, which is 10 orders here.

[0049] The linear prediction coefficient extraction formula is as follows:

[0050]

[0051] wherein a k represents the k-th order linear prediction coefficient, j is an integer index from 1 to p, and represents that the j-th order linear prediction coefficient is currently being solved.

[0052] Further, the feature processing module extracts spectral features from the audio features through convolution operation, extracts time domain features through full connection layer, and converts them into feature vectors suitable for classification tasks; the output of the convolution layer is down-sampled through the pooling operation; the full connection layer performs feature mapping, which is used to compress and map the extracted time domain features, and outputs the time sequence features;

[0053] The dimension of the output feature is:

[0054]

[0055] wherein K represents the convolution kernel size, s represents the pooling step, F input represents the input dimension.

[0056] Further, the time weighted attention mechanism module highlights the influence of key moments in the audio signal according to the feature weight of each time step of the time sequence data, in order to improve the response sensitivity of the model at key moments, and finally obtains a context vector containing time step information; the specific execution process is as follows:

[0057] 1) Weight calculation: for each time step t, the attention weight e t is calculated:

[0058] e t =tanh(W h ·h t +b h )

[0059] wherein W h represents the weight matrix, h t represents the hidden state of the current time step, b h represents the bias, and tanh(·) represents the hyperbolic tangent activation function.

[0060] 2) Normalized weight: in order to make the sum of the weights of all time steps equal to 1, the normalized attention weight a t is calculated:

[0061]

[0062] wherein e t represents the attention score of each time step, and at denotes the normalized attention weight;

[0063] 3) Weighted feature output: Weight the features of each time step by the attention weight, and finally get the context vector:

[0064]

[0065] where c represents the context vector, containing the information of important time steps, used for subsequent classification tasks.

[0066] Further, the bidirectional long short-term memory network module is used to generate a hidden state containing context information for each time step, at each time step, the forward and backward hidden states are spliced into a joint hidden state, the output of the bidirectional long short-term memory network is compressed and nonlinearly mapped through a fully connected layer to generate features required for classification, the features output by the fully connected layer are converted into prediction probabilities for each class, and the error between the prediction probabilities and the true labels is optimized through a cross-entropy loss function; the specific execution process is as follows:

[0067] 1) The forward long short-term memory network starts from the first time step t = 1 of the time series, processes step by step backward, and updates the hidden state and the memory state The update formula is as follows:

[0068]

[0069] where, denotes the output gate activation value of the current time step t, denotes the memory state of the current time step t, used to store long-term information before and after the time step t;

[0070] The backward long short-term memory network starts from the last time step t = T of the time series, processes step by step forward, and updates the hidden state and the memory state The calculation formula of the backward long short-term memory network is similar to that of the forward long short-term memory network, except that the processing order is reversed:

[0071]

[0072] 2) At each time step t, the hidden states of the forward long short-term memory network and the backward long short-term memory network are spliced into a joint hidden state h t :

[0073]

[0074] Finally, the output shape O of the bidirectional long short-term memory network is represented as:

[0075] O = B * T * (2 * H)

[0076] where B denotes batch size, T denotes the length of sequence, and H denotes the hidden state dimension of one-way LSTM network.

[0077] The output of the bidirectional LSTM network is compressed and nonlinearly mapped by the fully connected layer to generate the feature z required for classification:

[0078] z = σ (W z ·c+b z )

[0079] where W z ∈R d denotes the weight matrix, c d denotes the context vector, and b n,i denotes the bias, and σ denotes the activation function; after mapping by the fully connected layer, the dimension of the feature z becomes R n,i .

[0080] 3) The feature z output by the fully connected layer is converted into the predicted probability p (y = i | z) of each class by the Softmax layer:

[0081]

[0082] where C denotes the number of classes (4 classes of snoring, low hypopnea, airflow limitation, and pause), and p (y = i | z) denotes the predicted probability that the input feature z belongs to class i.

[0083] 4) The classification model uses the cross-entropy loss function to optimize the error between the predicted probability and the true label:

[0084]

[0085] where N denotes the batch size, C denotes the number of classes, y m denotes the true label (one-hot encoding) of the nth sample in class i, and p m denotes the predicted probability of the nth sample in class i.

[0086] The advantages and beneficial effects of the present application are:

[0087] The deep learning-based sleep apnea identification system of the present invention includes an audio signal feature extraction module, a feature processing module, a time-weighted attention mechanism module, and a bidirectional long short-term memory network (Bi-LSTM) module connected in sequence. By combining a temporal modeling method based on convolutional neural networks and bidirectional LSTM, the present invention can efficiently extract multi-dimensional features of audio signals while improving the accuracy and robustness of audio signal classification tasks, and realize the identification of four types of sleep apnea disorders: snoring, hypopnea, airflow limitation, and pauses. Attached Figure Description

[0088] Figure 1 This is a schematic diagram of the system structure in an embodiment of the present invention.

[0089] Figure 2 This is a flowchart illustrating the system execution process in an embodiment of the present invention. Detailed Implementation

[0090] The specific embodiments of the present invention will be described in detail below with reference to the accompanying drawings. It should be understood that the specific embodiments described herein are for illustration and explanation only and are not intended to limit the present invention.

[0091] like Figure 1 , Figure 2 As shown, the deep learning-based sleep apnea identification system includes a snoring input module, a sound detection module, an RK3558 embedded Linux module, and a data display module. The snoring input module monitors and captures snoring signals in real time and transmits them to the embedded RK3558 Linux system. These signals are then processed by the sound detection module. Finally, the data display module presents the processing results to the user in an intuitive way and provides diagnostic suggestions for possible sleep apnea disorders based on the analysis results. The system ensures efficient processing and accurate analysis of snoring signals, providing users with a reliable sleep health monitoring tool. The system can record and identify input snoring signals and display the identification and classification results on the data display module in a graphical interface, facilitating a clear understanding of sleep apnea disorders.

[0092] In this embodiment of the invention, the snoring input module monitors and identifies snoring signals in real time through a professional snoring detection microphone, and transmits and stores them in the RK3558 embedded Linux module; the sound detection module uses the aforementioned sleep apnea disorder identification and classification method; the RK3558 embedded Linux module provides the module with a computing platform and operating system support to ensure that each module can work together efficiently; the data display module is used to display the snoring detection results to the user in the form of a graphical interface, making it easier for the user to intuitively understand sleep apnea disorder.

[0093] The sound detection module comprises an audio signal feature extraction module, a feature processing module, a time-weighted attention mechanism module and a bidirectional long short-term memory (Bi-LSTM) module connected in sequence, and each module cooperates to realize efficient classification and identification of sleep respiratory disorders.

[0094] The audio signal feature extraction module is responsible for pre-processing, frequency domain conversion and feature extraction of the input sleep audio signal to generate audio features suitable for identification of 4 types of sleep respiratory disorder diseases, i.e. snoring, low ventilation, airflow limitation and pause.

[0095] Specifically, the audio signal feature extraction module is used to extract multiple key features from the original audio signal, including Mel frequency cepstral coefficient (MFCC), filter bank energy (Fbanks), short-time energy (STE) and linear prediction coefficient (LPC). MFCC can effectively represent the spectral characteristics of the audio signal, Fbanks reflects the frequency energy distribution of the audio signal, STE captures the instantaneous changes of the signal, and LPC is mainly used to describe the vocal characteristics of the audio signal. The feature extraction process is as follows:

[0096] 1) Pre-emphasis and framing of the audio signal, pre-emphasis is to compensate for the attenuation of low frequencies in the speech signal and emphasize high frequency information, framing is to analyze the short-time characteristics of the audio signal, each frame is windowed to reduce spectral leakage, and then the time domain signal is converted to a frequency domain signal to further reduce the influence of spectral leakage, and a power spectrum is generated based on each frame signal frequency domain conversion to estimate the periodogram of the power spectrum.

[0097] A high-pass filter is used for pre-emphasis processing; the high frequency components in the speech signal are compensated by subtracting the previous sampling point, and the pre-emphasis formula is:

[0098] y(t) = x(t) - αx(t-1)

[0099] Where t represents the time, x(t) represents the audio signal at the t time, y(t) represents the signal after pre-emphasis processing, and a represents the filter coefficient for adjusting the degree of low frequency suppression, which is typically 0.97. Through this processing, the attenuation of low frequencies in the speech signal is compensated, the high frequency information is emphasized, and the spectral resolution of the signal is improved.

[0100] In order to analyze the short-time characteristics of the audio signal, the present application divides it into multiple continuous short-time frames, and the framing process is realized by the following method:

[0101] x[n]→x m [n] = x[n + mH], 0 ≤ n < N

[0102] where x[n] represents the original discrete audio signal, n represents the index of the sample point within the frame, x m [m, n] represents the m-th frame signal after framing, m represents the frame index, H represents the frame shift (set as 50% of the frame length), and N represents the number of sample points per frame;

[0103] In order to reduce spectral leakage, a Kaiser window is added to each frame signal, and the window function is defined as:

[0104]

[0105] where I0 represents the zero-order modified Bessel function, β represents the window function parameter used to control the main lobe width and side lobe suppression, and n represents the index of the sample point within the frame;

[0106] After windowing, the time-domain signal is converted to the frequency domain signal using the Fast Fourier Transform (FFT) to reduce the effect of spectral leakage, and the calculation formula is:

[0107]

[0108] where x ′ [n] represents the normalized signal, k represents the frequency variable, X(k) represents the frequency domain function after the Fast Fourier Transform, n is the index variable, and represents the nth sample point within the current frame.

[0109] By performing frequency domain conversion on each frame signal and summing the squares of the frequency domain signals, the power spectrum is obtained:

[0110] P(k) = |X(k)| 2

[0111] 2) Apply the Mel filter bank to the power spectrum to obtain the energy of the filter bank;

[0112] The frequency response of the Mel filter is:

[0113]

[0114] where f m-1 , f m , and f m+1 represent the left and right boundary frequencies and the center frequency of the Mel filter, respectively, k represents the frequency index, and m represents the number of the Mel filter;

[0115] The output energy of each Mel filter is calculated by the following formula:

[0116]

[0117] where P(k) represents the power spectrum of the signal, H m (k) represents the frequency response of the Mel filter.

[0118] 3) Extract the Mel-frequency cepstral coefficients (MFCC) by discrete cosine transform (DCT) of the log filter bank energies;

[0119] In order to compress the dynamic range, the output energy of the filter bank is taken as the logarithm:

[0120]

[0121] By taking the discrete cosine transform (DCT) of the log filter bank energy, the Mel-frequency cepstral coefficients (MFCC) are extracted, and the formula is as follows:

[0122]

[0123] Where M represents the total number of filters, and the first 13 cepstral coefficients are taken as the Mel-frequency cepstral coefficients (MFCC) for normalization of the frequency range.

[0124] 4) Extract the filter bank energy features Fbanks, which provide spectral information of the audio signal and help distinguish different audio patterns;

[0125] The energy of the filter bank is directly output as another audio feature, and the energy feature Fbanks is obtained as follows:

[0126] F m = E m , 1≤m≤M

[0127] 5) Extract the short-time energy (STE) of each frame to capture the instantaneous intensity changes of the signal;

[0128] The formula for calculating the short-time energy of each frame of signal is:

[0129]

[0130] In order to improve the stability of the signal features and reduce the influence of external noise on the short-time energy, the short-time energy is normalized in the present application:

[0131]

[0132] Where max(E) represents the maximum value taken in the short-time energy sequence of the entire signal.

[0133] 6) Extract the linear prediction coefficients (LPC) by calculating the autocorrelation of the audio signal using the Levinson-Durbin algorithm;

[0134] The correlation uses the following correlation function:

[0135]

[0136] where p represents the LPC order, which determines the complexity and fitting ability of the model, and here is 10 orders;

[0137] The linear prediction coefficient extraction formula is as follows:

[0138]

[0139] where a k represents the kth order linear prediction coefficient, j is an integer index from 1 to p, and represents the jth order linear prediction coefficient currently being solved.

[0140] The feature processing module further processes the extracted audio features, extracts spectral features through convolution operation, and extracts time domain features through fully connected layer, and converts them into feature vectors suitable for classification tasks;

[0141] Specifically, the feature processing module extracts various time sequence features from the audio signal, including spectral features (MFCC and Fbanks) and time domain features (STE and LPC), extracts spectral features through convolution operation, and extracts time domain features through fully connected layer.

[0142] The convolutional network extracts effective information from spectral features (MFCC and Fbanks) through convolution operation, and the output of the convolutional layer is down-sampled through pooling operation; assuming that the convolution kernel size is K, the pooling step is s, and the input dimension is F input , then the dimension of the output feature is:

[0143]

[0144] When extracting time domain features (short-time energy STE and linear prediction coefficient LPC), a fully connected layer is used for feature mapping, which is used to compress and map the extracted time domain features, and output time sequence features.

[0145] The time-weighted attention mechanism module highlights the influence of key moments in the audio signal according to the feature weight of each time step of the time series data, in order to improve the response sensitivity of the model at key moments;

[0146] Specifically, the time-weighted attention mechanism module calculates the feature weight of each time step to highlight the information of important moments by weighting the features, and the specific execution process is as follows:

[0147] 1) Weight calculation: for each time step t, calculate the attention weight e t :

[0148] e t =tanh(W h ·h t +b h )

[0149] where W h denotes the weight matrix, h t denotes the hidden state of the current time step, b h denotes the bias, tanh(·) denotes the hyperbolic tangent activation function.

[0150] 2) Normalized weights: In order to make the sum of weights of all time steps equal to 1, the normalized attention weights α t are calculated:

[0151]

[0152] where e t denotes the attention score of each time step, α t denotes the normalized attention weight.

[0153] 3) Weighted feature output: The features of the time steps are weighted by the attention weights, and the final context vector c is obtained:

[0154]

[0155] where c denotes the context vector containing important time step information for subsequent classification tasks.

[0156] The Bi-LSTM module captures the dependency between the previous and subsequent time steps of the time series data by modeling the forward and backward directions, adjusts the network parameters according to the training results of the classification task, and obtains the classification results through forward propagation and completes the training of the network through the backpropagation algorithm.

[0157] Specifically, the Bi-LSTM module uses a bidirectional LSTM to capture the dependency between the previous and subsequent time steps, generating a hidden state containing context information for each time step. The specific implementation process is as follows:

[0158] 1) Forward LSTM starts from the first time step (t = 1) of the time series and processes backward step by step, updating the hidden state and memory state The update formula is as follows:

[0159]

[0160] where, denotes the output gate activation value of the current time step t, denotes the memory state of the current time step t, used to store long-term information before and after the time step t;

[0161] Backward LSTM starts from the last time step (t = T) of the time series and processes step by step forward, updating the hidden state and memory state The calculation formula of the backward LSTM is similar to that of the forward LSTM, except that the processing order is reversed:

[0162]

[0163] 2) At each time step t, the hidden states of the forward and backward LSTMs are spliced into a joint hidden state h t :

[0164]

[0165] Finally, the output shape O of the bidirectional LSTM is represented as:

[0166] O = B x T x (2 x H)

[0167] where B represents the batch size, T represents the length of the sequence, and H represents the hidden state dimension of the unidirectional LSTM.

[0168] The output of the bidirectional LSTM is compressed and nonlinearly mapped through a fully connected layer to generate the features z required for classification:

[0169] z = σ(W z ·c + b z )

[0170] where W represents the weight matrix, c represents the context vector, b z ∈ R d represents the bias, and σ represents the activation function; after mapping through the fully connected layer, the dimension of the feature z becomes R d .

[0171] 3) The feature z output by the fully connected layer is converted into the predicted probability p(y = i | z) of each class through a Softmax layer:

[0172]

[0173] where C represents the number of classes (4 classes: snoring, low ventilation, airflow limitation, and pause), and p(y = i | z) represents the predicted probability that the input feature z belongs to class i;

[0174] 4) The classification model uses the cross-entropy loss function to optimize the error between the predicted probability and the true label:

[0175]

[0176] wherein, N represents batch size, C represents category number, y n,i represents the real label (one-hot encoding) of the nth sample on the category i, p n,i represents the predicted probability of the nth sample on the category i.

[0177] The present application realizes efficient classification and identification of complex sleep breathing disorders through the synergistic design of the above modules, and realizes high accuracy and robustness.

[0178] The above examples are only used to illustrate the technical solutions of the present application, but not to limit them; although the present application has been described in detail with reference to the foregoing examples, those skilled in the art should understand that the technical solutions recorded in the foregoing examples can be modified, or some or all of the technical features can be replaced by equivalents; and these modifications or replacements do not make the essence of the corresponding technical solutions deviate from the scope of the technical solutions of the present application.

Claims

1. A deep learning-based sleep apnea recognition system, including a sound detection module, characterized in that: The sound detection module includes an audio signal feature extraction module, a feature processing module, a time-weighted attention mechanism module, and a bidirectional long short-term memory network module connected in sequence. The audio signal feature extraction module collects sleep breathing audio signals and extracts audio features. The feature processing module extracts various temporal features from the audio features, including spectral features and time-domain features; The audio features are extracted by convolutional operations to extract spectral features, and temporal features are extracted by fully connected layers, which are then transformed into feature vectors suitable for classification tasks. The output of the convolutional layer is downsampled by pooling operations. The fully connected layer performs feature mapping to compress and map the extracted temporal features, outputting temporal features. The dimensions of the output features are: in, Indicates the kernel size. Indicates the pooling step size. Indicates the input dimension; The time-weighted attention mechanism module weights the temporal features of the audio signal to highlight information at key moments in the audio signal; The bidirectional long short-term memory (LSTM) network module captures the dependencies between time points in an audio signal by modeling the forward and backward propagation of time-series data, generating hidden state representations containing contextual information. It adjusts network parameters based on the training results of the classification task. The classification result is obtained through forward propagation, and the network is trained using a backpropagation algorithm. This generates hidden states containing contextual information for each time step. At each time step, the forward and backward hidden states are concatenated into a joint hidden state. The output of the bidirectional LSM network is compressed and nonlinearly mapped through a fully connected layer to generate the features required for classification. The features output by the fully connected layer are converted into predicted probabilities for each category, and the error between the predicted probabilities and the true labels is optimized using a cross-entropy loss function. The specific execution process is as follows: 1) Feedforward Long Short-Term Memory Network from the first time step of the time series Begin by processing sequentially backwards, updating the hidden state. and memory state The updated formula is as follows: in, Indicates the current time step The output gate activation value, Indicates the current time step The memory state is used to store time steps. Long-term information before and after; Backward Long Short-Term Memory Network from the last time step of the time series Begin by processing step by step, updating the hidden state. and memory state The calculation formula for backward long short-term memory networks is the same as that for forward long short-term memory networks, only the processing order is reversed: 2) At each time step The hidden states of the forward Long Short-Term Memory (LSTM) network and the backward LSTM network are concatenated into a joint hidden state. : Ultimately, the output shape of the bidirectional long short-term memory network Represented as: in, Indicates batch size. Indicates the length of the sequence. This represents the hidden state dimension of a unidirectional long short-term memory network; The output of the bidirectional long short-term memory network is compressed and nonlinearly mapped using a fully connected layer to generate the features required for classification. : in, Represents the weight matrix. Represents the context vector. Indicates bias. This represents the activation function; after mapping through a fully connected layer, the features... The dimension becomes ; 3) The features output by the fully connected layer are processed through the Softmax layer. Converted to predicted probabilities for each category : in, Indicates the number of categories. Representing input features Category The predicted probability; 4) The classification model uses the cross-entropy loss function to optimize the error between the predicted probability and the true label: in, Indicates batch size. Indicates the number of categories. Indicates the first Each sample in category The real labels on Indicates the first Each sample in category The predicted probability.

2. The sleep apnea identification system based on deep learning according to claim 1, characterized in that: In the audio signal feature extraction module, the audio features are pre-emphasized and framed, each frame is windowed, the time-domain signal is converted into a frequency-domain signal, a power spectrum is generated based on the frequency domain conversion of each frame, and the key features in the audio signal are extracted based on the power spectrum and the output energy of the filter bank applied to the power spectrum.

3. The sleep apnea identification system based on deep learning according to claim 2, characterized in that: In the pre-emphasis and framing, the pre-emphasis formula is: Where t represents time. Indicates the first The audio signal at that moment, This indicates the signal after pre-emphasis processing. The filter coefficients indicate the degree of low-frequency suppression. It is divided into multiple consecutive short frames, and the framing process is implemented in the following way: in, This represents the original discrete audio signal. Indicates the index of the sampling point within the frame. Indicates the 1st frame after frame division Frame signal, Indicates the frame index. Indicates frame shift, This indicates the number of sample points per frame; A weighted Kaiser window is applied to each frame of the signal, and its window function is defined as follows: in, Represents the zeroth-order modified Bessel function. This represents the window function parameters, used to control the main lobe width and side lobe suppression. Indicates the index of the sampling point within the frame; After windowing, the time-domain signal is converted to the frequency-domain signal using the Fast Fourier Transform (FFT). The calculation formula is as follows: in, This represents the normalized signal, where k represents the frequency variable. This represents the frequency domain function after the Fast Fourier Transform. This is an index variable, representing the index of the current frame. One sampling point; By performing frequency domain transformation on each frame of the signal and summing the squares of the frequency domain signals, the power spectrum is obtained: 。 4. The sleep apnea identification system based on deep learning according to claim 2, characterized in that: The filter is a Mel filter, and the key feature includes Mel frequency cepstral coefficients, which are extracted by the discrete cosine transform of the energy of the logarithmic filter bank. The frequency response of the Mel filter is: in, These represent the left and right boundary frequencies and the center frequency of the Mel filter, respectively. Indicates frequency index, Indicates the number of the Mel filter; The output energy of each Mel filter is calculated using the following formula: in, Represents the power spectrum of the signal. This represents the frequency response of the Mel filter.

5. The sleep apnea identification system based on deep learning according to claim 2, characterized in that: The key feature includes filter bank energy, which is obtained by taking the energy of the directly output filter bank as another audio feature. Take the logarithm of the output energy of the filter bank: Cepstral coefficients are extracted by performing a discrete cosine transform on the energy of the logarithmic filter bank. The formula is as follows: in, This represents the total number of filters, and the first N' cepstral coefficients are taken as the Mel frequency cepstral coefficients.

6. The sleep apnea identification system based on deep learning according to claim 2, characterized in that: The key feature includes short-time energy. By extracting the short-time energy of each frame, the instantaneous intensity change of the signal is captured. At the same time, the short-time energy is normalized to obtain the normalized short-time energy. The formula for calculating the short-time energy of each frame of signal is: The short-time energy was normalized: in, This represents the maximum value obtained in the short-time energy sequence of the entire signal.

7. The sleep apnea identification system based on deep learning according to claim 2, characterized in that: The key feature includes linear prediction coefficients, which are extracted by calculating the autocorrelation of the audio signal. The correlation is expressed using the following correlation function: in, Indicates the order of LPC; The formula for extracting linear prediction coefficients is as follows: in, Indicates the first Linear prediction coefficients of order, From 1 to The integer index indicates that the solution is currently being performed on the i-th... Linear prediction coefficients of order 1.

8. The sleep apnea identification system based on deep learning according to claim 1, characterized in that: The time-weighted attention mechanism module, based on the feature weights of each time step in the time-series data, highlights the influence of key moments in the audio signal, ultimately obtaining a context vector containing time-step information; the specific execution process is as follows: 1) Weight calculation: For each time step Calculate attention weights : in, Represents the weight matrix. This indicates the hidden state at the current time step. Indicates bias. Represents the hyperbolic tangent activation function; 2) Normalized weights: To ensure that the sum of the weights at all time steps is 1, normalized attention weights are calculated. : in, This represents the attention score at each time step. This represents the normalized attention weights; 3) Weighted Feature Output: The features at each time step are weighted using attention weights to obtain the final context vector. in, This represents the context vector, which contains information about important time steps and is used for subsequent classification tasks.

Images