Sleep apnea detection method based on multi-view acoustic spectral clustering

Abstract

The invention discloses a sleep apnea detection method based on multi-view acoustic spectral clustering, and belongs to the technical field of digital signal processing, and the method comprises the steps: collecting sleep respiration audio data of a user, and carrying out the framing processing; performing fast Fourier transform on the audio data after framing processing to generate a time-frequency graph of each frame; filtering each frame of audio signal by using a Mel filter, and superposing energy in each filter frequency band to obtain a Mel spectrogram; performing continuous wavelet transform on each frame of audio signal, and obtaining a wavelet map according to a wavelet coefficient; constructing a multi-view three-dimensional tensor according to the time-frequency graph, the Mel spectrogram and the wavelet graph; clustering the multi-view three-dimensional tensor to obtain a feature vector of each time step; using the feature vector as input, and using a pre-trained convolutional attention Utime network to perform apnea detection on sleep respiration audio data of the user; aiming at the low sleep apnea detection precision in the prior art, the invention provides the precision.

Description

Technical Field

[0001] The present application relates to the technical field of digital signal processing, and more specifically, to a sleep apnea detection method based on multi-view acoustic spectrum clustering. Background Art

[0002] At present, audio signals have attracted extensive research in the field of sleep apnea detection due to their good non-invasiveness and ease of use. However, some studies directly estimate AHI or OSA severity categories without detecting specific OSA events, making it difficult to manually verify the output of the method and unable to meet clinical needs. At the same time, most studies have low temporal resolution for OSA detection and cannot achieve real-time detection. Therefore, it is of great significance to develop a real-time sleep apnea detection method based on audio signals.

[0003] Application No. CN202310438705.0 proposes a method for identifying sleep apnea events by analyzing nighttime sleep audio. It mainly includes an audio acquisition module, an audio signal feature extraction module, a feature annotation module, a training set construction module, and a statistical classification module. First, the audio signal is feature extracted, and the Mel frequency cepstral coefficient (MFCC) is calculated. The 13-dimensional MFCC features are extracted by performing short-time Fourier transform (STFT) and Mel filter bank processing on the audio signal. These include 1-dimensional signal frame energy and 12-dimensional DCT coefficients to capture information such as pitch, volume, and resonance peaks in the audio signal. Then, the extracted MFCC features are annotated using a feature annotation module, which is divided into two labels: normal sleep and apnea and hypopnea events. Finally, a training set for the time series classification algorithm model is constructed based on the annotated MFCC features, trained using the MiniRocket algorithm, and the classification task is completed using a linear classifier.

[0004] Although this method has achieved the recognition of sleep apnea events to a certain extent, it still has some obvious shortcomings. First, relying solely on MFCC features is not enough to fully capture all useful information in the audio signal, especially in complex noise environments, and the selection of features with fewer dimensions may lead to information loss or misjudgment. In addition, although the MiniRocket algorithm performs well in time series classification, its fixed convolution kernel structure and single feature aggregation method cannot effectively capture the local spatiotemporal correlation and dynamic change characteristics, which greatly limits the classification performance when processing complex and changeable whole-night audio data. In addition, the system uses a fragment classification method to divide the whole-night audio into multiple fragments for processing. This method may ignore the continuity information between fragments, thereby affecting the real-time and accuracy of detection. Because the connection relationship between fragments is ignored, some short-term apnea events may be missed or misjudged at the boundary. Finally, the post-processing steps of the system are relatively simple, and there is a lack of further optimization and verification of the detection results, which also limits its reliability and practicality in clinical applications. Summary of the invention

[0005] 1. Technical issues to be solved

[0006] In view of the low accuracy of sleep apnea detection in the prior art, this application provides a sleep apnea detection method based on multi-view acoustic spectrum clustering, which comprehensively utilizes time-frequency diagram, Mel spectrum diagram and wavelet Figure 3 Acoustic spectrum features are constructed by multi-view Figure 3 dimensional tensor and cluster it to obtain low-dimensional embedding vectors that represent the features of different time steps, and then use the pre-trained convolutional attention Utime network to detect apnea events, fully explore the complementary and correlation information between multi-view acoustic spectra, and learn more robust and discriminative acoustic feature representations, which shows better pause event detection performance on respiratory audio data and has higher detection accuracy.

[0007] 2. Technical solution

[0008] The purpose of this application is achieved through the following technical solutions.

[0009] One aspect of the present application provides a sleep apnea detection method based on multi-view acoustic spectrum clustering, comprising: collecting user sleep breathing audio data; and framing the collected audio data; calculating the energy spectrum of the audio data after framing through fast Fourier transform, and generating a time-frequency diagram of each frame according to the energy spectra of all frames; filtering each frame of audio signal with a Mel filter, and obtaining a Mel spectrum diagram after superimposing the energy in each filter band; performing continuous wavelet transform on each frame of audio signal, calculating the wavelet coefficients, and obtaining a wavelet diagram according to the wavelet coefficients; constructing a multi-view acoustic spectrum clustering method according to the time-frequency diagram, the Mel spectrum diagram and the wavelet diagram. Figure 3 dimensional tensor; for multi-view Figure 3 dimensional tensors are clustered to obtain the feature vector of each time step; the feature vector is used as input and the pre-trained convolutional attention Utime network is used to perform apnea detection on the user's sleep breathing audio data.

[0010] Among them, the Mel filter is a filter bank designed based on the auditory characteristics of the human ear. The human ear has different perception abilities for sounds of different frequencies, and the Mel scale is a frequency scale proposed based on this characteristic of the human ear. The Mel filter bank consists of a series of triangular filters, each of which is arranged at equal intervals on the Mel scale and has bandwidths of varying widths on the frequency axis. The filter bandwidth in the low-frequency region is narrower, and the filter bandwidth in the high-frequency region is wider, which is consistent with the frequency selectivity of the human ear. Filtering the audio signal through the Mel filter bank can obtain a frequency domain representation that better matches the perception of the human ear.

[0011] The time-frequency diagram is a commonly used method for representing audio signals in the time-frequency domain. By performing a short-time Fourier transform (STFT) on the audio signal, the energy distribution of the signal at different time points and frequency points can be obtained, forming a two-dimensional time-frequency matrix. The time-frequency diagram intuitively shows the change of signal energy over time and frequency, and can capture the dynamic characteristics of the audio. In the scheme, each frame of audio corresponds to a time-frequency diagram, which reflects the spectrum structure within the frame.

[0012] The Mel spectrum is a time-frequency representation obtained by transforming the frequency axis from a linear scale to a Mel scale based on the time-frequency graph. Specifically, each column (corresponding to a frequency point) in the time-frequency graph is filtered through a Mel filter bank, and the energy value output by the filter forms a column of the Mel spectrum. Due to the nonlinear arrangement of the Mel filter bank on the frequency axis, the Mel spectrum is more uniform in perception than the original time-frequency graph, and can better reflect the human ear's ability to distinguish different frequencies. The Mel spectrum is often used in tasks such as speech recognition and audio classification.

[0013] Wavelet graph is a time-frequency representation based on wavelet transform. Unlike Fourier transform which uses fixed sinusoidal basis functions, wavelet transform uses scalable and shiftable wavelet basis functions to decompose the signal. Through continuous wavelet transform, the wavelet coefficients of the signal at different scales (frequencies) and time points can be obtained, reflecting the local time-frequency characteristics of the signal. Compared with Fourier transform, wavelet transform has better time-frequency localization ability and can capture the instantaneous characteristics and singularity of the signal. In the scheme, each frame of audio is subjected to wavelet transform to obtain the corresponding wavelet graph, which enriches the information of multi-view representation.

[0014] The Utime network is a convolutional neural network architecture for sequence labeling tasks. It consists of two parts: an encoder and a decoder, forming a U-shaped network structure. The encoder extracts high-level features of the input sequence through convolution and pooling operations, and the decoder restores the resolution of the feature map through upsampling and jump connections, and finally generates a predicted label at each time step. The Utime network is widely used in speech recognition, named entity recognition and other fields, and can effectively model long-distance dependencies of sequence data. In the scheme, the convolutional attention-enhanced Utime network is used to model the feature sequence of breathing audio to achieve frame-level sleep apnea event detection.

[0015] Furthermore, based on the time-frequency graph, Mel spectrum graph and wavelet graph, a multi-view Figure 3 The method comprises: adjusting the sizes of the time-frequency graph, the Mel spectrum graph and the wavelet graph by using a bilinear interpolation method; arranging the adjusted time-frequency graph, the Mel spectrum graph and the wavelet graph in the order of the audio data after frame processing to obtain three three-dimensional tensors, each of which represents a view; for each view, flattening the image of each time step into a one-dimensional vector; using a Gaussian kernel function to calculate the similarity between each pair of time points in the one-dimensional vector in each view, and obtaining a similarity matrix of the corresponding view; performing weighted averaging on the similarity matrices of the three views to obtain a fused similarity matrix S; and obtaining a low-dimensional embedding vector U according to the similarity matrix S by constructing a Laplacian matrix and solving the eigenvalues ​​and eigenvectors.

[0016] Among them, in multi-view learning, view refers to different representations or descriptions of the same data object. Each view characterizes the characteristics of the data from different perspectives and provides complementary information. By fusing the information of multiple views, the intrinsic structure and properties of the data can be described more comprehensively and accurately. In the scheme, the time-frequency diagram, Mel spectrum diagram and wavelet diagram constitute the three views of sleep breathing audio data. They represent the audio signal from the perspectives of time-frequency energy distribution, perceptual spectrum, time-frequency localization, etc., reflecting the different characteristic aspects of the audio. Multi-view representation enhances the information content and diversity of the data, which helps to improve the performance of subsequent clustering and detection tasks. The low-dimensional embedding vector U is a compact data representation obtained by reducing the dimension of the similarity matrix by the spectral clustering algorithm. Specifically, the Laplace matrix L is first constructed according to the similarity matrix S, and then L is decomposed by eigenvalue to obtain a set of eigenvalues ​​and corresponding eigenvectors. The eigenvectors corresponding to the first k smallest non-zero eigenvalues ​​are selected to form a matrix U of N_t\times k, where N_t is the number of time steps and k is the dimension of the embedding space (usually much smaller than the dimension of the original data). Each row of U represents a low-dimensional embedding vector of a time step, reflecting the similarity relationship between this time step and other time steps.

[0017] Furthermore, according to the similarity matrix S, a low-dimensional embedding vector U is obtained by constructing a Laplacian matrix and solving the eigenvalues ​​and eigenvectors, including: constructing an adjacency matrix W according to the similarity matrix S; constructing a degree matrix D according to the adjacency matrix W by calculating the degree of each node; obtaining a normalized Laplacian matrix L according to the degree matrix D and the adjacency matrix W; performing eigenvalue decomposition on the normalized Laplacian matrix L, selecting the eigenvectors corresponding to the first k smallest non-zero eigenvalues, and normalizing the eigenvectors to obtain a low-dimensional embedding vector U.

[0018] Among them, the adjacency matrix is ​​a matrix representation used in graph theory to describe the graph structure. For a graph with N nodes, the adjacency matrix W is a square matrix of N\times N, in which the element W(i, j) represents the connection weight between node i and node j. If there is an edge connecting node i and node j, W(i, j) is the weight of the edge (for an unweighted graph, the weight is 1); if there is no edge connecting node i and node j, W(i, j) is 0. The adjacency matrix is ​​a symmetric matrix, that is, W(i, j) = W(j, i). The adjacency matrix fully describes the topological structure of the graph and the connection relationship between nodes. In the scheme, the fused similarity matrix S is assigned to the adjacency matrix W. Each node here corresponds to a time step, and the connection weight between nodes is the similarity between the corresponding time steps. Through the adjacency matrix W, the sleep breathing audio data is converted into an undirected weighted graph, in which the nodes represent time steps and the edges represent the similarity between time steps. The adjacency matrix W describes the structure and correlation of audio data in the time dimension, providing a basis for subsequent spectral clustering.

[0019] The degree matrix is ​​another matrix representation of a graph related to the adjacency matrix. For a graph with N nodes, the degree matrix D is an N\times N diagonal matrix, where the diagonal elements D(i, i) represent the degree of node i, that is, the number of edges connected to node i (for weighted graphs, the sum of the weights of the edges connected to node i); the non-diagonal elements D(i, j) are 0. The degree matrix describes the connection strength and importance of each node in the graph. In the scheme, the degree of each node (time step) is calculated according to the adjacency matrix W, that is, d_i=\sum_{j=1}^{N_t}W(i, j), where N_t is the number of time steps. The degrees of the nodes are formed into a diagonal matrix to obtain the degree matrix D. The degree matrix D reflects the centrality and influence of each time step in the graph structure. Nodes with large degrees occupy important positions in the graph and have more connections with other nodes; nodes with small degrees are relatively isolated and have weaker connections with other nodes. The degree matrix D is used together with the adjacency matrix W to construct the normalized Laplacian matrix to achieve spectral clustering.

[0020] Furthermore, for multi-view Figure 3 The low-dimensional embedding vector U is clustered by using a K-means clustering algorithm; for each cluster, the cluster center is used as the feature vector of the corresponding time step.

[0021] Among them, the K-means clustering algorithm is a commonly used unsupervised machine learning algorithm used to divide a data set into K clusters. The goal of the algorithm is to minimize the sum of the squares of the distances between each data point and its cluster center, thereby achieving compactness within the cluster and separability between clusters. The basic steps of the K-means algorithm are as follows: Randomly select K data points as initial cluster centers. For each data point in the data set, calculate the distance between it and each cluster center and assign it to the cluster represented by the closest cluster center. For each cluster, recalculate the cluster center, that is, solve the mean vector of all data points in the cluster. Repeat until the cluster center no longer changes significantly or the maximum number of iterations is reached.

[0022] In the scheme, the K-means algorithm is used to cluster the low-dimensional embedding vector U. Each time step corresponds to a row vector of U, which represents the coordinate of the time step in the spectral embedding space. The K-means algorithm divides these vectors into K clusters, each cluster representing a group of semantically similar time steps. The clustering process can be regarded as finding the natural structure of data distribution in the embedding space, and classifying similar time steps into one category. The clustering results can be used to extract the feature vector of each time step. Specifically, for each cluster, the mean of the vectors corresponding to all time steps in the cluster is calculated to obtain a cluster center vector. The center vector of the cluster to which each time step belongs is used as the feature vector of the time step.

[0023] Furthermore, the Gaussian kernel function is used to calculate the similarity between each pair of time points in the one-dimensional vector in each view, and the similarity matrix of the corresponding view is obtained, including: The similarity matrix expression is as follows:

[0024]

[0025]

[0026]

[0027] Among them, S X (i,j) represents the view of the time-frequency diagram; S Y (i, j) represents the view of the Mel spectrum graph; S Y (i, j) represents the view of the wavelet image; σ represents the bandwidth parameter of the Gaussian kernel, vec(T X,i ) represents the image T at the i-th time point X,i Flatten into a one-dimensional vector; vec(T Y,i ) represents the image T at the i-th time point Y,i Flatten into a one-dimensional vector; vec(T Z,i ) represents the image T at the i-th time point Z,i Flatten into a one-dimensional vector;

[0028] Furthermore, the energy spectrum of the audio data after frame processing is calculated by fast Fourier transform, and the time-frequency diagram of each frame is generated according to the energy spectrum of all frames, including:

[0029] Calculate the energy spectrum using the following formula:

[0030] X(i,k)=FFT[x i (m)]

[0031] E(i,k)=[X(i,k)] 2

[0032] Where i represents the i-th time point, k represents the k-th spectrum line in the frequency domain; E represents the two-dimensional matrix of energy spectrum, in which rows represent frequencies and columns represent times; FFT represents fast Fourier transform; x represents the audio data at the i-th time point;

[0033] Furthermore, each frame of audio signal is filtered using a Mel filter, and the energy in each filter band is superimposed to obtain a Mel spectrum diagram, including:

[0034] Energy superposition is achieved through the following formula:

[0035]

[0036] Among them, S(i,m) represents the energy after superposition; H m (k) represents the frequency response of the mth Mel filter; N represents the total number of spectral lines.

[0037] Furthermore, continuous wavelet transform is performed on each frame of audio signal to calculate wavelet coefficients, and a wavelet graph is obtained according to the wavelet coefficients, including:

[0038] Calculate the wavelet coefficients using the following formula:

[0039]

[0040] Where W(a,b) represents the wavelet coefficient; t represents time; a represents the scale factor; b represents the translation factor; w0 represents the center frequency of the mother wavelet;

[0041] Furthermore, according to the degree matrix D and the adjacency matrix W, the normalized Laplace matrix L is obtained, including:

[0042]

[0043] Where I represents the identity matrix.

[0044] Another aspect of the present application further provides a sleep apnea detection system based on multi-view acoustic spectrum clustering, which is used to execute a sleep apnea detection method based on multi-view acoustic spectrum clustering of the present application.

[0045] 3. Beneficial effects

[0046] Compared with the prior art, the advantages of this application are:

[0047] By processing the sleep breathing audio data by framing, calculating the energy spectrum to generate a time-frequency diagram, using the Mel filter to obtain the Mel spectrum diagram, and obtaining the wavelet diagram through continuous wavelet transform, the breathing audio signal can be fully characterized from three different perspectives: time domain, frequency domain, and time-frequency domain, and richer acoustic feature information can be extracted. By adjusting the size of the three spectrograms and arranging them in the order of the audio frames to obtain a three-dimensional tensor, flattening each frame image into a one-dimensional vector and calculating the similarity with a Gaussian kernel function, and constructing a similarity matrix for each view, it is possible to measure the correlation between different time steps while retaining the temporal structure information of the spectrogram, and prepare the data foundation for subsequent clustering.

[0048] The fusion matrix S is obtained by weighted averaging the similarity matrices of the three views, and the Laplacian matrix is ​​constructed based on S to solve the eigenvalues ​​and eigenvectors to obtain the low-dimensional embedding vector U, thus realizing the fusion and dimensionality reduction of multi-view acoustic spectrum features. By fusing the similarity information of different views, the intrinsic structure and data distribution of breathing audio in different feature domains can be comprehensively captured. The low-dimensional embedding vector after dimensionality reduction can better highlight the distinction between different categories of breathing segments.

[0049] The K-means algorithm is used to cluster the low-dimensional embedding vector U, and the cluster center is used as the feature vector of the corresponding time step. The feature vectors of all time steps are combined to obtain a two-dimensional matrix. The inherent structure and category information in the breathing audio can be adaptively discovered during the clustering process. There is no need to annotate the breathing segments in advance, which has the advantage of unsupervised learning. The feature matrix obtained by clustering contains the category information of each time step, which can be used as a compact representation of the breathing segment for subsequent classification.

[0050] Inputting the clustered feature matrix into the pre-trained convolutional attention Utime network for apnea event detection can fully utilize the feature extraction and classification capabilities of the deep learning model and automatically learn high-level features with strong discriminative power. At the same time, the convolutional attention module can focus on the key areas in the input feature matrix that have a greater impact on the classification results, thereby improving the feature representation capabilities. The Utime network adopts a U-Net-like structure and introduces jump connections in the encoding and decoding process, which can not only realize the hierarchical extraction of features, but also integrate information of different scales, thereby enhancing the network's feature representation and classification performance. BRIEF DESCRIPTION OF THE DRAWINGS

[0051] Figure 1 This is a technical flow chart of a sleep apnea detection method based on multi-view acoustic spectrum clustering in this application;

[0052] Figure 2 This is a schematic diagram of the convolutional attention Utime network of this application;

[0053] Figure 3 This is a sleep apnea detection result diagram of the present application;

[0054] Figure 4 This is the AHI estimation result diagram for this application. DETAILED DESCRIPTION

[0055] The present application is described in detail below in conjunction with the accompanying drawings and specific embodiments.

[0056] Figure 1 This is a technical flow chart of a sleep apnea detection method based on multi-view acoustic spectrum clustering proposed in this application. The real-time sleep apnea detection method based on multi-view acoustic spectrum clustering proposed in this application mainly includes three steps: the first step is multi-view acoustic spectrum clustering. First, the whole night sleep audio signal is collected through a microphone, and it is pre-processed and framed. Then, the time-frequency diagram, Mel spectrum diagram and wavelet Figure 3 Each frame of audio is characterized by three perspectives. Specifically, the time-frequency graph is obtained by short-time Fourier transform (STFT), which shows the energy distribution of the signal in the time-frequency plane; the Mel spectrum graph is obtained by mapping the spectrum to the Mel scale and filtering it, which simulates the auditory characteristics of the human ear; the wavelet graph is obtained by continuous wavelet transform (CWT), which provides the time-frequency representation of the signal at different scales. Then, the feature representations of the three spectrograms are constructed into a three-dimensional tensor, and the similarity between different time steps in the tensor is calculated by the kernel function to obtain the similarity matrix of each view. The similarity matrices of the three views are then weighted averaged and fused to obtain a comprehensive similarity matrix. Finally, the Laplacian matrix is ​​constructed based on the fused similarity matrix and the eigenvalues ​​and eigenvectors are solved to obtain the low-dimensional embedding representation of the audio clips at different time steps. Through spectral clustering, the clips with similar embedding representations are clustered into one category, realizing the adaptive segmentation and representation learning of the whole night audio.

[0057] The second step is sleep apnea detection. In this step, the convolutional attention Utime network is used to detect sleep apnea events for the audio clips belonging to each cluster in step S1. The network input is a feature matrix, which is composed of the embedding vectors of different clips obtained by clustering in step S1 in chronological order. The main body of the network adopts a U-Net-like structure, which contains multiple downsampling and upsampling modules, and can extract local and global context information at different scales. At the same time, a skip connection is introduced between the encoder and the decoder to integrate features at different levels. In addition, a convolutional attention module is embedded in each downsampling and upsampling module, and the features of the key areas are adaptively strengthened by generating attention weights consistent with the size of the input feature map. The network outputs a respiratory event prediction sequence of the same length as the input, indicating whether apnea has occurred in the corresponding clip at each time step. The network adopts an end-to-end training method, takes the labeled respiratory events as the supervision signal, optimizes the model parameters through the binary cross entropy loss function, and finally realizes the detection of second-level apnea events based on audio signals.

[0058] The third step is post-processing and AHI estimation. Since the diagnosis of apnea requires counting the number of events that occur per hour, the output of step S2 needs to be post-processed. First, the adjacent apnea segments are merged and the segments with too short duration are filtered out. Then, the total duration of apnea during the whole night of sleep is counted and divided by the total sleep time to obtain the average apnea duration in hours, that is, the AHI index. Finally, according to the AHI index The severity of SA of the subjects was divided into four levels: normal, mild, moderate and severe.

[0059] Specifically, the first step is to extract comprehensive acoustic features of the whole night audio signal from the time-frequency diagram, Mel spectrum diagram and wavelet diagram and three perspectives using multi-view acoustic spectrum clustering. First, the whole night audio signal is divided into multiple segments according to the length of 10 minutes, and each segment can be represented as x(n). Then each segment is framed and windowed to obtain the frame signal x_i(n), where i represents the frame number. The frame length is set to 25ms, and the corresponding number of sampling points is N=Fs0.025=200 points; the frame shift is set to 10ms, and the corresponding number of sampling points is M=Fs0.01=80 points. Assuming that the total length of each 10-minute segment is L sampling points, it can be divided into (LN) / M+1 frames, and each segment is about 60,000 frames.

[0060] Then perform Fast Fourier Transform (FFT) on each frame of signal and calculate the energy spectrum: X(i,k) = FFT[x i (m)]

[0061] E(i,k)=[X(i,k)] 2

[0062] Among them, i represents the i-th time point, k represents the k-th spectrum line in the frequency domain; E represents the two-dimensional matrix of energy spectrum, in which rows represent frequency and columns represent time; FFT represents fast Fourier transform; x represents the audio data at the i-th time point, so as to draw the time-frequency diagram of each frame.

[0063] In the frequency range [0, 0.5Fs], K triangular filters are uniformly generated according to the Mel frequency scale, and the frequency response of the kth filter is recorded as H k (f), k = 1, 2, ..., K. For the FFT result X(i, k) of the i-th frame signal, multiply it with the frequency response of each filter and sum them to obtain the energy S(i, m) of the k-th Mel band of the i-th frame:

[0064]

[0065] Among them, H m (k) is the frequency response of the mth Mel filter (there are M=13 in total). Take the logarithm of S(i,m) and normalize it to get the Mel spectrum.

[0066] For each frame signal x i (m) Apply continuous wavelet transform (CWT) to calculate wavelet coefficients by adjusting scale parameter a and translation parameter b. The wavelet coefficient W(a,b) is calculated as follows:

[0067]

[0068] Among them, ψ represents the complex conjugate of the wavelet basis function, and Morlet wavelet is selected as the wavelet basis function, and its expression is:

[0069]

[0070] The calculation formula of wavelet coefficient W(a,b) can be further derived as follows:

[0071]

[0072] After normalizing the wavelet coefficients, a wavelet graph is drawn. The color intensity of the image represents the magnitude of the wavelet coefficients, showing the variation pattern of the signal in time and frequency.

[0073] Adjust the size of the time-frequency graph, Mel spectrum graph, and wavelet graph generated for each frame and arrange them in the original frame order to obtain three three-dimensional tensors T X 、T Y 、T Z ∈R 60000×32×32For each view, the image at each time step is flattened into a one-dimensional vector, and a Gaussian kernel function is used to calculate the similarity matrix s between each pair of time points X 、s Y and Z :

[0074]

[0075]

[0076]

[0077] Where σ is the bandwidth parameter of the Gaussian kernel, vec(T X,i ) represents the image T at the i-th time point X,i Flatten into a 1D vector.

[0078] The three similarity matrices are added and averaged to form a comprehensive similarity matrix S, which is assigned to the adjacency matrix W. The degree of each node is calculated to construct the degree matrix D. Then calculate the normalized Laplacian matrix Solve the eigenvalues ​​and eigenvectors of the Laplacian matrix L, select the eigenvectors corresponding to the first k = 2048 smallest non-zero eigenvalues, and standardize them to form a new low-dimensional embedding U.

[0079] K-means algorithm is used for clustering, and the number of clusters is set to 128. For each cluster, its center value is the 1-dimensional feature vector at that time point, and the combination can obtain the 128-dimensional feature vector of each time step.

[0080] Figure 2This is a schematic diagram of the convolutional attention Utime network of the present application, and sleep apnea detection based on the convolutional attention Utime network (CBAM-Utime). The network is mainly composed of an encoding module, a decoding module, a convolutional attention module, and a jump connection unit. The specific implementation steps include: using the encoding module to extract the deep features of the input sequence layer by layer. The encoding module consists of six layers of convolutional blocks, and the convolutional blocks contain convolutional layers, batch normalization layers, maximum pooling layers, and activation functions. The convolution kernel size of the convolutional block is 5; the number of filters is 32, 48, 64, 96, 128, and 256 respectively; the corresponding pooling sizes are 8, 6, 4, 2, 2, and 1 respectively. The decoding module is used for upsampling layer by layer. The decoding module consists of five layers of upsampling blocks, and the upsampling blocks contain convolutional layers, batch normalization layers, upsampling layers, and activation functions. The upsampling block shares the same convolution kernel size and number of filters with the corresponding convolutional block, and uses the pooling size of the convolutional block as the upsampling factor. The number of feature channels in the decoding module decreases layer by layer, and finally has the same data dimension as the original input sequence. The convolutional attention module is used to calculate feature weights from the two dimensions of space and channel to improve the network's ability to represent input features. This module contains a channel attention submodule and a spatial attention submodule, and the input feature sequence passes through the two modules in turn. Assume that the input feature sequence is F∈R C×1×L , the channel weight distribution after the channel attention submodule is W c ∈R C×1×1 , the spatial weight distribution after the spatial attention submodule is W s ∈R C×1×L , the process through these two submodules can be expressed as: Among them, C is the number of channels of the feature sequence, L is the length of the feature sequence, is an element-by-element multiplication operation, and F' is the final output of the convolutional attention module.

[0081] Skip Connection is used to fuse the feature maps of the corresponding layers in the encoding module and the decoding module to comprehensively utilize the feature information of the shallow and deep layers. Specifically, the output of the i-th convolution block in the encoding module is concatenated with the output of the (6-i)-th upsampling block in the decoding module according to the channel dimension. Taking the 1st convolution block and the 5th upsampling block as examples, their output feature map sizes are: 1st convolution block: 32×(N t / 8), the fifth upsampling block: 32×(N t / 8), concatenate these two feature maps in the channel dimension to obtain a 64×(N t / 8) as the output of the skip connection. Similarly, feature maps of other layers are concatenated in this way. The output of the skip connection is then passed through a convolutional layer to adjust the number of channels to match the number of input channels of the next layer of the decoding module.

[0082] The last layer output of the decoding module (the same size as the original input sequence) is processed through two layers to obtain the final classification result: Fully connected layer (Linear): The feature map is transformed from 3K×N t Transformed to 2×N t , where 2 represents normal breathing (0) and sleep apnea (1). Convolutional layer (Conv2d): The convolution kernel size is 1x1, and the number of input and output channels is 2, which is equivalent to performing classification prediction frame by frame. Softmax activation function: Normalize the two logit values ​​of each frame to obtain the probabilities of normal breathing and sleep apnea. The one with the larger probability is taken as the predicted label of the current frame, and finally a length of N is obtained. t The sequence represents the breathing state every second, realizing sleep apnea event detection at the second level.

[0083] The obtained second-level sleep apnea detection results are post-processed to estimate the subject's AHI. The specific implementation steps are as follows: normal breathing is labeled as 0, sleep apnea is labeled as 1, and a single 0 in a continuous 1 segment is recorded as an abnormal value and corrected to 1; a continuous 1 segment is detected, and a duration of more than 10 seconds is recorded as a sleep apnea event; the number of sleep apneas in the subject throughout the night is divided by the number of sleep hours to obtain the subject's AHI. Figure 3 This is a sleep apnea detection result diagram of this application, Figure 4 The figure is the AHI estimation result of this application. From the figure, it can be seen that the method proposed in this application has a high consistency with PSG. Specifically, the mean absolute error (MAE) and root mean square error (RMSE) are both low, indicating that the deviation between the estimated value and the actual value is small. The correlation coefficient is as high as 0.954, indicating that there is a strong linear relationship between the estimated value and the actual value. The intrinsic inter-class covariance (ICC) and the determination coefficient R 2 The values ​​were 0.929 and 0.85, respectively, which further demonstrated the stability and reliability of this method among different individuals.

[0084] The invention of the present application and its implementation methods are described schematically above. The description is not restrictive. Without departing from the spirit or basic features of the present application, the present application can be implemented in other specific forms. What is shown in the accompanying drawings is only one of the implementation methods of the invention of the present application. The actual structure is not limited to this, and any figure mark in the claims should not limit the claims involved. Therefore, if a person of ordinary skill in the art is inspired by it, without departing from the purpose of the present invention, a structural method and an embodiment similar to the technical solution are designed without creativity, which should all belong to the scope of protection of the present application. In addition, the word "including" does not exclude other elements or steps, and the word "one" before the element does not exclude the inclusion of "multiple" elements. The multiple elements stated in the product claim can also be implemented by one element through software or hardware. The words first, second, etc. are used to indicate names, and do not indicate any specific order.

Claims

1. A sleep apnea detection method based on multi-view acoustic spectrum clustering, characterized in that: include: Collect the user's sleeping breathing audio data, and perform frame processing on the collected audio data to obtain each frame of audio data; For all the frame audio data after the frame processing, the energy spectrum of all frames is calculated by fast Fourier transform, and the time-frequency diagram of each frame is generated according to the energy spectrum of all frames; Each frame of audio data is filtered using a Mel filter, and the energy in each filter band is superimposed to obtain a Mel spectrum. Perform continuous wavelet transform on each frame of audio data, calculate the wavelet coefficients, and obtain the wavelet graph according to the wavelet coefficients; Construct a multi-view three-dimensional tensor based on the time-frequency graph, Mel spectrum graph and wavelet graph; Cluster the multi-view 3D tensors to obtain the feature vector for each time step; Taking the feature vector as input, the pre-trained convolutional attention Utime network is used to perform apnea detection on the user's sleep breathing audio data.

2. The sleep apnea detection method based on multi-view acoustic spectrum clustering according to claim 1, characterized in that: According to the time-frequency graph, Mel spectrum graph and wavelet graph, a multi-view three-dimensional tensor is constructed, including: Use bilinear interpolation to adjust the size of the time-frequency diagram, Mel spectrum diagram and wavelet diagram; Arrange the adjusted time-frequency graph, Mel spectrum graph and wavelet graph in the order of the audio data after frame processing to obtain three three-dimensional tensors, each of which represents a view; For each view, flatten the image at each time step into a one-dimensional vector; The Gaussian kernel function is used to calculate the similarity between each pair of time points in the one-dimensional vector in each view, and the similarity matrix of the corresponding view is obtained; Take the weighted average of the similarity matrices of the three views to obtain the fused similarity matrix S; According to the similarity matrix S, the low-dimensional embedding vector U is obtained by constructing the Laplacian matrix and solving the eigenvalues ​​and eigenvectors.

3. The sleep apnea detection method based on multi-view acoustic spectrum clustering according to claim 2, characterized in that: According to the similarity matrix S, by constructing the Laplacian matrix and solving the eigenvalues ​​and eigenvectors, a low-dimensional embedding vector U is obtained, including: According to the similarity matrix S, construct the adjacency matrix W; According to the adjacency matrix W, the degree matrix D is constructed by calculating the degree of each node; According to the degree matrix D and the adjacency matrix W, the normalized Laplace matrix L is obtained; Perform eigenvalue decomposition on the normalized Laplace matrix L, select the eigenvectors corresponding to the first k smallest non-zero eigenvalues, and standardize the eigenvectors to obtain the low-dimensional embedding vector U.

4. The sleep apnea detection method based on multi-view acoustic spectrum clustering according to claim 3, characterized in that: Cluster the multi-view 3D tensor to obtain the feature vector for each time step, including: The K-means clustering algorithm is used to cluster the low-dimensional embedding vector U; For each cluster, the cluster center is used as the feature vector of the corresponding time step.

5. The sleep apnea detection method based on multi-view acoustic spectrum clustering according to claim 4, characterized in that: The Gaussian kernel function is used to calculate the similarity between each pair of time points in the one-dimensional vector in each view, and the similarity matrix of the corresponding view is obtained, including: The similarity matrix expression is as follows: Among them, S X (i,j) represents the view of the time-frequency diagram; S Y (i, j) represents the view of the Mel spectrum graph; S Y (i, j) represents the view of the wavelet image; σ represents the bandwidth parameter of the Gaussian kernel, vec(T X,i ) represents the image T at the i-th time point X,i Flatten into a one-dimensional vector; vec(T Y,i ) represents the image T at the i-th time point Y,i Flatten into a one-dimensional vector; vec(T Z,i ) represents the image T at the i-th time point Z,i Flatten into a 1D vector.

6. The sleep apnea detection method based on multi-view acoustic spectrum clustering according to claim 5, characterized in that: The energy spectrum of the audio data after frame processing is calculated through fast Fourier transform. According to the energy spectrum of all frames, the time-frequency diagram of each frame is generated, including: Calculate the energy spectrum using the following formula: X(i,k)=FFT[x i (m)] E(i,k)=[X(i,k)] 2 Among them, i represents the i-th time point, k represents the k-th spectrum line in the frequency domain; E represents the two-dimensional matrix of energy spectrum, in which rows represent frequencies and columns represent time; FFT represents fast Fourier transform; x represents the audio data at the i-th time point.

7. The sleep apnea detection method based on multi-view acoustic spectrum clustering according to claim 6, characterized in that: Mel filters are used to filter each frame of audio signal. After the energy in each filter band is superimposed, a Mel spectrum is obtained, including: Energy superposition is achieved through the following formula: Among them, S(i,m) represents the energy after superposition; H m (k) represents the frequency response of the mth Mel filter; N represents the total number of spectral lines.

8. The sleep apnea detection method based on multi-view acoustic spectrum clustering according to claim 7, characterized in that: Perform continuous wavelet transform on each frame of audio signal, calculate the wavelet coefficients, and obtain the wavelet graph based on the wavelet coefficients, including: Calculate the wavelet coefficients using the following formula: Among them, W(a,b) represents the wavelet coefficient; t represents time; a represents the scale factor; b represents the translation factor; w0 represents the center frequency of the mother wavelet.

9. The sleep apnea detection method based on multi-view acoustic spectrum clustering according to claim 8, characterized in that: According to the degree matrix D and the adjacency matrix W, the normalized Laplace matrix L is obtained, including: Where I represents the identity matrix.

Images