Intelligent sleep apnea recognition method based on hybrid neural network

Abstract

The present application belongs to the technical field of sleep respiratory disease detection, and is a sleep apnea identification method based on a hybrid neural network. The method uses a millimeter wave radar to perceive human sleep in a surrounding bed environment, adopts spatial two-dimensional analysis to locate the chest cavity position, and solves the interference problem in radar sign extraction of different targets at the same distance. Shannon entropy of the respiratory signal is constructed as an iteration index of the whale optimization algorithm, and the minimum index is used to realize iteration optimization of the variational mode extraction parameter, so that different breathing states are matched with different parameters to accurately extract the main frequency narrowband respiratory signal. The respiratory time-frequency energy graph obtained through data processing is used as input, and a CNN network and a bidirectional long short-term memory module are used to jointly construct a feature extraction module, so that the application can identify the frequency change trend over time, capture more comprehensive features, and accurately identify target sleep apnea. In addition, the hybrid network can accurately classify unmanned scenes, apnea, low ventilation and normal breathing.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of non-contact radar sleep monitoring, in particular to a sleep apnea intelligent monitoring method for respiratory disorder patients at home based on a frequency-modulated continuous wave radar. BACKGROUND

[0002] High-quality sleep is essential for health and happiness. However, many diseases in sleep are fatal, such as sudden cardiac arrest, sleep apnea, asthma. A recent study also shows that respiratory disorders are an important cause of sleep infant sudden death syndrome (SIDS). Obstructive sleep apnea (OSA) is the most common type of sleep disorder, which is composed of multiple episodes of partial or complete respiratory obstruction (hypopnea, apnea) during sleep and undiagnosed problems. In many cases, patients with sleep apnea disease only have symptoms for a short time or in occasional situations, and long-term hospitalization and observation is undoubtedly unrealistic for such patients. Therefore, continuous and cost-effective apnea monitoring in the home sleep environment is crucial.

[0003] Traditional solutions mostly use polysomnography, electrocardiogram, etc. to make judgments and recognitions. However, the acquisition of such signals requires a large number of electrodes to be pasted on the measured person, which is not suitable for a home environment. Solutions based on pressure or acceleration sensors require contact with the human body, while lighting conditions limit solutions based on computer vision. Because there is no reliable portable home sleep monitoring system, Doppler radar systems, as a home sleep monitoring system, are increasingly attracting attention due to their non-contact and unobtrusive monitoring form.

[0004] The respiratory signal is acquired by using a millimeter wave radar for sleep apnea recognition. The advantage is that the user privacy is not disclosed, and the target position can be accurately positioned by using the super high position resolution, and the target position can be easily detected and the target information can be captured. In the radar target phase signal acquisition and processing, the distance FFT in the fast time axis is mostly used to directly select the target distance range in the distance domain information, and the phase information of the distance range is processed subsequently. The method of positioning the target only from the distance not only extracts more environmental clutter in the signal, but also causes the position of the human chest vibration to be not accurately positioned. When there are multiple human chest targets in the same distance gate, the radar signals of different targets are mixed, which affects the detection of the respiratory pause of a single target. In the aspect of the input of the feature signal of the recognition and detection, compared with the heartbeat and the heart rate variability (HRV), the respiratory signal is rarely used due to its large individual difference and state fluctuation. The inaccurate extraction of the respiratory signal can easily cause the recognition error of the respiratory pause of a part of the special respiratory mode group, thereby causing the low accuracy of the sleep apnea detection. At the same time, in the process of the sleep apnea detection, the features such as the respiratory depth and the respiratory area ratio are extracted to distinguish different sleep disorder events, such as the respiratory pause, the hypopnea, and the normal respiration. There is a limitation of single feature, which causes the detection accuracy to be unable to be further improved. For the above reasons, an effective technical solution is proposed. SUMMARY

[0005] (I) Technical problems solved

[0006] In view of the above problems, the purpose of the present application is to provide a sleep apnea patient intelligent monitoring method based on a frequency-modulated continuous wave radar, which can accurately monitor the sleep apnea condition of a patient without touching the skin surface of the human body and ensuring the sleep comfort. The radar sensor adopts a two-transmitting and four-receiving mode for collection, and locates the position of the chest vibration target in the range-azimuth angle plane, solving the problem of interference in extracting radar signs of different targets at the same distance. The WOA-VME method is used to directly extract the breathing signal, so that the main frequency narrowband breathing signal can be accurately extracted even if the breathing frequency state of the human body changes frequently. The short-time Fourier transform and sliding window method are combined to process and segment the signal, and the time-frequency energy graph features of the target radar signal are extracted, so that the network model can identify the trend of the frequency change over time and capture more comprehensive features. The hybrid neural network model is constructed by adding the Bi-LSTM module to the CNN, and the four-classification of the unmanned scene, apnea, low ventilation and normal breathing is performed, so that the target sleep apnea-low ventilation detection is more accurate. The system has the advantages of low cost, simple layout, high reliability and the like, and can be widely applied to various indoor sleep environments.

[0007] (Two) Technical solutions

[0008] In order to achieve the above purpose, the present application specifically adopts the following technical solutions:

[0009] A sleep apnea intelligent identification method based on a frequency-modulated continuous wave radar, comprising the following steps,

[0010] S1, a chest vibration target positioning method using spatial two-dimensional analysis is used to extract radar sleep sign signals, the vibration of the chest part of the human body in the bed is captured to determine the target area of the extracted signal and extract the phase information of the area, and then the sleep sign signal is obtained through signal processing;

[0011] S2, the WOA is added to the VME method to adaptively optimize the experience setting parameters, the narrowband single mode extraction is realized with the highest frequency component of the breathing signal as the center frequency, and the sleep breathing signal is directly and accurately extracted from the sleep sign signal;

[0012] S3, the sliding window method and the STFT are used to obtain the sleep breathing segment data to be input into the classification neural network;

[0013] S4, introduce Bi-LSTM module in the convolutional neural network (CNN) model based on the respiratory signal time-frequency energy spectrum as input to optimize it to capture more comprehensive time sequence features before and after, and improve the apnea detection accuracy of the model under different sleep postures and different sleep stages;

[0014] Further, the S1 specific steps are as follows:

[0015] S11, use a frequency-modulated continuous wave radar with a working frequency of 60GHz, and the measured person freely and comfortably lies on the test bed. The millimeter wave radar sensor is placed on the wall or the ceiling as high as possible to better detect the body signs of the person in the bed. The frequency-modulated continuous wave radar sensor transmits millimeter wave electromagnetic waves for detection. After the radar waves are reflected by the body movement and the chest, the device collects the echo signals to obtain the radar raw data;

[0016] S12, extract the millimeter wave radar raw echo information and process the original echo information in the distance-angle domain.

[0017] The radar frequency modulation bandwidth is 4GHz, two transmitting antennas and four receiving antennas are used, the time division multiplexing multi-transmission and multi-reception mode is used to improve the angle resolution, the angle FFT is performed on the basis of target distance estimation, and the angle estimation of multiple targets is further realized. The frequency-modulated continuous wave radar transmits electromagnetic waves in a time division multiplexing (TDM) manner on two transmitting (TX) antennas; and receives through four receiving (RX) channels, effectively forming eight pairs of virtual arrays, each virtual channel corresponding to a combination of signals received by an equivalent antenna. Thus, a MIMO configuration is applied to achieve better azimuth resolution; distance FFT is performed on each of these transmission-reception link (TX-RX) pairs to obtain radar echo distance dimension data, which can separate objects occupying different distance gates from each other. For each distance gate of the distance dimension data, a second FFT is performed along the antenna pair (TX-RX) dimension to obtain a distance-azimuth angle plane, so that objects falling within the same distance range but having different angles can be separated;

[0018] S13, obtain the signal matrix of the distance-azimuth angle plane from the radar system , wherein represents the distance unit, represents the azimuth angle unit, represents the time. For the signal at each distance and azimuth angle position, the phase at multiple time points is extracted;

[0019] S14, calculate the phase mean value in the time dimension , and perform vector mean cancellation calculation on the original signal through the following formula .

[0020]

[0021] After phase cancellation, the signal matrix Dynamically changing targets (thoracic vibration signals caused by breathing and heartbeat) are preserved, while stable background noise is suppressed;

[0022] S15. The cell average constant false alarm rate (CA-CFAR) algorithm is used on the range-azimuth plane to determine the region of the vibrating target, so as to filter out the reflection of static objects and further extract the phase information of the target region.

[0023] S16. Demodulate the I / Q orthogonal two-channel complex data acquired by the radar at the target location using the extended DAM algorithm. The DAM algorithm is extended by adding an integration process to suppress high-frequency noise introduced by the differentiator in the DAM algorithm. The extended DAM algorithm uses the following discrete form expression. :

[0024]

[0025] in Indicates at a point in time The phase value obtained from demodulation. This represents the index of the sampling point in the time series. Indicates the first The in-phase components of each sampling point Indicates the first The in-phase component at each sampling point is the real part of the radar received signal. Indicates the first Orthogonal components of each sampling point Indicates the first The orthogonal components of each sampling point are the imaginary part of the radar received signal. Indicates the first The signal power at each sampling point is used for normalization to avoid the impact of amplitude variations on phase demodulation.

[0026] The extended DAM demodulation method was used to recover the thoracic vibration phase of the detected target area, thus obtaining the human sleep vital signs signal.

[0027] Furthermore, the specific steps of S2 are as follows:

[0028] S21. Directly extract the desired modality of breathing signals from sleep signs signals. and residual signal Separation is performed to ensure that the respiratory signal converges at the center frequency and that the residual signal energy is minimized in the frequency band of the respiratory signal. The desired modal respiratory signal is extracted using the following minimization standard formula.

[0029]

[0030]

[0031] wherein is the center angular frequency of the desired modal respiratory signal, is a balancing factor, is time, is the Dirac distribution function, is the complex unit, is the impulse response function of the filter;

[0032] S22, the following augmented Lagrangian function is established Solve the variational modal extraction minimization formula

[0033]

[0034] wherein is the sleep respiratory signal to be solved, is the center angular frequency of the desired modal respiratory signal, is the residual signal, is the Lagrange multiplier to be iteratively updated, which is essentially a function for enforcing the constraint condition, is the frequency, , , , and is the frequency domain signal after Fourier transform, is the frequency response of the filter, that is, the Fourier transform of the Lagrange multiplier. Combining constitutes a high-pass filter in the frequency domain, filters out the negative frequency component, and retains the positive frequency component;

[0035] S23, the formula is solved by using the multiplier alternating direction method (ADMM) The parameters are updated by the ADMM formula and the formula Iteratively update the desired modal respiratory signal and its center frequency

[0036]

[0037]

[0038] wherein is the current iteration number, is the frequency domain representation of the desired modal respiratory signal after the th iteration,​ For the first The center frequency of the respiratory signal during the next iteration. Finally, the update equation for the Lagrange multipliers is obtained using the dual ascent method. for:

[0039]

[0040] in These are update parameters used to control the multiplier amplitude. For the first Frequency domain representation of the Lagrange multipliers after the next iteration;

[0041] S24. Extract the initial center frequency of the input parameters from the variational mode. and balance parameters The optimal parameters are obtained by substituting the position function into the population-based whale optimization algorithm. The Shannon entropy value of the desired modal respiratory signal is constructed as the fitness function, which serves as the iterative index for the whale optimization algorithm, continuously minimizing it to achieve adaptive iterative optimization. The optimal positions of the lead whale and other individuals are updated through three predation methods: spiral encirclement predation, bubble net predation, and capture-and-search predation. With iteration, the optimal initial center frequency is continuously approached. and balance parameters ;

[0042] S25. The breathing extraction method based on WOA-VME designed above can directly extract a single narrowband breathing signal component from complex radar sleep signs signals, thereby achieving accurate extraction of sleep breathing signals.

[0043] Furthermore, the specific steps of S3 are as follows:

[0044] S31. Use a fixed sliding window of 15 seconds (375 points) to capture the sleep breathing signal, with each sliding step being 1 second (25 points), to obtain one-dimensional feature data of the breathing signal;

[0045] S32. Set the sampling frequency to 25Hz, select a spectral window length of 64 sampling points, and set the overlap length to the spectral window length minus 1, meaning that 63 data points overlap in each calculation window. Apply the Kaiser window for windowing processing. The shape factor is set to 3 to balance the resolution in the time and frequency domains;

[0046] S33. Perform a Fast Fourier Transform (FFT), setting the FFT length to 100 times the window length to improve frequency resolution. The specific calculation for the Short Time Fourier Transform (STFT) can be found in the formula. Represented as:

[0047]

[0048] wherein is the original signal, is a window function, and are time and frequency indices, respectively, is the short-time Fourier transform (STFT) output in time-frequency domain.

[0049] S34, further processing the one-dimensional breathing feature data obtained in the sleep environment through a short-time Fourier transform-based time-frequency analysis method to obtain two-dimensional breathing time-frequency energy spectrum data, which contains time-varying frequency characteristics and is more conducive to the identification and processing of sleep apnea;

[0050] Further, the S4 specific steps are as follows:

[0051] S41, designing a CNN network model based on time-frequency spectrum input according to the data obtained by S34 processing, the network processing input data through three feature extraction modules, each feature extraction module consisting of a convolution layer, an activation layer (ReLU) and a pooling layer. The convolution layer is responsible for capturing local features, the activation layer introduces a nonlinear function to learn complex features, and the pooling layer is responsible for dimension reduction and compression of features. In this way, the level of features is extracted and deepened;

[0052] S42, the CNN network model design input sample size is 128*128*3, the three convolution layers have 32, 64 and 128 convolution kernels respectively, the convolution kernel size is also 3*3, the moving step is 1, and the training method adopts the commonly used stochastic gradient descent method in deep learning;

[0053] S43, adding a bidirectional long short-term memory module after the three convolution layers to optimize the CNN, thereby forming a CNN+Bi-LSTM model, which sends the one-dimensional features flattened after the CNN output into the Bi-LSTM module for secondary processing, and strengthens the model's capture of forward and backward time sequence features. Two fully connected layers with ReLU activation function are used to further integrate the high-dimensional features output by Bi-LSTM. Finally, the length of the Softmax output layer is 4, i.e. corresponding to 4 categories, respectively identifying and classifying the four situations of no one scene, apnea, low respiratory and normal breathing;

[0054] S44, in order to ensure better learning and training effect, the cross-entropy loss is combined with regularization to form a hybrid loss function containing data loss and model complexity, improve the generalization ability of the model on unseen data, and reduce the risk of overfitting. The cross-entropy loss function calculation expression is as shown in the formula

[0055] ​

[0056] wherein C is the number of categories, is a one-hot encoding vector, indicating whether sample i belongs to category C. On the basis of the cross-entropy loss function, the sum of the absolute values of the model weights and the sum of the squares of the model weights are added as penalty terms to reduce the model complexity and prevent overfitting, and the hybrid loss function expression is as follows:

[0057]

[0058] wherein is a hyperparameter of the regularization strength, is the weight of the model. and is a regularization strength parameter for balancing the weights of the data loss and the regularization term in the loss function. The difference between the prediction result and the true label is measured by calculating the hybrid loss function, guiding the model parameter update, and optimizing the model performance;

[0059] (Three) beneficial effects

[0060] Compared with the prior art, the present application provides a sleep apnea intelligent recognition method based on a hybrid neural network, which has the following beneficial effects:

[0061] The present application realizes a sleep apnea detection method based on a hybrid neural network, which is a classification and recognition method using a deep learning network model, and can more accurately recognize and detect sleep breathing under different sleeping positions; the frequency-modulated continuous wave radar sensor is used for detection, which is simple and lightweight in hardware, and can accurately obtain information in weak light or even no light conditions or when the device is blocked by bedding and other obstacles; the present application realizes adaptive narrowband respiratory signal extraction under different human respiratory differences and different respiratory frequency fluctuation states, and maximally and truly obtains the sleep respiratory signal of the measured human body.

[0062] The system of the present application has high detection accuracy, and can realize real-time, long-term and non-contact monitoring, and can be widely applied in family bedrooms, hospitals and nursing homes and other places, and is easy for doctors to accurately grasp the patient's condition. BRIEF DESCRIPTION OF DRAWINGS

[0063] In order to more clearly illustrate the technical solutions in the embodiments of the present application, the following will briefly introduce the drawings needed to be used in the embodiments or prior art description. Obviously, the drawings in the following description are only some embodiments of the present application, and other drawings can be obtained by those skilled in the art without creative labor.

[0064] Figure 1 is the flow chart of the system and method of the present application;

[0065] Figure 2 A flowchart of a neural network sleep apnea intelligent recognition method based on radar respiratory signal extraction and CNN+Bi-LSTM is provided for the present application.

[0066] Figure 3 A schematic diagram of an FMCW radar modulation system is provided for the present application.

[0067] Figure 4 A schematic diagram of FMCW radar transmitting signal and echo signal is provided for the present application.

[0068] Figure 5 An algorithm flowchart for optimizing VME parameters using whale algorithm is used for the present application. DETAILED DESCRIPTION

[0069] To make the purposes, technical solutions and advantages of the embodiments of the present application clearer, the technical solutions in the embodiments of the present application will be described clearly and completely below with reference to the drawings in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, rather than all the embodiments of the present application. Based on the embodiments in the present application, all other embodiments obtained by those skilled in the art without creative labor fall within the scope of protection of the present application.

[0070] The present application will be further described below with reference to the embodiments and the drawings of the specification.

[0071] EMBODIMENT

[0072] As Figure 1 and Figure 2 described, the present application provides a sleep apnea intelligent recognition method based on a hybrid neural network, and the implementation method is as follows:

[0073] S1, a chest vibration target positioning method using spatial two-dimensional analysis is used to extract radar sleep sign signals, the vibration of the chest part of the human body in the sleep state is captured to determine the target area of the extracted signal and extract the phase information of the area, and then the sleep sign signal is obtained through signal processing;

[0074] S11, a frequency-modulated continuous wave radar with a working frequency of 60GHz is used, the measured person freely and comfortably lies on the test bed, the millimeter wave radar sensor is placed on the wall surface or the ceiling, and the sensor can better detect the body signs of the person in the surrounding bed. The frequency-modulated continuous wave radar sensor transmits millimeter wave electromagnetic waves for detection, after the radar waves are reflected by the body movement and the chest, the device collects the echo signal and obtains the radar raw data;

[0075] S12, the millimeter wave radar raw echo information is extracted, and the raw echo information is processed in the range-angle domain.

[0076] The radar uses a frequency modulation bandwidth of 4GHz, uses two transmitting antennas and four receiving antennas, and improves the angle resolution in a time-division multiplexing multi-transmitting and multi-receiving manner. Angle FFT is performed on the basis of target distance estimation, and further multi-target angle estimation is realized. The frequency modulation continuous wave radar transmits electromagnetic waves in a time-division multiplexing (TDM) manner on two transmitting (TX) antennas; and receives through four receiving (RX) channels, effectively forming eight pairs of virtual arrays, each virtual channel corresponding to a combination of signals received by an equivalent antenna. Thus, a MIMO configuration is applied to realize better azimuth angle resolution; distance FFT is performed on each of these transmitting-receiving link (TX-RX) pairs to obtain radar echo distance dimension data, which can separate objects occupying different distance gates from each other. For each distance gate of the distance dimension data, a second FFT is performed along the antenna pair (TX-RX) dimension to obtain a distance-azimuth angle plane, so that objects falling within the same distance range but having different angles can be separated;

[0077] S13, obtaining a signal matrix of the distance-azimuth angle plane from the radar system , wherein represents a distance unit, represents an azimuth angle unit, represents time. For each distance and azimuth angle position, the phase of multiple time points is extracted ;

[0078] S14, calculating a phase mean value in the time dimension , performing vector mean cancellation calculation on the original signal through the following formula

[0079]

[0080] After phase cancellation, the signal matrix in which the target (chest cavity vibration signal caused by respiration and heartbeat) dynamically changes is retained, and the stable background noise is suppressed;

[0081] S15, determining a vibration target region on the distance-azimuth angle plane by using a cell average constant false alarm rate (CA-CFAR) algorithm to filter out reflections of static objects, and further extracting phase information of the target region;

[0082] S16, performing DACM algorithm demodulation on I / Q quadrature two-channel complex data collected by the radar at the target position, extending the DACM algorithm, and increasing an integral process to suppress high-frequency noise caused by a differentiator in the DACM algorithm. The extended DACM algorithm is expressed in the following discrete form :​

[0083]

[0084] The detected thoracic vibration phase of the target region is recovered by using the extended DACM demodulation method, and a human sleep vital sign signal is obtained;

[0085] S2, a whale optimization algorithm (WOA) is added to a variational mode extraction (VME) method to adaptively optimize experience setting parameters, narrowband single mode extraction is realized with a highest frequency component of a breathing signal as a center frequency, and a sleep breathing signal is directly and accurately extracted from a sleep vital sign signal;

[0086] S21, a desired mode breathing signal and a residual signal are separated from the sleep vital sign signal, so that the breathing signal converges at the center frequency and the residual signal energy is minimized at the frequency band where the breathing signal is located. The desired mode breathing signal is extracted by minimizing the following standard formula

[0087]

[0088]

[0089] wherein is a center angular frequency of the desired mode breathing signal, is a balance factor, is time, is a Dirac distribution function, is a complex unit, is an impulse response function of a filter, is all other components except the target breathing signal after decomposition;

[0090] S22, the following augmented Lagrange function is established the variational mode extraction minimization formula is solved

[0091]

[0092] wherein is a sleep breathing signal to be solved, is a center angular frequency of the desired mode breathing signal, is a residual signal, is a Lagrange multiplier to be iteratively updated, which is essentially a function for enforcing the constraint condition, is a frequency, 、 、 、 and for the Fourier transformed frequency domain signal, i.e. the frequency response of the filter, i.e. the Fourier transform of the Lagrange multiplier. combining to form a high-pass filter in the frequency domain, filtering out the negative frequency components and retaining the positive frequency components;

[0093] S23, solving the formula using the multiplier Alternating Direction Method (ADMM) parameters, through the ADMM formula and the formula iteratively updating the expected modal respiratory signal and its center frequency

[0094]

[0095]

[0096] wherein is the current iteration number, is the frequency domain representation of the expected modal respiratory signal after the th iteration, is the frequency domain representation of the expected modal respiratory signal after the th iteration, is the center frequency of the respiratory signal at the th iteration, is the center frequency of the respiratory signal at the th iteration. Finally, the updating equation of the Lagrange multiplier is obtained by the dual ascent method is:

[0097]

[0098] wherein is an updating parameter used to control the amplitude of the multiplier, is the frequency domain representation of the Lagrange multiplier after the th iteration, is the frequency domain representation of the Lagrange multiplier after the th iteration;

[0099] S24, taking the input parameter initial center frequency and the balancing parameter as the position function into the population-based whale optimization algorithm to obtain the optimal parameters. The Shannon entropy value of the expected modal respiratory signal is constructed as the fitness function, which is constantly minimized as the iteration index of the whale optimization algorithm to realize adaptive iterative optimization. The optimal positions of the leader whale and the remaining individuals are updated through three predation modes of spiral surrounding predation, bubble net predation and post-capture search predation of the whale algorithm. With the iteration, the optimal initial center frequency is constantly approached And balance parameters ;

[0100] S25, the single narrow-band respiratory signal component is directly extracted from the complex radar sleep sign signal by using the WOA-VME-based respiratory extraction method designed above, and the sleep respiratory signal is accurately extracted;

[0101] S3, the sleep respiratory segment data to be input into the classification neural network is obtained by using the sliding window method and the short-time Fourier transform (STFT);

[0102] S31, the sleep respiratory signal is intercepted by using a fixed sliding window of 15 seconds (375 points), and the respiratory signal one-dimensional feature data is obtained by taking 1 second (25 points) as the sliding step each time;

[0103] S32, the sampling frequency is set to 25 Hz, the spectrum window length of 64 sampling points is selected, and the overlap length is the spectrum window length minus 1, that is, there are 63 data points overlapping in the window each time. Kaiser window is applied for windowing processing, and the shape factor of Kaiser window is set to 3 to balance the resolution of time domain and frequency domain;

[0104] S33, fast Fourier transform (FFT) is performed, and the FFT length is set to 100 times the window length to improve the frequency resolution. The specific calculation of the short-time Fourier transform (STFT) can be seen from the formula

[0105]

[0106] wherein is the original signal, is the window function, and are the time and frequency indexes respectively, is the time-frequency domain output result of the short-time Fourier transform (STFT).

[0107] S34, the one-dimensional respiratory feature data obtained in the sleep environment is further processed by the time-frequency analysis method based on the short-time Fourier transform, and the two-dimensional respiratory time-frequency energy spectrum data is obtained, which contains time-varying frequency characteristics and is more conducive to the identification and processing of sleep apnea;

[0108] S4, the Bi-LSTM module is introduced into the convolutional neural network (CNN) model based on the respiratory signal time-frequency energy spectrum as input for optimization, so that it can capture more comprehensive front and back time sequence characteristics, and improve the apnea detection accuracy of the model under different sleep postures and different sleep stages;

[0109] ​​S41. Based on the data obtained in S34, design a CNN network model based on time-spectrum input. The network processes the input data through three feature extraction modules, each consisting of a convolutional layer, an activation layer (ReLU), and a pooling layer. The convolutional layer is responsible for capturing local features, the activation layer introduces a non-linear function to learn complex features, and the pooling layer is responsible for dimensionality reduction and feature compression. In this way, the features are extracted and deepened at different levels.

[0110] The S42 CNN network model is designed with an input sample size of 128*128*3. The three convolutional layers have 32, 64 and 128 convolutional kernels respectively, and the kernel size is also 3*3. The stride is 1. The training method is the stochastic gradient descent method commonly used in deep learning.

[0111] S43. A bidirectional long short-term memory module is added after the three convolutional layers to optimize the CNN, thus forming a CNN+Bi-LSTM model. This model feeds the flattened one-dimensional features of the CNN output into the Bi-LSTM module for secondary processing, enhancing the model's ability to capture forward and backward time series features. Two fully connected layers with ReLU activation functions are used to further integrate the high-dimensional features output by the Bi-LSTM. Finally, the length of the Softmax output layer is 4, corresponding to 4 categories, which are used to identify and classify four situations: unmanned scenes, sleep apnea, hypoventilation, and normal breathing.

[0112] S44. To ensure better learning and training results, cross-entropy loss is combined with regularization to form a hybrid loss function that includes data loss and model complexity. This improves the model's generalization ability on unseen data and reduces the risk of overfitting. The cross-entropy loss function is calculated as follows: As shown:

[0113]

[0114] in It is the number of categories. It is a one-hot encoded vector indicating whether sample X belongs to the class. , Model for the first The sample belongs to the first The predicted probability of the class, N, and the total number of samples involved in calculating the loss are used. A penalty term is added to the cross-entropy loss function, consisting of the sum of the absolute values ​​of the model weights and the sum of the squares of the model weights. This reduces model complexity and prevents overfitting. The expression for its mixed loss function is shown below:

[0115]

[0116] in is a hyper-parameter of the regularization strength, is a weight of the model. and is a regularization strength parameter, used to balance the weight of data loss and regularization term in the loss function. The difference between the prediction result and the true label is measured by calculating the hybrid loss function, guiding the model parameter update, and optimizing the model performance;

[0117] In this embodiment, as shown in Figure 2 , the millimeter wave radar acquires data in the enclosed bed sleep environment, obtains the I / Q phase information of the target through the designed distance and angle FFT, vector mean cancellation and single unit average constant false alarm in the range-azimuth plane, and obtains the sleep sign signal through the extended DACM demodulation. The sleep sign signal is obtained by using the adaptive sleep breathing extraction method based on WOA-VME to obtain the sleep breathing signal. Further, the sleep breathing signal is segmented by sliding window, and the two-dimensional sleep breathing feature data is obtained by short-time Fourier transform. The data set is constructed by collecting the unmanned sleep scene data, normal breathing data, low ventilation data and apnea data in different sleep posture states by the radar, the data set is divided into training set, verification set and test set according to the proportion of 6:2:2, the designed CNN+Bi-LSTM classification and recognition network is trained, and the best network model is saved;

[0118] In this embodiment, as shown in Figure 4 , a frequency-modulated continuous wave radar is used, the subject freely and comfortably lies on the test bed, and the millimeter wave radar sensor is placed on the ceiling opposite to the upper body of the subject, and the radar echoes of 10 volunteers in different sleep postures are collected. In this study, the signal of the radar is:

[0119]

[0120] wherein is the amplitude of the transmitted signal of the millimeter wave radar. In addition, is the starting frequency of the signal, B is the signal bandwidth, and T is the pulse signal. The receiving antenna acquires the reflection signals of all targets in the radar detection range. Assuming that the distance between the i-th target and the radar is , we have:

[0121]

[0122] In this embodiment, as shown in Figure 5 , the algorithm flow of using the whale algorithm to optimize the VME parameter is shown, which is specifically:

[0123] When the VME decomposition is performed, the initial center frequency and the penalty parameter The best parameters of VME are searched by WOA, the randomness and complexity of the signal are reflected by Shannon entropy, and the VME decomposition parameters are optimized by minimizing the Shannon entropy as the fitness function. The essence of Shannon entropy is to describe the degree of information uncertainty. The greater the value, the less information the extracted signal contains; the smaller the value, the more information the extracted signal contains. The sparsity of the modal function, i.e. the VME decomposition effect, is reflected by the size of the Shannon entropy.

[0124] The extracted modal signal contains more noise, the periodic respiratory signal is not obvious, the randomness and complexity of the signal are large, and the Shannon entropy value is large. If the extracted modal signal component contains more respiratory signals, the randomness and complexity of the signal are small, and the Shannon entropy value is small. When the whale group is at a certain position (corresponding to a group of parameter combinations and ), the Shannon entropy value of the modal component extracted by VME under the condition of this position is calculated. In order to search for the global optimal component, i.e. to strip out the most ideal respiratory signal from the respiratory signal pool obtained from each parameter, the minimum entropy value is taken as the fitness value in the optimization process, and the minimization of the respiratory signal entropy value is taken as the final optimization target.

[0125] The whale algorithm updates the optimal position of the leader whale and the remaining individuals through three hunting methods. With the iteration, the optimal solution is constantly approached.

[0126] (1) Spiral surrounding hunting. The position of the initialized leader whale is taken as the optimal hunting position in the current group, and all whales will gradually surround the position of the leader whale to gradually approach the optimal solution. The position updating formula is:

[0127]

[0128]

[0129] wherein is the distance between the individual to be solved and the random individual in the current group; is a swing factor, wherein is a random number between 0 and 1; is the position of the leader whale; is the current iteration number; is the position of the remaining whale; is the position of the individual in the next generation group; is a convergence factor, , wherein is the maximum iteration number.

[0130] (2) Construct bubble net predation. In the process of predation, the positions of the remaining individuals and the distance between the remaining individuals and the leader whale must also be calculated. The remaining individuals will spiral around the leader whale and spit out bubbles to prey on the prey. In this process, the position update formula is:

[0131]

[0132]

[0133] wherein is a logarithmic spiral constant; is a random number in [-1, 1]; is the distance between the current whale individual and the leader whale.

[0134] The spiral surrounding predation and the construction of the bubble net predation of the whale group in the process of predation are carried out simultaneously, so the probabilities of the two predation methods are both set to 50%. The position formula of the whale is updated as

[0135]

[0136] wherein is a random number between [-1, 1].

[0137] (3) Search for predation after capture. When the whale group successfully captures food, if the convergence factor , the whale individual will separate from the leader whale and carry out individual random search predation. This random search mechanism can avoid the whale algorithm from falling into local optimum; if , the whale individual will continue to follow the leader whale to carry out spiral surrounding predation and bubble net predation. The position update formula in the search process is:

[0138]

[0139]

[0140] wherein is the random predation position of the whale individual.

[0141] Finally, it should be noted that the above only describes the preferred embodiments of the present application and is not intended to limit the present application. Although the present application has been described in detail with reference to the foregoing embodiments, those skilled in the art can still modify the technical solutions described in the foregoing embodiments or make equivalent replacements to some technical features. Any modification, equivalent replacement, improvement, etc. made within the spirit and principles of the present application shall be included in the protection scope of the present application.

Claims

1. A method for intelligent sleep apnea recognition based on a hybrid neural network, wherein the method uses frequency-modulated continuous wave radar to sense human sleep in a bedside environment, characterized in that, Includes the following steps: S1. A spatial two-dimensional analysis method for thoracic vibration target localization is used to extract radar sleep signs signals. By capturing the vibration of the thoracic cavity in the human body during sleep within the bed, the target area of ​​the extracted signal is determined and the phase information of the area is extracted. Then, the sleep signs signals are obtained through signal processing. S2. The Whale Algorithm (WOA) is added to the Variational Modality Extraction (VME) method to adaptively optimize the empirically set parameters, so as to realize narrowband single-mode extraction with the highest frequency component of the respiratory signal as the center frequency, and directly and accurately extract the sleep respiratory signal from the sleep sign signal. S3. The sleep breathing segment data to be input into the classification neural network is obtained by using the sliding window method and short-time Fourier transform (STFT); S4. Introduce a Bi-LSTM module into the convolutional neural network (CNN) model based on the time-frequency energy spectrum of respiratory signals as input to optimize it, so that it can capture more comprehensive temporal features and improve the accuracy of apnea detection under different sleep postures and different sleep stages. The specific process of the chest cavity vibration target localization method using spatial two-dimensional analysis in S1 is as follows: S11. Using a frequency modulated continuous wave radar with a working frequency of 60GHz, the subject lies freely and comfortably on the test bed. The millimeter-wave radar sensor is placed on the ceiling to better detect human vital signs around the bed. The frequency modulated continuous wave radar sensor emits millimeter-wave electromagnetic waves for detection. After the radar waves pass through body movement and chest cavity reflection, the equipment collects the echo signal and obtains the raw radar data. S12. Extract the raw echo information from the millimeter-wave radar and perform range-angle domain processing on the raw echo information. The radar employs a 4GHz frequency modulation bandwidth, using two transmit antennas and four receive antennas. Time-division multiplexing (TDM) improves angular resolution. Angle FFT is performed based on target range estimation to further achieve multi-target angle estimation. The frequency-modulated continuous wave radar transmits electromagnetic waves on two transmit (TX) antennas using TDM and receives them through four receive (RX) channels, effectively forming eight pairs of virtual arrays. Each virtual channel corresponds to a combination of signals received by an equivalent antenna, thus applying MIMO configuration to achieve better azimuth resolution. Range FFT is performed on each of these transmit-receive links (TX-RX) to obtain radar echo range dimension data. This allows objects occupying different range gates to be separated. For each range gate in the range dimension data, a second FFT is performed along the antenna pair (TX-RX) dimension to obtain the range-azimuth plane, thereby separating objects falling within the same range range but with different angles. S13. Obtain the signal matrix of the range-azimuth plane from the radar system. ,in Represents distance unit, Represents the azimuth unit. Representing time, for each distance and azimuth The signal at the location is used to extract the phase at multiple time points. ; S14. Calculate the phase mean over the time dimension. The vector mean cancellation of the original signal is calculated using the following formula: ; After phase cancellation, the signal matrix Dynamically changing targets are preserved, while stable background noise is suppressed; S15. The cell average constant false alarm rate (CA-CFAR) algorithm is used on the range-azimuth plane to determine the region of the vibrating target, so as to filter out the reflection of static objects and further extract the phase information of the target region. S16. Demodulate the I / Q orthogonal two-channel complex data acquired by the radar in the target area using the extended DAM algorithm. The extended DAM algorithm uses the following discrete form expression: ; in Indicates at a point in time The phase value obtained from demodulation. This represents the index of the sampling point in the time series. Indicates the first The in-phase components of each sampling point Indicates the first The in-phase component at each sampling point is the real part of the radar received signal. Indicates the first Orthogonal components of each sampling point Indicates the first The orthogonal components of each sampling point are the imaginary part of the radar received signal. Indicates the first The signal power of each sampling point is used for normalization to avoid the influence of amplitude changes on phase demodulation. The extended DAM demodulation method was used to recover the thoracic vibration phase of the detected target area, thus obtaining the human sleep vital signs signal.

2. The intelligent sleep apnea recognition method based on a hybrid neural network according to claim 1, characterized in that: The improved variational modal respiratory signal extraction method based on WOA in S2 is as follows: S21. From sleep signs and signals The desired modal breathing signal is directly transmitted in the middle. and residual signal Separation is performed to ensure that the respiratory signal converges at the center frequency and the residual signal energy is minimized in the frequency band where the respiratory signal is located. The desired modal respiratory signal is extracted using the following minimization standard formula. ; ; ; in The center angular frequency of the desired modal respiratory signal. As a balance factor, For time, Let be the Dirac distribution function. For complex units, Let be the impulse response function of the filter; S22. Establish the following augmented Lagrange function. Solve for the variational mode extraction minimization formula; ; in For the sleep breathing signals to be sought, The center angular frequency of the desired modal respiratory signal. For residual signals, These are Lagrange multipliers that are iteratively updated; essentially, they are functions used to enforce constraints. For frequency, , , , and The frequency domain signal after Fourier transform. That is, the frequency response of the filter. That is, the Fourier transform of the Lagrange multipliers. These components are combined to form a high-pass filter in the frequency domain, which filters out negative frequency components and retains positive frequency components. S23. Solve the problem described in S22 using the Alternating Direction Multiplier Method (ADMM). The parameters are iteratively updated using the following two formulas from ADMM to determine the desired modal respiratory signal and its center frequency; ; ; in This represents the current iteration number. For the first The frequency domain representation of the desired modal breathing signal after the next iteration. For the first The frequency domain representation of the desired modal breathing signal after the next iteration. For the first The center frequency of the respiratory signal during the next iteration No. The center frequency of the respiratory signal during the next iteration, and finally, the update equation of the Lagrange multipliers is obtained by the dual ascent method: ; in These are update parameters used to control the multiplier amplitude. For the first Frequency domain representation of the Lagrange multipliers after the next iteration For the first Frequency domain representation of the Lagrange multipliers after the next iteration; S24. Extract the initial center frequency of the input parameters from the variational mode. and balance parameters The optimal parameters are obtained by substituting the position function into the population-based whale optimization algorithm. The Shannon entropy value of the expected modal respiratory signal is constructed as the fitness function. This fitness function serves as the iterative index of the whale optimization algorithm, continuously minimizing it to achieve adaptive iterative optimization. The optimal positions of the leader whale and other individuals are updated through three predation methods of the whale algorithm: spiral encirclement predation, bubble net predation, and predation after capture. As the iteration progresses, the optimal initial center frequency is continuously approached. and balance parameters ; S25. The breathing extraction method based on WOA-VME designed above can directly extract a single narrowband breathing signal component from complex radar sleep signs signals, thereby achieving accurate extraction of sleep breathing signals.

3. The intelligent sleep apnea recognition method based on a hybrid neural network according to claim 1, characterized in that: The method for obtaining sleep breathing characteristic data in S3 is as follows: S31. Use a fixed sliding window of 15 seconds to capture the sleep breathing signal, with each sliding step being 1 second, to obtain one-dimensional feature data of the breathing signal; S32. Set the sampling frequency to 25Hz, select a spectral window length of 64 sampling points, and set the overlap length to the spectral window length minus 1, meaning that 63 data points overlap in each calculation. Apply a Kaiser window for windowing processing. The shape factor is set to 3 to balance the resolution in the time and frequency domains; S33. Perform a Fast Fourier Transform (FFT), setting the FFT length to 100 times the window length to improve frequency resolution. The Short Time Fourier Transform (STFT) is calculated as follows: ; in The original signal, For window functions, and They are time and frequency indices, respectively. It is the time-frequency domain output of the Short Time Fourier Transform (STFT); S34. The one-dimensional respiratory feature data obtained in the sleep environment is further processed by the time-frequency analysis method based on short-time Fourier transform to obtain two-dimensional respiratory time-frequency energy spectrum data. This data contains time-varying frequency features, which is more conducive to the identification and processing of sleep apnea.

4. The intelligent sleep apnea recognition method based on a hybrid neural network according to claim 1, characterized in that: The sleep breathing state classification method based on hybrid neural networks in S4 is as follows: S41. Based on the data obtained from S34, design a CNN network model based on time-frequency input. The network processes the input data through three feature extraction modules. Each feature extraction module consists of a convolutional layer, an activation layer (ReLU), and a pooling layer. The convolutional layer is responsible for capturing local features, the activation layer introduces a nonlinear function to learn complex features, and the pooling layer is responsible for dimensionality reduction and feature compression. In this way, the features are extracted and deepened. The S42 CNN network model is designed with an input sample size of 128*128*3. The three convolutional layers have 32, 64 and 128 convolutional kernels respectively, and the kernel size is also 3*3. The stride is 1. The training method is the stochastic gradient descent method commonly used in deep learning. S43. After the three convolutional layers, a bidirectional long short-term memory module is added to optimize the CNN, thus forming a CNN+Bi-LSTM model. This model feeds the flattened one-dimensional features of the CNN output into the Bi-LSTM module for secondary processing, which enhances the model's capture of forward and backward time series features. Two fully connected layers with ReLU activation function are used to further integrate the high-dimensional features output by Bi-LSTM. Finally, the length of the Softmax output layer is 4, which corresponds to 4 categories, and identifies and classifies four situations: no-man's-land, sleep apnea, hypoventilation, and normal breathing. S44. To ensure better learning and training results, cross-entropy loss is combined with regularization to form a hybrid loss function that includes data loss and model complexity. This improves the model's generalization ability on unseen data and reduces the risk of overfitting. The cross-entropy loss function is calculated as follows: ; in It is the number of categories. It is a one-hot encoded vector indicating whether sample X belongs to the class. , Model for the first The sample belongs to the first The predicted probability of the class, N, and the total number of samples involved in calculating the loss are used. A penalty term is added to the cross-entropy loss function, consisting of the sum of the absolute values ​​of the model weights and the sum of the squares of the model weights, to reduce model complexity and prevent overfitting. The expression for its mixed loss function is as follows: ; in It is a hyperparameter of regularization strength. These are the model weights. and It is a regularization strength parameter used to balance the weights of data loss and regularization terms in the loss function. By calculating this mixed loss function, the difference between the predicted results and the true labels is measured, which guides the updating of model parameters and optimizes model performance.

Images