Neural network model training method, oral-nasal airflow signal reconstruction method and device

Abstract

The application provides a neural network model training method, an oral-nasal airflow signal reconstruction method and equipment. The method is applied to the field of radar signal processing. The method comprises the following steps: acquiring multiple chest and abdominal movement signals collected by a millimeter wave radar on a target object; processing the multiple chest and abdominal movement signals by using a neural network model to obtain a reconstructed oral-nasal airflow signal and a prediction result of apnea; calculating a regression error according to the reconstructed oral-nasal airflow signal and a real oral-nasal airflow signal, and calculating an apnea prediction error according to the prediction result of apnea and an apnea label; performing weighted calculation according to the regression error and the apnea prediction error to obtain an integrated loss; and performing back propagation and gradient updating on the neural network model according to the integrated loss, and optimizing model parameters. The oral-nasal airflow signal reconstructed by the neural network model is more fine, comprehensive and accurate.

Description

Technical Field

[0001] This invention relates to the field of radar signal processing, specifically to a neural network model training method, a method and device for reconstructing nasal and oral airflow signals. Background Technology

[0002] Oral and nasal airflow signals are among the most important indicators in human sleep monitoring. Accurately capturing these signals is of high clinical value for diagnosing sleep apnea and other respiratory diseases. Traditional measurement methods use respiratory flow temperature or pressure sensors, which require patients to wear nasal cannulas. This contact-based measurement process causes significant discomfort to the monitored individual, making it difficult to apply in long-term health management scenarios.

[0003] Given the limitations of traditional measurement methods, millimeter-wave radar has emerged as a new, user-friendly monitoring technology. It can acquire information about a subject's breathing status without physical contact. By analyzing the range, azimuth, elevation angle, and Doppler information in the echo signal, it can detect micro-motion signals at specific parts of the human body, and then successfully separate respiratory information from these signals.

[0004] However, current millimeter-wave radar technology reflects the respiratory movement of a monitored object by extracting chest and abdominal motion signals. However, the extracted chest and abdominal motion signals are significantly distorted compared to the airflow signals from the mouth and nose, and the details are severely lacking, making it difficult to meet the needs of refined assessment. Summary of the Invention

[0005] In view of this, the present invention provides a method for training a neural network model, comprising:

[0006] Acquire multiple chest and abdominal motion signals collected by millimeter-wave radar from the target object;

[0007] The multiple chest and abdominal motion signals are processed using a neural network model to obtain reconstructed oral and nasal airflow signals and apnea prediction results. The neural network model performs feature fusion on the multiple chest and abdominal motion signals to obtain a fused feature signal. The fused feature signal is then subjected to temporal feature capture to obtain a target temporal signal. The target temporal signal is then subjected to feature fitting and linear mapping to obtain reconstructed oral and nasal airflow signals and apnea prediction results.

[0008] The regression error is calculated based on the reconstructed nasal and oral airflow signals and the actual nasal and oral airflow signals; the apnea prediction error is calculated based on the apnea prediction results and the apnea labels.

[0009] The integrated loss is obtained by weighting the regression error and the apnea prediction error.

[0010] The neural network model is backpropagated and its gradients are updated based on the integration loss, and the model parameters are optimized.

[0011] Optionally, the plurality of chest and abdominal motion signals include a plurality of distance gate breathing signals of chest and abdominal position movements of the target object, wherein each distance gate breathing signal is a signal sequence of a preset duration collected in the corresponding distance gate spatial region at a preset sampling rate.

[0012] Optionally, the step of calculating the regression error based on the reconstructed nasal airflow signal and the actual nasal airflow signal includes:

[0013] Acquire real nasal and oral airflow signals from the target subject using polysomnography;

[0014] The regression error is obtained by calculating the mean absolute error based on the reconstructed nasal and oral airflow signals and the actual nasal and oral airflow signals.

[0015] Optionally, the step of performing feature fitting and linear mapping on the target time-series signal to obtain the reconstructed oral and nasal airflow signal includes:

[0016] The target time series signal is subjected to dimensionality reduction projection to obtain a time series signal of a first preset dimension;

[0017] The reconstructed nasal airflow signal is obtained by mapping the time-series signal of the first preset dimension.

[0018] Optionally, the step of performing feature fitting and linear mapping on the target time-series signal to obtain the apnea prediction result includes:

[0019] The target time series signal is subjected to dimensionality reduction projection to obtain a time series signal with a second preset dimension;

[0020] The time-series signal of the second preset dimension is mapped to obtain the apnea prediction result.

[0021] Optionally, the neural network model includes a convolutional network, a bidirectional long short-term memory neural network, and an analysis model. The convolutional network is used to fuse the features of the multiple chest and abdominal motion signals to obtain a fused feature signal. The bidirectional long short-term memory neural network is used to capture the temporal features of the fused feature signal to obtain a target temporal signal. The analysis model is used to perform feature fitting and linear mapping on the target temporal signal to obtain a reconstructed oral and nasal airflow signal and a sleep apnea prediction result.

[0022] Alternatively, the integration loss can be calculated using the following method:

[0023] ,

[0024] in, Indicates integration loss, Indicates the weighting coefficient. This represents the regression error. This indicates the error in predicting sleep apnea.

[0025] A second aspect of the present invention provides a method for reconstructing airflow signals from the mouth and nose, comprising:

[0026] Acquire multiple target chest and abdominal motion signals collected by millimeter-wave radar from the target object;

[0027] The neural network model trained using any one of the methods described above is used to process the chest and abdominal motion signals of the multiple targets to obtain the reconstructed airflow signals from the mouth and nose of the targets.

[0028] A third aspect of the present invention provides a neural network model training device, the device comprising: a processor and a memory connected to the processor; wherein the memory stores instructions executable by the processor, the instructions being executed by the processor to cause the processor to perform the neural network model training method described above.

[0029] A fourth aspect of the present invention provides an oral and nasal airflow signal reconstruction device, the device comprising: a processor and a memory connected to the processor; wherein the memory stores instructions executable by the processor, the instructions being executed by the processor to cause the processor to perform the above-described oral and nasal airflow signal reconstruction method.

[0030] This invention fuses chest and abdominal motion signals acquired by millimeter-wave radar using a neural network model, then captures their temporal features, and performs dual-task processing on the temporal signals. Through feature fitting and mapping, it reconstructs the nasal and oral airflow signals and predicts apnea. Next, it compares the reconstructed nasal and oral airflow signals with the actual nasal and oral airflow signals to calculate the regression error, and compares the apnea prediction results with apnea labels to obtain the apnea prediction error. Then, it performs a weighted calculation of the two errors to obtain the integration loss. Finally, it optimizes the model parameters based on the integration loss and continuously updates the neural network model.

[0031] This method, in reconstructing nasal and oral airflow signals, employs an auxiliary task for predicting apnea events and utilizes ensemble loss for parameter optimization. This guides the neural network model to learn the signal characteristics during apnea periods, enabling it to not only closely approximate the original nasal and oral airflow signals during normal breathing but also accurately identify features during apnea. Therefore, the nasal and oral airflow signals reconstructed by the neural network model possess higher precision, presenting a more comprehensive and accurate signal that encompasses signal waveforms under various conditions, including normal breathing and apnea.

[0032] Furthermore, in terms of monitoring comfort, millimeter-wave radar enables seamless monitoring, eliminating the need for the target to wear equipment, significantly improving comfort, especially suitable for long-term monitoring scenarios such as sleep. Regarding signal detail, the airflow signals reconstructed by the neural network model are more consistent with the actual signal patterns, offering richer details compared to traditional radar breathing waveforms, providing strong support for subsequent analyses such as sleep apnea detection and lung function disease assessment.

[0033] This invention utilizes millimeter-wave radar to acquire chest and abdominal motion signals of a target object, enabling non-contact data acquisition, avoiding interference with the target object, and improving the convenience and comfort of data acquisition. The chest and abdominal motion signals are reconstructed using a trained neural network model to obtain the target's reconstructed nasal and oral airflow signals. Because the neural network model is trained through dual tasks, it can obtain nasal and oral airflow signals under normal breathing conditions as well as signal details under apnea conditions, resulting in more refined and comprehensive reconstructed nasal and oral airflow signals, thus providing strong support for related diagnosis and research. Attached Figure Description

[0034] To more clearly illustrate the specific embodiments of the present invention or the technical solutions in the prior art, the drawings used in the description of the specific embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are some embodiments of the present invention. For those skilled in the art, other drawings can be obtained from these drawings without creative effort.

[0035] Figure 1 This is a flowchart of the neural network model training method in an embodiment of the present invention;

[0036] Figure 2 This is a comparison diagram of nasal airflow pressure signals under different reconstructions in an embodiment of the present invention;

[0037] Figure 3 This is a structural diagram of the convolutional network in an embodiment of the present invention;

[0038] Figure 4 This is a structural diagram of the oral-nasal airflow regression model in an embodiment of the present invention;

[0039] Figure 5 This is a structural diagram of the sleep apnea event prediction model in an embodiment of the present invention. Detailed Implementation

[0040] The technical solution of the present invention will now be clearly and completely described with reference to the accompanying drawings. Obviously, the described embodiments are only some, not all, of the embodiments of the present invention. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0041] In the description of this invention, it should be noted that the terms "center," "upper," "lower," "left," "right," "vertical," "horizontal," "inner," and "outer," etc., indicate the orientation or positional relationship based on the orientation or positional relationship shown in the accompanying drawings. They are used only for the convenience of describing the invention and for simplifying the description, and do not indicate or imply that the device or element referred to must have a specific orientation, or be constructed and operated in a specific orientation. Therefore, they should not be construed as limitations on the invention. Furthermore, the terms "first," "second," and "third" are used for descriptive purposes only and should not be construed as indicating or implying relative importance.

[0042] Furthermore, the technical features involved in the different embodiments of the present invention described below can be combined with each other as long as they do not conflict with each other.

[0043] like Figure 1 As shown, this embodiment of the invention provides a neural network model training method, which is executed by an electronic device such as a computer or server, and specifically includes:

[0044] S1 acquires multiple chest and abdominal motion signals collected by the millimeter-wave radar from the target object.

[0045] In this embodiment, the scenario for acquiring multiple chest and abdominal motion signals is as follows: a millimeter-wave radar is placed at a height of 1m above the head of the target object's bed to acquire data from the target object and preprocess the echo signals to obtain multiple respiratory velocity waveforms that characterize the chest and abdominal motion of the target object. In addition to the respiratory velocity waveforms, other waveforms that characterize respiratory motion, such as phase waveforms and displacement waveforms, can also be extracted from the radar signals.

[0046] S2. A neural network model is used to process multiple chest and abdominal motion signals to obtain reconstructed oral and nasal airflow signals and apnea prediction results. Specifically, the neural network model performs feature fusion on multiple chest and abdominal motion signals to obtain fused feature signals, captures temporal features on the fused feature signals to obtain target temporal signals, and performs feature fitting and linear mapping on the target temporal signals to obtain reconstructed oral and nasal airflow signals and apnea prediction results.

[0047] After acquiring multiple chest and abdominal motion signals from millimeter-wave radar, a pre-defined neural network model performs a dual task. First, multiple chest and abdominal motion signals are fused to obtain a fused feature signal. Then, the temporal features in the fused feature signal are captured, ensuring they match the sampling frequency and length of the multiple chest and abdominal motion signals to form a target temporal signal. Next, feature fitting and linear mapping are performed on the target temporal signal to reconstruct the nasal and oral airflow signals and predict whether apnea has occurred. This yields the reconstructed nasal and oral airflow signals and the apnea prediction results. The reconstructed nasal and oral airflow signals can be nasal airflow pressure signals, or oral airflow pressure signals, nasal airflow temperature signals, and oral airflow temperature signals, etc.

[0048] S3 calculates the regression error based on the reconstructed nasal and oral airflow signals and the actual nasal and oral airflow signals, and calculates the apnea prediction error based on the apnea prediction results and apnea labels.

[0049] In this embodiment, the actual nasal and oral airflow signal needs to be obtained based on the reconstructed nasal and oral airflow signal, which can be a nasal and oral airflow pressure signal or a nasal and oral airflow temperature signal, etc. Apnea tags are typically determined and labeled by professionals according to medical standards based on the actual nasal and oral airflow signal. For example, when a target subject's breathing stops for a certain duration, they will be labeled as being in a state of apnea; this labeling information is the apnea tag, which serves as the benchmark for judging the accuracy of apnea prediction results.

[0050] The reconstructed nasal and oral airflow signals are compared with the actual nasal and oral airflow signals to calculate the regression error, which reflects the degree of difference between the reconstructed and actual signals. Simultaneously, the apnea prediction results are compared with apnea labels to calculate the apnea prediction error, which reflects the accuracy of the prediction results.

[0051] S4 is calculated by weighting the regression error and the apnea prediction error to obtain the integration loss.

[0052] To comprehensively consider the impact of regression error and apnea prediction error on the neural network model, it is necessary to perform a weighted calculation based on the two losses to obtain an integrated loss value that can reflect the overall performance of the neural network model in both reconstructing the oral and nasal airflow signals and predicting apnea.

[0053] S5 performs backpropagation and gradient updates on the neural network model based on the integration loss, and optimizes the model parameters.

[0054] By adding an auxiliary task of apnea event detection, multi-task learning is achieved, allowing the neural network model to allocate attention to learning the airflow characteristics of the target object's mouth and nose during apnea. Parameter optimization using ensemble loss allows the neural network model to focus more on learning features during apnea periods.

[0055] like Figure 2 As shown, taking nasal airflow pressure signal as an example, (a) is the real nasal airflow pressure signal, (b) is the traditional single-task reconstructed nasal airflow pressure signal, and (c) is the multi-task reconstructed nasal airflow pressure signal of this embodiment. In the real nasal airflow pressure signal, the signal amplitude attenuation indicates that the target object has apnea, and the signal characteristic during apnea is that the amplitude is close to 0. During normal breathing, the nasal airflow pressure signals reconstructed by single-task learning and multi-task learning in this embodiment are close to the real nasal airflow pressure signal. However, during apnea, only multi-task learning in this embodiment accurately identifies and is close to the real nasal airflow pressure signal, and the signal reconstruction capability is more accurate and comprehensive.

[0056] This embodiment utilizes a neural network model to fuse chest and abdominal motion signals acquired by millimeter-wave radar, then captures their temporal features, and performs dual-task processing on the temporal signals. Through feature fitting and mapping, it reconstructs the nasal and oral airflow signals and predicts apnea. Next, it compares the reconstructed nasal and oral airflow signals with the actual nasal and oral airflow signals to calculate the regression error, and compares the apnea prediction results with the apnea label to obtain the apnea prediction error. Then, it performs a weighted calculation of the two errors to obtain the integration loss. Finally, it optimizes the model parameters based on the integration loss and continuously updates the neural network model.

[0057] This method, in reconstructing nasal and oral airflow signals, employs an auxiliary task for predicting apnea events and utilizes ensemble loss for parameter optimization. This guides the neural network model to learn the signal characteristics during apnea periods, enabling it to not only closely approximate the original nasal and oral airflow signals during normal breathing but also accurately identify features during apnea. Therefore, the nasal and oral airflow signals reconstructed by the neural network model possess higher precision, presenting a more comprehensive and accurate signal that encompasses signal waveforms under various conditions, including normal breathing and apnea.

[0058] Furthermore, in terms of monitoring comfort, millimeter-wave radar enables seamless monitoring, eliminating the need for the target to wear equipment, significantly improving comfort, especially suitable for long-term monitoring scenarios such as sleep. Regarding signal detail, the airflow signals reconstructed by the neural network model are more consistent with the actual signal patterns, offering richer details compared to traditional radar breathing waveforms, providing strong support for subsequent analyses such as sleep apnea detection and lung function disease assessment.

[0059] In some optional embodiments of this example, the multiple chest and abdominal movement signals in step S1 include multiple distance gate breathing signals of the target object's chest and abdominal position movement, wherein each distance gate breathing signal is a signal sequence of a preset duration collected in the corresponding distance gate spatial region according to a preset sampling rate.

[0060] Multiple chest and abdominal motion signals are obtained by preprocessing the echo signals collected by millimeter-wave radar. These signals characterize multiple respiratory waveforms of the target object's chest and abdominal motion (which can be respiratory velocity waveforms, phase waveforms, displacement waveforms, etc.).

[0061] To comprehensively and meticulously capture the respiratory characteristics of the target subject, 40 respiratory signals corresponding to different distance gates are selected as an example. The selection of these distance gates ensures that the monitoring range covers the respiratory details of the target subject to the greatest extent possible. The respiratory signal for each distance gate is processed according to a preset sampling rate and duration. For example, a sampling rate of 50Hz means acquiring 50 data points per second; the preset duration is 12 minutes, or 720 seconds. Thus, the respiratory signal for each distance gate is a signal sequence formed by continuously acquiring data at a sampling rate of 50Hz for 12 minutes within the spatial region corresponding to that distance gate. Each sequence contains 50 × 720 = 36,000 data points. These 40 respiratory signals from different distance gates are then integrated, resulting in a signal segment data format of 40 × 36,000.

[0062] In this embodiment, the multiple chest and abdominal motion signals in step S1 encompass multiple range-gate respiratory signals representing the chest and abdominal movements of the target object. Each range-gate respiratory signal is a signal sequence acquired in its corresponding spatial region at a preset sampling rate and duration. After preprocessing the echo signals acquired by the millimeter-wave radar, various respiratory waveforms characterizing the chest and abdominal movements of the target object can be obtained. By selecting a certain number of respiratory signals corresponding to range gates, the monitoring range can cover the respiratory details of the target object to the greatest extent. Acquiring data at a preset sampling rate and duration, and integrating the respiratory signal data format, achieves comprehensive, detailed, and accurate capture and quantification of the respiratory characteristics of the target object, providing a rich and reliable data foundation for subsequent more accurate analysis of respiratory status.

[0063] In some optional embodiments of this example, step S3, which calculates the regression error based on the reconstructed nasal and oral airflow signals and the actual nasal and oral airflow signals, includes:

[0064] Acquire real nasal and oral airflow signals from the target subject using polysomnography.

[0065] In this embodiment, the actual nasal and oral airflow signal is acquired through polysomnography (PSG), the gold standard medical device for acquiring respiratory signals. It primarily utilizes thermal sensors (or pressure sensors), which are placed at the target subject's mouth and nose. When the target subject breathes, the temperature (or pressure) of the exhaled airflow at the mouth and nose changes. The thermal sensor detects the temperature change, and the pressure sensor detects the pressure change. In this way, the signal acquired by the PSG is presented as a temperature change curve (or pressure change curve), and these curves represent the target subject's actual breathing situation; this signal is the actual nasal and oral airflow signal.

[0066] The mean absolute error is calculated based on the reconstructed nasal and oral airflow signals and the actual nasal and oral airflow signals, and the regression error is obtained.

[0067] To evaluate the accuracy of the reconstructed nasal and oral airflow signals from the neural network model, it is necessary to compare them with real nasal and oral airflow signals. In this embodiment, the calculated mean absolute error reflects the overall deviation between the reconstructed nasal and oral airflow signals and the real nasal and oral airflow signals. The smaller the regression error, the closer the reconstructed signal is to the real signal, and the higher the accuracy of the reconstruction method. Besides the mean absolute error, the regression error can also be calculated using other methods.

[0068] In some optional embodiments of this example, step S2 involves feature fitting and linear mapping of the target time-series signal to obtain a reconstructed oral and nasal airflow signal, including:

[0069] The target time series signal is dimensionality reduced and projected to obtain a time series signal with a first preset dimension.

[0070] First, the target time-series signal is subjected to dimensionality reduction projection to summarize features. In this way, some unnecessary information can be removed, and the most critical features for reconstructing the nasal and oral airflow signal can be retained.

[0071] The time-series signal of the first preset dimension is mapped to obtain the reconstructed airflow signal from the mouth and nose.

[0072] By projecting the time-series signal of the first preset dimension onto a 1-dimensional signal, the result of this 1-dimensional signal corresponds to the predicted value of the airflow signal of the mouth and nose at the current time point. By concatenating the predicted values ​​of multiple time points (e.g., 36,000 time points), the reconstructed airflow signal of the mouth and nose can be obtained.

[0073] This embodiment processes the target time-series signal to reconstruct the nasal and oral airflow signal. Dimensionality reduction projection removes redundant information, retains key features, reduces complexity and computational load, and avoids interference. Then, through result mapping, high-dimensional features are accurately converted into low-dimensional predictions, and the predicted values ​​from multiple time points are concatenated to accurately reconstruct the nasal and oral airflow signal.

[0074] In some optional embodiments of this example, step S2 involves feature fitting and linear mapping of the target time-series signal to obtain the apnea prediction result, including:

[0075] The target time series signal is dimensionality reduced and projected to obtain a time series signal with a second preset dimension.

[0076] First, the target time-series signal is subjected to dimensionality reduction projection to summarize features. In this way, some unnecessary information can be removed, and the most critical features for reconstructing the nasal and oral airflow signal can be retained to obtain the time-series signal of the second preset dimension.

[0077] The time-series signal of the second preset dimension is mapped to obtain the apnea prediction result.

[0078] By projecting the time-series signal of the second preset dimension onto the 1-dimensional signal, the result of this 1-dimensional signal corresponds to the predicted value of whether the current time point is the time point of apnea. By concatenating the predicted values ​​of multiple time points (e.g., 36,000 time points), the apnea prediction result can be obtained.

[0079] This embodiment processes the target time-series signal to obtain apnea prediction results. Dimensional reduction projection removes redundant information and retains key features to obtain a second preset dimension time-series signal, reducing complexity, computational load, and avoiding interference. Then, through result mapping, the high-dimensional features are accurately converted into low-dimensional predictions, and the prediction values ​​at multiple time points are concatenated to accurately output apnea prediction results, assisting in training the reconstruction of the nasal and oral signals.

[0080] In some optional embodiments of this example, the neural network model includes a convolutional network, a bidirectional long short-term memory neural network, and an analysis model. The convolutional network is used to fuse multiple chest and abdominal motion signals to obtain a fused feature signal. The bidirectional long short-term memory neural network is used to capture the temporal features of the fused feature signal to obtain a target temporal signal. The analysis model is used to perform feature fitting and linear mapping on the target temporal signal to obtain a reconstructed oral and nasal airflow signal and a sleep apnea prediction result.

[0081] For example, the convolutional network includes a first convolutional layer, a first activation function layer, a first max pooling layer, a second convolutional layer, a second activation function layer, a second max pooling layer, and an output layer. The first convolutional layer is used to convolve the airflow signal from the mouth and nose to increase the channel dimension of the airflow signal. The first activation function layer is used to enhance the nonlinear fitting ability of the features processed by the first convolutional layer. The first max pooling layer is used to expand the receptive field of the second convolutional layer. The second convolutional layer is used to convolve the features processed by the first max pooling layer to further increase the channel dimension. The second activation function layer is used to enhance the nonlinear fitting ability of the features processed by the second convolutional layer. The second max pooling layer is used to expand the receptive field of the output layer. The output layer is used to output the fused feature signal.

[0082] like Figure 3 As shown, a shallow convolutional network is used to fuse features of chest and abdominal motion signals with 40 distance gates. First, a first convolutional layer (Conv1D) with a kernel size of 3 is used to integrate the respiratory signal features of 40 channels. Here, 64 convolutional kernels are used, so the channel dimension increases to 64 after passing through the first convolutional layer (Conv1D). Then, the nonlinear fitting ability is enhanced by the first activation function layer (ReLU), followed by the first max pooling layer (MaxPool1D) (k=3, s=1, p=1) to expand the receptive field. The above process is set up in two layers, and 128 convolutional kernels are used in the second convolutional layer to further increase the dimension to express the features. Then, after passing through the second activation function layer (ReLU) and the second max pooling layer (MaxPool1D) (k=5, s=1, p=1), the final output layer outputs a 128×36000 fused feature signal, providing sufficient data support for subsequent temporal modeling.

[0083] The convolutional network in this embodiment consists of multiple convolutional layers, activation function layers, max pooling layers, and an output layer. Convolutional layers enhance the channel dimension of the nasal and oral airflow signals to represent features; activation function layers enhance the nonlinear fitting ability of the features; max pooling layers expand the receptive field of subsequent layers; and finally, a fused feature signal is output, providing sufficient data support for subsequent temporal modeling.

[0084] A bidirectional long short-term memory (Bi-LSTM) neural network is used to process the fused feature signal after convolutional network fusion. Bi-LSTM uses point-by-point output to ensure that the sampling frequency and signal length of the subsequent reconstructed signal are consistent with the input chest and abdominal motion signal. In this example, the data time dimension is 12 minutes, ensuring that Bi-LSTM can acquire sufficient temporal relationships for signal reconstruction. This example uses a two-layer Bi-LSTM, resulting in a final output signal dimension of 256×36000, used for subsequent signal reconstruction and apnea event prediction tasks.

[0085] This embodiment uses a bidirectional long short-term memory neural network (Bi-LSTM) to process the fused feature signal of the convolutional network. Point-by-point output ensures that the sampling frequency and length of the reconstructed signal are consistent with the input chest and abdominal motion signal. Sufficient data temporal dimension allows the Bi-LSTM to acquire adequate temporal relationships for signal reconstruction. The two-layer Bi-LSTM configuration allows the final output signal to be used for subsequent signal reconstruction and apnea event prediction tasks.

[0086] The analysis model includes a nasal airflow regression model and a sleep apnea event prediction model. The nasal airflow regression model includes a first linear layer, a third activation function layer, a first random deactivation layer, and a second linear layer. The first linear layer is used to perform dimensionality reduction projection on the target time series signal. The third activation function layer is used to enhance the nonlinear fitting ability of the features processed by the first linear layer. The first random deactivation layer is used to reduce the overfitting problem of the nasal airflow regression model. The second linear layer is used to map the results of the features processed by the first linear layer to obtain the reconstructed nasal airflow signal.

[0087] like Figure 4 As shown, the nasal airflow regression model first uses a first linear layer to project the target time-series signal with a signal dimension of 256×36000, projecting the original 256-dimensional target time-series signal to a first preset dimension (e.g., 64-dimensional) to summarize signal features. Then, a third activation function layer (ReLU) is used to enhance the nonlinear fitting ability of the features processed by the first linear layer. Next, a first random deactivation layer (Dropout) is used to reduce the overfitting problem of the nasal airflow regression model. Finally, a second linear layer is used to map the features processed by the first linear layer, projecting the 64-dimensional feature signal to 1 dimension, which corresponds to the predicted value of the nasal airflow signal at the current time point. By concatenating the predicted values ​​of multiple time points (e.g., 36000 time points), the reconstructed nasal airflow signal can be obtained.

[0088] The nasal airflow regression model in this embodiment consists of a first linear layer, a third activation function layer, a first random deactivation layer, and a second linear layer. The first linear layer performs dimensionality reduction projection on the target time-series signal to summarize features; the third activation function layer enhances the nonlinear fitting ability of the features; the first random deactivation layer reduces overfitting; and the second linear layer maps the results, ultimately obtaining the reconstructed nasal airflow signal. The nasal airflow regression model achieves effective processing of the target time-series signal through the collaborative work of each layer. Dimensionality reduction projection summarizes features, enhances nonlinear fitting ability, reduces the risk of overfitting, and ultimately accurately maps and outputs the reconstructed nasal airflow signal.

[0089] The sleep apnea event prediction model consists of a third linear layer, a fourth activation function layer, a second random deactivation layer, a fourth linear layer, and a prediction result mapping layer. The third linear layer is used to perform dimensionality reduction projection on the target time-series signal. The fourth activation function layer is used to enhance the nonlinear fitting ability of the features processed by the third linear layer. The second random deactivation layer is used to reduce the overfitting problem of the sleep apnea event prediction model. The fourth linear layer is used to map the features processed by the third linear layer. The prediction result mapping layer is used to output the result mapped by the fourth linear layer to obtain the sleep apnea prediction result.

[0090] like Figure 5 As shown, the apnea event prediction model first uses a third linear layer to project the target time series signal with a signal dimension of 256×36000 to a second preset dimension (e.g., 32 dimensions) to summarize signal features. Then, a fourth activation function layer (ReLU) is used to enhance the nonlinear fitting ability of the features processed by the third linear layer. Next, a second random deactivation layer (Dropout) is used to reduce the overfitting problem of the apnea event prediction model. Finally, a fourth linear layer is used to map the features processed by the third linear layer, projecting the 32-dimensional feature signal to 1 dimension. Then, a prediction result mapping layer (Sigmoid) is used to output the apnea prediction result mapped by the fourth linear layer, which corresponds to the predicted value of whether the current time point is the apnea time point. By concatenating the prediction values ​​of multiple time points (e.g., 36000 time points), the apnea prediction result can be obtained.

[0091] The apnea event prediction model in this embodiment consists of a third linear layer, a fourth activation function layer, a second random deactivation layer, and a prediction result mapping layer. The model uses the third linear layer to reduce the dimensionality of the target time-series signal through projection and feature summarization. The fourth activation function layer enhances the nonlinear fitting ability of the features. The second random deactivation layer reduces overfitting. The fourth linear layer performs result mapping, and finally, the prediction result mapping layer outputs the apnea prediction result. This model can effectively process the target time-series signal, accurately identify apnea events during signal reconstruction, and has a more comprehensive signal reconstruction capability, helping to promptly detect potential apnea events and protect people's respiratory health.

[0092] In some optional implementations of this embodiment, the integration loss is calculated in the following manner:

[0093] ,

[0094] in, Indicates integration loss, Indicates the weighting coefficient. Indicates the regression error. This indicates the error in predicting sleep apnea.

[0095] In this embodiment, regression error The apnea prediction error is calculated based on the mean absolute error between the reconstructed nasal and oral airflow signals and the actual nasal and oral airflow signals. The apnea prediction error is calculated based on the cross-entropy loss of the apnea prediction results and apnea labels. The regression error is integrated with the apnea prediction error. In this example, we set... The integrated loss is used for backpropagation and gradient updates. In this example, the Adam optimizer is used to optimize the model parameters, the learning rate can be set to 0.001, and the neural network model can stop training after 200 epochs, using an early stopping mechanism.

[0096] This embodiment integrates regression error and apnea prediction error for backpropagation and gradient update. It also employs an early apnea mechanism to optimize model parameters, effectively improving the training performance and efficiency of the neural network model.

[0097] This invention also provides a method for reconstructing airflow signals from the mouth and nose, which is executed by an electronic device such as a computer or server, and specifically includes:

[0098] Acquire multiple target chest and abdominal motion signals collected by millimeter-wave radar.

[0099] The neural network model trained using the above-mentioned neural network model training method is used to process the chest and abdominal motion signals of multiple targets to obtain the reconstructed airflow signals of the mouth and nose of the targets.

[0100] This invention utilizes millimeter-wave radar to acquire chest and abdominal motion signals of a target object, enabling non-contact data acquisition, avoiding interference with the target object, and improving the convenience and comfort of data acquisition. The chest and abdominal motion signals are reconstructed using a trained neural network model to obtain the target's reconstructed nasal and oral airflow signals. Because the neural network model is trained through dual tasks, it can obtain nasal and oral airflow signals under normal breathing conditions as well as signal details under apnea conditions, resulting in more refined and comprehensive reconstructed nasal and oral airflow signals, thus providing strong support for related diagnosis and research.

[0101] Those skilled in the art will understand that embodiments of the present invention can be provided as methods, systems, or computer program products. Therefore, the present invention can take the form of a completely hardware embodiment, a completely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, the present invention can take the form of a computer program product embodied on one or more computer-usable storage media (including, but not limited to, disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0102] This invention is described with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of the invention. It will be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, special-purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, generate instructions for implementing the flowchart illustrations and / or block diagrams. Figure 1 One or more processes and / or boxes Figure 1 A device that provides the functions specified in one or more boxes.

[0103] These computer program instructions may also be stored in a computer-readable storage medium that can direct a computer or other programmable data processing device to function in a particular manner, such that the instructions stored in the computer-readable storage medium produce an article of manufacture including instruction means, which are implemented in a process Figure 1 One or more processes and / or boxes Figure 1 The function specified in one or more boxes.

[0104] These computer program instructions may also be loaded onto a computer or other programmable data processing equipment to cause a series of operational steps to be performed on the computer or other programmable equipment to produce a computer-implemented process, thereby providing instructions that execute on the computer or other programmable equipment for implementing the process. Figure 1 One or more processes and / or boxes Figure 1 The steps of the function specified in one or more boxes.

[0105] Obviously, the above embodiments are merely illustrative examples for clear explanation and are not intended to limit the implementation. Those skilled in the art will recognize that other variations or modifications can be made based on the above description. It is neither necessary nor possible to exhaustively list all possible implementations here. However, obvious variations or modifications derived therefrom are still within the scope of protection of this invention.

Claims

1. A method for training a neural network model, characterized in that, include: Acquire multiple chest and abdominal motion signals collected by millimeter-wave radar from the target object; The multiple chest and abdominal motion signals are processed using a neural network model to obtain reconstructed oral and nasal airflow signals and apnea prediction results. The neural network model performs feature fusion on the multiple chest and abdominal motion signals to obtain a fused feature signal. The fused feature signal is then subjected to temporal feature capture to obtain a target temporal signal. The target time series signal is subjected to dimensionality reduction projection to obtain a time series signal of the first preset dimension. The time series signal of the first preset dimension is then mapped to obtain the reconstructed oral and nasal airflow signal. The target time series signal is subjected to dimensionality reduction projection to obtain a time series signal of the second preset dimension. The result of the second preset dimension time series signal is mapped to obtain the sleep apnea prediction result. The regression error is calculated based on the reconstructed nasal and oral airflow signals and the actual nasal and oral airflow signals; the apnea prediction error is calculated based on the apnea prediction results and the apnea labels. The integrated loss is obtained by weighting the regression error and the apnea prediction error. The neural network model is backpropagated and its gradients are updated based on the integration loss, and the model parameters are optimized.

2. The method according to claim 1, characterized in that, The multiple chest and abdominal movement signals include multiple distance gate breathing signals of the target object's chest and abdominal position movements, wherein each distance gate breathing signal is a signal sequence of a preset duration collected in the corresponding distance gate spatial region at a preset sampling rate.

3. The method according to claim 1, characterized in that, The calculation of regression error based on the reconstructed nasal and oral airflow signals and the actual nasal and oral airflow signals includes: Acquire real nasal and oral airflow signals from the target subject using polysomnography; The regression error is obtained by calculating the mean absolute error based on the reconstructed nasal and oral airflow signals and the actual nasal and oral airflow signals.

4. The method according to claim 1, characterized in that, The neural network model includes a convolutional network, a bidirectional long short-term memory neural network, and an analysis model. The convolutional network is used to fuse the features of the multiple chest and abdominal motion signals to obtain a fused feature signal. The bidirectional long short-term memory neural network is used to capture the temporal features of the fused feature signal to obtain a target temporal signal. The analysis model is used to perform feature fitting and linear mapping on the target temporal signal to obtain a reconstructed oral and nasal airflow signal and a sleep apnea prediction result.

5. The method according to claim 1, characterized in that, The integration loss is calculated as follows: ， in, Indicates integration loss, Indicates the weighting coefficient. This represents the regression error. This indicates the error in predicting sleep apnea.

6. A method for reconstructing airflow signals from the mouth and nose, characterized in that, include: Acquire multiple target chest and abdominal motion signals collected by millimeter-wave radar from the target object; The neural network model trained using any one of claims 1-5 is used to process the chest and abdominal motion signals of the plurality of targets to obtain the reconstructed airflow signals of the mouth and nose of the targets.

7. A neural network model training device, characterized in that, include: A processor and a memory connected to the processor; wherein the memory stores instructions executable by the processor, the instructions being executed by the processor to cause the processor to perform the neural network model training method as described in any one of claims 1-5.

8. A device for reconstructing airflow signals from the mouth and nose, characterized in that, include: A processor and a memory connected to the processor; wherein the memory stores instructions executable by the processor, the instructions being executed by the processor to cause the processor to perform the oral and nasal airflow signal reconstruction method as described in claim 6.

Images