Neural network model training method, mouth-nose airflow signal reconstruction method and equipment

Abstract

The invention provides a neural network model training method, a mouth-nose airflow signal reconstruction method and equipment. The method is applied to radar signal processing. The method comprises the following steps: acquiring a plurality of thoracico-abdominal movement signals acquired by a millimeter wave radar for a target object; processing the plurality of thoracico-abdominal movement signals by using a neural network model to obtain a reconstructed mouth-nose airflow signal and an apnea prediction result; calculating a regression error according to the reconstructed mouth-nose airflow signal and the real mouth-nose airflow signal, and calculating an apnea prediction error according to an apnea prediction result and an apnea label; performing weighted calculation according to the regression error and the apnea prediction error to obtain integration loss; and performing back propagation and gradient updating on the neural network model according to the integration loss, and optimizing model parameters. The mouth-nose airflow signal reconstructed by the neural network model is finer, more comprehensive and more accurate.

Description

TECHNICAL FIELD

[0001] The present application relates to the field of radar signal processing, in particular to a neural network model training method, an oral-nasal airflow signal reconstruction method and equipment BACKGROUND

[0002] Oral-nasal airflow signal is one of the most important indicators in human sleep monitoring, and accurately capturing the oral-nasal airflow signal has high clinical value for diagnosing sleep apnea and other respiratory diseases of patients. The traditional measurement method uses a respiratory flow temperature sensor or a pressure sensor, which requires the patient to wear a nasal catheter and the like. This contact type measurement process will cause obvious discomfort to the monitored object, making it difficult to apply to long-term health management scenarios.

[0003] In view of the limitations of the traditional measurement method, millimeter wave radar emerges as a new user-friendly monitoring technology. It can obtain the respiratory condition of the monitored object without contacting the monitored object. Through the distance, azimuth, pitch angle and Doppler information in the echo signal, the micro-motion signal of a specific part of the human body can be obtained, and then the respiratory information can be successfully separated from the micro-motion signal.

[0004] However, the current millimeter wave radar technology reflects the respiratory movement of the monitored object by extracting the chest and abdomen movement signal of the monitored object, but the shape of the extracted chest and abdomen movement signal is greatly distorted compared with the oral-nasal airflow signal, and the details are seriously insufficient, which is difficult to meet the fine evaluation requirement. SUMMARY

[0005] Therefore, the present application provides a neural network model training method, comprising: obtaining a plurality of chest and abdomen movement signals collected by a millimeter wave radar on a target object; processing the plurality of chest and abdomen movement signals by using a neural network model to obtain a reconstructed oral-nasal airflow signal and an apnea prediction result, wherein the neural network model performs feature fusion on the plurality of chest and abdomen movement signals to obtain a fusion feature signal, performs time sequence feature capture on the fusion feature signal to obtain a target time sequence signal, and performs feature fitting and linear mapping on the target time sequence signal to obtain the reconstructed oral-nasal airflow signal and the apnea prediction result; calculating a regression error according to the reconstructed oral-nasal airflow signal and a true oral-nasal airflow signal, and calculating an apnea prediction error according to the apnea prediction result and an apnea label; performing weighted calculation according to the regression error and the apnea prediction error to obtain an integrated loss; performing back propagation and gradient update on the neural network model according to the integrated loss, and optimizing model parameters.

[0006] Optionally, the plurality of thoraco-abdominal movement signals comprise a plurality of distance-gated respiration signals of thoraco-abdominal part movement of the target object, wherein each distance-gated respiration signal is a signal sequence of a preset time length collected in a corresponding distance-gated spatial region according to a preset sampling rate.

[0007] Optionally, the calculating the regression error according to the reconstructed oral-nasal airflow signal and the real oral-nasal airflow signal comprises: acquiring a real oral-nasal airflow signal collected by a polysomnogram on the target object; calculating a mean absolute error according to the reconstructed oral-nasal airflow signal and the real oral-nasal airflow signal to obtain the regression error.

[0008] Optionally, the feature fitting and linear mapping of the target time sequence signal to obtain the reconstructed oral-nasal airflow signal comprises: dimensionality reduction projection on the target time sequence signal to obtain a time sequence signal of a first preset dimension; result mapping on the time sequence signal of the first preset dimension to obtain the reconstructed oral-nasal airflow signal.

[0009] Optionally, the feature fitting and linear mapping of the target time sequence signal to obtain the apnea prediction result comprises: dimensionality reduction projection on the target time sequence signal to obtain a time sequence signal of a second preset dimension; result mapping on the time sequence signal of the second preset dimension to obtain the apnea prediction result.

[0010] Optionally, the neural network model comprises a convolutional network, a bidirectional long short-term memory neural network and an analysis model, wherein the convolutional network is used for feature fusion on the plurality of thoraco-abdominal movement signals to obtain a fusion feature signal, the bidirectional long short-term memory neural network is used for time sequence feature capture on the fusion feature signal to obtain a target time sequence signal, and the analysis model is used for feature fitting and linear mapping on the target time sequence signal to obtain a reconstructed oral-nasal airflow signal and an apnea prediction result.

[0011] Optionally, the integrated loss is calculated in the following manner: , wherein, represents the integrated loss, represents a weight coefficient, represents the regression error, represents an apnea prediction error.

[0012] The second aspect of the present application provides a method for reconstructing an oral-nasal airflow signal, comprising: acquire a plurality of target thoraco-abdominal movement signals collected by the millimeter wave radar on the target object; process the plurality of target thoraco-abdominal movement signals by using the trained neural network model to obtain a target reconstructed oral-nasal airflow signal.

[0013] The third aspect of the present application provides a neural network model training device, comprising a processor and a memory connected to the processor; wherein the memory stores instructions executable by the processor, and the instructions are executed by the processor to enable the processor to perform the neural network model training method described above.

[0014] The fourth aspect of the present application provides an oral-nasal airflow signal reconstruction device, comprising a processor and a memory connected to the processor; wherein the memory stores instructions executable by the processor, and the instructions are executed by the processor to enable the processor to perform the oral-nasal airflow signal reconstruction method described above.

[0015] The present application fuses thoraco-abdominal movement signals collected by the millimeter wave radar by using a neural network model, then captures the timing characteristics, and further performs double-task processing on the timing signals, reconstructs the outlet-nose airflow signal through feature fitting and mapping, and realizes the prediction of apnea. Then, the reconstructed oral-nasal airflow signal is compared with the real oral-nasal airflow signal to calculate the regression error, and the apnea prediction result is compared with the apnea label to obtain the apnea prediction error. Then, the two kinds of errors are weighted to obtain the integrated loss. Finally, the model parameters are optimized according to the integrated loss, and the neural network model is continuously updated.

[0016] In the oral-nasal airflow signal reconstruction, the present method designs an auxiliary task for predicting apnea events, and optimizes the parameters by using the integrated loss to guide the neural network model to learn the signal characteristics in the apnea period. Not only can the reconstructed oral-nasal airflow signal be close to the original signal during normal breathing, but also can accurately identify the characteristics during apnea. Therefore, the oral-nasal airflow signal reconstructed by the neural network model has higher precision and presents a more comprehensive and accurate signal, which can cover the signal waveforms in various situations such as normal breathing and apnea.

[0017] In addition, in terms of monitoring comfort, the millimeter wave radar is used for non-invasive monitoring, and the target object does not need to wear equipment, which greatly improves the comfort, especially suitable for long-term monitoring scenes such as sleep. In terms of signal details, the oral-nasal airflow signal reconstructed by the neural network model is more consistent with the real signal form, and compared with the traditional radar breathing waveform, it has more details, which provides strong support for subsequent apnea detection, lung function disease evaluation and other analysis.

[0018] The application collects the chest and abdomen movement signals of a target object by using a millimeter wave radar, can obtain data in a non-contact manner, avoids interference to the target object, and improves the convenience and comfort of data collection. The chest and abdomen movement signals are reconstructed by a trained neural network model to obtain target reconstructed oral and nasal airflow signals. Since the neural network model is obtained by double-task training, the oral and nasal airflow signals in a normal breathing state can be obtained, and the signal details in a respiratory pause state are also concerned, so that the reconstructed oral and nasal airflow signals are more detailed and comprehensive, thereby providing strong support for related diagnosis and research. BRIEF DESCRIPTION OF DRAWINGS

[0019] In order to more clearly illustrate the technical solutions in the specific embodiments of the present application or the prior art, the drawings needed in the specific embodiments or prior art description will be briefly introduced as follows. Obviously, the drawings in the following description are some embodiments of the present application, and other drawings can be obtained by those skilled in the art without creative labor.

[0020] Figure 1 The flow chart of the neural network model training method in the embodiment of the present application; Figure 2 The comparison chart of the nasal airflow pressure signal under different reconstructions in the embodiment of the present application; Figure 3 The structure chart of the convolution network in the embodiment of the present application; Figure 4 The structure chart of the oral and nasal airflow regression model in the embodiment of the present application; Figure 5 The structure chart of the respiratory pause event prediction model in the embodiment of the present application. DETAILED DESCRIPTION

[0021] The technical solutions of the present application will be described in detail below with reference to the drawings. Obviously, the described embodiments are some embodiments of the present application, not all embodiments. Based on the embodiments in the present application, all other embodiments obtained by those skilled in the art without creative labor are within the scope of protection of the present application.

[0022] In the description of the present application, it should be noted that the terms "center", "upper", "lower", "left", "right", "vertical", "horizontal", "inner", "outer" and the like indicate the orientation or positional relationship based on the orientation or positional relationship shown in the drawings, and are only for the convenience of describing the present application and simplifying the description, and therefore cannot be understood as indicating or implying that the devices or elements referred to must have a particular orientation, be constructed and operated in a particular orientation, and therefore cannot be understood as limiting the present application. In addition, the terms "first", "second", "third" are only for descriptive purposes and cannot be understood as indicating or implying relative importance.

[0023] In addition, the technical features involved in the different embodiments of the application described below can be combined with each other as long as there is no conflict.

[0024] As Figure 1 shown, the embodiment of the present application provides a neural network model training method, which is executed by a computer or a server and the like electronic device, and specifically includes: S1, obtaining a plurality of chest and abdominal motion signals collected by a millimeter wave radar on a target object.

[0025] The scene of collecting a plurality of chest and abdominal motion signals in this embodiment is that the millimeter wave radar is placed at a height of 1m from the head of the target object to collect data on the target object, and the echo signal is preprocessed to obtain a plurality of breathing speed waveforms representing the chest and abdominal motion of the target object. In addition to the breathing speed waveform, other waveforms representing the breathing motion can also be extracted from the radar signal, such as phase waveform, displacement waveform, etc.

[0026] S2, processing the plurality of chest and abdominal motion signals using a neural network model to obtain reconstructed oral and nasal airflow signals and apnea prediction results, wherein the neural network model performs feature fusion on the plurality of chest and abdominal motion signals to obtain a fused feature signal, captures the time sequence features of the fused feature signal to obtain a target time sequence signal, and performs feature fitting and linear mapping on the target time sequence signal to obtain the reconstructed oral and nasal airflow signals and the apnea prediction results.

[0027] After obtaining the plurality of chest and abdominal motion signals collected by the millimeter wave radar, a pre-set neural network model is used to perform a double task. First, the plurality of chest and abdominal motion signals are fused to obtain a fused feature signal, and then the time sequence features in the fused feature signal are captured to make them consistent with the sampling frequency and length of the plurality of chest and abdominal motion signals to form a target time sequence signal. Then, by performing feature fitting and linear mapping on the target time sequence signal, on the one hand, the outlet and nasal airflow signals are reconstructed, and on the other hand, it is predicted whether to apnea, so as to obtain the reconstructed oral and nasal airflow signals and the apnea prediction results. The reconstructed oral and nasal airflow signals can be nasal airflow pressure signals, and can also be oral airflow pressure signals, nasal airflow temperature signals, oral airflow temperature signals, etc.

[0028] S3, calculate regression error according to the reconstructed oronasal airflow signal and the real oronasal airflow signal, and calculate apnea prediction error according to the apnea prediction result and the apnea label.

[0029] The real oronasal airflow signal of the embodiment needs to be obtained according to the form of the reconstructed oronasal airflow signal, which can be an oronasal airflow pressure signal, or an oronasal airflow temperature signal, etc. The apnea label is usually judged and labeled by professionals according to medical standards on the real oronasal airflow signal. For example, when the target object stops breathing for a certain length of time, it will be marked as being in an apnea state. This marking information is the apnea label, which is the basis for judging the accuracy of the apnea prediction result.

[0030] The reconstructed oronasal airflow signal is compared with the real oronasal airflow signal to calculate the regression error, which reflects the difference between the reconstructed signal and the real signal. At the same time, the apnea prediction result is compared with the apnea label to calculate the apnea prediction error, which reflects the accuracy of the prediction result.

[0031] S4, weighted calculation according to the regression error and the apnea prediction error to obtain the integrated loss.

[0032] In order to comprehensively consider the influence of the regression error and the apnea prediction error on the neural network model, the two losses need to be weighted calculated to finally obtain an integrated loss value that can reflect the performance of the neural network model in both reconstructing the oronasal airflow signal and predicting apnea.

[0033] S5, according to the integrated loss, the neural network model is back propagated and gradient updated, and the model parameters are optimized.

[0034] By adding the auxiliary task of apnea event detection, multi-task learning is performed, and the neural network model will allocate attention to learn the oronasal airflow features of the target object during apnea. Using the integrated loss to optimize the parameters can make the neural network model pay more attention to feature learning during the apnea period.

[0035] As Figure 2The nasal airflow pressure signal is taken as an example, and the true nasal airflow pressure signal, the traditional single-task reconstructed nasal airflow pressure signal, and the multi-task reconstructed nasal airflow pressure signal of the embodiment are shown in (a), (b), and (c) respectively. In the true nasal airflow pressure signal, the signal amplitude attenuation indicates that the target object has an apnea, and the signal feature during the apnea is that the amplitude is close to 0. During normal breathing, the reconstructed nasal airflow pressure signals of the single-task learning and the multi-task learning of the embodiment are close to the true nasal airflow pressure signal, but only the multi-task learning of the embodiment can accurately identify the apnea and is close to the true nasal airflow pressure signal, and the signal reconstruction capability is more accurate and comprehensive.

[0036] In the embodiment, the neural network model is used to fuse the chest and abdominal movement signals collected by the millimeter wave radar, and then the time sequence features are captured, and then the dual-task processing is performed on the time sequence signals, the outlet nasal airflow signal is reconstructed through feature fitting and mapping, and the apnea prediction is realized. Then, the reconstructed oral and nasal airflow signals are compared with the true oral and nasal airflow signals to calculate the regression error, and the apnea prediction result is compared with the apnea label to obtain the apnea prediction error. Then, the two kinds of errors are weighted to obtain the integrated loss. Finally, the model parameters are optimized according to the integrated loss, and the neural network model is continuously updated.

[0037] In the method, the auxiliary task of apnea event prediction is designed, and the parameter optimization is performed by using the integrated loss to guide the neural network model to learn the signal features in the apnea period. Therefore, the reconstructed oral and nasal airflow signals of the neural network model have higher fineness, and present more comprehensive and accurate signals, which can cover the signal waveforms in normal breathing and apnea.

[0038] In addition, in terms of monitoring comfort, the millimeter wave radar is used for non-invasive monitoring, and the target object does not need to wear equipment, which greatly improves the comfort, and is especially suitable for long-term monitoring scenes such as sleep. In terms of signal details, the oral and nasal airflow signals reconstructed by the neural network model are more consistent with the true signal form, and compared with the traditional radar breathing waveform, the details are more abundant, which provides strong support for subsequent apnea detection, lung function disease evaluation, and other analyses.

[0039] In some optional embodiments of the embodiment, the plurality of chest and abdominal movement signals in step S1 include a plurality of distance gate respiratory signals of the chest and abdominal parts of the target object, and each distance gate respiratory signal is a signal sequence collected in a corresponding distance gate space region according to a preset sampling rate and for a preset time length.

[0040] Multiple chest and abdominal motion signals are obtained by preprocessing the echo signals collected by millimeter-wave radar. These signals characterize multiple respiratory waveforms of the target object's chest and abdominal motion (which can be respiratory velocity waveforms, phase waveforms, displacement waveforms, etc.). To comprehensively and meticulously capture the respiratory characteristics of the target subject, 40 respiratory signals corresponding to different distance gates are selected as an example. The selection of these distance gates ensures that the monitoring range covers the respiratory details of the target subject to the greatest extent possible. The respiratory signal for each distance gate is processed according to a preset sampling rate and duration. For example, a sampling rate of 50Hz means acquiring 50 data points per second; the preset duration is 12 minutes, or 720 seconds. Thus, the respiratory signal for each distance gate is a signal sequence formed by continuously acquiring data at a sampling rate of 50Hz for 12 minutes within the spatial region corresponding to that distance gate. Each sequence contains 50 × 720 = 36,000 data points. These 40 respiratory signals from different distance gates are then integrated, resulting in a signal segment data format of 40 × 36,000.

[0041] In this embodiment, the multiple chest and abdominal motion signals in step S1 encompass multiple range-gate respiratory signals representing the chest and abdominal movements of the target object. Each range-gate respiratory signal is a signal sequence acquired in its corresponding spatial region at a preset sampling rate and duration. After preprocessing the echo signals acquired by the millimeter-wave radar, various respiratory waveforms characterizing the chest and abdominal movements of the target object can be obtained. By selecting a certain number of respiratory signals corresponding to range gates, the monitoring range can cover the respiratory details of the target object to the greatest extent. Acquiring data at a preset sampling rate and duration, and integrating the respiratory signal data format, achieves comprehensive, detailed, and accurate capture and quantification of the respiratory characteristics of the target object, providing a rich and reliable data foundation for subsequent more accurate analysis of respiratory status.

[0042] In some optional embodiments of this example, step S3, which calculates the regression error based on the reconstructed nasal and oral airflow signals and the actual nasal and oral airflow signals, includes: Acquire real nasal and oral airflow signals from the target subject using polysomnography.

[0043] In this embodiment, the actual nasal and oral airflow signal is acquired through polysomnography (PSG), the gold standard medical device for acquiring respiratory signals. It primarily utilizes thermal sensors (or pressure sensors), which are placed at the target subject's mouth and nose. When the target subject breathes, the temperature (or pressure) of the exhaled airflow at the mouth and nose changes. The thermal sensor detects the temperature change, and the pressure sensor detects the pressure change. In this way, the signal acquired by the PSG is presented as a temperature change curve (or pressure change curve), and these curves represent the target subject's actual breathing situation; this signal is the actual nasal and oral airflow signal.

[0044] The mean absolute error is calculated based on the reconstructed nasal and oral airflow signals and the actual nasal and oral airflow signals, and the regression error is obtained.

[0045] To evaluate the accuracy of the reconstructed nasal and oral airflow signals from the neural network model, it is necessary to compare them with real nasal and oral airflow signals. In this embodiment, the calculated mean absolute error reflects the overall deviation between the reconstructed nasal and oral airflow signals and the real nasal and oral airflow signals. The smaller the regression error, the closer the reconstructed signal is to the real signal, and the higher the accuracy of the reconstruction method. Besides the mean absolute error, the regression error can also be calculated using other methods.

[0046] In some optional embodiments of this example, step S2 involves feature fitting and linear mapping of the target time-series signal to obtain a reconstructed oral and nasal airflow signal, including: The target time series signal is dimensionality reduced and projected to obtain a time series signal with a first preset dimension.

[0047] First, the target time-series signal is subjected to dimensionality reduction projection to summarize features. In this way, some unnecessary information can be removed, and the most critical features for reconstructing the nasal and oral airflow signal can be retained.

[0048] The time-series signal of the first preset dimension is mapped to obtain the reconstructed airflow signal from the mouth and nose.

[0049] By projecting the time-series signal of the first preset dimension onto a 1-dimensional signal, the result of this 1-dimensional signal corresponds to the predicted value of the airflow signal of the mouth and nose at the current time point. By concatenating the predicted values ​​of multiple time points (e.g., 36,000 time points), the reconstructed airflow signal of the mouth and nose can be obtained.

[0050] This embodiment processes the target time-series signal to reconstruct the nasal and oral airflow signal. Dimensionality reduction projection removes redundant information, retains key features, reduces complexity and computational load, and avoids interference. Then, through result mapping, high-dimensional features are accurately converted into low-dimensional predictions, and the predicted values ​​from multiple time points are concatenated to accurately reconstruct the nasal and oral airflow signal.

[0051] In some optional embodiments of this example, step S2 involves feature fitting and linear mapping of the target time-series signal to obtain the apnea prediction result, including: The target time series signal is dimensionality reduced and projected to obtain a time series signal with a second preset dimension.

[0052] First, the target time-series signal is subjected to dimensionality reduction projection to summarize features. In this way, some unnecessary information can be removed, and the most critical features for reconstructing the nasal and oral airflow signal can be retained to obtain the time-series signal of the second preset dimension.

[0053] The time-series signal of the second preset dimension is mapped to obtain the apnea prediction result.

[0054] By projecting the time-series signal of the second preset dimension onto the 1-dimensional signal, the result of this 1-dimensional signal corresponds to the predicted value of whether the current time point is the time point of apnea. By concatenating the predicted values ​​of multiple time points (e.g., 36,000 time points), the apnea prediction result can be obtained.

[0055] This embodiment processes the target time-series signal to obtain apnea prediction results. Dimensional reduction projection removes redundant information and retains key features to obtain a second preset dimension time-series signal, reducing complexity, computational load, and avoiding interference. Then, through result mapping, the high-dimensional features are accurately converted into low-dimensional predictions, and the prediction values ​​at multiple time points are concatenated to accurately output apnea prediction results, assisting in training the reconstruction of the nasal and oral signals.

[0056] In some optional embodiments of this example, the neural network model includes a convolutional network, a bidirectional long short-term memory neural network, and an analysis model. The convolutional network is used to fuse multiple chest and abdominal motion signals to obtain a fused feature signal. The bidirectional long short-term memory neural network is used to capture the temporal features of the fused feature signal to obtain a target temporal signal. The analysis model is used to perform feature fitting and linear mapping on the target temporal signal to obtain a reconstructed oral and nasal airflow signal and a sleep apnea prediction result.

[0057] For example, the convolutional network includes a first convolutional layer, a first activation function layer, a first max pooling layer, a second convolutional layer, a second activation function layer, a second max pooling layer, and an output layer. The first convolutional layer is used to convolve the airflow signal from the mouth and nose to increase the channel dimension of the airflow signal. The first activation function layer is used to enhance the nonlinear fitting ability of the features processed by the first convolutional layer. The first max pooling layer is used to expand the receptive field of the second convolutional layer. The second convolutional layer is used to convolve the features processed by the first max pooling layer to further increase the channel dimension. The second activation function layer is used to enhance the nonlinear fitting ability of the features processed by the second convolutional layer. The second max pooling layer is used to expand the receptive field of the output layer. The output layer is used to output the fused feature signal.

[0058] like Figure 3As shown, a shallow convolutional network is used to fuse features of chest and abdominal motion signals with 40 distance gates. First, a first convolutional layer (Conv1D) with a kernel size of 3 is used to integrate the respiratory signal features of 40 channels. Here, 64 convolutional kernels are used, so the channel dimension increases to 64 after passing through the first convolutional layer (Conv1D). Then, the nonlinear fitting ability is enhanced by the first activation function layer (ReLU), followed by the first max pooling layer (MaxPool1D) (k=3, s=1, p=1) to expand the receptive field. The above process is set up in two layers, and 128 convolutional kernels are used in the second convolutional layer to further increase the dimension to express the features. Then, after passing through the second activation function layer (ReLU) and the second max pooling layer (MaxPool1D) (k=5, s=1, p=1), the final output layer outputs a 128×36000 fused feature signal, providing sufficient data support for subsequent temporal modeling.

[0059] The convolutional network in this embodiment consists of multiple convolutional layers, activation function layers, max pooling layers, and an output layer. Convolutional layers enhance the channel dimension of the nasal and oral airflow signals to represent features; activation function layers enhance the nonlinear fitting ability of the features; max pooling layers expand the receptive field of subsequent layers; and finally, a fused feature signal is output, providing sufficient data support for subsequent temporal modeling.

[0060] A bidirectional long short-term memory (Bi-LSTM) neural network is used to process the fused feature signal after convolutional network fusion. Bi-LSTM uses point-by-point output to ensure that the sampling frequency and signal length of the subsequent reconstructed signal are consistent with the input chest and abdominal motion signal. In this example, the data time dimension is 12 minutes, ensuring that Bi-LSTM can acquire sufficient temporal relationships for signal reconstruction. This example uses a two-layer Bi-LSTM, resulting in a final output signal dimension of 256×36000, used for subsequent signal reconstruction and apnea event prediction tasks.

[0061] This embodiment uses a bidirectional long short-term memory neural network (Bi-LSTM) to process the fused feature signal of the convolutional network. Point-by-point output ensures that the sampling frequency and length of the reconstructed signal are consistent with the input chest and abdominal motion signal. Sufficient data temporal dimension allows the Bi-LSTM to acquire adequate temporal relationships for signal reconstruction. The two-layer Bi-LSTM configuration allows the final output signal to be used for subsequent signal reconstruction and apnea event prediction tasks.

[0062] The analysis model includes a nasal airflow regression model and a sleep apnea event prediction model. The nasal airflow regression model includes a first linear layer, a third activation function layer, a first random deactivation layer, and a second linear layer. The first linear layer is used to perform dimensionality reduction projection on the target time series signal. The third activation function layer is used to enhance the nonlinear fitting ability of the features processed by the first linear layer. The first random deactivation layer is used to reduce the overfitting problem of the nasal airflow regression model. The second linear layer is used to map the results of the features processed by the first linear layer to obtain the reconstructed nasal airflow signal.

[0063] like Figure 4 As shown, the nasal airflow regression model first uses a first linear layer to project the target time-series signal with a signal dimension of 256×36000, projecting the original 256-dimensional target time-series signal to a first preset dimension (e.g., 64-dimensional) to summarize signal features. Then, a third activation function layer (ReLU) is used to enhance the nonlinear fitting ability of the features processed by the first linear layer. Next, a first random deactivation layer (Dropout) is used to reduce the overfitting problem of the nasal airflow regression model. Finally, a second linear layer is used to map the features processed by the first linear layer, projecting the 64-dimensional feature signal to 1 dimension, which corresponds to the predicted value of the nasal airflow signal at the current time point. By concatenating the predicted values ​​of multiple time points (e.g., 36000 time points), the reconstructed nasal airflow signal can be obtained.

[0064] The nasal airflow regression model in this embodiment consists of a first linear layer, a third activation function layer, a first random deactivation layer, and a second linear layer. The first linear layer performs dimensionality reduction projection on the target time-series signal to summarize features; the third activation function layer enhances the nonlinear fitting ability of the features; the first random deactivation layer reduces overfitting; and the second linear layer maps the results, ultimately obtaining the reconstructed nasal airflow signal. The nasal airflow regression model achieves effective processing of the target time-series signal through the collaborative work of each layer. Dimensionality reduction projection summarizes features, enhances nonlinear fitting ability, reduces the risk of overfitting, and ultimately accurately maps and outputs the reconstructed nasal airflow signal.

[0065] The sleep apnea event prediction model consists of a third linear layer, a fourth activation function layer, a second random deactivation layer, a fourth linear layer, and a prediction result mapping layer. The third linear layer is used to perform dimensionality reduction projection on the target time-series signal. The fourth activation function layer is used to enhance the nonlinear fitting ability of the features processed by the third linear layer. The second random deactivation layer is used to reduce the overfitting problem of the sleep apnea event prediction model. The fourth linear layer is used to map the features processed by the third linear layer. The prediction result mapping layer is used to output the result mapped by the fourth linear layer to obtain the sleep apnea prediction result.

[0066] likeFigure 5 As shown, the apnea event prediction model first uses a third linear layer to project the target time series signal with a signal dimension of 256×36000 to a second preset dimension (e.g., 32 dimensions) to summarize signal features. Then, a fourth activation function layer (ReLU) is used to enhance the nonlinear fitting ability of the features processed by the third linear layer. Next, a second random deactivation layer (Dropout) is used to reduce the overfitting problem of the apnea event prediction model. Finally, a fourth linear layer is used to map the features processed by the third linear layer, projecting the 32-dimensional feature signal to 1 dimension. Then, a prediction result mapping layer (Sigmoid) is used to output the apnea prediction result mapped by the fourth linear layer, which corresponds to the predicted value of whether the current time point is the apnea time point. By concatenating the prediction values ​​of multiple time points (e.g., 36000 time points), the apnea prediction result can be obtained.

[0067] The apnea event prediction model in this embodiment consists of a third linear layer, a fourth activation function layer, a second random deactivation layer, and a prediction result mapping layer. The model uses the third linear layer to reduce the dimensionality of the target time-series signal through projection and feature summarization. The fourth activation function layer enhances the nonlinear fitting ability of the features. The second random deactivation layer reduces overfitting. The fourth linear layer performs result mapping, and finally, the prediction result mapping layer outputs the apnea prediction result. This model can effectively process the target time-series signal, accurately identify apnea events during signal reconstruction, and has a more comprehensive signal reconstruction capability, helping to promptly detect potential apnea events and protect people's respiratory health.

[0068] In some optional implementations of this embodiment, the integration loss is calculated in the following manner: , in, Indicates integration loss, Indicates the weighting coefficient. Indicates the regression error. This indicates the error in predicting sleep apnea.

[0069] In this embodiment, regression error The apnea prediction error is calculated based on the mean absolute error between the reconstructed nasal and oral airflow signals and the actual nasal and oral airflow signals. The apnea prediction error is calculated based on the cross-entropy loss of the apnea prediction results and apnea labels. The regression error is integrated with the apnea prediction error. In this example, we set... The integrated loss is used for backpropagation and gradient updates. In this example, the Adam optimizer is used to optimize the model parameters, the learning rate can be set to 0.001, and the neural network model can stop training after 200 epochs, using an early stopping mechanism.

[0070] This embodiment integrates regression error and apnea prediction error for backpropagation and gradient update. It also employs an early apnea mechanism to optimize model parameters, effectively improving the training performance and efficiency of the neural network model.

[0071] This invention also provides a method for reconstructing airflow signals from the mouth and nose, which is executed by an electronic device such as a computer or server, and specifically includes: Acquire multiple target chest and abdominal motion signals collected by millimeter-wave radar.

[0072] The neural network model trained using the above-mentioned neural network model training method is used to process the chest and abdominal motion signals of multiple targets to obtain the reconstructed airflow signals of the mouth and nose of the targets.

[0073] This invention utilizes millimeter-wave radar to acquire chest and abdominal motion signals of a target object, enabling non-contact data acquisition, avoiding interference with the target object, and improving the convenience and comfort of data acquisition. The chest and abdominal motion signals are reconstructed using a trained neural network model to obtain the target's reconstructed nasal and oral airflow signals. Because the neural network model is trained through dual tasks, it can obtain nasal and oral airflow signals under normal breathing conditions as well as signal details under apnea conditions, resulting in more refined and comprehensive reconstructed nasal and oral airflow signals, thus providing strong support for related diagnosis and research.

[0074] Those skilled in the art will understand that embodiments of the present invention can be provided as methods, systems, or computer program products. Therefore, the present invention can take the form of a completely hardware embodiment, a completely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, the present invention can take the form of a computer program product embodied on one or more computer-usable storage media (including, but not limited to, disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0075] This invention is described with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of the invention. It will be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, special-purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, generate instructions for implementing the flowchart illustrations and / or block diagrams. Figure 1 One or more processes and / or boxes Figure 1 A device that provides the functions specified in one or more boxes.

[0076] These computer program instructions may also be stored in a computer-readable storage medium that can direct a computer or other programmable data processing device to function in a particular manner, such that the instructions stored in the computer-readable storage medium produce an article of manufacture including instruction means, which are implemented in a process Figure 1 One or more processes and / or boxes Figure 1 The function specified in one or more boxes.

[0077] These computer program instructions may also be loaded onto a computer or other programmable data processing equipment to cause a series of operational steps to be performed on the computer or other programmable equipment to produce a computer-implemented process, thereby providing instructions that execute on the computer or other programmable equipment for implementing the process. Figure 1 One or more processes and / or boxes Figure 1 The steps of the function specified in one or more boxes.

[0078] Obviously, the above embodiments are merely illustrative examples for clear explanation and are not intended to limit the implementation. Those skilled in the art will recognize that other variations or modifications can be made based on the above description. It is neither necessary nor possible to exhaustively list all possible implementations here. However, obvious variations or modifications derived therefrom are still within the scope of protection of this invention.

Claims

1. A method for training a neural network model, characterized in that, include: Acquire multiple chest and abdominal motion signals collected by millimeter-wave radar from the target object; The multiple chest and abdominal motion signals are processed using a neural network model to obtain reconstructed oral and nasal airflow signals and apnea prediction results. The neural network model performs feature fusion on the multiple chest and abdominal motion signals to obtain a fused feature signal. The fused feature signal is then subjected to temporal feature capture to obtain a target temporal signal. The target temporal signal is then subjected to feature fitting and linear mapping to obtain reconstructed oral and nasal airflow signals and apnea prediction results. The regression error is calculated based on the reconstructed nasal and oral airflow signals and the actual nasal and oral airflow signals; the apnea prediction error is calculated based on the apnea prediction results and the apnea labels. The integrated loss is obtained by weighting the regression error and the apnea prediction error. The neural network model is backpropagated and its gradients are updated based on the integration loss, and the model parameters are optimized.

2. The method according to claim 1, characterized in that, The multiple chest and abdominal movement signals include multiple distance gate breathing signals of the target object's chest and abdominal position movements, wherein each distance gate breathing signal is a signal sequence of a preset duration collected in the corresponding distance gate spatial region at a preset sampling rate.

3. The method according to claim 1, characterized in that, The calculation of regression error based on the reconstructed nasal airflow signal and the actual nasal airflow signal includes: Acquire real nasal and oral airflow signals from the target subject using polysomnography; The regression error is obtained by calculating the mean absolute error based on the reconstructed nasal and oral airflow signals and the actual nasal and oral airflow signals.

4. The method according to claim 1, characterized in that, The step of performing feature fitting and linear mapping on the target time-series signal to obtain the reconstructed oral and nasal airflow signal includes: The target time series signal is subjected to dimensionality reduction projection to obtain a time series signal of a first preset dimension; The reconstructed nasal airflow signal is obtained by mapping the time-series signal of the first preset dimension.

5. The method according to claim 1, characterized in that, The step of performing feature fitting and linear mapping on the target time-series signal to obtain the apnea prediction result includes: The target time series signal is subjected to dimensionality reduction projection to obtain a time series signal with a second preset dimension; The time-series signal of the second preset dimension is mapped to obtain the apnea prediction result.

6. The method according to claim 1, characterized in that, The neural network model includes a convolutional network, a bidirectional long short-term memory neural network, and an analysis model. The convolutional network is used to fuse the features of the multiple chest and abdominal motion signals to obtain a fused feature signal. The bidirectional long short-term memory neural network is used to capture the temporal features of the fused feature signal to obtain a target temporal signal. The analysis model is used to perform feature fitting and linear mapping on the target temporal signal to obtain a reconstructed oral and nasal airflow signal and a sleep apnea prediction result.

7. The method according to claim 1, characterized in that, The integration loss is calculated as follows: ， in, Indicates integration loss, Indicates the weighting coefficient. This represents the regression error. This indicates the error in predicting sleep apnea.

8. A method for reconstructing airflow signals from the mouth and nose, characterized in that, include: Acquire multiple target chest and abdominal motion signals collected by millimeter-wave radar from the target object; The neural network model trained using any one of claims 1-7 is used to process the chest and abdominal motion signals of the plurality of targets to obtain the reconstructed airflow signals of the mouth and nose of the targets.

9. A neural network model training device, characterized in that, include: A processor and a memory connected to the processor; wherein the memory stores instructions executable by the processor, the instructions being executed by the processor to cause the processor to perform the neural network model training method as described in any one of claims 1-7.

10. A device for reconstructing airflow signals from the mouth and nose, characterized in that, include: A processor and a memory connected to the processor; wherein the memory stores instructions executable by the processor, the instructions being executed by the processor to cause the processor to perform the oral and nasal airflow signal reconstruction method as described in claim 8.

Images