Attention-based thoracico-abdominal movement signal segmentation method and system and storage medium

Abstract

The invention relates to the technical field of biomedical radar signal processing and artificial intelligence crossing, in particular to an attention-based thoracico-abdominal movement signal segmentation method and system and a storage medium, and the segmentation method comprises the following steps: obtaining a physiological signal obtained by detecting a human body through a millimeter wave radar; performing feature extraction on the acquired physiological signals to obtain a multi-scale feature map; performing fusion of time domain attention, space domain attention and channel attention on the multi-scale feature map by using a constructed attention fusion model to obtain a fusion signal based on attention enhancement; and inputting the fusion signal based on attention enhancement into the segmentation network model to obtain a respiratory movement signal and a thoracico-abdominal movement signal. According to the method, the FMCW millimeter wave radar signal is processed by using the deep learning model, the related characteristics of the thoracic and abdominal movement are enhanced through an attention mechanism, the thoracic and abdominal movement signal is accurately segmented, and reliable input is provided for subsequent respiratory rate and heart rate calculation.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of biomedical radar signal processing and artificial intelligence, in particular to a chest and abdominal motion signal segmentation method and system based on attention and a storage medium. BACKGROUND

[0002] Respiration rate and heart rate are important vital signs of the human body, and real-time monitoring is of great significance for health management and early warning of diseases. Traditional detection methods (such as chest straps, optical sensors, etc.) usually require direct contact with the human body, which not only affects the comfort of the user, but also is limited by environmental conditions.

[0003] FMCW millimeter wave radar, as a non-contact and non-invasive monitoring method, can capture the tiny displacement of the body surface (such as chest and abdominal motion caused by respiration or heartbeat) by transmitting and receiving millimeter wave signals, and has great potential in the field of vital sign monitoring. However, due to the influence of background noise and multipath effect on radar signals, how to accurately separate the chest and abdominal motion signals from complex millimeter wave signals becomes a difficult point in the prior art. SUMMARY

[0004] The present application aims to overcome the defects of the prior art and provide a chest and abdominal motion signal segmentation method and system based on attention, which solves the difficult problem of how to accurately separate the chest and abdominal motion signals from millimeter wave signals.

[0005] The technical solution to achieve the above-mentioned purpose is: The present application provides a chest and abdominal motion signal segmentation method based on attention, comprising the following steps: Obtaining physiological signals obtained by a millimeter wave radar detecting a human body; Extracting features from the obtained physiological signals to obtain multi-scale feature maps; Constructing an attention fusion model, and using the constructed attention fusion model to fuse time domain attention, spatial domain attention and channel attention of the extracted multi-scale feature maps to obtain an attention-enhanced fusion signal; Providing a segmentation network model, inputting the attention-enhanced fusion signal into the provided segmentation network model, and then obtaining the respiration motion signal and the chest and abdominal motion signal output by the segmentation network model.

[0006] The further improvement of the chest and abdominal motion signal segmentation method based on attention of the present application is that the construction of the attention fusion model comprises the following steps: Constructing a time domain attention calculation model, where GRU is a gated recurrent unit, W tis a time sequence feature weight matrix, F t is a multi-scale feature map F l is a time sequence feature sequence compressed in a spatial dimension; constructing a spatial domain attention calculation model, wherein W s is a spatial convolution kernel parameter, F s is a multi-scale feature map F l is a spatial feature map aggregated in a time dimension; constructing a channel attention calculation model, wherein W c1 is a weight matrix of a global average pooling feature, W c2 is a weight matrix of a global maximum pooling feature, F c is a multi-scale feature map F l is a channel feature globally pooled in a space-time dimension; constructing a space-time cross-attention calculation model, , constructing a space-time correlation matrix, wherein d k is an attention dimension normalization factor, Q st is a space-time query matrix, K st is a space-time key matrix, W q is a query projection weight matrix, W k is a key projection weight matrix, H is a spatial dimension, C is a feature channel number, and T is a time frame number; constructing an attention fusion calculation model, wherein, is an element-level multiplication, is a tensor outer product, is an attention weight coefficient, and the multi-scale feature map extracted is fused in a time domain attention, a spatial domain attention, and a channel attention by using the constructed attention fusion calculation model to obtain a fusion signal based on attention enhancement.

[0007] The further improvement of the chest and abdominal motion signal segmentation method based on attention of the present application is that the segmentation network model comprises the following steps: Based on the U-Net network architecture, the encoder part is set as: wherein the DownBlock l comprises a convolution layer and a down-sampling: ; The decoder part of the jump connection and feature fusion is set as: wherein the UpBlock l comprises an up-sampling and feature splicing: , ; The split output layer is set as: , A three-dimensional probability map is generated: .

[0008] The further improvement of the chest and abdominal motion signal segmentation method based on attention is that the feature extraction of the obtained physiological signal comprises the following steps: A multi-scale feature extraction module is constructed: , is the original distance, speed and angle three-dimensional feature map, ResBlock l is a residual block, MaxPool l-1 is a maximum pooling; The original distance, speed and angle three-dimensional feature map is obtained by calculating the obtained physiological signal, and the calculation formula is as follows: The calculation formula of distance dimension data is , is the frequency of the difference frequency signal, , wherein is the frequency modulation rate, T c is the coherence processing time, and the distance resolution , c is the speed of light, and B is the bandwidth, is the obtained physiological signal, and the expression is: , wherein N is the target point number, A n is the amplitude of the nth target point, R n is the target distance of the nth target point, v n is the radial velocity of the nth target point, is the initial phase, is Gaussian white noise, is the wavelength; The calculation formula of speed dimension data is , wherein is the kth range spectrum, and the velocity resolution is , T s is the frame interval, and f d is the Doppler frequency; The calculation formula of angle dimension data is , wherein the angle resolution is , M is the number of antennas, d is the antenna spacing, S m is the range Doppler spectrum value output by the mth antenna channel, is the azimuth angle of the target relative to the normal direction of the radar antenna array, and m is the antenna index.

[0009] The further improvement of the chest and abdominal motion signal segmentation method based on attention is that the root mean square and the approximate entropy of the adjacent respiratory cycle difference of the obtained respiratory motion signal and the chest and abdominal motion signal are calculated to quantify the complexity of the respiratory mode. The calculation formula of the root mean square is: Wherein, is the duration of the i th respiratory cycle. The calculation formula of the approximate entropy is: Wherein, is the template matching probability.

[0010] The application further provides a storage medium, wherein the storage medium stores a program of the chest and abdominal motion signal segmentation method based on attention, When the program of the chest and abdominal motion signal segmentation method based on attention is executed by the processor, the steps of the chest and abdominal motion signal segmentation method based on attention are realized.

[0011] The application further provides a chest and abdominal motion signal segmentation system based on attention, comprising: An acquisition unit is configured to acquire physiological signals obtained by a millimeter wave radar when detecting a human body; A feature extraction unit is connected with the acquisition unit and configured to perform feature extraction on the acquired physiological signals to obtain a multi-scale feature map; An attention fusion model is connected with the feature extraction unit and configured to perform fusion of time domain attention, space domain attention and channel attention on the extracted multi-scale feature map to obtain an enhanced fusion signal based on attention enhancement; A segmentation network model is connected with the attention fusion model and configured to perform segmentation on the enhanced fusion signal to obtain a respiratory motion signal and a chest and abdominal motion signal.

[0012] The further improvement of the chest and abdominal motion signal segmentation system based on attention is that the attention fusion model comprises: A time domain attention calculation model, Wherein, GRU is a gated recurrent unit, W t is a time sequence feature weight matrix, F t is a multi-scale feature map F l compressed in a spatial dimension; A space domain attention calculation model, Wherein, W s is a spatial convolution kernel parameter, F s is a multi-scale feature map F l aggregated in a time dimension; A channel attention calculation model, Wherein, Wc1 W is a weight matrix of the global average pooling feature c2 F is a weight matrix of the global maximum pooling feature c F is a multi-scale feature map l Channel features after spatio-temporal dimension global pooling A spatio-temporal cross-attention calculation model, A spatio-temporal correlation matrix, where d k Q is an attention dimension normalization factor, st K is a spatio-temporal query matrix, st W is a spatio-temporal key matrix, q W is a query projection weight matrix, k H is a key projection weight matrix, W is a spatial dimension, C is a feature channel number, and T is a time frame number; An attention fusion calculation model, wherein, is an element-level multiplication, is a tensor outer product, is an attention weight coefficient, which is used to fuse the time domain attention, spatial domain attention and channel attention of the extracted multi-scale feature map to obtain a fusion signal based on attention enhancement.

[0013] The further improvement of the chest and abdominal motion signal segmentation system based on attention of the application is that the feature extraction unit comprises a multi-scale feature extraction module, and the multi-scale feature extraction module is: wherein, is a three-dimensional feature map of original distance, speed and angle, and ResBlock l is a residual block, and MaxPool l-1 is maximum pooling; The three-dimensional feature map of original distance, speed and angle is obtained by calculating the acquired physiological signal, and the calculation formula is as follows: The calculation formula of distance dimension data is , is the frequency of the difference frequency signal, wherein, is a frequency modulation rate, Tc is a coherence processing time, and the distance resolution c is the speed of light, and B is the bandwidth, is the acquired physiological signal, and the expression is: wherein N is the target point number, A n is the amplitude of the nth target point, R n is the target distance of the nth target point, v n is the radial velocity of the nth target point, ​is an initial phase, is a Gaussian white noise, is a wavelength; The calculation formula of the velocity dimension data is wherein is the kth frame distance spectrum, and the velocity resolution is , T s is a frame interval, f d is a Doppler frequency; The calculation formula of the angle dimension data is wherein the angle resolution is , M is the number of antennas, d is the antenna spacing, S m is the distance Doppler spectrum value output by the mth antenna channel, is the azimuth angle of the target relative to the normal direction of the radar antenna array, and m is the antenna index.

[0014] Further improvement of the chest and abdominal motion signal segmentation system based on attention of the application is that it further comprises a quantitative calculation unit connected with the segmentation network model, which is used for calculating the root mean square of the difference value of adjacent breathing cycles and the approximate entropy of the obtained respiratory motion signal and chest and abdominal motion signal to quantify the complexity of the breathing pattern. The calculation formula of the root mean square is: wherein is the duration of the ith breathing cycle; The calculation formula of the approximate entropy is: wherein is the template matching probability.

[0015] The chest and abdominal motion signal segmentation method, system and storage medium based on attention of the application have the following beneficial effects: The chest and abdominal motion signal segmentation method, system and storage medium based on attention of the application utilize a deep learning model to process FMCW millimeter wave radar signals, enhance the relevant features of chest and abdominal motion through an attention mechanism, accurately segment the chest and abdominal motion signals, and provide reliable input for subsequent breathing frequency and heart rate calculation.

[0016] The chest and abdominal motion signal segmentation method, system and storage medium based on attention of the application are particularly suitable for sleep respiration monitoring, chronic obstructive pulmonary disease (COPD) evaluation and postoperative rehabilitation exercise analysis, and provide high spatiotemporal resolution quantitative biomarker data for clinical decision-making. BRIEF DESCRIPTION OF DRAWINGS

[0017] Figure 1 is a flowchart of the chest and abdominal motion signal segmentation method based on attention of the application.

[0018] Figure 2A system diagram of the chest and abdominal motion signal segmentation system based on attention of the present application. DETAILED DESCRIPTION

[0019] The present application will be further described below in conjunction with the drawings and specific embodiments.

[0020] Referring to Figure 1 , the present application provides a chest and abdominal motion signal segmentation method and system based on attention, which is used to solve the technical difficulties of chest and abdominal motion segmentation in millimeter wave radar signals. The present application aims to use deep learning technology combined with attention mechanism to efficiently and accurately segment chest and abdominal motion from physiological signals (such as respiratory signals, body surface motion signals, etc.), and to provide accurate data and feature support for respiratory monitoring, disease diagnosis and rehabilitation evaluation models. The chest and abdominal motion signal segmentation method and system based on attention of the present application will be described below in conjunction with the drawings.

[0021] Referring to Figure 2 , a system diagram of the chest and abdominal motion signal segmentation system based on attention of the present application is shown. The chest and abdominal motion signal segmentation system based on attention of the present application will be described below in conjunction with Figure 2 .

[0022] As shown in Figure 2 , the chest and abdominal motion signal segmentation system based on attention of the present application includes an acquisition unit 21, a feature extraction unit 22, an attention fusion model 23 and a segmentation network model 24, wherein the acquisition unit 21 is connected with the feature extraction unit 22, the feature extraction unit 22 is connected with the attention fusion model 23, and the attention fusion model 23 is connected with the segmentation network model 24. The acquisition unit 21 is used to acquire physiological signals obtained by a millimeter wave radar detecting a human body, which includes respiratory signals, body surface motion signals, etc. The millimeter wave radar can emit and receive millimeter wave signals to capture small displacements on the surface of the human body, such as chest and abdominal movements caused by breathing or heartbeat.

[0023] The FMCW millimeter wave radar collects data by emitting a linear frequency modulation signal, and its emission signal can be expressed as: , wherein, is the initial frequency, is the frequency modulation, , B is the bandwidth, T is the scanning period, A is the amplitude, and j is the imaginary unit, which is used to construct the complex domain FMCW millimeter wave radar difference frequency signal. The core function is to completely retain the signal change information caused by small movements of the chest and abdomen, such as breathing, heartbeat fluctuations, etc. Compared with the problem that traditional real number domain signal processing easily loses details, the complex representation provides key phase dimension data support for subsequent accurate segmentation of chest and abdominal motion signals, and is the basic mathematical carrier for realizing high-resolution extraction of chest and abdominal motion.

[0024] The difference frequency signal of the mixed signal is: , Wherein, N is the target point number, A n is the amplitude of the nth target point, R n is the target distance of the nth target point, v n is the radial velocity of the nth target point, is the initial phase, is the Gaussian white noise, is the wavelength, determined by the center frequency of the FMCW millimeter wave radar used The center frequency of the radar is its inherent parameter, which is determined when the radar is designed and produced, and the wavelength can be calculated according to the relationship between the wavelength and the frequency ; The difference frequency signal is the physiological signal to be segmented by the chest and abdominal motion signal segmentation system based on attention of the application, and the difference frequency signal is one of the millimeter wave signals received by the millimeter wave radar.

[0025] The feature extraction unit 22 is used for feature extraction of the acquired physiological signal to obtain a multi-scale feature map; The attention fusion model 23 is used for fusion of time domain attention, spatial domain attention and channel attention on the extracted multi-scale feature map to obtain an attention-enhanced fusion signal; The segmentation network model 24 is used for segmentation of the enhanced fusion signal to obtain a respiratory motion signal and a chest and abdominal motion signal.

[0026] The chest and abdominal motion signal segmentation system based on attention of the application further comprises a processing unit connected with the above-mentioned acquisition unit 21, feature extraction unit 22, attention fusion model 23 and segmentation network model 24, the processing unit is used for sending the physiological signal acquired by the acquisition unit 21 to the feature extraction unit 22 for feature extraction, the processing unit is also used for sending the extracted features to the attention fusion model 23 for fusion of time domain attention, spatial domain attention and channel attention to obtain an attention-enhanced fusion signal, and the processing unit is also used for sending the enhanced fusion signal to the segmentation network model 24 for segmentation, and then receiving the respiratory motion signal and the chest and abdominal motion signal output by the segmentation network model 24.

[0027] In one specific embodiment of the application, the feature extraction unit 22 comprises a multi-scale feature extraction module, which is: , is the original distance, velocity and angle three-dimensional feature map, ResBlock l is a residual block, MaxPool l-1 is a maximum pooling; wherein the original distance, velocity and angle three-dimensional feature maps are obtained by calculating the acquired physiological signals, and the calculation formula is as follows: The calculation formula of the distance dimension data is , is the frequency of the difference frequency signal, wherein, is the frequency modulation, Tc is the coherent processing time, the distance resolution , c is the speed of light, B is the bandwidth, is the acquired physiological signal, and the expression is: wherein N is the target point number, A n is the amplitude of the nth target point, R n is the target distance of the nth target point, v n is the radial velocity of the nth target point, is the initial phase, is the Gaussian white noise, is the wavelength, determined by the center frequency of the FMCW millimeter wave radar used The center frequency of the radar is its inherent parameter, which has been determined when the radar is designed and produced, and the wavelength can be calculated according to the relationship between the wavelength and the frequency ; The calculation formula of the velocity dimension data is wherein is the kth range profile, and the velocity resolution is , T s is the frame interval, f d is the Doppler frequency (unit: Hz), used to represent the frequency offset caused by the radial motion of the target; The calculation formula of the angle dimension data is wherein the angle resolution is , M is the number of antennas, d is the antenna spacing, S m is is the range Doppler spectrum value output by the mth antenna channel, that is, the complex spectrum component after the signal received by the antenna is processed in the range dimension and the velocity dimension, is the azimuth angle of the target relative to the normal direction of the radar antenna array, and m is the antenna index.

[0028] ResBlock l contains convolution layers, batch normalization and ReLU functions, and the method is as follows: ReLU activation: , Residual connection: . wherein, This is a learnable weight matrix for the i-th convolutional layer in the l-th residual block (ResBlock). Its dimension matches the number of channels in the input feature map and the size of the convolutional kernel. During training, it can adaptively learn key features of chest and abdominal movement, such as the local signal differences in chest cavity rise and fall and abdominal cavity expansion during the respiratory cycle. This overcomes the limitation of traditional fixed-weight filtering in capturing dynamic features of chest and abdominal movement, and provides an optimizable parameter basis for the accurate extraction of multi-scale chest and abdominal movement features. This weight matrix and the attention weight coefficients (α, β, γ, δ) in the model are iteratively updated through an adaptive gradient algorithm with the goal of minimizing the cross-entropy loss function, that is, minimizing the deviation between the chest and abdominal movement segmentation results and the labeled data, and finally converge to the optimal value that is suitable for the chest and abdominal movement segmentation task. The output feature map of the i-th layer in the l-th residual block (ResBlock) is obtained by " The multidimensional tensor, processed by "convolutional feature extraction, batch normalization (BN) to eliminate gradient offset, and ReLU activation to enhance nonlinear expression," has dimensions [B, H, W, C], where B is the batch frame number, H / W is the spatial dimension, and C is the number of feature channels. Its core function is to preserve and enhance chest and abdominal motion-related features layer by layer, such as the chest and abdominal cavity boundary features in the distance dimension and the respiratory motion rate features in the velocity dimension. This lays a high-quality feature foundation for the subsequent "three-way attention mechanism" to focus on key regions and features. The value of i is the index of the convolutional layer inside the l-th residual block (ResBlock), satisfying 1≤i≤K (K is the total number of convolutional layers in a single ResBlock, and the value of K is related to the actual device parameters). This layered design can achieve gradual enhancement of chest and abdominal motion features. The shallow layer can capture basic distance / velocity features, and the deep layer can extract subtle differences in chest and abdominal cavity motion features, adapting to the multi-scale and non-stationary signal characteristics of chest and abdominal motion and avoiding the problem of insufficient feature extraction at a single level.

[0029] In one specific embodiment of the present invention, the attention fusion model 23 includes: Temporal attention computation model In this context, GRU stands for Gated Cyclic Unit, which captures the temporal dependence of respiratory movements and outputs temporal weights At to enhance feature differentiation between the inspiratory and expiratory phases. Wt is the temporal feature weight matrix. C represents the number of feature channels, used to perform linear transformation on the temporal features output by the GRU; The data is obtained automatically through the model training process. During the training phase, the cross-entropy loss function of chest and abdominal motion segmentation is used as the optimization objective, and the Adam optimizer is employed, with iterative updates via backpropagation. The parameter values ​​are calculated until the model converges, and Ft is the multi-scale feature map F. l Temporal feature sequences after spatial dimension compression T is the number of time frames, and C is the number of feature channels; a spatial attention calculation model, where Ws is a spatial convolution kernel parameter, k is the size of the convolution kernel, and C is the number of feature channels; The acquisition method is: automatically learned through the model training process. In the training stage, the segmentation loss function is taken as the target, and the gradient descent iteration is updated until the segmentation accuracy of the model on the verification set no longer improves, F s is a multi-scale feature map F l a spatial feature map aggregated in the time dimension, HxW is the spatial size, and C is the number of feature channels. By aggregating the time dimension information, the convolution layer can more accurately locate the spatial boundary of the chest and abdominal region. When performing spatial attention calculation, the convolution kernel can be adjusted according to the sampling rate, or it can be artificially set. On the premise of meeting the calculation accuracy requirement, the convolution kernel is as small as possible to reduce the calculation amount. For example, if the sampling rate is 40Hz, a 3x3 convolution kernel can be set to generate a spatial weight map to highlight the features of the chest and abdominal region and accurately locate the boundary region of the ribs and abdominal cavity.

[0030] a channel attention calculation model, where W c1 is a weight matrix of the global average pooling feature, W c2 is a weight matrix of the global maximum pooling feature, r is the channel compression rate; and The acquisition method is: automatically learned through the model end-to-end learning. In the training process, the cross-entropy loss combined with the chest and abdominal phase difference consistency loss is taken as the total target, and the parameters are updated through back propagation until the model converges. Fc is a channel feature of a multi-scale feature map Fl aggregated in the global space-time dimension, C is the number of feature channels. This channel attention mechanism is used to enhance the features related to breathing.

[0031] a space-time cross-attention calculation model, , a space-time correlation matrix, where d k is an attention dimension normalization factor, Q st is a space-time query matrix, used to represent the query features of each time frame, K st is a space-time key matrix, used to represent the key features of each spatial position, and W q is a query projection weight matrix, W is used to map input features to a query matrix. k The key projection weight matrix, , is used to map input features to a key matrix, where H×W is the spatial size, C is the number of feature channels, and T is the number of time frames. The acquisition method: obtained through self-learning during the model training process. During the training phase, these values ​​are compared with those in the model. The optimization process involves multiple components, with the overall objective being a combination of the cross-entropy loss function for chest and abdominal motion segmentation and a physiological constraint loss. The Adam optimizer is employed, iteratively updating parameter values ​​via backpropagation until the model's segmentation accuracy on the validation set no longer improves, at which point convergence is achieved. . It is through input features and respectively Obtained through matrix operations. Specifically, the processed features containing spatiotemporal information are respectively compared with... Multiplying, and after adjusting for the appropriate dimensions, yields... This information is then used to calculate the subsequent spatiotemporal correlation matrix to capture the dynamic correlation features between time frames and spatial locations. By calculating the feature similarity between each time frame and spatial location using the spatiotemporal correlation matrix, dynamic correlations such as "the change of the anterior convexity area of ​​the chest cavity over time during inhalation" can be captured.

[0032] Attention fusion computation model This is used to fuse temporal attention, spatial attention, and channel attention into the extracted multi-scale feature maps to obtain an attention-enhanced fused signal. This is element-wise multiplication. For tensor outer product, These are the attention weight coefficients; spatiotemporal-channel attention weight fusion is achieved through tensor outer product, and the weight coefficients are optimized using an adaptive gradient algorithm. , in, This is element-wise division, EMA is the exponential moving average, and η is the learning rate. It is the minimum constant. This is the cross-entropy loss function.

[0033] In one specific embodiment of the present invention, the segmentation network model 24 adopts a variant of the U-Net network architecture, that is, based on the U-Net network architecture, the encoder part is set as follows: DownBlock l Includes convolutional layers and downsampling: ; The decoder part for skip connections and feature fusion is configured as follows: UpBlock lThe upsampling and feature stitching are included: , ; The split output layer is set as: , The three-dimensional probability map is generated: , which represents the segmentation probability of the background, chest cavity and abdominal cavity three types of regions.

[0034] When the attention-enhanced fusion signal is segmented, the attention-enhanced fusion signal is input into the segmentation network model 24, the segmentation network model 24 generates a three-dimensional probability map for the attention-enhanced fusion signal, and then the three-dimensional probability map is spatio-temporally integrated to extract the respiratory motion signal:

[0035] Wherein, r i is a spatial coordinate, is a segmentation probability.

[0036] The Hilbert transform is used to calculate the phase, and then the Kalman filter is used for smoothing, so that the thoraco-abdominal phase difference can be obtained:

[0037] Wherein, F is a state transition matrix, H is an observation matrix, is a Kalman gain.

[0038] In one specific embodiment of the present application, the attention-based thoraco-abdominal motion signal segmentation system of the present application further comprises a quantitative calculation unit connected with the segmentation network model, for calculating the root mean square of the difference value of adjacent respiratory cycles and the approximate entropy of the obtained respiratory motion signal and thoraco-abdominal motion signal, to quantify the complexity of the respiratory pattern. The calculation formula of the root mean square is: , wherein is the duration of the i th respiratory cycle. The calculation formula of the approximate entropy is: , wherein is the template matching probability.

[0039] The segmentation system of the present application realizes the accurate segmentation of thoraco-abdominal motion based on spatio-temporal attention segmentation, and provides reliable technical support for respiratory monitoring and disease diagnosis.

[0040] The present application also provides an attention-based thoraco-abdominal motion signal segmentation method, comprising the following steps: Obtaining physiological signals obtained by detecting human body by millimeter wave radar; Performing feature extraction on the obtained physiological signals to obtain multi-scale feature maps; An attention fusion model is constructed, and the multi-scale feature maps extracted are fused in time domain attention, space domain attention and channel attention by using the constructed attention fusion model to obtain a fusion signal based on attention enhancement. A segmentation network model is provided, and the fusion signal based on attention enhancement is input into the provided segmentation network model to obtain a respiratory motion signal and a thoraco-abdominal motion signal output by the segmentation network model.

[0041] In one specific embodiment of the present application, the construction of the attention fusion model includes the following steps: A time domain attention calculation model is constructed, wherein GRU is a gated recurrent unit, W t is a time sequence feature weight matrix, F t is a multi-scale feature map F l is a time sequence feature sequence compressed in spatial dimension; A space domain attention calculation model is constructed, wherein W s is a spatial convolution kernel parameter, F s is a multi-scale feature map F l is a spatial feature map aggregated in time dimension; A channel attention calculation model is constructed, wherein W c1 is a weight matrix of global average pooling features, W c2 is a weight matrix of global maximum pooling features, F c is a multi-scale feature map F l is a channel feature globally pooled in time and space dimensions; A time-space cross attention calculation model is constructed, , A time-space correlation matrix is constructed, wherein d k is an attention dimension normalization factor, Q st is a time-space query matrix, K st is a time-space key matrix, W q is a query projection weight matrix, W k is a key projection weight matrix, HxW is a spatial dimension, C is a feature channel number, and T is a time frame number; An attention fusion calculation model is constructed, wherein, is an element-level multiplication, is a tensor outer product, is an attention weight coefficient, and the multi-scale feature maps extracted are fused in time domain attention, space domain attention and channel attention by using the constructed attention fusion calculation model to obtain a fusion signal based on attention enhancement.

[0042] In one specific embodiment of the present application, the segmentation network model is provided comprising the following steps: Based on the U-Net network architecture, the encoder part is set as: DownBlock l which contains convolutional layers and down-sampling: ; The decoder part of the jump connection and feature fusion is set as: UpBlock l which contains up-sampling and feature concatenation: , ; The segmentation output layer is set as: , to generate a three-dimensional probability map: .

[0043] When segmenting the attention-enhanced fusion signal, the attention-enhanced fusion signal is input into the segmentation network model, the segmentation network model generates a three-dimensional probability map from the attention-enhanced fusion signal, and then performs spatio-temporal integration on the three-dimensional probability map to extract the respiratory motion signal:

[0044] where r i is the spatial coordinate, is the segmentation probability.

[0045] After calculating the phase by Hilbert transform and smoothing by Kalman filter, the thoraco-abdominal phase difference can be obtained:

[0046] where F is the state transition matrix, H is the observation matrix, is the Kalman gain.

[0047] In one specific embodiment of the present application, the feature extraction of the acquired physiological signal comprises the following steps: A multi-scale feature extraction module is constructed: wherein, is the original distance, velocity and angle three-dimensional feature map, ResBlock l is a residual block, MaxPool l-1 is a maximum pooling; The original distance, velocity and angle three-dimensional feature map is obtained by calculating the acquired physiological signal, and the calculation formula is as follows: The calculation formula of distance dimension data is , is the frequency of the difference frequency signal, ,in, To adjust the frequency, T c For coherent processing time, distance resolution c is the speed of light, and B is the bandwidth. The acquired physiological signal is expressed as: Where N is the number of target points, A n R is the magnitude of the nth target point. n v is the target distance for the nth target point. n Let n be the radial velocity of the nth target point. For the initial phase, It is Gaussian white noise. Wavelength; The formula for calculating velocity dimension data is as follows: ,in Let k be the range spectrum, and let velocity resolution be... T s f is the frame interval. d The frequency is the Doppler frequency. The formula for calculating angular dimension data is as follows: , where the angular resolution is M represents the number of antennas, d represents the antenna spacing, and Sm represents the range Doppler spectrum value output by the m-th antenna channel. Let m be the azimuth angle of the target relative to the normal direction of the radar antenna array, and m be the antenna index.

[0048] In one specific embodiment of the present invention, the method further includes calculating the root mean square and approximate entropy of the difference between adjacent respiratory cycles on the obtained respiratory motion signal and chest and abdominal motion signal, so as to quantify the complexity of the respiratory pattern. The formula for calculating the root mean square is: ,in The duration of the i-th respiratory cycle; The formula for calculating approximate entropy is:

[0049] ,in This represents the template matching probability.

[0050] The present invention also provides a storage medium storing a program for an attention-based chest and abdominal motion signal segmentation method, wherein the program for the attention-based chest and abdominal motion signal segmentation method is executed by a processor and implements the steps of the attention-based chest and abdominal motion signal segmentation method.

[0051] The application is described in detail above with reference to the drawings. Those skilled in the art can make various changes to the application according to the above description. Therefore, some details in the embodiments should not be regarded as limiting the application, and the scope of protection of the application is defined by the appended claims.

Claims

1. An attention-based thoraco-abdominal motion signal segmentation method, characterized in that, The method comprises the following steps: acquiring physiological signals obtained by detecting a human body by a millimeter wave radar; extracting features of the acquired physiological signals to obtain multi-scale feature maps; constructing an attention fusion model, and fusing time domain attention, space domain attention and channel attention of the extracted multi-scale feature maps by using the constructed attention fusion model to obtain an attention-enhanced fusion signal; providing a segmentation network model, inputting the attention-enhanced fusion signal into the provided segmentation network model, and then obtaining respiratory motion signals and thoraco-abdominal motion signals output by the segmentation network model.

2. The attention-based thoraco-abdominal motion signal segmentation method of claim 1, wherein, The method of constructing the attention fusion model comprises the following steps: a time domain attention calculation model is constructed, wherein GRU is a gating recurrent unit, W t is a time sequence feature weight matrix, F t is a multi-scale feature map F l a time sequence feature sequence compressed in a spatial dimension constructing a spatial attention calculation model, wherein W s for spatial convolution kernel parameters, F s for multi-scale feature maps F l spatial feature maps aggregated through a time dimension constructing a channel attention calculation model, wherein W c1 is a weight matrix of the global average pooling feature, W c2 is a weight matrix of the global maximum pooling feature, F c is a multi-scale feature map F l channel features after spatio-temporal dimension global pooling; constructing a spatio-temporal cross-attention computational model, , constructing a spatio-temporal correlation matrix, where d k is an attention dimension normalization factor, Q st is a spatio-temporal query matrix, K st is a spatio-temporal key matrix, W q is a query projection weight matrix, W k is a key projection weight matrix, H W is a spatial dimension, C is a number of feature channels, T is a number of time frames;​ constructing an attention fusion calculation model, wherein, is an element-level multiplication, is a tensor outer product, is an attention weight coefficient, and the multi-scale feature maps extracted are fused by using the constructed attention fusion calculation model to obtain a fusion signal based on attention enhancement.

3. The attention-based thoraco-abdominal motion signal segmentation method of claim 1, wherein, The method of providing the segmentation network model comprises the following steps: Based on the U-Net network architecture, the encoder part is set as: DownBlock l contains a convolutional layer and down-sampling: ; The decoder part of the skip connection and feature fusion is set as: UpBlock l contains upsampling and feature stitching: , ; The set partition output layer is set as: , generating a three-dimensional probability map: .

4. The attention-based thoraco-abdominal motion signal segmentation method of claim 1, wherein, The method of extracting features of the acquired physiological signals comprises the following steps: A multi-scale feature extraction module is constructed: wherein, is the original distance, speed and angle three-dimensional feature map, ResBlock l is a residual block, MaxPool l-1 is a maximum pooling; The original distance, speed and angle three-dimensional feature maps are obtained by calculating the acquired physiological signals, and the calculation formula is as follows: The formula for calculating distance dimension data is as follows: , The frequency of the difference frequency signal. ,in, To adjust the frequency, T c For coherent processing time, distance resolution , c At the speed of light, B For bandwidth, The acquired physiological signal is expressed as: ,in N For the target number of points, A n For the first n The magnitude of each target point R n For the first n The target distance of each target point v n For the first n The radial velocity of each target point For the initial phase, It is Gaussian white noise. Wavelength; The calculation formula of the velocity dimension data is wherein is the distance spectrum of the first k frame, and the velocity resolution is , T s is the frame interval, f d is the Doppler frequency; The calculation formula of the angle dimension data is , wherein the angle resolution is , M is the number of antennas, d is the antenna spacing, S m is the distance Doppler spectrum value output by the mth antenna channel, is the azimuth angle of the target relative to the normal direction of the radar antenna array, m is the antenna index.

5. The attention-based thoraco-abdominal motion signal segmentation method of claim 1, wherein, The method further comprises calculating a root mean square and an approximate entropy of a difference value of adjacent respiratory cycles of the obtained respiratory motion signals and thoraco-abdominal motion signals to quantify a respiratory pattern complexity; The formula for the root mean square is: where is the duration of the i th breathing cycle; The calculation formula of the approximate entropy is as follows: where is the template matching probability.

6. A storage medium, characterized by The storage medium stores a program of an attention-based thoraco-abdominal motion signal segmentation method, The program of the attention-based thoraco-abdominal motion signal segmentation method is executed by the processor to realize the steps of the attention-based thoraco-abdominal motion signal segmentation method in any one of claims 1 to 5.

7. An attention-based thoraco-abdominal motion signal segmentation system, characterized by, The method comprises: an acquisition unit configured to acquire physiological signals obtained by detecting a human body by a millimeter wave radar; a feature extraction unit connected with the acquisition unit and configured to extract features of the acquired physiological signals to obtain multi-scale feature maps; an attention fusion model connected with the feature extraction unit and configured to fuse time domain attention, space domain attention and channel attention of the extracted multi-scale feature maps to obtain an attention-enhanced fusion signal; a segmentation network model connected with the attention fusion model and configured to segment the enhanced fusion signal to obtain respiratory motion signals and thoraco-abdominal motion signals.

8. The attention-based chest-abdominal movement signal segmentation system of claim 7, wherein, The attention fusion model comprises: a time domain attention computation model, wherein GRU is a gated recurrent unit, W t is a temporal feature weight matrix, F t is a multi-scale feature map F l a time series of spatial dimension compressed temporal features; spatial attention computation model, wherein W s for spatial convolution kernel parameters, F s for multi-scale feature maps F l spatial feature maps aggregated along the temporal dimension; Channel attention computation model, wherein W c1 is a weight matrix for the global average pooling feature, W c2 is a weight matrix for the global max pooling feature, F c is a multi-scale feature map F l channel feature after spatio-temporal dimension global pooling; spatio-temporal cross-attention computation model, , spatial-temporal correlation matrix, where d k is an attention dimension normalization factor, Q st is a spatial-temporal query matrix, K st is a spatial-temporal key matrix, W q is a query projection weight matrix, W k is a key projection weight matrix, H W is a spatial dimension, C is a number of feature channels, T is a number of time frames;​ Attention fusion computing model, wherein, is an element-wise multiplication, is a tensor outer product, is an attention weight coefficient, used for fusing the extracted multi-scale feature maps in time domain attention, spatial domain attention and channel attention to obtain an attention-enhanced fusion signal.

9. The attention-based chest-abdominal movement signal segmentation system of claim 7, wherein, The feature extraction unit comprises a multi-scale feature extraction module, and the multi-scale feature extraction module is: wherein, is an original distance, speed and angle three-dimensional feature map, ResBlock l is a residual block, MaxPool l-1 is maximum pooling; The original distance, speed and angle three-dimensional feature maps are obtained by calculating the acquired physiological signals, and the calculation formula is as follows: The formula for calculating distance dimension data is as follows: , The frequency of the difference frequency signal. ,in, To adjust the frequency, T c For coherent processing time, distance resolution , c At the speed of light, B For bandwidth, The acquired physiological signal is expressed as: ,in N For the target number of points, A n For the first n The magnitude of each target point R n For the first n The target distance of each target point v n For the first n The radial velocity of each target point For the initial phase, It is Gaussian white noise. Wavelength; The calculation formula of the velocity dimension data is wherein is the first k frame distance spectrum, and the velocity resolution is , T s is the frame interval, f d is the Doppler frequency; The calculation formula of the angle dimension data is , wherein the angle resolution is , M is the number of antennas, d is the antenna spacing, S m is the distance Doppler spectrum value output by the mth antenna channel, is the azimuth angle of the target relative to the normal direction of the radar antenna array, m is the antenna index.

10. The attention-based chest-abdominal movement signal segmentation system of claim 7, wherein, The method further comprises a quantitative calculation unit connected with the segmentation network model and configured to calculate a root mean square and an approximate entropy of a difference value of adjacent respiratory cycles of the obtained respiratory motion signals and thoraco-abdominal motion signals to quantify a respiratory pattern complexity; The formula for the root mean square is: where is the duration of the i th breathing cycle; The calculation formula of approximate entropy is: wherein is the template matching probability.

Images