Non-contact fetal movement monitoring and identifying method and system based on millimeter wave radar

Abstract

The invention discloses a non-contact fetal movement monitoring identification method and system based on a millimeter wave radar, and the radar technology and deep learning are combined and applied to the field of fetal movement monitoring, and the millimeter wave radar is used for transmitting a detection signal to the abdomen of a pregnant woman and receiving an echo signal; then a series of operations such as scattering separation, time-frequency spectrogram generation, feature extraction, fetal movement recognition and storage are performed, and the advantages of the radar technology and deep learning are fully exerted. Complicated features in fetal movement signals are automatically learned and extracted through a deep learning algorithm, the recognition capability and accuracy of the fetal movement signals are improved, the method adapts to individual differences of different pregnant women and fetuses, more personalized fetal movement monitoring services are provided for the pregnant women, and higher convenience and reliability are achieved. The fetal movement condition can be accurately monitored in real time, abnormal fetal movement can be found in time, timely and accurate fetal health information can be provided for pregnant women and doctors, the risk of fetuses can be reduced, and the quality and level of fetal monitoring in the perinatal period can be improved.

Description

Technical Field

[0001] The present invention relates to the technical field of fetal electronic monitoring, and in particular to a non-contact fetal movement monitoring and identification method and system based on millimeter-wave radar. Background Art

[0002] In the monitoring of the health status of fetuses during the perinatal period, fetal movement is a key clinical indicator. The currently widely used and relatively reliable method in hospitals is to use ultrasonic equipment to image the fetus in order to observe the activities of fetal movement in utero in real time. However, this method is expensive and must be carried out with the assistance of professionals. At present, routine prenatal examinations in clinical practice are usually carried out once every 2-3 weeks. However, once the fetal movement decreases, the fetus may die within a very short period of time. When the pregnant woman notices the abnormality and goes to the hospital for examination, the fetus may have already been harmed. Therefore, the current main method for monitoring fetal movement is for pregnant women to regularly record the number of fetal movements subjectively every day by themselves. However, due to the uncertainty of fetal movement, pregnant women need to spend time learning how to accurately count fetal movements, and even after certain training, the reliability of this self-counting method will still fluctuate greatly due to various factors such as the living habits of pregnant women and the degree of concentration of attention. In addition, this method requires pregnant women to lie still for a long time and concentrate, but many pregnant women cannot persist or are hindered from self-monitoring due to reasons such as inconvenience in work and life.

[0003] In addition, existing wearable fetal movement monitoring devices have problems such as discomfort for pregnant women to wear, inconvenient wearing, easy interference of signals, and low recognition accuracy. And radar, as a non-contact measurement technology, has unique advantages such as strong penetration ability for materials such as clothing, good environmental adaptability, all-weather detection, and privacy protection.

[0004] Deep learning has significant advantages in fetal movement monitoring. It can automatically learn and extract complex features in fetal movement signals, improving the recognition ability and accuracy of fetal movement signals. Through the learning and analysis of a large amount of fetal movement data, the deep learning model can continuously optimize itself to adapt to the individual differences of different pregnant women and fetuses, thereby providing more personalized fetal movement monitoring services for pregnant women.

[0005] Combining radar technology with deep learning and applying it to the field of fetal movement monitoring is expected to further overcome the limitations of existing methods, providing more convenient, accurate and reliable fetal movement monitoring means for pregnant women, and better meeting the needs of fetal health monitoring in different scenarios. Summary of the Invention

[0006] In view of this, the purpose of the present invention is to provide a non-contact fetal movement monitoring and identification method based on millimeter-wave radar, which uses deep learning to realize the extraction and analysis of fetal movement signals, improving the quality and level of perinatal fetal monitoring.

[0007] To achieve the above object, the present invention provides the following technical solutions:

[0008] The non-contact fetal movement monitoring and recognition method based on millimeter-wave radar provided by the present invention includes the following steps:

[0009] S1 Use a millimeter-wave radar to emit a detection signal towards the pregnant woman's abdomen, and receive the echo signal returned by the abdomen to obtain a signal containing the pregnant woman's breathing, heartbeat, and fetal movement. Divide it into three frequency bands, which are respectively represented as three matrices R1, R2, and R3;

[0010] S2 Perform PCA scattering separation on the time-domain echo signals of the three frequency bands respectively, filter out the breathing and heartbeat signals, and extract the fetal movement signals;

[0011] S3 Perform short-time Fourier transform on the fetal movement signals to obtain the time-frequency spectrogram of the fetal movement signals;

[0012] S4 Input the fetal movement time-domain signal into a long short-term memory network to extract time-domain features, input the time-frequency spectrogram into a convolutional neural network to extract time-frequency features, and complete the fusion of time-domain features and time-frequency domain features;

[0013] S5 Pass the fusion features of the three frequency bands through a fully connected classification layer respectively to output a probability matrix;

[0014] S6 Complete the fusion of the recognition results of the sub-networks of the three frequency bands at the decision layer;

[0015] S7 After being processed by the argmax function, the column subscript of the maximum value in the probability vector is taken out as the final output result;

[0016] S8 Count and store the fetal movements within a unit time, and count and plot a line graph of the fetal movement times for consecutive days.

[0017] Further, the radar echo signal in step S1 is expressed as:

[0018]

[0019] Among them, m is the number of various signals, σ i is the scattering coefficient of the scattering part corresponding to the i-th type of signal, τ i is the echo delay of the i-th type of signal at a certain moment, A r is the amplitude of the radar echo signal, f0 is the starting frequency of transmission, B w is the signal bandwidth, and K is the total number of sampling points of each chirp signal;

[0020] Divide the echo signal into 3 frequency bands evenly according to K sampling points,

[0021] The first sampling point to the th sampling point is the first frequency band,

[0022] The first sampling point to the second sampling point is the second frequency band.

[0023] The second sampling point to the Kth sampling point is the third frequency band.

[0024] The matrices R1, R2, and R3 respectively correspond to the data of the first, second, and third frequency bands, that is, the abdominal echo signal matrix R.

[0025] Perform principal component analysis on the abdominal echo signal matrix R stacked by each step frequency;

[0026] Extract the second principal component component, that is, the fetal movement signal, and filter out other principal component components, that is, the human respiration and heartbeat echoes and environmental clutter.

[0027] Further, the PCA scattering separation method in step S2 reorganizes the originally linearly correlated frequency vectors in matrix R into the uncorrelated row vectors of the new matrix X in the order of contribution degree through orthogonal transformation, and separates the uncorrelated components from the multi-frequency data;

[0028] Separate the fetal movement signal from the human vital signals, that is, the respiration and heartbeat signals, and the environmental clutter signals, and its corresponding central frequency is concentrated at the central frequency point of this frequency band;

[0029] The specific method of using PCA to obtain the recombined matrix X is to find the covariance matrix C of R S and find the S eigenvalues of C, and arrange the unit eigenvectors e1, e2,..., e corresponding to the eigenvalues in descending order N in rows to form matrix E, and then map the original signal to the new feature space E to obtain the new component matrix X = (x1, x2,..., x N ) T , that is, the recombination of the original signal, and the formula is expressed as:

[0030] X = ER

[0031] The row vectors of the obtained recombined matrix X are uncorrelated with each other, are the linear weighted recombination of the row vectors of the original matrix S, and their arrangement order is the order of the contribution values to the observed signal.

[0032] Further, the long short-term memory network in S4 includes several long short-term memory units, and each unit includes: a forgetting gate, an input gate, an output gate, and a memory unit;

[0033] (1) Input gate:

[0034] The calculation formula is

[0035] i t = σ(W i · [h t-1 , x t + b i )

[0036] where σ is the sigmoid function, W i is the weight matrix corresponding to the input data x t h t-1 is the hidden state at the previous moment, b i is the bias term, and the activation value i t of the input gate is calculated, representing the part of the input data that needs to be updated to the memory cell; at the same time

[0037]

[0038] where tanh is the hyperbolic tangent function, W c is another weight matrix, b c is the corresponding bias term, and the update information under the control of the input gate is calculated to determine the new information content to be written into the memory cell;

[0039] (2) Forget gate:

[0040] The formula for calculating the activation value of the forget gate is

[0041] f t = σ(W f · [h t-1 , x t + b f )

[0042] where W f is the weight matrix corresponding to the forget gate, b f is the bias term, and the value f t of the forget gate determines the information to be forgotten in the memory cell;

[0043] (3) Memory cell:

[0044] The formula for updating the memory cell state is

[0045]

[0046] By multiplying the output f t of the forget gate with the memory cell state c t-1 at the previous moment, and adding the update information under the control of the input gate, that is, the same as in the calculation of the input gate , the update of the memory cell state is achieved;

[0047] (4) Output gate:

[0048] The calculation formula for the activation value of the output gate is

[0049] o t = σ(W o · [h t-1 , x t + b o )

[0050] where W o is the output gate weight matrix, and b o is the bias term; the formula for the final hidden state output is

[0051] h t = o t · tanh(c t )

[0052] That is, the activation value o of the output gate t is dot-multiplied with the current memory cell state c after tanh processing t to obtain the hidden state output h t ; the output feature sequence of the long short-term memory network is represented as

[0053] h LSTM = {h1, h2,..., h T}

[0054] where is the feature vector at the t-th time step, T is the sequence length, and d is the dimension of the feature vector;

[0055] Furthermore, the convolutional neural network in step S4 includes a number of cascaded convolutional layers and max pooling layers; the convolutional layer is also provided with a batch normalization + ReLU activation function layer;

[0056] The output of the CNN is represented as

[0057] F CNN = {F1, F2,..., F C},

[0058] where, is the feature map of the c-th channel, C is the total number of feature maps, and H and W are the height and width of each feature map respectively.

[0059] Furthermore, the feature fusion in step S4 is completed by combining the attention mechanism, including the following steps:

[0060] S41 uses the temporal attention mechanism for the output feature sequence h LSTM of the LSTM:

[0061] By querying q t , key k t and value v t to calculate the weight α for each time step t t :

[0062] q t = W q h t , k t = W k h t , v t = W v h t

[0063]

[0064] where W q , W k and W v are learned weight matrices, d is the dimension of the query and key vectors, and score(q t , k t ) is the attention score; then, based on the calculated temporal attention weight α t , the temporal feature sequence is weighted:

[0065]

[0066] where is the weighted temporal feature;

[0067] S42 applies the channel attention mechanism to the output feature F CNN = {F1, F2,..., F C}:

[0068] By querying q c , key k c and value v c to calculate the weight β for each channel c c :

[0069] q c = W q′ global_avg(F c ), k c = W k′ global_avg(F c ), v c = F c

[0070]

[0071] where global_avg(F c) is the result after global average pooling of the feature map F c , W q′ , W k′ are the learned weight matrices, and d c is the dimension of the query and key vectors;

[0072] According to the calculated channel attention weight β c , the time-frequency features are weighted and flattened into a one-dimensional vector

[0073]

[0074] S43 Feature Fusion:

[0075] The weighted time-domain features and the time-frequency features are weighted and summed:

[0076]

[0077] where λ1 and λ2 are parameters that can be learned and adjusted, controlling the fusion ratio of the time-domain features and the time-frequency features.

[0078] Further, the fully connected classification layer described in S5 includes two fully connected layers that respectively select ReLU and Softmax as activation functions. The Softmax function outputs a two-dimensional result vector that classifies the signal into fetal movement and stillness categories. The two dimensions respectively represent the probability values that the input is recognized as fetal movement and stillness. The sum of the probabilities of each dimension is 1, expressed as

[0079] p 1T + p 1F = 1

[0080] p 2T + p 2F = 1

[0081] p 3T + p 3F = 1

[0082] where p 1T , p 1F represent the probability values of fetal movement and stillness output by the sub-network of the first frequency band;

[0083] p 2T , p 2F represent the probability values of fetal movement and stillness output by the sub-network of the second frequency band;

[0084] p 3T , p 3F represent the probability values of fetal movement and stillness output by the sub-network of the third frequency band.

[0085] Furthermore, the decision-level fusion described in S6 weights and sums the probability vectors output by the three sub-networks, and uses the softmax function as the activation function for normalization to obtain the final recognition probability vector, which is expressed by the formula:

[0086] z T = μ1p 1T + μ2p 2T + μ3p 3T

[0087] z F = μ1p 1F + μ2p 2F + μ3p 3F

[0088] p T = softmax(z T )

[0089] p F = softmax(z F )

[0090] p = (p T , p F )

[0091] Among them, p T and p F respectively represent the final output fetal movement and stillness probability values;

[0092] μ1, μ2, and μ3 are parameters adjusted during learning respectively.

[0093] The non-contact fetal movement monitoring and counting system based on millimeter-wave radar provided by the present invention includes a memory, a processor, a display, and a computer program stored on the memory and executable on the processor. When the processor executes the program, the above method is implemented. The memory stores the historical fetal movement data, and the display displays the fetal movement line graph.

[0094] The beneficial effects of the present invention are as follows:

[0095] The non-contact fetal movement monitoring and recognition method and system based on millimeter-wave radar provided by the present invention combines radar technology with deep learning and applies it to the field of fetal movement monitoring. By using a millimeter-wave radar to transmit a detection signal to the pregnant woman's abdomen and receive the echo signal, and then performing a series of operations such as scattering separation, time-frequency spectrogram generation, feature extraction, fetal movement recognition and storage, etc., it gives full play to the advantages of radar technology and deep learning. The non-contact measurement characteristic of the radar brings great convenience to pregnant women. There is no need to attach the sensor to a specific position on the pregnant woman's body, and the operation is simple, realizing real-time and convenient home monitoring and avoiding potential sudden risks.

[0096] Among them, the deep learning algorithm can automatically learn and extract complex features in the fetal movement signals, avoiding the limitations of manual feature extraction in traditional methods, improving the recognition ability and accuracy of fetal movement signals, adapting to the individual differences of different pregnant women and fetuses, and providing more personalized fetal movement monitoring services for pregnant women. Compared with the existing fetal movement monitoring methods, this method has higher accuracy, convenience and reliability. It can monitor fetal movement in real time and accurately, detect abnormal fetal movement in time, provide more timely and accurate fetal health information for pregnant women and doctors, help reduce the risks of fetuses, and improve the quality and level of perinatal fetal monitoring.

[0097] This method uses a millimeter-wave radar to transmit detection signals to the abdomen of a pregnant woman and receives the echo signals returned from the abdomen, which are divided into three frequency bands;

[0098] Perform scattering separation on the time-domain echo signals of the three frequency bands respectively, filter out respiratory and heartbeat signals and noise, and extract fetal movement signals;

[0099] Perform short-time Fourier transform on the three segments of fetal movement signals respectively to obtain the time-frequency spectrogram of the fetal movement signals;

[0100] Input the fetal movement time-domain signals into a long short-term memory network to extract time-domain features, input the time-frequency spectrogram into a convolutional neural network to extract time-frequency features, and complete the fusion of time-domain features and time-frequency domain features;

[0101] Output probability vectors for the fusion features of the three frequency bands respectively through a fully connected classification layer;

[0102] Fuse the recognition results of the sub-networks of the three frequency bands at the decision layer to obtain the final recognition result;

[0103] Count and store the fetal movements within a unit time, and count the number of fetal movements for consecutive days and draw a line chart.

[0104] This method uses radar and deep learning technologies to accurately and conveniently monitor fetal movements in a non-contact manner, effectively reducing manual monitoring errors, improving monitoring efficiency and accuracy, better meeting the fetal health monitoring needs in different scenarios, providing strong support for pregnant women to understand the situation of the fetus in real time, helping to detect abnormal fetal states in time, and ensuring the safety of the fetus.

[0105] Other advantages, objectives and features of the present invention will be described to some extent in the subsequent specification, and to some extent, will be obvious to those skilled in the art based on the study of the following text, or can be taught from the practice of the present invention. The objectives and other advantages of the present invention can be realized and obtained through the following specification. BRIEF DESCRIPTION OF THE DRAWINGS

[0106] To make the objectives, technical solutions, and beneficial effects of the present invention clearer, the following drawings are provided for illustration of the present invention.

[0107] Figure 1 is the overall flowchart of this method.

[0108] Figure 2 is the LSTM cell structure.

[0109] Figure 3 is the radar echo signal of fetal movement.

[0110] Figure 4 is the radar echo signal containing fetal movement, breathing, and heartbeat.

[0111] Figure 5 is the time-frequency spectrum diagram of the fetal movement signal.

[0112] Figure 6 is the time-frequency spectrum diagram of the signal containing fetal movement, breathing, and heartbeat. Detailed implementation manners

[0113] The present invention will be further described below in conjunction with the drawings and specific embodiments, so that those skilled in the art can better understand the present invention and implement it, but the specific embodiments cited are not intended to limit the present invention.

[0114] Embodiment 1

[0115] As Figure 1 shown, Figure 1 is the overall flowchart of this method. The non-contact fetal movement monitoring and recognition method based on millimeter-wave radar provided in this embodiment includes the following steps:

[0116] S1: Use a millimeter-wave radar to emit a detection signal to the abdomen of a pregnant woman, and receive the echo signal returned from the abdomen to obtain a multi-frequency sampling signal including the breathing, heartbeat, and fetal movement of the pregnant woman, and evenly divide it into three matrices R1, R2, and R3 according to the frequency bands;

[0117] The pregnant woman sits quietly 3-6 meters away from the radar, and the radar is directly facing the abdomen of the pregnant woman for continuous acquisition for a period of time; in this embodiment, the pregnant woman sits quietly 5 meters away from the radar, and the radar is directly facing the abdomen of the pregnant woman. Each 1 minute is a segment, and continuous acquisition is carried out for 60 minutes; the millimeter-wave radar echo sampling signal can be expressed as:

[0118]

[0119] where p is the number of various signals, σ i is the scattering coefficient of the scattering part corresponding to the i-th type of signal, τ i is the echo delay of the i-th type of signal at a certain moment, A r is the amplitude of the radar echo signal, f0 is the starting frequency of transmission, B wis the signal bandwidth, and K is the total number of sampling points for each chirp signal;

[0120] In this embodiment, the echo signal is evenly divided into three frequency bands according to K sampling points,

[0121] The first sampling point to the th sampling point is the first frequency band, and the th sampling point to the th sampling point is the second frequency band, and the th sampling point to the Kth sampling point is the third frequency band;

[0122] Specifically, the transmit start frequency f0 of the millimeter-wave radar provided in this embodiment is 77 GHz, the signal bandwidth B w is 4 GHz, the total number of sampling points K is taken as 256, and the echo signal is evenly divided into three frequency bands according to 256 sampling points,

[0123] The first to 85th sampling points are the first frequency band,

[0124] The 86th to 170th sampling points are the second frequency band,

[0125] The 171st to 255th sampling points are the third frequency band,

[0126] Matrices R1, R2, and R3 respectively correspond to the data of the first, second, and third frequency bands, hereinafter collectively referred to as R. The columns of the matrix R represent the respective stepped frequencies, and the rows represent the time-period changes of the echo data.

[0127] S2: Perform scattering separation on the time-domain echo signal, filter out the respiratory and heartbeat signals, and extract the fetal movement signal.

[0128] Perform principal component analysis on the abdominal echo signal matrix R stacked according to each stepped frequency, extract the second principal component component, that is, the fetal movement signal, and filter out other principal component components, that is, the human respiratory and heartbeat echoes and environmental clutter.

[0129] Among them, principal component analysis (PCA scattering separation) reorganizes the originally linearly correlated frequency vectors in matrix R into the uncorrelated row vectors of the new matrix X in the order of contribution degree through orthogonal transformation, so as to separate the uncorrelated components from the multi-frequency data. Therefore, the fetal movement signal is separated from the human vital signs (breathing, heartbeat), environmental clutter, etc., and its corresponding center frequency is concentrated at the center frequency point of this frequency band.

[0130] The specific method of using PCA scattering separation to obtain the reconstructed matrix X is to find the covariance matrix C of R S and find the eigenvalues of C S and arrange the unit eigenvectors e1, e2,..., e corresponding to the eigenvalues in descending orderN Arrange them in rows to form matrix E, and then map the original signal into the new feature space E to obtain the new component matrix X = (x1, x2,..., x N ) T , that is, the recombination of the original signal, and the formula is expressed as:

[0131] X = E·R

[0132] The row vectors of the obtained recombination matrix X are mutually uncorrelated, which is a linear weighted recombination of the row vectors of the original matrix R, and their arrangement order is the order of the contribution values to the observed signal.

[0133] As Figure 3 shown, it is the extracted fetal movement signal. As a comparison, Figure 4 it is the time-domain signal containing fetal movement and respiratory and heartbeat signals, that is, before PCA scattering separation. It can be seen that the fetal movement signal is effectively extracted from the respiratory and heartbeat signals.

[0134] S3: Perform short-time Fourier transform on the fetal movement signal to obtain the time-frequency spectrogram of the fetal movement signal.

[0135] Among them, the short-time Fourier transform selects a Hamming window with a length of 17 as the window function. As Figure 5 shown, it is the time-frequency spectrogram of the extracted fetal movement signal. As a comparison, Figure 6 it is the time-frequency spectrogram containing fetal movement and respiratory and heartbeat signals, that is, before PCA scattering separation. It can be seen that in the time-frequency domain, the fetal movement signal is effectively extracted from the respiratory and heartbeat signals.

[0136] S4: Input the fetal movement time-domain signal into a long short-term memory network to extract time-domain features, input the time-frequency spectrogram into a convolutional neural network to extract time-frequency features, and complete the fusion of time-domain features and frequency-domain features.

[0137] The long short-term memory network contains several long short-term memory units. As Figure 2 shown, the long short-term memory unit includes: forget gate, input gate, output gate and memory unit.

[0138] (1) Input gate:

[0139] The calculation formula is

[0140] i t = σ(W i ·[h t-1 , x t + b i ),

[0141] where σ is the sigmoid function, and W i is the weight matrix corresponding to the input data x t , ht-1 is the hidden state at the previous moment, b i is the bias term. The activation value i of the input gate is calculated by this formula t , indicating the part of the input data that needs to be updated to the memory cell; at the same time

[0142]

[0143] where tanh is the hyperbolic tangent function, W c is another weight matrix, b c is the corresponding bias term. The updated information under the control of the input gate is calculated by this formula to determine the content of the new information to be written into the memory cell.

[0144] (2) Forget gate:

[0145] The formula for calculating the activation value of the forget gate is

[0146] f t = σ(W f · [h t-1 , x t + b f ),

[0147] where W f is the weight matrix corresponding to the forget gate, b f is the bias term. The value f of the forget gate t determines which information in the memory cell needs to be forgotten.

[0148] (3) Memory cell:

[0149] The formula for updating the memory cell state is

[0150]

[0151] By multiplying the output f of the forget gate t by the memory cell state c at the previous moment t-1 and adding the updated information under the control of the input gate (the same as in the input gate calculation ), the update of the memory cell state is realized.

[0152] (4) Output gate:

[0153] The formula for calculating the activation value of the output gate is

[0154] o t = σ(W o · [h t-1 , x t + b o );

[0155] Where W o is the output gate weight matrix, and b o is the bias term; the formula for the final hidden state output is

[0156] h t = o t · tanh(c t ),

[0157] That is, the output gate activation value o t is dot - multiplied with the current memory cell state c t after being processed by tanh to obtain the hidden state output h t .

[0158] Finally, the output feature sequence of the long - short - term memory network is represented as h LSTM = {h1, h2,..., h T}, where h t ∈ i d is the feature vector at the t - th time step, T is the sequence length, and d is the dimension of the feature vector.

[0159] The convolutional neural network includes a number of serially connected convolutional layers and max - pooling layers; the convolutional layers are also provided with batch normalization + ReLU activation function layers;

[0160] The convolutional neural network in this embodiment is composed of 3 two - dimensional convolutional layers and 3 pooling layers cross - stacked with each other. The convolutional layers are provided with multiple convolutional kernels with a size of 3 and a stride of 1. The pooling layers select max - pooling with a pooling kernel size of 2 and a stride of 2. Each convolutional layer is followed by a batch normalization + ReLU activation function layer.

[0161] The output of the CNN is represented as

[0162] F CNN = {F1, F2,..., F C},

[0163] where, is the feature map of the c - th channel, C is the total number of feature maps, and H and W are the height and width of each feature map respectively;

[0164] The feature fusion is completed by combining the attention mechanism and includes the following steps:

[0165] S41 uses the temporal attention mechanism for the output feature sequence h LSTM of the LSTM:

[0166] Calculate the weight α t for each time step t through the query q t , key k t and value v t, to represent the importance of the features at the t-th time step:

[0167] q t = W q h t , k t = W k h t , v t = W v h t

[0168]

[0169] where W q , W k and W v are learnable weight matrices, d is the dimension of the query and key vectors, and score(q t , k t ) is the attention score;

[0170] Then, according to the calculated temporal attention weight α t , the temporal feature sequence is weighted:

[0171]

[0172] where is the weighted temporal feature;

[0173] S42 applies the channel attention mechanism to the output features F CNN = {F1, F2,..., F C}:

[0174] By query q c , key k c and value v c , the weight β c of each channel c is calculated to represent the importance of the c-th feature map:

[0175] q c = W q′ global_avg(F c ), k c = W k′ global_avg(F c ), v c = F c

[0176]

[0177] where global_avg(F c ) is the global average of the feature map F cThe result after global average pooling, W q′ , W k′ are the learned weight matrices, and d c is the dimension of the query and key vectors. In this embodiment, d c takes the number of channels of the feature map;

[0178] According to the calculated channel attention weight β c , weight the time-frequency features and flatten them into a one-dimensional vector

[0179] S43 Feature fusion:

[0180] Perform weighted summation on the weighted time-domain features and the time-frequency features :

[0181]

[0182] where λ1 and λ2 are parameters that can be learned and adjusted, controlling the fusion ratio of the time-domain features and the time-frequency features.

[0183] S5: Output the probability matrix for the fusion features of the three frequency bands through the fully connected classification layer respectively.

[0184] The fully connected classification layer includes two fully connected layers that respectively use ReLU and Softmax as activation functions. The Softmax function outputs a two-dimensional result vector to classify the signal into two categories: fetal movement and stillness. The two dimensions respectively represent the probability values that the input is recognized as fetal movement and stillness, and the sum of the probabilities of each dimension is 1, expressed as

[0185] p 1T + p 1F = 1, p 2T + p 2F = 1, p 3T + p 3F = 1,

[0186] where p 1T , p 1F represent the probability values of fetal movement and stillness output by the sub-network of the first frequency band; p 2T , p 2F represent the probability values of fetal movement and stillness output by the sub-network of the second frequency band; p 3T , p 3F represent the probability values of fetal movement and stillness output by the sub-network of the third frequency band.

[0187] S6: Fuse the recognition results of the sub-networks of the three frequency bands at the decision layer.

[0188] The probability vectors output by the three sub-networks are weighted and summed, and the softmax function is used as the activation function for normalization to obtain the final recognition probability vector, which is expressed by the formula:

[0189] z T = μ1p 1T + μ2p 2T + μ3p 3T ,

[0190] z F = μ1p 1F + μ2p 2F + μ3p 3F ,

[0191] p T = softmax(z T ),

[0192] p F = softmax(z F ),

[0193] p = (p T , p F ),

[0194] where p T and p F represent the fetal movement and rest probability values of the final output respectively, and μ1, μ2, and μ3 are the parameters adjusted during learning.

[0195] S7: The final output result is obtained through the argmax function.

[0196] The argmax function mentioned above represents taking out the column subscript of the maximum value in the vector. The maximum probability indicates the motion category that the classifier believes the data corresponds to the model believes is the most likely for the current input data. Taking out its subscript gives the prediction label of the classifier.

[0197] The argmax function in step S7 represents taking out the column subscript of the maximum value in the probability vector. The maximum value in the probability vector indicates the motion category that is the most likely for the current input data in the corresponding model of the data output by the classifier. Taking out its subscript gives the prediction label of the classifier.

[0198] S8: Count and store the fetal movements within a unit time, and count the number of fetal movements over consecutive days and plot a line graph.

[0199] Real-time monitoring of fetal activities, accumulating data to assist in immediate clinical diagnosis. By reviewing historical data and combining with other examinations, the fetal development status can be accurately judged. Moreover, it can intuitively display the long-term trend of fetal movements to facilitate the discovery of abnormalities, help doctors evaluate the fetal state individually and analyze the impact of special situations, while enhancing the pregnant woman's self-monitoring awareness, promptly detecting sudden abnormalities, and ensuring the healthy development of the fetus.

[0200] This embodiment also provides a non-contact fetal movement monitoring and counting system based on millimeter-wave radar, including a memory, a processor, a display, and a computer program stored on the memory and operable on the processor. When the processor executes the program, the above method is implemented. The memory stores the historical data of fetal movements, and the display displays the line graph of fetal movements.

[0201] The above-described embodiments are only preferred embodiments given to fully illustrate the present invention, and the protection scope of the present invention is not limited thereto. Equivalent substitutions or transformations made by those skilled in the art on the basis of the present invention are all within the protection scope of the present invention. The protection scope of the present invention is subject to the claims.

Claims

1. A non-contact fetal movement monitoring and recognition method based on millimeter-wave radar, characterized in that: It includes the following steps: S1 Use a millimeter-wave radar to emit detection signals towards the pregnant woman's abdomen, and receive the echo signals returned from the abdomen to obtain signals containing the pregnant woman's breathing, heartbeat, and fetal movement. Divide them into three frequency bands, which are respectively represented as three matrices R1, R2, and R3; S2 Perform PCA scattering separation on the time-domain echo signals of the three frequency bands respectively, filter out the breathing and heartbeat signals, and extract the fetal movement signals; S3 Perform short-time Fourier transform on the fetal movement signals to obtain the time-frequency spectrogram of the fetal movement signals; S4 Input the fetal movement time-domain signals into a long short-term memory network to extract time-domain features, input the time-frequency spectrogram into a convolutional neural network to extract time-frequency features, and complete the fusion of time-domain features and time-frequency domain features; S5 Output probability matrices for the fusion features of the three frequency bands respectively through a fully connected classification layer; S6 Complete the fusion of the recognition results of the sub-networks of the three frequency bands at the decision layer; S7 After being processed by the argmax function, the column subscript that takes out the maximum value in the probability vector is the final output result; S8 Count and store the fetal movements within a unit time, and count and plot a line graph for the fetal movement counts over consecutive days.

2. The non-contact fetal movement monitoring and recognition method based on millimeter-wave radar according to claim 1, characterized in that: The radar echo signal in step S1 is expressed as: Among them, m is the number of various signals, and σ i is the scattering coefficient of the scattering part corresponding to the i-th signal, and τ i is the echo time delay of the i-th signal at a certain moment, A r is the amplitude of the radar echo signal, f0 is the starting frequency of transmission, and B w is the signal bandwidth, and K is the total number of sampling points of each chirp signal; Divide the echo signal into 3 frequency bands evenly according to K sampling points, The first sampling point to the th sampling point is the first frequency band, The th sampling point to the th sampling point is the second frequency band, The first sampling point to the Kth sampling point is the third frequency band, The matrices R1, R2, and R3 respectively correspond to the data of the first, second, and third frequency bands, that is, the abdominal echo signal matrix R.

3. The non-contact fetal movement monitoring and recognition method based on millimeter-wave radar according to claim 1, characterized in that: The scattering separation method in S2 includes the following steps: Perform principal component analysis on the abdominal echo signal matrix R stacked according to each stepped frequency; Extract the second principal component component, that is, the fetal movement signal, and filter out other principal component components, that is, the human breathing and heartbeat echoes and environmental clutter.

4. The non-contact fetal movement monitoring and recognition method based on millimeter-wave radar according to claim 1, characterized in that: The PCA scattering separation method in step S2 reorganizes the originally linearly correlated frequency vectors in matrix R into the uncorrelated row vectors of the new matrix X in the order of contribution degree through orthogonal transformation, and separates the uncorrelated components from the multi-frequency data; Separate the fetal movement signals from the human vital signals, that is, the breathing and heartbeat signals and environmental clutter signals, and their corresponding center frequencies are concentrated at the center frequency points of this frequency band; The specific method for obtaining the reconstructed matrix X using PCA is to find the covariance matrix C of R S and calculate C S of the eigenvalues, and arrange the unit eigenvectors e1, e2,..., e corresponding to the eigenvalues in descending order N Arrange them row by row to form the matrix E, and then map the original signal into the new feature space E to obtain the new component matrix X = (x1, x2,..., x N ) T , that is, the recombination of the original signal, and the formula is expressed as: X = E·R The row vectors of the obtained reconstructed matrix X are uncorrelated with each other, which is a linear weighted reconstruction of the row vectors of the original matrix R, and their arrangement order is the order of their contribution values to the observed signals.

5. The non-contact fetal movement monitoring and recognition method based on millimeter-wave radar according to claim 1, characterized in that: The long short-term memory network in S4 includes several long short-term memory units, and each unit includes: a forgetting gate, an input gate, an output gate, and a memory unit; (1) Input gate: The calculation formula is i t = σ(W i · [h t-1 , x t + b i ) where σ is the sigmoid function, W i is the weight matrix corresponding to the input data x t h t-1 is the hidden state at the previous moment, b i is the bias term, and the activation value i of the input gate is calculated t , indicating the part of the input data that needs to be updated to the memory cell; meanwhile where tanh is the hyperbolic tangent function, and W c is another weight matrix, and b c is the corresponding bias term, and the updated information under the control of the input gate is calculated to determine the new information content to be written into the memory cell; (2) Forgetting gate: The formula for calculating the forgetting gate activation value is f t = σ(W f · [h t-1 , x t + b f ) Among which W f is the weight matrix corresponding to the forget gate, and b f is the bias term. The value f of the forget gate t determines the information to be forgotten in the memory cell; (3) Memory unit: The memory unit state update formula is By multiplying the output f of the forget gate t with the state c of the memory cell at the previous moment t-1 and adding the updated information under the control of the input gate i.e., the same as in the calculation of the input gate the update of the state of the memory cell is realized; (4) Output gate: The formula for calculating the output gate activation value is Among them, W o is the output gate weight matrix, and b o is the bias term; the formula for the final hidden state output is h t = o t ·tanh(c t ) That is, the output gate activation value o t is dot - multiplied with the current memory cell state c after tanh processing t to obtain the hidden state output h t ; The output feature sequence of the long short-term memory network is expressed as h LSTM = {h1, h2,..., h T} Among them, is the feature vector at the t-th time step, represents the real number field, T is the sequence length, and d is the dimension of the feature vector.

6. The non-contact fetal movement monitoring and recognition method based on millimeter-wave radar according to claim 1, characterized in that: The convolutional neural network in step S4 includes several cascaded convolutional layers and max pooling layers; the convolutional layers are also provided with batch normalization + ReLU activation function layers; Express the output of the CNN as F CNN = {F1, F2,..., F C}, Among them, is the feature map of the c-th channel, C is the total number of feature maps, and H and W are the height and width of each feature map respectively.

7. The non-contact fetal movement monitoring and recognition method based on millimeter-wave radar according to claim 1, characterized in that: The feature fusion in step S4 is completed in combination with the attention mechanism, and includes the following steps: S41 performs time attention mechanism on the output feature sequence h of the LSTM LSTM using the time attention mechanism: by querying q t , key k t and value v t to calculate the weight α for each time step t t : q t = W q h t , k t = W k h t , v t = W v h t Among them, W q , W k and W v are the weight matrices for learning, d is the dimension of the query and key vectors, and score(q t , k t ) is the attention score; Then, according to the calculated temporal attention weight α t , the time-domain feature sequence is weighted: Among them, is the weighted time-domain feature; S42 applies the channel attention mechanism to the output features F of the CNN CNN ={F1, F2, ..., F C} By querying q c , key k c and value v c to calculate the weight β of each channel c c : q c = W q′ global_avg(F c ), k c = W k′ global_avg(F c ), v c = F c Among them, global_avg(F c ) is the result after global average pooling on the feature map F c , W q′ , W k′ are the learned weight matrices, and d c is the dimension of the query and key vectors; According to the calculated channel attention weight β c , weight the time-frequency features and flatten them into a one-dimensional vector : S43 Feature fusion: The weighted time-domain features and time-frequency features are weighted and summed up: Among them, λ1 and λ2 are parameters that can be learned and adjusted, and control the fusion ratio of time-domain features and time-frequency features.

8. The non-contact fetal movement monitoring and recognition method based on millimeter-wave radar according to claim 1, characterized in that: The fully-connected classification layer in step S5 includes two fully-connected layers that respectively use ReLU and Softmax as activation functions. The Softmax function outputs a two-dimensional result vector to classify the signal into two categories: fetal movement and stillness. The two dimensions respectively represent the probabilities that the input is recognized as fetal movement and stillness, and the sum of the probabilities of each dimension is 1, which is expressed as p 1T +p 1F = 1 p 2T +p 2F = 1 p 3T +p 3F = 1 Among them, p 1T , p 1F represent the probability values of fetal movement and stillness output by the sub-network of the first frequency band; p 2T and p 2F represent the probability values of fetal movement and stillness output by the sub-network of the second frequency band; p 3T and p 3F represent the probability values of fetal movement and stillness output by the sub-network of the third frequency band.

9. The non-contact fetal movement monitoring and recognition method based on millimeter-wave radar according to claim 1, characterized in that: The decision-level fusion in step S6 weights and sums the probability vectors output by the three sub-networks, and uses the softmax function as the activation function for normalization to obtain the final recognition probability vector. The formula is expressed as: z T = μ1p 1T + μ2p 2T + μ3p 3T z F = μ1p 1F + μ2p 2F + μ3p 3F p T = softmax(z T ) p F = softmax(z F ) p = (p T , p F ) Among them, p T and p F respectively represent the final output fetal movement and stillness probability values; μ1, μ2, and μ3 are parameters adjusted during learning respectively.

10. A non-contact fetal movement monitoring and counting system based on millimeter-wave radar, comprising a memory, a processor, a display, and a computer program stored on the memory and executable on the processor, characterized in that, When the processor executes the program, it implements the method described in any one of claims 1 to 9 above. The memory stores the historical fetal movement data, and the display shows the fetal movement line graph.

Images