A non-contact emotion recognition method based on frequency-modulated continuous wave radar

Abstract

The application discloses a kind of non-contact emotion recognition methods based on frequency-modulated continuous wave radar, belong to high-tech transformation traditional industry and non-contact emotion monitoring field.The method mainly includes the following processes: (1) by transmitting antenna array transmitting frequency-modulated continuous wave signal and by receiving antenna array receiving;(2) by MVDR algorithm, extract the echo signal from the chest wall of monitoring object in radar echo signal;(3) from the phase of radar echo signal, demodulate the motion signal of the chest wall of monitoring object;(4) by filter, extract the respiratory signal from the chest wall motion signal;(5) extract the duration of successive heartbeat from the motion signal of the chest wall of monitoring object;(6) calculate the physiological characteristics for emotion recognition;(7) train random forest machine learning model to realize emotion recognition.The application uses microwave as detection medium, has many advantages, such as convenient to use, low requirement to environment, fast measurement and higher accuracy.

Description

Technical Field

[0001] This invention belongs to the field of high-tech transformation of traditional industries and non-contact emotion monitoring, specifically involving a non-contact emotion recognition method based on frequency-modulated continuous wave radar. Background Technology

[0002] In recent years, research on non-contact emotion recognition has flourished, with various non-contact emotion recognition methods emerging. Traditional emotion recognition mainly relies on monitoring electroencephalograms (EEGs) or facial expressions. However, both have significant limitations in practical applications. For EEGs, acquiring signals requires the subject to wear an EEG cap, a complex process that demands specialized knowledge and greatly limits its applicability. Furthermore, the discomfort caused by the EEG cap may affect the subject's emotions. For facial expressions, obtaining information requires a certain level of ambient brightness, and this method is susceptible to subjective deception by the subject, such as a "fake smile." These limitations significantly restrict the application of emotion recognition in daily life.

[0003] Accurately identifying the emotions of monitored subjects is of great significance in modern life. For example, applying emotion recognition in psychological diagnosis and treatment can help therapists more accurately grasp the emotional state of patients; applying emotion recognition in the psychological assessment of soldiers can provide more precise psychological counseling and improve soldiers' combat performance; applying emotion recognition in home scenarios can adjust the working status of smart home appliances, such as air conditioning temperature and lighting brightness, in a timely manner according to the owner's mood.

[0004] On the other hand, compared to facial expressions, human physiological characteristics (such as respiration, heart rate, and heart rate variability) are more difficult to manipulate subjectively. Therefore, emotion recognition based on these physiological characteristics theoretically has higher accuracy. Furthermore, acquiring the physiological characteristics of the monitored object using frequency-modulated continuous wave radar is not affected by ambient lighting conditions. Based on this, this invention proposes an emotion recognition method based on frequency-modulated continuous wave radar, which uses microwaves as the detection medium and has many advantages, including ease of use, low environmental requirements, rapid measurement, and high accuracy. Summary of the Invention

[0005] To address the issues of low measurement accuracy and limited application conditions in existing non-contact emotion recognition methods, this invention proposes a non-contact emotion recognition method based on frequency-modulated continuous wave radar.

[0006] The specific technical solution adopted in this invention is as follows:

[0007] This invention provides a non-contact emotion recognition method based on frequency-modulated continuous wave radar, as detailed below:

[0008] S1: The transmitting antenna array transmits periodic linear frequency modulated waves to the area where the monitored object is located, and the corresponding echo signals are collected by the receiving antenna array;

[0009] S2: Construct a virtual antenna array based on the geometric features of the transmitting antenna array and the receiving antenna array; based on the virtual antenna array, extract the radar echo signal from the direction of the monitored object from the echo signal acquired in step S1 using the MVDR beamforming method.

[0010] S3: Demodulate the chest wall motion signal of the monitored object from the phase of the radar echo signal;

[0011] S4: Extract respiratory signals from the chest wall motion signals using a filter;

[0012] S5: Extract the duration of successive heartbeats from the chest wall motion signal of the monitored object using a non-contact heart rate variability monitoring method based on frequency modulated continuous wave radar;

[0013] S6: Based on the chest wall motion signal mentioned in step S3, the respiratory signal mentioned in step S4, and the duration of each heartbeat mentioned in step S5, calculate the time domain features, frequency domain features, and nonlinear domain features respectively to construct a feature vector for emotion recognition.

[0014] S7: Based on the feature vector described in step S6, train a random forest machine learning model for emotion recognition to complete the emotion classification task.

[0015] Preferably, the pulse frequency modulation range of the linear frequency modulated wave is from 77 GHz to 81 GHz, and the duration of a single pulse is 50 μs; the fast time axis sampling frequency is 4 MHz, the number of sampling points is 128, and the slow time axis sampling frequency is 100 Hz.

[0016] Preferably, the transmitting antenna array comprises 3 transmitting antennas, with a horizontal distance of λ between any two adjacent transmitting antennas; the receiving antenna array comprises 4 receiving antennas, with a horizontal distance of λ between any two adjacent receiving antennas. The virtual antenna array contains 12 virtual antennas located on the same plane. Taking the virtual antenna numbered 1 as the origin, the coordinates of the virtual antennas numbered 2 to 12 are respectively: (λ / 20), (λ0), (3λ / 20), (2λ0), (5λ / 20), (3λ0), (7λ / 20), (λλ / 2), (3λ / 2λ / 2), (2λλ / 2), (5λ / 2λ / 2); where λ is the wavelength of the linear frequency modulated wave.

[0017] Preferably, the MVDR beamforming method in S2 uses only the echo signals received by the virtual antennas numbered 1-8 in the virtual antenna array.

[0018] Preferably, the MVDR beamforming method in step S2 is as follows:

[0019] S2-1: The radar echo signals received by the 8 virtual antennas in the virtual antenna array after ADC conversion are denoted as x. k Let Tx(n) be the received signal vector, where k = 1, 2, ..., 8.

[0020] T(n)=[x1(n) x2(n) … x8(n)];

[0021] S2-2: Based on the pre-measured azimuth angle θ0 of the monitored object relative to the virtual antenna array, construct the beamforming steering vector α(θ0).

[0022]

[0023] S2-3: Based on the guiding vector α(θ0) described in step S2-2, create the optimal weight vector ω(n).

[0024]

[0025] Where matrix R(n) is the covariance matrix at time n, calculated according to the following formula.

[0026] R(n) = T H (n)T(n);

[0027] S2-4: Based on the optimal weight vector ω(n) described in step S2-3, extract the radar echo signal x(n) from the direction of the monitored object.

[0028] x(n)=ω H (n)T H (n).

[0029] Preferably, step S3 is as follows:

[0030] S3-1: The radar echo signal x(n) extracted in step S2 is a complex exponential signal, and its mathematical expression is:

[0031]

[0032] Where A represents the amplitude of the signal, which is a constant; f represents the frequency of the signal. Indicates the phase of the signal;

[0033] S3-2: Solve for the phase of x(n) in step S3-1 using the arctangent function. And untangle it;

[0034] S3-3: Based on the results of step S3-2, according to Relationship with the chest wall motion signal D(n) of the monitored object

[0035]

[0036] The chest wall motion signal D(n) of the monitored object is obtained.

[0037] Preferably, the filter is a Butterworth bandpass filter with an order of 20 and a passband of [0.1Hz-0.5Hz].

[0038] As a preferred embodiment, the non-contact heart rate variability monitoring method based on frequency-modulated continuous wave radar in step S5 is as follows:

[0039] S8-1: Transmits periodic linear frequency modulated waves to the area where the monitored object is located, and uses a receiving antenna to collect the corresponding radar echo signals;

[0040] S8-2: The acquired radar echo signals are preprocessed by sequentially using frequency mixing and fast Fourier transform.

[0041] S8-3: Extract the location of the heart of the monitored object from the preprocessed radar echo signal;

[0042] S8-4: Using a phase correlation method, extract the motion information of the heart region of the monitored object from the position obtained in step S8-3, and calculate its acceleration signal accordingly;

[0043] S8-5: The obtained acceleration signal is smoothed by calculating the short-time average power, and the position of the segmentation point between each heartbeat is estimated by peak detection.

[0044] S8-6: Based on the position of the segmentation point between each heartbeat estimated in step S8-5, generate an acceleration template signal for a single heartbeat; then use this acceleration template signal to precisely segment the acceleration signal obtained in step S8-4 to obtain the heartbeat interval of each heartbeat;

[0045] S8-7: Based on the heart rate interval obtained in step S8-6, calculate the heart rate variability index to achieve non-contact heart rate variability monitoring.

[0046] Preferably, step S6 is as follows:

[0047] Based on the chest wall motion signal D(n) described in step S3, the respiratory signal B(n) described in step S4, and the duration H(m) of each heartbeat described in step S5, a feature vector C for this set of samples is created based on the calculated 26 time-domain features, 23 frequency-domain features, and 13 nonlinear features.

[0048] The time-domain features include: the variance of D(n); the mean of the absolute values ​​of the first-order differences of D(n); the mean of the absolute values ​​of the normalized first-order differences of D(n); the mean of the absolute values ​​of the second-order differences of D(n); the mean of the absolute values ​​of the normalized second-order differences of D(n); the mean of the time intervals between adjacent peak points in B(n); the mean of the time intervals between adjacent valley points in B(n); the variance of the time intervals between adjacent peak points in B(n); the variance of the time intervals between adjacent valley points in B(n); the reciprocal of the mean of the time intervals between adjacent peak points in B(n); the reciprocal of the mean of the time intervals between adjacent valley points in B(n); and the first-order differences of B(n). Mean of the absolute values ​​of B(n); mean of the absolute values ​​of the normalized first-order differences of B(n); mean of the absolute values ​​of the second-order differences of B(n); mean of the absolute values ​​of the normalized second-order differences of B(n); mean of H(m); reciprocal of the mean of H(m); standard deviation of H(m); root mean square of the first-order differences of H(m); mean of the absolute values ​​of the first-order differences of H(m); mean of the absolute values ​​of the first-order differences of H(m); mean of the absolute values ​​of the normalized first-order differences of H(m); mean of the absolute values ​​of the second-order differences of H(m); mean of the absolute values ​​of the normalized second-order differences of H(m); skewness of H(m); kurtosis of H(m); proportion of terms with absolute values ​​greater than 50ms in the first-order differences of H(m).

[0049] The frequency domain characteristics include: the average amplitude of frequency components less than 0.1 Hz in the spectrum of D(n); the average amplitude of frequency components in the spectrum of D(n) between [0.1 Hz and 0.2 Hz); the average amplitude of frequency components in the spectrum of D(n) between [0.2 Hz and 0.3 Hz); the average amplitude of frequency components in the spectrum of D(n) between [0.3 Hz and 0.4 Hz); the average amplitude of frequency components less than 0.1 Hz in the spectrum of B(n); the average amplitude of frequency components in the spectrum of B(n) between [0.1 Hz and 0.2 Hz); the average amplitude of frequency components in the spectrum of B(n) between [0.2 Hz and 0.3 Hz); the average amplitude of frequency components in the spectrum of B(n) between [0.3 Hz and 0.4 Hz]. The average amplitude of the frequency components within the range of 0.4 Hz; the average amplitude of the frequency components in the spectrum of B(n) between 0.4 Hz and 0.9 Hz; the average amplitude of the frequency components in the spectrum of B(n) between 0.9 Hz and 1.5 Hz; the ratio of the average amplitude of the frequency components in the spectrum of B(n) between 0.1 Hz and 0.4 Hz to the average amplitude of the frequency components in the spectrum of B(n) between 0.4 Hz and 1.5 Hz; the sum of the amplitudes of the frequency components less than 0.04 Hz in the spectrum of H(m); the sum of the amplitudes of the frequency components in the spectrum of H(m) between 0.04 Hz and 0.15 Hz; the sum of the amplitudes of the frequency components in the spectrum of D(n) between 0.15 Hz and 0.4 Hz; H( The frequency with the largest amplitude among the frequency components less than 0.04 Hz in the spectrum of H(m); the frequency with the largest amplitude among the frequency components between 0.04 Hz and 0.15 Hz in the spectrum of D(n); the frequency with the largest amplitude among the frequency components between 0.15 Hz and 0.4 Hz in the spectrum of H(m); the ratio of the sum of the amplitudes of the frequency components between 0.04 Hz and 0.15 Hz in the spectrum of H(m) to the sum of the amplitudes of the frequency components less than 0.4 Hz in the spectrum of H(m); the ratio of the sum of the amplitudes of the frequency components between 0.04 Hz and 0.15 Hz in the spectrum of H(m) to the sum of the amplitudes of the frequency components less than 0.4 Hz in the spectrum of H(m); the frequency of H(m) between 0.15 Hz and 0.15 Hz in the spectrum of D(n); the frequency with the largest amplitude among the frequency components less than 0.4 Hz in the spectrum of D(n); the frequency with the largest amplitude among the frequency components between 0.15 Hz and 0.15 Hz in the spectrum of H(m ... The ratio of the sum of amplitudes of frequency components in the range of 5Hz to 0.4Hz to the sum of amplitudes of frequency components in the H(m) spectrum less than 0.4Hz; the ratio of the sum of amplitudes of frequency components in the H(m) spectrum less than 0.4Hz to the sum of amplitudes of frequency components in the H(m) spectrum between 0.04Hz and 0.15Hz; the ratio of the sum of amplitudes of frequency components in the H(m) spectrum less than 0.4Hz to the sum of amplitudes of frequency components in the H(m) spectrum between 0.15Hz and 0.4Hz; the ratio of the sum of amplitudes of frequency components in the H(m) spectrum between 0.04Hz and 0.15Hz to the sum of amplitudes of frequency components in the H(m) spectrum between 0.15Hz and 0.4Hz.

[0050] The nonlinear characteristics include: the approximate entropy of D(n); the first-order detrended fluctuation analysis coefficient of D(n); the second-order detrended fluctuation analysis coefficient of D(n); the approximate entropy of B(n); the first-order detrended fluctuation analysis coefficient of B(n); the second-order detrended fluctuation analysis coefficient of B(n); the approximate entropy of H(m); the first-order detrended fluctuation analysis coefficient of H(m); the second-order detrended fluctuation analysis coefficient of H(m); SD1 in the Poincaré scatter plot of H(m); SD2 in the Poincaré scatter plot of H(m); the product of SD1 and SD2 in the Poincaré scatter plot of H(m); and the ratio of SD1 to SD2 in the Poincaré scatter plot of H(m).

[0051] Preferably, in the random forest machine learning model of step S7, the number of decision trees is 200, and each node is set to generate at least 2 samples in any child node after branching.

[0052] Compared with the prior art, the present invention has the following advantages:

[0053] This invention, through an innovative design of a radar echo signal processing algorithm, extracts the motion signal of the chest wall, the respiratory signal, and the heart rate variability information of the monitored subject. Based on this, random forest machine learning is used to identify the emotions of the monitored subject. This invention uses microwaves as the detection medium, which has many advantages such as ease of use, low environmental requirements, rapid measurement, and high accuracy. Attached Figure Description

[0054] Figure 1 The diagram shows the transmitting antenna array (a), the receiving antenna array (b), and the virtual antenna array (c) constructed based on the two in the embodiment.

[0055] Figure 2 This is a flowchart of the method of the present invention;

[0056] Figure 3 The figure shows the recognition accuracy under different emotion combinations in the example. Detailed Implementation

[0057] The present invention will be further described and illustrated below with reference to the accompanying drawings and specific embodiments. The technical features of each embodiment of the present invention can be combined accordingly, provided that there is no mutual conflict.

[0058] like Figure 2As shown, this invention provides a non-contact emotion recognition method based on frequency-modulated continuous wave radar. The method mainly includes the following processes: (1) transmitting frequency-modulated continuous wave signals by a transmitting antenna array and receiving signals by a receiving antenna array; (2) extracting the echo signal from the chest wall of the monitored object from the radar echo signal using the Minimum Variance Distortionless Response (MVDR) algorithm; (3) demodulating the motion signal of the chest wall of the monitored object from the phase of the radar echo signal; (4) extracting the respiratory signal from the chest wall motion signal using a filter; (5) extracting the duration of successive heartbeats from the motion signal of the chest wall of the monitored object; (6) calculating the physiological features used for emotion recognition; and (7) training a random forest machine learning model to achieve emotion recognition.

[0059] The steps of the method of the present invention will be described in detail below.

[0060] S1: The transmitting antenna array transmits periodic linear frequency modulated waves to the area where the monitored object is located, and the corresponding echo signals are collected by the receiving antenna array.

[0061] In this embodiment, the pulse frequency modulation range of the linear frequency modulated wave is from 77GHz to 81GHz, and the duration of a single pulse is 50μs; the fast time axis sampling frequency is 4MHz, and the number of sampling points is 128; the slow time axis sampling frequency is 100Hz.

[0062] S2: Construct a virtual antenna array based on the geometric characteristics of the transmitting and receiving antenna arrays.

[0063] like Figure 1 The figures show the transmitting antenna array (a), receiving antenna array (b), and virtual antenna array (c) constructed in this embodiment. The transmitting antenna array contains 3 transmitting antennas, with a horizontal distance of λ (where λ is the wavelength of the linear frequency modulated wave) between adjacent transmitting antennas; the receiving antenna array contains 4 receiving antennas, with a horizontal distance of [missing information - likely a value].

[0064] During emotion recognition, the radar position is adjusted so that the plane of the antenna array is parallel to the chest wall surface of the monitored subject, and both the transmitting and receiving antenna arrays are aligned along the central axis of the human body. The angle θ0 between the position of the monitored subject's chest wall and the normal vector of the plane containing the antenna array is measured beforehand using a protractor.

[0065] Based on a virtual antenna array, radar echo signals from the direction of the monitored object are extracted from the echo signals acquired in step S1 using the MVDR beamforming method.

[0066] In this embodiment, the MVDR beamforming method only employs Figure 1 (c) shows the echo signals received by virtual antennas numbered 1-8 in the virtual antenna array. This virtual antenna array contains a total of 12 virtual antennas, all located on the same plane. Taking virtual antenna number 1 as the origin, the virtual coordinates of antennas numbered 2-12 are: (λ / 20), (λ0), (3λ / 20), (2λ0), (5λ / 20), (3λ0), (7λ / 20), (λλ / 2), (3λ / 2λ / 2), (2λλ / 2), (5λ / 2λ / 2). Here, λ is the wavelength of the linear frequency modulated wave.

[0067] The specific method for MVDR beamforming in step S2 is as follows:

[0068] S2-1: Note Figure 1 The radar echo signals received by the 8 virtual antennas numbered 1-8 in (c) after ADC conversion are denoted as x. k Let Tx(n) be the received signal vector, where k = 1, 2, ..., 8.

[0069] T(n)=[x1(n) x2(n) … x8(n)];

[0070] S2-2: Based on the pre-measured azimuth angle θ0 of the monitored object relative to the virtual antenna array, construct the beamforming steering vector α(θ0).

[0071]

[0072] S2-3: Based on the guiding vector α(θ0) described in step S2-2, create the optimal weight vector ω(n).

[0073]

[0074] Where matrix R(n) is the covariance matrix at time n, calculated according to the following formula.

[0075] R(n) = T H (n)T(n);

[0076] S2-4: Based on the optimal weight vector ω(n) described in step S2-3, extract the radar echo signal x(n) from the direction of the monitored object.

[0077] x(n)=ω H (n)T H (n).

[0078] S3: Demodulate the chest wall motion signal of the monitored object from the phase of the radar echo signal in step S2. In this embodiment, this step is specifically as follows:

[0079] S3-1: The radar echo signal x(n) extracted in step S2 is a complex exponential signal, and its mathematical expression is:

[0080]

[0081] Where A represents the amplitude of the signal, which is a constant; f represents the frequency of the signal. Indicates the phase of the signal;

[0082] S3-2: Solve for the phase of x(n) in step S3-1 using the arctangent function. And untangle it;

[0083] S3-3: Based on the results of step S3-2, according to Relationship with the chest wall motion signal D(n) of the monitored object

[0084]

[0085] The chest wall motion signal D(n) of the monitored object is obtained.

[0086] S4: Extract the respiratory signal from the chest wall motion signal in step S3 using a filter.

[0087] Since the respiratory signal dominates the chest wall motion signal D(n) of the monitored object, a high-quality respiratory signal can be extracted using a simple bandpass filter. The filter used in this embodiment is a Butterworth bandpass filter with an order of 20 and a passband of [0.1Hz 0.5Hz].

[0088] S5: Since the amplitude of the heartbeat signal in the chest wall motion signal D(n) of the monitored object is very weak, a non-contact heart rate variability monitoring method based on frequency-modulated continuous wave radar is needed to amplify the heartbeat signal contained in the chest wall motion signal of the monitored object in step S3 and extract the duration of each heartbeat. Let the extracted duration of each heartbeat be H(m), where the index m represents the number of heartbeats.

[0089] The non-contact heart rate variability monitoring method based on frequency-modulated continuous wave radar in this step adopts Chinese invention patent application number CN202210539306.9 and publication number CN114732390A. The method will be briefly described below:

[0090] S8-1: Transmits periodic linear frequency modulated waves to the area where the monitored object is located, and uses a receiving antenna to collect the corresponding radar echo signals.

[0091] In step S8-1, the pulse frequency modulation range of the linear frequency modulated wave is 77GHz to 81GHz, and the duration of a single pulse is 50µs; the fast time axis sampling frequency is 4MHz, and the number of sampling points is 128; the slow time axis sampling frequency is 100Hz.

[0092] S8-2: The acquired radar echo signals are preprocessed by sequentially using frequency mixing and fast Fourier transform.

[0093] In step S8-2, the output after mixing is an intermediate frequency signal; the amplitude of the intermediate frequency signal is a constant, and the frequency and phase are both proportional to the distance between the reflecting object and the receiving antenna array.

[0094] S8-3: Extract the location of the heart region of the monitored object from the preprocessed radar echo signal. This step is detailed below:

[0095] S83-1: For the nth received radar echo signal, after preprocessing in step S8-2, extract the phase of its kth element.

[0096] S83-2: Select a time window of length T, and process it sequentially according to the method in step S83-1 to obtain the sequence Φ. k :

[0097]

[0098] S83-3: For the obtained sequence Φ k The entanglement was de-entangled and its spectrum was calculated using Fast Fourier Transform. Its energy E in the heartbeat frequency band [0.8Hz, 2Hz] was statistically analyzed. k ;

[0099] S83-4: Iterate through k to find E k The maximum value is used to determine the distance between the heart of the monitored object and the radar antenna, which is the location of the heart of the monitored object.

[0100] S8-4: Using a phase correlation method, extract the motion information of the heart region of the monitored object from the position obtained in step S8-3, and calculate its acceleration signal accordingly. This step is detailed below:

[0101] According to the formula Extract the phase sequence reflecting the change in distance between the heart region of the monitored object and the receiving antenna array. After untangling, the corresponding body motion signal sequence {R} is obtained. m}; where λ is the wavelength of the emitted frequency-modulated continuous pulse;

[0102] Then through the formula

[0103]

[0104] The corresponding acceleration signal sequence {a} is obtained. m}; where Δt is the sampling interval of the slow time axis, R m+1 For the body motion signal sequence {R m The (m+1)th term of}, R m For the body motion signal sequence {R m The m-th term of}, R m-1 For the body motion signal sequence {R m The (m-1)th term of}.

[0105] S8-5: The obtained acceleration signal is smoothed by calculating the short-time average power, and the position of the segmentation point between each heartbeat is estimated by peak detection. This step is detailed below:

[0106] S85-1: Define the average power P of the acceleration signal in a neighborhood of length L at time n0. L (n0) is

[0107]

[0108] Where L is half the length of the time window used to calculate the average power, and a i For the acceleration signal sequence {a m The i-th term in};

[0109] Let L1 be the number of sampling points corresponding to half the duration of a single heartbeat cycle, and let L2 = 0.4L1; process the acceleration signal sequence {a} item by item according to the following formula. m}, to obtain the smoothed heartbeat signal sequence {Y m};

[0110]

[0111] Where L1 and L2 represent the different time window lengths selected when calculating the average power. This represents the average power of the acceleration signal within the L1-length neighborhood at time m; This represents the average power of the acceleration signal within the L2-length neighborhood at time m;

[0112] S85-2: For the smoothed heartbeat signal sequence {Y m The index t of all its maxima in the sequence is found by peak detection.

[0113] t = [t1, t2, ... t i …,t I ],

[0114] The obtained t iThat is, the index in the sequence corresponding to the dividing point between each heartbeat; where I represents the total number of maximum points found.

[0115] S8-6: Based on the estimated positions of the segmentation points between each heartbeat obtained in step S8-5, an acceleration template signal for a single heartbeat is generated; subsequently, this acceleration template signal is used to precisely segment the acceleration signal obtained in step S8-4 to obtain the interval between each heartbeat. This step is detailed as follows:

[0116] S86-1: Based on the estimated value t of the position of each heartbeat cycle segmentation point obtained in step S8-5, the acceleration signal sequence {a m The data is divided into (I-1) segments:

[0117]

[0118] S86-2: Averaging the data segment obtained in step S86-1 yields the heart rate acceleration template signal.

[0119]

[0120] During calculation, interpolation is used to ensure the integrity of each acceleration signal segment in S86-1. Consistent length;

[0121] S86-3: Select a new set of splitting points S:

[0122] S = [s1, s2, ..., s I ],

[0123] A set of acceleration signal sequences {a m The new truncation:

[0124]

[0125] S86-4: The degree of matching between the new truncation obtained in step S86-3 and the acceleration template signal is defined by the following formula:

[0126]

[0127] Interpolation is used during calculation to ensure that each Same length as Template;

[0128] S86-5: Repeat steps S86-3 to S86-4 to calculate the degree of matching between the truncation and acceleration template signal corresponding to each set of segmentation points, and find the set of segmentation points S with the smallest D. s As the optimal segmentation under the current template signal; based on S sFollow step S86-2 to generate a new template signal;

[0129] S86-6: Repeat the process described in S86-5 until D calculated by step S86-4 in a certain repetition is less than the preset threshold. The set of split points S at this time is the final position of the split points between each heartbeat.

[0130] S8-7: Based on the heart rate interval obtained in step S8-6, calculate the heart rate variability index to achieve non-contact heart rate variability monitoring.

[0131] S6: Based on the chest wall motion signal obtained in step S3, the respiratory signal obtained in step S4, and the duration of each heartbeat obtained in step S5, calculate the time domain features, frequency domain features, and nonlinear domain features respectively, and construct a feature vector for emotion recognition.

[0132] Before calculation, the chest wall motion signal D(n) obtained in step S3, the respiratory signal B(n) obtained in step S4, and the duration H(m) of each heartbeat obtained in step S5 are simultaneously divided into data segments with a time length of 1 minute, and each data segment is considered as a sample. Using D(n), B(n), and H(m) from each sample, a total of 62 eigenvalues ​​are calculated, including 26 time-domain eigenvalues, 23 frequency-domain eigenvalues, and 13 nonlinear eigenvalues. These eigenvalues ​​are listed below:

[0133] Time-domain eigenvalues: (1) Variance of D(n); (2) Mean of the absolute value of the first difference of D(n); (3) Mean of the absolute value of the normalized first difference of D(n); (4) Mean of the absolute value of the second difference of D(n); (5) Mean of the absolute value of the normalized second difference of D(n); (6) Mean of the time interval between adjacent peak points in B(n); (7) Mean of the time interval between adjacent valley points in B(n); (8) Variance of the time interval between adjacent peak points in B(n); (9) Variance of the time interval between adjacent valley points in B(n); (10) Reciprocal of the mean of the time interval between adjacent peak points in B(n); (11) Reciprocal of the mean of the time interval between adjacent valley points in B(n); (12) Mean of the absolute value of the first difference of B(n); (1 3) Mean of the absolute values ​​of the first-order differences of B(n) after normalization; (14) Mean of the absolute values ​​of the second-order differences of B(n); (15) Mean of the absolute values ​​of the second-order differences of B(n) after normalization; (16) Mean of H(m); (17) Reciprocal of the mean of H(m); (18) Standard deviation of H(m); (19) Root mean square of the first-order differences of H(m); (20) H(m) (21) Mean of the absolute values ​​of the first-order differences of H(m) after normalization; (22) Mean of the absolute values ​​of the second-order differences of H(m); (23) Mean of the absolute values ​​of the second-order differences of H(m) after normalization; (24) Skewness of H(m); (25) Kurtosis of H(m); (26) Proportion of terms with absolute values ​​greater than 50ms in the first-order differences of H(m).

[0134] Frequency domain characteristic values: (1) The average amplitude of frequency components less than 0.1Hz in the spectrum of D(n); (2) The average amplitude of frequency components in the spectrum of D(n) between [0.1Hz and 0.2Hz); (3) The average amplitude of frequency components in the spectrum of D(n) between [0.2Hz and 0.3Hz); (4) The average amplitude of frequency components in the spectrum of D(n) between [0.3Hz and 0.4Hz); (5) The average amplitude of frequency components less than 0.1Hz in the spectrum of B(n); (6) The average amplitude of frequency components in the spectrum of B(n) between [0.1Hz and 0.2Hz); (7) The average amplitude of frequency components in the spectrum of B(n) between [0.2Hz and 0.3Hz); (8) B (9) The average amplitude of the frequency components in the spectrum of B(n) between [0.3Hz and 0.4Hz); (10) The average amplitude of the frequency components in the spectrum of B(n) between [0.4Hz and 0.9Hz); (11) The average amplitude of the frequency components in the spectrum of B(n) between [0.9Hz and 1.5Hz); (12) The ratio of the average amplitude of the frequency components in the spectrum of B(n) between [0.1Hz and 0.4Hz] to the average amplitude of the frequency components in the spectrum between [0.4Hz and 1.5Hz]; (13) The sum of the amplitudes of the frequency components less than 0.04Hz in the spectrum of H(m); (14) The sum of the amplitudes of the frequency components in the spectrum of H(m) between [0.04Hz and 0.15Hz]; (15) The average amplitude of the frequency components in the spectrum of D(n). (15) The sum of the amplitudes of the frequency components whose spectrum is between [0.15Hz and 0.4Hz); (16) The frequency with the largest amplitude among the frequency components with a frequency range less than 0.04Hz in the spectrum of H(m); (17) The frequency with the largest amplitude among the frequency components with a frequency range between [0.04Hz and 0.15Hz] in the spectrum of H(m); (18) The ratio of the sum of the amplitudes of the frequency components with a frequency range between [0.04Hz and 0.15Hz] in the spectrum of H(m) to the sum of the amplitudes of the frequency components with a frequency range less than 0.4Hz in the spectrum of H(m); (19) The amplitude of the frequency components with a frequency range between [0.04Hz and 0.15Hz] in the spectrum of H(m). (20) The ratio of the sum of the amplitudes of the frequency components of H(m) with a spectrum less than 0.4 Hz to the sum of the amplitudes of the frequency components of H(m) with a spectrum between [0.15 Hz and 0.4 Hz]; (21) The ratio of the sum of the amplitudes of the frequency components of H(m) with a spectrum less than 0.4 Hz to the sum of the amplitudes of the frequency components of H(m) with a spectrum between [0.04 Hz and 0.15 Hz]; (22) The ratio of the sum of the amplitudes of the frequency components of H(m) with a spectrum less than 0.4 Hz to the sum of the amplitudes of the frequency components of H(m) with a spectrum between [0.15 Hz and 0.4 Hz]; (23) The ratio of the sum of the amplitudes of the frequency components of H(m) with a spectrum between [0.04 Hz and 0.4 Hz];The sum of the amplitudes of the frequency components (15 Hz) is the ratio of the sum of the amplitudes of the frequency components (0.15 Hz - 0.4 Hz) in the spectrum of H(m).

[0135] Nonlinear eigenvalues: (1) Approximate entropy of D(n); (2) First-order detrending fluctuation analysis coefficient of D(n); (3) Second-order detrending fluctuation analysis coefficient of D(n); (4) Approximate entropy of B(n); (5) First-order detrending fluctuation analysis coefficient of B(n); (6) Second-order detrending fluctuation analysis coefficient of B(n); (7) Approximate entropy of H(m); (8) First-order detrending fluctuation analysis coefficient of H(m); (9) Second-order detrending fluctuation analysis coefficient of H(m); (10) SD1 in the Poincaré scatter plot of H(m); (11) SD2 in the Poincaré scatter plot of H(m); (12) Product of SD1 and SD2 in the Poincaré scatter plot of H(m); (13) Ratio of SD1 to SD2 in the Poincaré scatter plot of H(m).

[0136] S7: Based on the feature vector described in step S6, train a random forest machine learning model for emotion recognition to complete the emotion classification task.

[0137] In this embodiment, corresponding feature values ​​are calculated for each group of samples collected in the experiment. Each group of samples includes the emotional state of the monitored subjects as reported through a questionnaire at the time of sample collection, which is used as a training dataset to train the random forest machine learning model for emotion recognition. The random forest model used contains 200 decision trees. To balance training time and recognition accuracy, the minimum number of samples contained in any child node generated after branching from each node is set to 2.

[0138] When emotion recognition is required, it is only necessary to collect the physiological characteristic signals of the monitored object for 1 minute through frequency-modulated continuous wave radar, and then calculate the above 62 feature values ​​as input to the random forest machine learning model to recognize the emotion of the monitored object.

[0139] In this embodiment, the sample set contains 348 samples with a duration of 1 minute, including 90 happy samples, 58 sad samples, 90 fear samples, and 110 relaxed samples. These 348 samples in the test set come from the same 6 subjects in the training set. The out-of-bag error assessment method is used to evaluate the emotion recognition performance of the random forest machine learning model established in step S7. The recognition accuracy for different combinations of the four emotions is obtained as follows: Figure 3 As shown, the overall accuracy rate for recognizing the four emotions of happiness, sadness, fear, and ease is 59.4%.

[0140] The embodiments described above are merely preferred embodiments of the present invention and are not intended to limit the invention. Those skilled in the art can make various changes and modifications without departing from the spirit and scope of the invention. Therefore, all technical solutions obtained through equivalent substitution or transformation fall within the protection scope of the present invention.

Claims

1. A non-contact emotion recognition method based on frequency-modulated continuous wave radar, characterized in that, Specifically as follows: S1: The transmitting antenna array transmits periodic linear frequency modulated waves to the area where the monitored object is located, and the corresponding echo signals are collected by the receiving antenna array; S2: Construct a virtual antenna array based on the geometric features of the transmitting antenna array and the receiving antenna array; based on the virtual antenna array, extract the radar echo signal from the direction of the monitored object from the echo signal acquired in step S1 using the MVDR beamforming method. S3: Demodulate the chest wall motion signal of the monitored object from the phase of the radar echo signal; S4: Extract respiratory signals from the chest wall motion signals using a filter; S5: Extract the duration of successive heartbeats from the chest wall motion signal of the monitored object using a non-contact heart rate variability monitoring method based on frequency modulated continuous wave radar; S6: Based on the chest wall motion signal mentioned in step S3, the respiratory signal mentioned in step S4, and the duration of each heartbeat mentioned in step S5, calculate the time domain features, frequency domain features, and nonlinear domain features respectively to construct a feature vector for emotion recognition. S7: Based on the feature vector described in step S6, train a random forest machine learning model for emotion recognition to complete the emotion classification task; The nonlinear domain features include: Approximate entropy; The first-order detrended fluctuation analysis coefficient; The second-order detrended volatility analysis coefficient; Approximate entropy; The first-order detrended fluctuation analysis coefficient; The second-order detrended volatility analysis coefficient; Approximate entropy; The first-order detrended fluctuation analysis coefficient; The second-order detrended volatility analysis coefficient; Poincaré scatter plot ; Poincaré scatter plot ; Poincaré scatter plot and The product; Poincaré scatter plot and The ratio; where, This is a signal of chest wall movement. This is a breathing signal. The duration of each heartbeat.

2. The non-contact emotion recognition method based on frequency-modulated continuous wave radar according to claim 1, characterized in that, The linear frequency modulated wave has a pulse frequency modulation range of 77GHz to 81GHz, and a single pulse duration of 50μs; the fast time axis sampling frequency is 4MHz, and the number of sampling points is 128; the slow time axis sampling frequency is 100Hz.

3. The non-contact emotion recognition method based on frequency-modulated continuous wave radar according to claim 1, characterized in that, The transmitting antenna array comprises three transmitting antennas, with a horizontal distance between any two adjacent transmitting antennas. The receiving antenna array comprises four receiving antennas, with a horizontal distance of [missing information] between any two adjacent receiving antennas. The virtual antenna array comprises 12 virtual antennas located on the same plane. Taking virtual antenna number 1 as the origin, the coordinates of virtual antennas numbered 2 through 12 are as follows: , , , , , , , , , , ;in, The wavelength of the linear frequency modulated wave.

4. The non-contact emotion recognition method based on frequency-modulated continuous wave radar according to claim 3, characterized in that, The MVDR beamforming method in S2 uses only the echo signals received by the virtual antennas numbered 1-8 in the virtual antenna array.

5. The non-contact emotion recognition method based on frequency-modulated continuous wave radar according to claim 1, characterized in that, The specific method for MVDR beamforming in step S2 is as follows: S2-1: The radar echo signals received by the 8 virtual antennas in the virtual antenna array after ADC conversion are respectively denoted as... ,in Let the received signal vector be denoted. for ； S2-2: Based on the pre-measured orientation angle of the monitored object relative to the virtual antenna array Constructing the beamforming guide vector , ； S2-3: Based on the guiding vector described in step S2-2 Create the optimal weight vector , ； Where the matrix for The covariance matrix at time t is calculated according to the following formula. ； S2-4: Based on the optimal weight vector described in step S2-3 Extract radar echo signals from the direction of the monitored object. , 。 6. The non-contact emotion recognition method based on frequency-modulated continuous wave radar according to claim 1, characterized in that, Step S3 is as follows: S3-1: Based on the radar echo signal extracted in step S2 It is a complex exponential signal, and its mathematical expression is: ， in This represents the amplitude of the signal and is a constant. This indicates the frequency of the signal. Indicates the phase of the signal; S3-2: Solve step S3-1 using the arctangent function. phase And untangle it; S3-3: Based on the results of step S3-2, according to Chest wall motion signals of the monitored subject Relationship ， Obtain chest wall motion signals from the monitored subject .

7. The non-contact emotion recognition method based on frequency-modulated continuous wave radar according to claim 1, characterized in that, The filter is a Butterworth bandpass filter with an order of 20 and a passband of [missing information]. .

8. A non-contact emotion recognition method based on frequency-modulated continuous wave radar according to claim 1, characterized in that, The specific method for non-contact heart rate variability monitoring based on frequency-modulated continuous wave radar in step S5 is as follows: S8-1: Transmits periodic linear frequency modulated waves to the area where the monitored object is located, and uses a receiving antenna to collect the corresponding radar echo signals; S8-2: The acquired radar echo signals are preprocessed by sequentially using frequency mixing and fast Fourier transform. S8-3: Extract the location of the heart of the monitored object from the preprocessed radar echo signal; S8-4: Using a phase correlation method, extract the motion information of the heart region of the monitored object from the position obtained in step S8-3, and calculate its acceleration signal accordingly; S8-5: The obtained acceleration signal is smoothed by calculating the short-time average power, and the position of the segmentation point between each heartbeat is estimated by peak detection. S8-6: Based on the positions of the segmentation points between each heartbeat estimated in step S8-5, generate the acceleration template signal of a single heartbeat; Then, the acceleration template signal is used to precisely segment the acceleration signal obtained in step S8-4 to obtain the interval between heartbeats for each heartbeat. S8-7: Based on the heart rate interval obtained in step S8-6, calculate the heart rate variability index to achieve non-contact heart rate variability monitoring.

9. A non-contact emotion recognition method based on frequency-modulated continuous wave radar according to claim 1, characterized in that, Step S6 is as follows: According to the chest wall motion signal described in step S3 The respiratory signal mentioned in step S4 and the duration of each heartbeat as described in step S5 Based on the calculated 26 time-domain features, 23 frequency-domain features, and 13 nonlinear features, a feature vector for this set of samples is created. ; The time-domain features include: The variance; The mean of the absolute values ​​of the first-order differences; normalized The mean of the absolute values ​​of the first-order differences; The mean of the absolute values ​​of the second difference; normalized The mean of the absolute values ​​of the second difference; The average time interval between adjacent peak points; The average time interval between adjacent valley points; The variance of the time interval between adjacent peak points; The variance of the time interval between adjacent valley points; The reciprocal of the mean of the time intervals between adjacent peak points; The reciprocal of the mean of the time intervals between adjacent valley points; The mean of the absolute values ​​of the first-order differences; normalized The mean of the absolute values ​​of the first-order differences; The mean of the absolute values ​​of the second difference; normalized The mean of the absolute values ​​of the second difference; The mean; The reciprocal of the mean; Standard deviation; The root mean square value of the first difference; The mean of the absolute values ​​of the first-order differences; normalized The mean of the absolute values ​​of the first-order differences; The mean of the absolute values ​​of the second difference; normalized The mean of the absolute values ​​of the second difference; skewness; The ravine; The absolute value of the first difference is greater than The proportion of items; The frequency domain features include: The spectrum of less than The average amplitude of the frequency components; The spectrum is between The average amplitude of the frequency components; The spectrum is between The average amplitude of the frequency components; The spectrum is between The average amplitude of the frequency components; The spectrum of less than The average amplitude of the frequency components; The spectrum is between The average amplitude of the frequency components; The spectrum is between The average amplitude of the frequency components; The spectrum is between The average amplitude of the frequency components; The spectrum is between The average amplitude of the frequency components; The spectrum is between The average amplitude of the frequency components; The spectrum is between The average amplitude of the frequency components and between The ratio of the average amplitude of the frequency components; The spectrum of less than The sum of the amplitudes of the frequency components; The spectrum is between The sum of the amplitudes of the frequency components; The spectrum is between The sum of the amplitudes of the frequency components; The spectrum of less than The frequency with the largest amplitude among the frequency components; The spectrum is between The frequency with the largest amplitude among the frequency components; The spectrum is between The frequency with the largest amplitude among the frequency components; The spectrum is between The sum of the amplitudes of the frequency components and Spectrum less The ratio of the sum of the amplitudes of the frequency components; The spectrum is between The sum of the amplitudes of the frequency components and Spectrum less The ratio of the sum of the amplitudes of the frequency components; The spectrum is between The sum of the amplitudes of the frequency components and Spectrum less The ratio of the sum of the amplitudes of the frequency components; Spectrum less The sum of the amplitudes of the frequency components and The spectrum is between The ratio of the sum of the amplitudes of the frequency components; Spectrum less The sum of the amplitudes of the frequency components and The spectrum is between The ratio of the sum of the amplitudes of the frequency components; Spectrum between The sum of the amplitudes of the frequency components and The spectrum is between The ratio of the sum of the amplitudes of the frequency components.

10. A non-contact emotion recognition method based on frequency-modulated continuous wave radar according to claim 1, characterized in that, In the random forest machine learning model of step S7, the number of decision trees is 200, and it is set that any child node generated after branching of each node contains at least 2 samples.

Images