In-vehicle liveness detection method based on multi-dimensional features of radar echo signals

Abstract

This invention discloses an in-vehicle liveness detection method based on multi-dimensional features of radar echo signals. Utilizing millimeter-wave radar, this method better respects user privacy. It performs liveness detection by processing reflected echoes from targets inside the vehicle in a short time, offering strong real-time performance, independence from lighting conditions, and high environmental adaptability. The invention extracts multi-dimensional features from radar echo signals to generate a training set, which is then input into a support vector machine to train a classification model. The trained classification model determines whether a target is alive, detecting situations such as children left inside the vehicle or unauthorized intrusions, thus issuing alarms to reduce accidents. This invention can be applied to multi-person liveness detection in multiple areas within a vehicle, achieving 100% accuracy.

Description

Technical Field

[0001] This invention relates to a method for detecting live bodies inside a vehicle. Background Technology

[0002] With the development of automotive technology, cars have gradually become a part of everyday life, greatly facilitating people's travel. However, many safety hazards have also arisen, such as children and pets suffocating or dying from heatstroke after being forgotten in cars, and thieves breaking into vehicles to steal valuables. To address these safety hazards and prevent risks, research into in-vehicle liveness detection technology has significant practical implications.

[0003] Millimeter-wave radar has the characteristics of all-day and all-weather operation, as well as advantages such as low cost, small size, high resolution, and strong anti-interference ability, and can be used for in-vehicle liveness detection.

[0004] Existing liveness detection methods include camera imaging, detection by extracting respiratory and heartbeat signals, and temperature sensor detection. Camera imaging detection is limited by installation location, easily affected by obstructions, and poses some risk to user privacy. Liveness detection by extracting respiratory and heartbeat information involves filtering the processed radar echo signal and extracting frequencies in the corresponding band to determine if the person is alive. However, phase information is sensitive and easily affected by the environment, and the long cycle of respiratory signals results in poor real-time performance and a high risk of false positives. Temperature sensor detection also has certain environmental requirements for accuracy. Summary of the Invention

[0005] The technical problem this invention aims to solve is the low accuracy of existing live animal detection methods, which are easily affected by environmental factors.

[0006] To solve the above-mentioned technical problems, the technical solution of the present invention is to provide an in-vehicle liveness detection method based on multi-dimensional features of radar echo signals, characterized by comprising the following steps:

[0007] Step 1: Install millimeter-wave radar inside the vehicle. The millimeter-wave radar emits millimeter-wave signals that can cover the entire interior of the vehicle and has I channels.

[0008] Step 2: Divide the interior space into different target detection areas. Each target detection area corresponds to one of the seats in the vehicle. Based on the installation position of the millimeter-wave radar, set the corresponding radial distance unit range dist1~dist2 and the corresponding angle unit range θ1~θ2 for each target detection area.

[0009] Step 3: Acquire the reflected echo signals. Within each detection period, process the echo signals of a total frame length of K frames to extract the target distance features obj for each target detection area. dist-diff-totalTarget angle features obj angle-diff-total and target energy characteristics Power var Then, the data is input into the trained classification model, which outputs the detection result of whether a live object exists in each target detection region. Specifically, the target distance feature `obj` of the current target detection region is extracted. dist-diff-total Target angle features obj anglk-diff-total and target energy characteristics Power var Includes the following steps:

[0010] Step 301: Perform a one-dimensional Fourier transform on the echo signal of each channel of each frame to obtain the distance-time spectrum y. 1d Filter out the distance-time spectrum y of each frame of echo signal 1d Obtaining static clutter filtering data for each frame of echo signal. And filter out static clutter data Perform a Fourier transform in the velocity dimension to obtain the distance-Doppler data of each frame of echo signal;

[0011] Step 302: Based on the multi-channel accumulated range-Doppler data, extract the maximum energy value of the current target detection area within the radial distance cell range dist1 to dist2. This leads to the attainment of the maximum energy value. The corresponding radial distance element obj radical-dist (k) and velocity unit obj vel (k), k = 1, 2, ..., K;

[0012] Step 303: Perform an angle-dimensional Fourier transform on the range-Doppler data to obtain angle spectrum data, and retrieve the target energy peak value within the angle cell range θ1 to θ2 of the current target detection area. Thus, the peak energy of the target can be obtained. The corresponding angle unit obj angle (k), where the target energy peak value is... Represented as:

[0013]

[0014]

[0015] In the formula: θ is the angular cell index of the target detection area, θ∈[θ1,θ2]; c is the angular cell index, c=1,2,…,Q, Q is the number of points in the angular Fourier transform; y 2d[a,b,i] represents the range-Doppler data, where a is the range cell index, a = 1, 2, ..., N, and N is the number of sampling points in the fast time dimension; b is the velocity cell index, b = 1, 2, ..., M, and M is the number of sampling points in the slow time dimension.

[0016] Step 304: Detect the K groups of distance units obj within the current detection period. radical-dist (k), Angle unit obj angle (k) and target energy peak Process and transform into features:

[0017] Calculate the distance unit obj between two consecutive frames within the current detection period. radical-dist (k) The sum of the differences of obj dist-diff-total Let this be denoted as the target distance feature, then we have:

[0018]

[0019] Calculate the angle unit obj for two consecutive frames within the current detection period. angle The sum of the differences in ()obj anglk-diff-total Let this be the target angle feature, then we have:

[0020]

[0021] Calculate the intra-frame energy variance Power within the current detection period. var As a characteristic of the target energy, we have:

[0022]

[0023] In the formula,

[0024] Preferably, in step 301, the distance-time spectrum y obtained from the echo signal of the i-th virtual channel of each frame of echo signal is... 1d Represented as y 1d [a,m,i], then we have:

[0025]

[0026] In the formula: y[n,m,i] is the echo signal of the nth sampling point of the mth chirp of the i-th channel of each frame echo signal.

[0027] Preferably, y[n,m,i] is represented as:

[0028]

[0029] In the formula: m = 1, 2, ..., M, where M is the number of sampling points in the slow time dimension; n = 1, 2, ..., N, where N is the number of sampling points in the fast time dimension; AR To receive signal energy; f b T is the beat frequency; f T is the sampling interval of the fast time-dimensional ADC; λ is the wavelength; s d represents the slow-time sampling interval; R represents the radial distance of the target; i Let θ be the distance of the i-th channel relative to the first channel; ′ The angle corresponding to the target.

[0030] Preferably, in step 301, the method for filtering out stationary clutter is to analyze the distance-time spectrum y of each frame of echo signal. 1d The difference between it and its mean in the fast time dimension is as follows:

[0031]

[0032]

[0033] In the formula: For distance-time spectrum y 1d [a,m,i] represents the one-dimensional Fourier transform data after filtering out static clutter. For distance-time spectrum y 1d The mean of [a,m,i] over the fast time dimension.

[0034] Preferably, in step 301, through the... The obtained distance-Doppler data is represented as y 2d [a,b,i], then we have:

[0035] Preferably, in step 302, the maximum energy value Represented as:

[0036]

[0037]

[0038] In the formula: a = dist1, dist1+1, ..., dist2; max(·) is the maximum value function; |·| is the modulo operation; y 2d(k) [a,b,i] represents the range-Doppler data of the k-th frame echo signal; y 2d_u(k) [a,b] represents the range-Doppler data accumulated from the echo signal of the k-th frame.

[0039] Preferably, in step 303, the y 2d The angle spectrum data obtained by performing a Fourier transform on [a,b,i] in the angle dimension is represented as y. 3d [a,b,c], then:

[0040]

[0041] In the formula: c is the angular unit index, c = 1, 2, ..., Q, and Q is the number of points in the angular Fourier transform.

[0042] Preferably, in step 3, the classification model is established and trained based on support vector machines.

[0043] This invention utilizes millimeter-wave radar, which better respects user privacy. It performs liveness detection by processing reflected echoes from targets inside the vehicle in a short time, offering strong real-time performance, unaffected by lighting conditions, and high environmental adaptability. This invention extracts multi-dimensional features from radar echo signals to generate a training set, which is then input into a support vector machine to train a classification model. The trained classification model determines whether a target is alive, detecting situations such as children left inside the cabin or unauthorized intrusions, thus issuing alarms to reduce accidents. This invention can be applied to multi-person liveness detection in multiple areas inside a vehicle, achieving 100% accuracy. Attached Figure Description

[0044] Figure 1 This is a flowchart of the method disclosed in this invention;

[0045] Figure 2 This indicates the angle range covered by the driver's seat in the front.

[0046] Figure 3 This illustrates the comparison of target distance features with and without a living organism present;

[0047] Figure 4 This illustrates the comparison of target angle features with and without a living organism present.

[0048] Figure 5 This illustrates the comparison of target energy characteristics with and without a living organism. Detailed Implementation

[0049] The present invention will be further illustrated below with reference to specific embodiments. It should be understood that these embodiments are for illustrative purposes only and are not intended to limit the scope of the invention. Furthermore, it should be understood that after reading the teachings of this invention, those skilled in the art can make various alterations or modifications to the invention, and these equivalent forms also fall within the scope defined by the appended claims.

[0050] This invention discloses an in-vehicle liveness detection method based on multi-dimensional features of radar echo signals. This method extracts energy features from data collected under different conditions, generates a training set for classification training, produces a classification model, and then uses this model to process the test set, outputting liveness detection results. Figure 1 As shown, the specific steps include:

[0051] A MIMO-based millimeter-wave radar is installed in a fixed location inside the vehicle to ensure that the emitted millimeter-wave signal can cover the entire interior. The transmit / receive mode is one transmitter and four receivers. The vehicle's interior space is divided into different target detection zones based on the number of seats, with each zone corresponding to a seat, ensuring that only one target can exist within each zone. Each target detection zone is assigned a corresponding radial distance unit range of dist1 to dist2 and a corresponding angular unit range of θ1 to θ2. For example... Figure 1 As shown, by setting the area angle unit range θ1~θ2, the driver's seat and the passenger seat in the front can be effectively separated.

[0052] The millimeter-wave radar transmits signals at a frame rate of K frames / s and has I virtual channels. Reflected echo signals are acquired. The output time for liveness detection results is set to 1 second, as required. The echo signals, totaling K frames within 1 second, are processed to obtain the target distance, target angle, and target energy characteristics for each target detection area. A classification model is then used to detect the presence of a liveness target in the current detection area.

[0053] Determining whether a live organism is present in the current target detection area includes the following steps:

[0054] Let y[n,m] represent the nth sample point of the mth chirp in each virtual channel of the echo signal in each frame, and let y[n,m,i] represent the echo signal of the nth sample point of the mth chirp in the i-th virtual channel of the echo signal in each frame. Then we have:

[0055]

[0056]

[0057] In the formula: m = 1, 2, ..., M, where M is the number of sampling points in the slow time dimension; n = 1, 2, ..., N, where N is the number of sampling points in the fast time dimension; i = 1, 2, ..., I, where I is the number of channels; A R To receive signal energy; f b T is the beat frequency; f T is the sampling interval of the fast time-dimensional ADC; λ is the wavelength; s d represents the slow-time sampling interval; R represents the radial distance of the target; i Let θ be the distance between the i-th virtual channel and the first virtual channel; ′ The angle corresponding to the target.

[0058] Perform a one-dimensional Fourier transform on the echo signal of each virtual channel of each frame to obtain the distance-time spectrum y. 1dThe distance-time spectrum y is obtained for the echo signal of the i-th virtual channel of each frame of echo signal. 1d Represented as y 1d [a,m,i], then we have:

[0059]

[0060] In the formula, 'a' is the distance cell index.

[0061] Filtering out the distance-time spectrum y of each frame of echo signal 1d Obtaining static clutter filtering data for each frame of echo signal. To eliminate environmental interference. In this embodiment, the specific method for filtering out stationary clutter is to analyze the distance-time spectrum y of each frame of echo signal. 1d The difference between it and its mean in the fast time dimension is as follows:

[0062]

[0063]

[0064] In the formula: For y 1d [a,m,i] represents the one-dimensional Fourier transform data after filtering out static clutter. For y 1d The mean of [a,m,i] over the fast time dimension.

[0065] Filtering data from stationary clutter Performing a Fourier transform in the velocity dimension yields the range-Doppler data for each frame of the echo signal. The obtained distance-Doppler data is represented as y 2d [a,b,i], then we have:

[0066]

[0067] In the formula, b is the velocity element index.

[0068] Distance-Doppler data y for each frame's multi-channel range 2d To accumulate knowledge, one must consider:

[0069]

[0070] In the formula: y 2d_(k) This represents the range-Doppler data accumulated from the echo signal of the k-th frame.

[0071] Based on the accumulated range-Doppler data, the maximum energy and corresponding radial distance information of the current target detection region (the radial distance cell range is dist1 to dist2) are extracted, as shown in the following formula:

[0072]

[0073] In the formula: a = dist1, ...,ist2; For y 2d_u(k) The maximum value within the current detection area, 2d_(k) [a,b] represents the range-Doppler data accumulated from the echo signal of the k-th frame. k = 1, 2, ..., K; max(·) is the maximum value function; |·| is the modulo operation;

[0074] get The corresponding radial distance unit obj radical-dist (k) and velocity unit obj vel (k).

[0075] To determine the different locations within different angular ranges, a Fourier transform in the angular dimension is performed on the range-Doppler data. For y... 2d The data obtained by performing a Fourier transform on [a,b,i] in the angular dimension is represented as y. 3d [a,b,c], then:

[0076]

[0077] In the formula: c is the angular unit index, c = 1, 2, ..., Q, and Q is the number of points in the angular Fourier transform.

[0078] Based on y 3d [a,b,c], retrieves the target energy peak value of the current target detection region. Then we have:

[0079]

[0080] In the formula: denoted as the peak value of the target energy detected in the current target detection area by the echo signal of the k-th frame.

[0081] get The corresponding angle unit obj angle ().

[0082] The K sets of distance, angle, and energy information for each target detected per second are processed and converted into features:

[0083] Calculate the sum of the differences in target range cell changes between two consecutive frames, obj dist-diff-total Let this be denoted as the target distance feature, then we have:

[0084]

[0085] Calculate the sum of the differences in the target angle unit changes between two consecutive frames, obj angle-diff-total Let this be denoted as the target angle feature, then we have:

[0086]

[0087] Calculate the power variance per second within a frame. var As a characteristic of the target energy, we have:

[0088]

[0089]

[0090] Figures 3 to 5 This illustrates the comparison of relevant features obtained from two sets of 24-second data collected in scenarios where a live object is present and a live object is not present in the same target detection area.

[0091] The target distance features, target angle features, and target energy features of the current target detection area are input into the trained classification model, and the classification model outputs the detection result of whether there is a live object in the current target detection area.

[0092] In this embodiment, the classification model is trained using the following method:

[0093] Multiple sets of data were collected for scenarios where a live object was present and not present within the same target detection area inside the vehicle. Each second, three features of the detected target within the current detection area were generated, resulting in 400 samples, with 50% for live objects and 50% for non-live objects, which served as the training set. The results were classified as 1 (live object present) and 0 (live object not present). The training set and results were then input into a support vector machine to train and generate a classification model.

[0094] Multiple sets of test data were collected, generating 100 sets of feature samples, with 50% containing live individuals and 50% not, forming the test set. The test set was input into the trained classification model for classification, and the results were compared with the actual results to obtain 100% accuracy. The confusion matrix is ​​shown in Table 1 below.

[0095]

[0096] Table 1

[0097] Compared with commonly used breathing and heartbeat detection methods, the above-mentioned energy-based liveness detection method has the advantages of short processing cycle and strong real-time performance. Compared with camera imaging methods, it is less affected by obstructions. Compared with temperature sensors, it is less affected by the environment and the results are more stable.

Claims

1. A method for in-vehicle liveness detection based on multi-dimensional features of radar echo signals, characterized in that, Includes the following steps: Step 1: Install millimeter-wave radar inside the vehicle. The millimeter-wave radar emits millimeter-wave signals that can cover the entire interior of the vehicle and has I channels. Step 2: Divide the interior space into different target detection areas. Each target detection area corresponds to one of the seats in the vehicle. Based on the installation position of the millimeter-wave radar, set the corresponding radial distance unit range dist1~dist2 and the corresponding angle unit range θ1~θ2 for each target detection area. Step 3: Acquire the reflected echo signals. Within each detection period, process the echo signals of a total frame length of K frames to extract the target distance features obj for each target detection area. dist-diff-total Target angle features obj angle-diff-total and target energy characteristics Power var Then, the data is input into the trained classification model, which outputs the detection result of whether a live object exists in each target detection region. Specifically, the target distance feature `obj` of the current target detection region is extracted. dist-diff-total Target angle features obj angle-diff-total and target energy characteristics Power var Includes the following steps: Step 301: Perform a one-dimensional Fourier transform on the echo signal of each channel of each frame to obtain the distance-time spectrum y. 1d Filter out the distance-time spectrum y of each frame of echo signal 1d Obtaining static clutter filtering data for each frame of echo signal. And filter out static clutter data Perform a Fourier transform in the velocity dimension to obtain the distance-Doppler data of each frame of echo signal; Step 302: Based on the accumulated range-Doppler data, extract the maximum energy value of the current target detection area within the radial range of dist1 to dist2. Thus, the maximum energy value is obtained. The corresponding radial distance element obj radical-dist (k) and velocity unit obj vel (k), k = 1, 2, ..., K; Step 303: Perform an angle-dimensional Fourier transform on the range-Doppler data to obtain angle spectrum data, and retrieve the target energy peak value within the angle cell range θ1 to θ2 of the current target detection area. Thus, the peak energy of the target can be obtained. The corresponding angle unit obj angle (k) and distance unit obj radical-dist (k), where the target energy peak value is... Expressed as: In the formula: θ is the angular cell index of the target detection area, θ∈[θ1,θ2]; c is the angular cell index, c=1,2,…,Q, Q is the number of points in the angular Fourier transform; y 2d [a,b,i] represents the range-Doppler data, where a is the range cell index, a = 1, 2, ..., N, and N is the number of sampling points in the fast time dimension; b is the velocity cell index, b = 1, 2, ..., M, and M is the number of sampling points in the slow time dimension; Step 304: Detect the K groups of distance units obj within the current detection period. radical-dist (k), Angle unit obj angle (k) and target energy peak Process and transform into features: Calculate the distance unit obj between two consecutive frames within the current detection period. radical-dist (k) The sum of the differences of obj dist-diff-total Let this be denoted as the target distance feature, then we have: Calculate the angle unit obj for two consecutive frames within the current detection period. angle (k) The sum of the differences of obj angle-diff-total Let this be denoted as the target angle feature, then we have: Calculate the intra-frame energy variance Power within the current detection period. var As a characteristic of the target energy, we have: In the formula, 2. The in-vehicle liveness detection method based on multi-dimensional features of radar echo signals as described in claim 1, characterized in that, In step 301, the distance-time spectrum y obtained from the echo signal of the i-th virtual channel of each frame of echo signal is... 1d Represented as y 1d [a,m,i], then we have: In the formula: y[n,m,i] is the echo signal of the nth sampling point of the mth chirp of the i-th channel of each frame echo signal.

3. The in-vehicle liveness detection method based on multi-dimensional features of radar echo signals as described in claim 2, characterized in that, The term y[n,m,i] is represented as: In the formula: m = 1, 2, ..., M, where M is the number of sampling points in the slow time dimension; n = 1, 2, ..., N, where N is the number of sampling points in the fast time dimension; A R To receive signal energy; f b T is the beat frequency; f T is the sampling interval of the fast time-dimensional ADC; λ is the wavelength; s d represents the slow-time sampling interval; R represents the radial distance of the target; i Let θ be the distance of the i-th channel relative to the first channel; ′ The angle corresponding to the target.

4. The in-vehicle liveness detection method based on multi-dimensional features of radar echo signals as described in claim 3, characterized in that, In step 301, the method for filtering out stationary clutter is to analyze the distance-time spectrum y of each frame of echo signal. 1d The difference between it and its mean in the fast time dimension is as follows: In the formula: For distance-time spectrum y 1d [a,m,i] represents the one-dimensional Fourier transform data after filtering out static clutter. For distance-time spectrum y 1d The mean of [a,m,i] over the fast time dimension.

5. The in-vehicle liveness detection method based on multi-dimensional features of radar echo signals as described in claim 4, characterized in that, In step 301, through the aforementioned The obtained distance-Doppler data is represented as y 2d [a,b,i], then we have:

6. The in-vehicle liveness detection method based on multi-dimensional features of radar echo signals as described in claim 1, characterized in that, In step 302, the maximum energy value Expressed as: In the formula: a = dist1, dist1+1, ..., dist2; max(·) is the maximum value function; |·| is the modulo operation; y 2d(k) [a,b,i] represents the range-Doppler data of the k-th frame echo signal; y 2d_accu(k) [a,b] represents the range-Doppler data accumulated from the echo signal of the k-th frame.

7. The in-vehicle liveness detection method based on multi-dimensional features of radar echo signals as described in claim 5, characterized in that, In step 303, for the y 2d The angle spectrum data obtained by performing a Fourier transform on [a,b,i] in the angle dimension is represented as y. 3d [a,b,c], then: In the formula: c is the angular unit index, c = 1, 2, ..., Q, and Q is the number of points in the angular Fourier transform.

8. The in-vehicle liveness detection method based on multi-dimensional features of radar echo signals as described in claim 1, characterized in that, In step 3, the classification model is established and trained based on support vector machines.

Images