Human body detection method, device and equipment based on dual-technology sensor and medium

Abstract

The invention discloses a human body detection method, device and equipment based on a dual-technology sensor, and a medium, and the method comprises the steps: carrying out the synchronous collection, feature decoupling extraction and feature fusion of a millimeter wave radar and a pyroelectric infrared signal, and carrying out the real-time classification of an end side through a decision tree model; the system can intelligently judge whether a human body exists or not according to inherent micro-motion-thermal radiation coupling fingerprints of a target instead of an isolated single-mode threshold value, fusion features contain the double physical essence of whether the target has biological motion and a heat source or not, and the fusion features are more accurate. Therefore, only one interference source (for example, the fan only has micro-motion and no heat source, and the heater only has heat source and no micro-motion) can be effectively eliminated, the false alarm rate caused by main environment interference sources such as green plant shaking, curtain swinging and fan rotation is reduced, and the detection accuracy is improved.

Description

Technical Field

[0001] This invention relates to smart home technology, and more particularly to a human body detection method, device, equipment, and medium based on dual-technology sensors. Background Technology

[0002] In the field of smart homes, indoor human body detection technology is widely used, thereby enabling interaction with other smart devices.

[0003] Most existing indoor human detection technologies employ millimeter-wave radar and pyroelectric infrared sensors for human detection. For example, when the indoor environment is unoccupied, the millimeter-wave radar is in sleep mode, while the pyroelectric infrared sensor continues to operate. When the pyroelectric infrared sensor detects someone entering, it triggers the millimeter-wave radar to operate, and the pyroelectric infrared sensor enters sleep mode, while the millimeter-wave radar continuously monitors the movement of people indoors.

[0004] Existing solutions primarily rely on simple logical judgments (such as speed thresholds, distance thresholds, etc.) or signal processing based on a single sensor (radar only or infrared only). However, in actual deployments, they are easily affected by environmental interference sources (e.g., swaying plants, curtains, fans, pet activity, and heating sources). Existing solutions struggle to effectively distinguish between these complex environmental disturbances and real targets, leading to false alarms. Summary of the Invention

[0005] This invention provides a human body detection method, device, equipment, and medium based on dual-technology sensors to reduce the false alarm rate caused by major environmental interference sources such as swaying plants, swaying curtains, and rotating fans, and to improve detection accuracy.

[0006] In a first aspect, the present invention provides a human body detection method based on a dual-technology sensor, comprising:

[0007] Acquire millimeter-wave radar data and pyroelectric infrared data within the current time window;

[0008] Extract radar features from the millimeter-wave radar data;

[0009] Infrared features are extracted from the pyroelectric infrared data;

[0010] The radar features and the infrared features are fused to obtain the fused features;

[0011] The fused features are input into a pre-trained decision tree model to obtain human detection results.

[0012] Optionally, extracting radar features from the millimeter-wave radar data includes:

[0013] The spectral entropy, periodic intensity, and Doppler frequency are extracted from the millimeter-wave radar data as micro-Doppler features.

[0014] The linearity and randomness of the target trajectory are extracted from the millimeter-wave radar data as motion trajectory features;

[0015] The density and dispersion of the target point cloud are extracted from the millimeter-wave radar data as point cloud morphological features.

[0016] The fluctuation frequency and fluctuation amplitude of the signal energy are extracted from the millimeter-wave radar data as energy change characteristics;

[0017] The micro-Doppler features, the motion trajectory features, the point cloud morphology features, and the energy change features are combined to form radar features.

[0018] Optionally, spectral entropy, periodic intensity, and Doppler frequency are extracted from the millimeter-wave radar data as micro-Doppler features, including:

[0019] A short-time Fourier transform is performed on the slow-time signal of the millimeter-wave radar data to obtain the time spectrum;

[0020] For each moment in the time spectrum, the frequency distribution of the signal intensity at that moment is normalized to a probability distribution;

[0021] The entropy value of the radar signal at that moment is calculated based on the probability distribution.

[0022] The mean value of the radar signal entropy at all times is calculated as the spectral entropy;

[0023] Calculate the autocorrelation function of the slow-time signal of the millimeter-wave radar data;

[0024] The entropy value of the autocorrelation function in the time domain is calculated as the periodicity intensity;

[0025] The frequency corresponding to the maximum signal intensity in the time spectrum and the average value of all signal intensities are taken as the Doppler frequency.

[0026] Optionally, the linearity and randomness of the target trajectory are extracted from the millimeter-wave radar data as motion trajectory features, including:

[0027] The target's location sequence is obtained through a tracking algorithm;

[0028] The position sequence was fitted using the least squares method to obtain a fitted straight line;

[0029] Calculate the distance from each point in the position sequence to the fitted straight line;

[0030] For each point in the position sequence, the fitting error of that point is calculated based on the distance as the linearity of the target trajectory;

[0031] Calculate the autocorrelation function of the position sequence;

[0032] The entropy value of the autocorrelation function in the time domain is calculated as randomness.

[0033] Optionally, the density and dispersion of the target point cloud are extracted from the millimeter-wave radar data as point cloud morphological features, including:

[0034] A range Doppler map is generated as a point cloud map based on the millimeter-wave radar data;

[0035] Calculate the density of the target point cloud in the point cloud map;

[0036] The coordinates of the target point cloud in the point cloud map are constructed into a coordinate matrix;

[0037] Calculate the covariance matrix of the coordinate matrix;

[0038] Calculate the eigenvalues ​​of the covariance matrix;

[0039] The entropy of the eigenvalues ​​is calculated as the dispersion of the target point cloud.

[0040] Optionally, the fluctuation frequency and fluctuation amplitude of the signal energy are extracted from the millimeter-wave radar data as energy change characteristics, including:

[0041] Calculate the power spectrum of the slow-time signal of the millimeter-wave radar data;

[0042] The coefficient of variation of the power spectrum is calculated as the fluctuation amplitude;

[0043] The frequency corresponding to the maximum power value in the power spectrum is taken as the fluctuation frequency.

[0044] Optionally, extracting infrared features from the pyroelectric infrared data includes:

[0045] The rise / fall slope and number of peaks of the signal are extracted from the pyroelectric infrared data as signal waveform features.

[0046] Extract the absolute energy level of the signal from the pyroelectric infrared data;

[0047] The signal waveform characteristics and the absolute energy level are combined to form an infrared feature.

[0048] Secondly, the present invention also provides a human body detection device based on a dual-technology sensor, comprising:

[0049] The data acquisition module is used to acquire millimeter-wave radar data and pyroelectric infrared data within the current time window;

[0050] A radar feature extraction module is used to extract radar features from the millimeter-wave radar data;

[0051] An infrared feature extraction module is used to extract infrared features from the pyroelectric infrared data;

[0052] A feature fusion module is used to fuse the radar features and the infrared features to obtain fused features;

[0053] The result prediction module is used to input the fused features into a pre-trained decision tree model to obtain human detection results.

[0054] Thirdly, the present invention also provides an electronic device, comprising:

[0055] One or more processors;

[0056] Storage device for storing one or more programs;

[0057] When the one or more programs are executed by the one or more processors, the one or more processors implement the human body detection method based on dual-technology sensors as described in the first aspect of the present invention.

[0058] Fourthly, the present invention also provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the human body detection method based on dual-technology sensors as described in the first aspect of the present invention.

[0059] The human detection method based on dual-technology sensors provided by this invention synchronously acquires, decouples, extracts, and fuses features from millimeter-wave radar and pyroelectric infrared signals, and performs real-time classification on the edge based on a decision tree model. This enables the system to intelligently determine the presence or absence of a human based on the inherent micro-motion-thermal radiation coupled fingerprint of the target, rather than an isolated single-mode threshold. The fused features imply whether the target possesses the dual physical essence of "biological movement + heat source", thus effectively eliminating interference sources that only have one aspect (such as a fan with only micro-motion and no heat source, or a heater with only heat source and no micro-motion). This reduces the false alarm rate caused by major environmental interference sources such as swaying plants, swaying curtains, and rotating fans, and improves the detection accuracy.

[0060] It should be understood that the description in this section is not intended to identify key or essential features of the embodiments of the present invention, nor is it intended to limit the scope of the invention. Other features of the invention will become readily apparent from the following description. Attached Figure Description

[0061] To more clearly illustrate the technical solutions in the embodiments of the present invention, the accompanying drawings used in the description of the embodiments will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0062] Figure 1 A flowchart of a human body detection method based on a dual-detection sensor provided by the present invention;

[0063] Figure 2 A schematic diagram of the structure of a human body detection device based on a dual-detection sensor provided by the present invention;

[0064] Figure 3 This is a schematic diagram of the structure of an electronic device provided by the present invention.

[0065] The accompanying drawings illustrate specific embodiments of this application, which will be described in more detail below. These drawings and descriptions are not intended to limit the scope of the concept in any way, but rather to illustrate the concept of this application to those skilled in the art through reference to particular embodiments. Detailed Implementation

[0066] To enable those skilled in the art to better understand the present invention, the technical solutions of the present invention will be clearly and completely described below with reference to the accompanying drawings of the embodiments of the present invention. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort should fall within the scope of protection of the present invention.

[0067] It should be noted that the terms "first," "second," etc., in the specification, claims, and accompanying drawings of this invention are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that such data can be interchanged where appropriate so that the embodiments of the invention described herein can be implemented in orders other than those illustrated or described herein. Furthermore, the terms "comprising" and "having," and any variations thereof, are intended to cover a non-exclusive inclusion; for example, a process, method, system, product, or apparatus that comprises a series of steps or units is not necessarily limited to those steps or units explicitly listed, but may include other steps or units not explicitly listed or inherent to such processes, methods, products, or apparatus.

[0068] Figure 1The flowchart illustrates a human detection method based on a dual-technology sensor provided by this invention. This embodiment is applicable to human detection in indoor environments. The method can be executed by the human detection device based on a dual-technology sensor provided by this invention. This device can be implemented in software and / or hardware, and is typically configured in electronic devices, such as… Figure 1 As shown, the human detection method based on dual-technology sensors includes the following steps:

[0069] S101. Obtain millimeter-wave radar data and pyroelectric infrared data within the current time window.

[0070] In this embodiment of the invention, the "current time window" refers to a time segment used for a single detection decision. Its length is set according to the target motion characteristics and the real-time requirements of the system response, ranging from 200ms to 2s, and can be selected as 500ms. This window length is sufficient to cover a complete cycle of human walking gait (0.8–1.2s) while taking into account the computing resource constraints of the embedded end. The millimeter-wave radar data is the raw baseband IQ signal acquired under the Frequency Modulated Continuous Wave (FMCW) mode, containing a two-dimensional structure of fast time (range dimension) and slow time (time dimension). The pyroelectric infrared data is a discrete voltage sequence after the pyroelectric infrared sensor is conditioned by the internal operational amplifier and sampled by the ADC, with a sampling rate of not less than 50Hz to ensure the capture of the slope characteristics of thermal radiation changes caused by typical human movement. The two data streams are synchronized through hardware mechanisms (such as shared master control clock triggering, GPIO synchronization pulse or timestamp alignment) to ensure strict alignment of the time axis, with the synchronization error controlled within ±10ms, avoiding distortion of fused features due to timing misalignment. This synchronous acquisition method enables subsequent feature extraction to be performed under a unified time reference, providing a foundation for constructing collaborative response features between radar and PIR (such as trigger time difference and energy trend consistency).

[0071] S102. Extract radar features from millimeter-wave radar data.

[0072] In this embodiment of the invention, "radar features" are low-dimensional, highly discriminative vectors characterizing the target's motion dynamics and spatial morphology. Their composition does not depend on absolute positioning information such as target distance and azimuth, but focuses on the target's intrinsic attributes. This feature set includes four sub-features: micro-Doppler features, motion trajectory features, point cloud morphology features, and energy change features. All sub-features undergo normalization processing (e.g., Z-score normalization or Min-Max scaling) to eliminate the impact of differences in dimensions and numerical ranges on subsequent fusion and classification. Micro-Doppler features reflect the frequency modulation effect caused by local components of the target (such as limb swaying), including spectral entropy (quantifying the disorder of time-frequency energy distribution), periodicity intensity (characterizing the time-domain regularity of the autocorrelation function), and Doppler frequency (indicating the subject's motion velocity). Motion trajectory features characterize the geometric and statistical properties of the target's overall displacement path; linearity describes the degree to which the trajectory deviates from a straight line, and randomness reflects the unpredictability of the motion direction and speed. Point cloud morphology features are constructed based on range-Doppler maps. Density represents the number of effective points per unit area, and dispersion is measured by the entropy of the covariance matrix eigenvalues ​​to determine the anisotropy of the target spatial distribution. Energy variation features are extracted from the power spectrum of slow-time signals, including fluctuation amplitude (coefficient of variation) and fluctuation frequency (dominant frequency), reflecting the rhythmicity of signal energy evolution over time. These four types of sub-features are concatenated to form radar feature vectors with dimensions of 12–20, such as: [spectral entropy, periodicity intensity, average Doppler frequency, linearity, randomness, density, dispersion, fluctuation amplitude, fluctuation frequency].

[0073] Radar features are extracted from millimeter-wave radar data, including: extracting spectral entropy, periodic intensity, and Doppler frequency as micro-Doppler features; extracting linearity and randomness of target trajectories as motion trajectory features; extracting density and dispersion of target point clouds as point cloud morphology features; and extracting fluctuation frequency and amplitude of signal energy as energy change features. The micro-Doppler features, motion trajectory features, point cloud morphology features, and energy change features are then combined to form radar features.

[0074] Spectral entropy is used to characterize the degree of disorder in the frequency distribution of slow-time signals from millimeter-wave radar. This feature is highly sensitive to non-stationary, non-periodic disturbances (such as cluttered echoes caused by wind blowing through vegetation), while micro-Doppler signals generated by human walking typically exhibit moderate entropy values—neither completely random (such as noise) nor highly concentrated (such as the rotation of fan blades). For example, a short-time Fourier transform can be performed on the slow-time signal of millimeter-wave radar data to obtain the time spectrum. For each moment in the time spectrum, the frequency distribution of the signal strength at that moment is normalized to a probability distribution. Based on this probability distribution, the entropy value of the radar signal at that moment is calculated, and the mean of the entropy values ​​of the radar signal at all moments is calculated as the spectral entropy.

[0075] Slow-time signals refer to one-dimensional time-series echo sequences acquired by millimeter-wave radar within a fixed range bin or angle bin, along the direction of the pulse repetition interval (PRI). Their sampling rate is typically from hundreds of Hz to several kHz, reflecting the motion modulation characteristics of the target in the slow-time dimension. This signal can be obtained by extracting it from selected target cells after range compression and constant false alarm rate (CFAR) detection from the original intermediate frequency (IF) signal.

[0076] The Short-Time Fourier Transform (STFT) is a method that performs a Fast Fourier Transform (FFT) on each segment of a windowed, segmented slow-time signal to generate a two-dimensional time-frequency energy distribution matrix. The window functions used include the Hanning window, Hamming window, or Kaiser window, with window lengths ranging from 32 to 128 points and overlap rates from 50% to 75%, balancing time-frequency resolution and computational efficiency. In embedded deployments, a window length of 64 points and an overlap rate of 62.5% correspond to a time window of approximately 20ms and a step size of 10ms, meeting the resolution requirements for human gait cycles (0.8–1.2s) and fan blade rotation cycles (0.05–0.5s).

[0077] Normalizing the frequency distribution of the signal strength at a given moment into a probability distribution means calculating the square of the amplitude (i.e., the power spectral density estimate) |S(f,t0)| at each frequency point of the frequency vector S(f,t0) corresponding to a fixed moment t0 in the time spectrum obtained by STFT. 2 Then perform L¹ norm normalization on the vector: p(f) = |S(f,t0)| 2 / Σ k |S(f k ,t0)| 2This normalization process eliminates the influence of absolute energy differences, highlighting the spectral structure characteristics. It also allows for the construction of a discrete probability mass function by setting ∑p(f) = 1.

[0078] Calculating the radar signal entropy at that moment based on this probability distribution refers to calculating the Shannon entropy: H(t0) = -Σ k p(f k )·log2p(f k ), where p(f k ) represents the probability value of the k-th frequency point after normalization; when p(f k When )=0, it is agreed that 0·log20 = 0; this entropy value characterizes the uncertainty or dispersion of the radar echo in the frequency dimension at that moment—a high entropy value indicates that the energy is dispersed across a wide frequency band (such as disordered tree branches swaying or air conditioning airflow disturbance), while a low entropy value indicates that the energy is concentrated in a few frequencies (such as the 1–5 Hz micro-Doppler spectrum dominated by human walking, or single-frequency or multi-frequency harmonics from fan blade rotation). In low-power scenarios, approximate entropy or sample entropy can be used instead of Shannon entropy, which only requires comparing the threshold relationship of the amplitude difference between adjacent frequency points, avoiding floating-point logarithmic operations and reducing MCU resource consumption.

[0079] The mean of the radar signal entropy values ​​at all times is used as the spectral entropy, which refers to the average of the entropy values ​​of the radar signal at all times (t1…t) over the entire time spectrum. T Calculate H(t) respectively i Then calculate the arithmetic mean: H_spectrum = (1 / T)·Σ i H(t i T is set to 32–128, corresponding to an analysis window of approximately 0.3–1.0 seconds, covering the typical human step cycle and the steady-state response period of interference sources. This mean operation suppresses transient outlier interference and enhances statistical stability.

[0080] Periodicity intensity is used to quantify the repetitive nature of radar signals in the time domain. It is obtained by calculating the autocorrelation function of a slow-time signal and then taking the Shannon entropy from its time-domain distribution. The lower the entropy value, the sharper the autocorrelation peak and the stronger the periodicity. For example, the rotation of a fan corresponds to an extremely low periodicity intensity entropy value (<0.8 bit), the swaying of curtains exhibits moderate periodicity (entropy value of approximately 1.2–1.6 bits), while human gait, although rhythmic, is affected by posture changes, and its periodicity intensity entropy value is usually higher than 1.8 bits. For instance, the autocorrelation function of a slow-time signal from millimeter-wave radar data can be calculated, and the entropy value of the autocorrelation function in the time domain can be used as the periodicity intensity.

[0081] Calculating the autocorrelation function of slow-time signals from millimeter-wave radar data refers to calculating the biased autocorrelation function of the slow-time signal x[n] (n=0,1,…,N-1): R x [m] = (1 / N)·Σ n x[n]·x[n+m], m∈[-M,M], M≤N / 2. In embedded systems, a fast convolution method is preferred: R is calculated using FFT. x [m] = IFFT{|FFT{x[n]}| 2 This significantly reduces computational complexity. The autocorrelation function reflects the similarity of signals under different time delays, and its main lobe width and attenuation characteristics are directly related to the stability of the motion rhythm.

[0082] Calculating the entropy value of the autocorrelation function in the time domain as the periodicity intensity means taking the autocorrelation function R... x [m] is considered as a one-dimensional time-series signal of length 2M+1, and its absolute value |R x [m]|(or squared|R) x [m]| 2 The same normalization and Shannon entropy calculation process as described above is performed for the spectral entropy. The smaller the resulting entropy value H_periodicity, the sharper the autocorrelation function and the more concentrated the main lobe, corresponding to a strongly periodic signal (such as a fan rotating at a constant speed); conversely, the larger the H_periodicity, the longer the tail of the autocorrelation function and the more diffuse the energy, corresponding to non-periodic or random motion (such as a pet running around or curtains fluttering erratically). This index does not depend on the absolute value of the Doppler frequency shift, but only characterizes the time-domain repeatability, thus having a natural robustness to radar installation position shifts and target distance changes. In another embodiment, the periodicity intensity can also be defined as the ratio of the peak value of the main lobe to the mean value of the sidelobe of the autocorrelation function (peak-to-sidelobe ratio), or the reciprocal of the full width at half maximum (FWHM) of the main lobe. Both can be mapped to equivalent entropy values ​​through table lookup or linear fitting, compatible with existing decision tree input dimensions.

[0083] Doppler frequency is used to reflect the overall radial velocity of a target. It is characterized by a combination of the frequency value corresponding to the maximum signal strength in the time spectrum (i.e., the dominant Doppler frequency offset) and the average Doppler frequency obtained by weighting the signal strength of all effective frequency points. The former is sensitive to instantaneous high-speed motion (such as waving a hand), while the latter stably represents a continuous displacement trend (such as walking). In practical deployment, the calculation of Doppler frequency can be limited to the effective frequency band with a signal-to-noise ratio greater than 10 dB, excluding the thermal noise-dominated region.

[0084] Taking the frequency corresponding to the maximum signal intensity in the time-frequency spectrum and the average of all signal intensities as the Doppler frequency means locating the global maximum power point (f_max, t_max) in the entire frame's time-frequency spectrum and extracting its frequency coordinate f_max as the dominant Doppler frequency; simultaneously, the power values ​​|S(f,t)| of all T×F time-frequency units are... 2 Calculate the arithmetic mean, denoted as μ_power, and then calculate the weighted average frequency f_avg = Σ_{f,t}|S(f,t)| 2 ·f / Σ_{f,t}|S(f,t)| 2 The f_avg reflects the overall energy center of gravity frequency; together they constitute the Doppler frequency feature vector [f_max, f_avg]; f_max is sensitive to the main motion component (such as the translation of the human torso), while f_avg is more robust to the superposition of micro-motions.

[0085] The linearity of the target trajectory is used to characterize the consistency of the target's direction of motion in space. This is achieved by analyzing the position sequence (x) output by the tracking algorithm. t , y t , z t Fit a least-squares line to the interval t=1,…,N, and calculate the perpendicular distance d from each point to the line. i Therefore, linearity is defined as the linearity of all d. i The reciprocal of the standard deviation (or 1-(std(d)) i The linearity of the trajectory is denoted by `max_distance`, where a larger value indicates a trajectory closer to a straight line. Normal human walking exhibits significantly higher linearity than the swaying of curtains (without a fixed direction) or the shaking of greenery (divergent motion). As an alternative implementation, linearity can also be characterized by the trajectory curvature integral, the deviation of the Hausdorff distance from the ideal straight line, or the variance proportion of the first principal component in Principal Component Analysis (PCA). The position sequence can originate from the output of any applicable tracking algorithm, such as Kalman filtering, Joint Probabilistic Data Association (JPDA), or Multiple Hypothesis Tracking (MHT). For example, the target's position sequence can be obtained through a tracking algorithm, fitted using the least squares method to obtain a fitted straight line, and the distance from each point in the position sequence to the fitted straight line can be calculated. For each point in the position sequence, the fitting error based on the distance is used as the linearity of the target trajectory.

[0086] The "tracking algorithm" refers to a processing method that correlates and continues the trajectory of target responses in the target point cloud or range-Doppler image detected by millimeter-wave radar within a continuous time window, outputting a temporal coordinate sequence of the target in two-dimensional or three-dimensional space. This algorithm can be selected from classic tracking frameworks such as Kalman Filter, Extended Kalman Filter (EKF), Unscented Kalman Filter (UKF), Particle Filter, or Joint Probabilistic Data Association (JPDA) based on data association. The position sequence is in the form of a set of coordinates at discrete time points {(x1, y1), (x2, y2), …, (x... n , y n )}, where x i y i These represent the estimated positions (in meters) of the target in the horizontal and vertical directions at time i, respectively. The sampling interval is Δt, with a value range of 50–200 ms. The sequence length n can be dynamically set according to the actual scene. For example, n = 16–64 frames, corresponding to a time window length of 0.8–12.8 s, taking into account both the periodicity of human gait and the persistence of environmental disturbances.

[0087] The least squares method is a mathematical optimization technique used to find the parameters of a line that minimizes the sum of the squared perpendicular distances from each point to the fitted line, given a set of coordinate points. For a two-dimensional position sequence {(x... i , y i The fitted straight line is expressed as y = ax + b, and its coefficients a (slope) and b (intercept) are solved analytically using the formulas: a = cov(x,y) / var(x), b = y0 - a·x0, where cov(x,y) is the covariance of x and y, var(x) is the variance of x, and x0 and y0 are the means of the x and y coordinates, respectively. This fitting process does not presuppose the direction of the straight line and can adapt to human walking paths of any orientation.

[0088] The distance from a point to a line refers to the distance from the midpoint (x, y) in Euclidean space. i , y i The perpendicular distance from d to the line y = ax + b is calculated using the formula d. i = |ax i - y i + b| / √(a 2 +1). This distance characterizes the degree to which the target deviates from the overall motion trend at that moment, and is a fundamental geometric quantity for quantifying the regularity of the trajectory. All d i This forms the distance sequence {d1, d2, …, d n The statistical distribution of the trajectory reflects the degree of curvature: when a human walks at a constant speed in a straight line, d iGenerally smaller and concentrated around zero; the movement of curtains or swaying of green plants will cause d i Large amplitude and drastic fluctuations.

[0089] The "fitting error" is defined here as the distance sequence {d}. i The normalized root mean square error (RMSE) of}, i.e., the linearity L = √[(1 / n)∑d i 2 ] / D0, where D0 is the spatial scale reference value of the position sequence, which can be a fixed proportion (e.g., 10%) of the distance from the centroid of the sequence to the farthest point, the diagonal length of the sequence bounding box, or the maximum radar detection range. This normalization process makes the linearity L a dimensionless quantity, with a value range of [0, 1]. The smaller L is, the closer the trajectory is to a straight line. L > 0.3 usually corresponds to strong curvature or circular motion.

[0090] The randomness of the target trajectory is used to describe the unpredictability of the motion path in time. This is achieved by constructing a one-dimensional projection sequence (such as radial distance r) from the position sequence. t = √(x t ²+y t ²+z t ²) or x-axis component x t The autocorrelation function R(τ) is calculated, and then the Shannon entropy is obtained by normalizing R(τ) to a probability distribution in the interval τ∈[0, T]. High random entropy values ​​(>2.0 bits) correspond to irregular perturbations (such as the random shaking of potted plant branches and leaves), low entropy values ​​(<1.2 bits) correspond to strong periodic movements (such as the uniform rotation of a ceiling fan), while human walking has a moderate random entropy value (approximately 1.4–1.7 bits) due to the natural variation in stride length and stride frequency. This feature complements the periodicity intensity: the former focuses on the repetitiveness of the time-domain structure, while the latter focuses on statistical uncertainty. For example, the autocorrelation function of the position sequence can be calculated, and then the entropy value of the autocorrelation function in the time domain can be calculated as randomness.

[0091] "Autocorrelation function" R xx (τ) represents the position sequence {x} i} or {y i} (or the magnitude r of its composite vector) i = √(x i ² +y i ²)) The similarity measure under time delay τ is defined as R xx (τ) = (1 / (n-|τ|))∑{n-|τ|} (x i – x0)(x i+τ –x0), τ ∈ [-(n-1), n-1]. This function characterizes the repetitiveness and memory of position changes over time: human walking has a gait frequency rhythm of 0.8–2.0 Hz, and its autocorrelation function shows a significant periodic peak; while wind-induced disturbances (such as the swaying of curtains) have a certain periodicity, but the phase drift is large and the decay is fast, and the rotation of a fan shows a highly stable single-frequency oscillation.

[0092] "Time domain entropy" refers to the value of the autocorrelation function R. xx The value of (τ) within the effective time delay range (τ ∈ [0, τ_max], where τ_max corresponds to 5–10 times the average gait period) is normalized to a probability distribution p(τ) = R. xx (τ) / ∑ j R xx (τ j Then calculate the Shannon entropy H = -∑ t p(τ t log2 p(τ) t This entropy value characterizes the information uncertainty of the autocorrelation structure: high entropy (H > 2.5 bits) indicates that R... xx (τ) Energy is diffused and lacks a main period, corresponding to irregular disturbances (such as plants shaking in turbulent water); low entropy (H < 1.5 bit) indicates that energy is concentrated in a few time delays, corresponding to strong periodic motion (such as a fan rotating at a constant speed or a human walking in a regular pattern).

[0093] The density of the target point cloud is used to characterize the number of radar detection points per unit area / volume. The target point cloud is obtained by performing CFAR (Constant False Alarm Rate) detection and clustering (such as DBSCAN or Euclidean range clustering) on ​​the raw millimeter-wave radar data, and then applying range-Doppler. Figure 3 In a 3D coordinate system, the density is calculated by dividing the number of points within the target's bounding box by its geometric area (or volume). High-density point clouds are commonly found on human torsos (highly reflective surfaces), while low-density points are found on slender objects (such as curtain edges or tree branches). The density threshold can be adaptively adjusted within the range of 5–20 points / (m²·Hz). For example, a range Doppler map is generated based on millimeter-wave radar data as a point cloud map, and the density of the target point cloud in the point cloud map is statistically analyzed.

[0094] The range-Doppler map is a two-dimensional time-frequency energy distribution map formed by performing range-dimensional Fast Fourier Transform (Range FFT) and Doppler-dimensional Fast Fourier Transform (Doppler FFT) on millimeter-wave radar signals. The horizontal axis represents the distance between the target and the radar (in meters), and the vertical axis represents the radial velocity of the target along the radar's line-of-sight (in m / s). Pixel grayscale or color intensity characterizes the energy amplitude of the echo signal within the range-velocity cell. This map can be directly used as a spatial representation of point clouds without additional point cloud clustering or 3D reconstruction, making it suitable for low-latency processing in embedded systems. Its generation process includes: sequentially performing range FFT, CFAR (constant false alarm rate) detection, Doppler FFT, and incoherent accumulation on the ADC sampling data acquired in each transmit antenna cycle, finally outputting a normalized range-Doppler amplitude matrix.

[0095] The density of a target point cloud refers to the ratio of the number of pixels identified as valid targets by CFAR in the range-Doppler map to the area of ​​the smallest bounding rectangle they occupy (in pixels), i.e.: Density = Total number of target pixels / Area of ​​the smallest bounding rectangle. This metric reflects the density of target clustering in the range-velocity domain: when a person walks or stands, the limb motion coupling is strong, and the echo energy is concentrated in narrow range cells and a limited Doppler frequency band, forming a high-density cluster; while interference sources such as swaying branches and leaves of green plants and rotating fan blades, due to the independent motion of each part and inconsistent phase, cause energy to be dispersed in multiple range cells and a wide Doppler frequency band, exhibiting a low-density, fragmented distribution.

[0096] The dispersion of the target point cloud is used to characterize the degree of anisotropy in the spatial distribution of the point cloud. This is achieved by calculating the covariance matrix C = X from the target point cloud coordinate matrix X. T X / (n-1), then solve for the three eigenvalues ​​of C: λ1≥λ2≥λ3, and finally obtain the eigenvalue distribution entropy H = -∑(λ i / ∑λ j log2(λ) i / ∑λ j Characterization: High entropy values ​​(>1.0 bit) reflect spherical or uniform diffusion of the point cloud (like leaves blowing in the wind), low entropy values ​​(<0.3 bit) reflect strong directional extension (like head-to-toe axial stretching when a person is standing), and medium entropy values ​​(0.4–0.8 bit) correspond to typical human postures. This feature, along with linearity, determines the geometric properties of the moving subject: for example, the swaying of curtains may have medium linearity but extremely high dispersion, while human walking exhibits a combination of medium-high linearity and medium-low dispersion. For example, the coordinates of the target point cloud in the point cloud map are constructed as a coordinate matrix, the covariance matrix of the coordinate matrix is ​​calculated, the eigenvalues ​​of the covariance matrix are calculated, and the entropy of the eigenvalues ​​is used as the dispersion of the target point cloud.

[0097] Constructing a coordinate matrix from the target point cloud coordinates in the point cloud image means: extracting the integer coordinate pairs (distance index, Doppler index) of all pixels identified as targets by CFAR, and forming an N×2 matrix X = [x1, x2; x3, x4;…; x…]. 2N-1 , x 2N The matrix [] represents the total number of target pixels, with each row corresponding to the distance index (column 1) and Doppler index (column 2) of a point. This matrix implicitly contains the spatial configuration information of the target in the joint range-velocity domain, which forms the basis for subsequent statistical modeling.

[0098] The covariance matrix is ​​a 2×2 real symmetric matrix calculated by zero-mean normalizing the coordinate matrix X. The diagonal elements of this matrix represent the variance of the target point cloud in the distance and Doppler dimensions, respectively, while the off-diagonal elements represent the covariance between the two dimensions. Overall, it characterizes the expansion direction, anisotropy, and principal component distribution of the point cloud in the distance-velocity plane. For example, a human target typically exhibits small distance variance (limbs concentrated in the same distance layer), moderate Doppler variance (periodic gait), and near-zero covariance (weak correlation between distance and velocity changes); while the overall movement of a curtain exhibits large distance variance (different pieces of fabric at different distances), small Doppler variance (uniform translation), and significant covariance (coupling between distance shift and velocity change).

[0099] The eigenvalues ​​of the covariance matrix are two non-negative real numbers λ1 ≥ λ2 ≥ 0 obtained by solving det(C - λI) = 0, which correspond to the variance of the point cloud in the first and second principal component directions, respectively. λ1 reflects the energy concentration in the direction of maximum expansion of the point cloud, and λ2 reflects the dispersion in the orthogonal direction; the ratio of the two, λ1 / λ2, can quantify the shape anisotropy of the point cloud (e.g., λ1 much greater than λ2 indicates a thin strip shape, commonly seen in the trajectory of walking human bodies; λ1 ≈ λ2 indicates an approximately circular shape, commonly seen in the interference of fan rotation).

[0100] The entropy of an eigenvalue is defined as: H = -Σp i log2 p i , where p i = λ i / (λ1 + λ2) represents the normalized eigenvalue probability distribution. This entropy value strictly lies between 0 (complete anisotropy, only one eigenvalue is non-zero, the point cloud is linear) and 1 (complete isotropy, two eigenvalues ​​are equal, the point cloud is disk-shaped), mathematically rigorously characterizing the uniformity and disorder of the spatial distribution of the point cloud. Human targets, due to coordinated limb movements, form compact, highly directional point cloud clusters in the distance Doppler image, with a highly skewed eigenvalue distribution (p1 is much greater than p2), and the entropy value approaches 0. Green plants or curtains, due to asynchronous movements of multiple parts, have a wide point cloud coverage area and chaotic directions, with a more balanced eigenvalue distribution (p1 ≈ p2), and the entropy value approaches 1. This indicator is particularly sensitive to fan interference—its rotating blades form a symmetrical ring distribution around the zero Doppler line in the distance Doppler image, with λ1 and λ2 being close, and the entropy value significantly higher than that of a human body.

[0101] The fluctuation frequency of signal energy is used to identify the dominant rhythm of radar echo power change over time. This is achieved by calculating the power spectrum P(f) of a slow-time signal s(t) and taking the frequency f corresponding to the main peak of P(f). max As a fluctuation frequency; this frequency is directly related to the physical motion frequency of the interference source—f corresponds to the fan speed. max ≈10–30 Hz, air conditioning airflow disturbance corresponds to f max The frequency range is approximately 0.5–3 Hz, while human respiration is approximately 0.2–0.3 Hz. For example, the power spectrum of a slow-time signal from millimeter-wave radar data is calculated, and the coefficient of variation of the power spectrum is calculated as the fluctuation amplitude.

[0102] When calculating the power spectrum, the periodogram method or a modified Welch method is used: the slow-time signal is segmented and windowed (e.g., Hanning window), overlapped (50% overlap), and subjected to FFT transformation. The power spectrum of each segment is averaged to suppress spectral leakage and estimate variance. The number of FFT points is 256 or 512, and the frequency resolution is 0.1 Hz to 0.5 Hz, which is sufficient to distinguish between fans (speed 300–1800 rpm, corresponding to a fundamental frequency of 5–30 Hz), air conditioning airflow disturbances (<2 Hz), and random noise (wideband flat spectrum). The power spectrum is a real-valued non-negative function, with the horizontal axis representing frequency f (unit: Hz) and the vertical axis representing power spectral density S(f) (unit: dB or linear power unit). Its shape directly characterizes the frequency domain distribution characteristics of energy changes.

[0103] The coefficient of variation (CV) is defined as the ratio of the standard deviation σ of the power spectrum amplitude to the mean μ, i.e., CV = σ / μ, and is dimensionless. This index is used to quantify the dispersion of the power spectrum energy distribution, avoiding the problem that the absolute amplitude is affected by system parameters such as radar gain and target range. During calculation, the amplitude of the power spectrum S(f) at all frequency points within the effective analysis frequency band (0.1 Hz–50 Hz) is statistically analyzed: first, the zero-frequency DC component is removed, and then the mean μ and standard deviation σ of the remaining frequency points S(f) are calculated. When strong periodic interference exists (such as fan blade rotation), S(f) shows a peak at a specific frequency, causing σ to increase significantly while μ remains relatively stable, resulting in a higher CV value (>0.8). Under white noise or weak disturbances, S(f) is approximately uniformly distributed, and the CV value is lower (<0.3).

[0104] The amplitude of signal energy fluctuation is used to measure the severity of slow-time signal power fluctuations, and is defined as the coefficient of variation (CV = standard deviation / mean) of the power spectrum P(f). A high CV value (>0.6) corresponds to strong impulse interference (such as fan starting and stopping, sudden drafts on curtains), while a low CV value (<0.2) corresponds to steady-state heat sources or slowly moving targets (such as radiator radiation, slow-moving pedestrians). This parameter, together with the fluctuation frequency, constitutes the dynamic fingerprint of energy. For example, the frequency corresponding to the maximum power value in the power spectrum is taken as the fluctuation frequency.

[0105] "The frequency corresponding to the maximum power value" refers to the frequency point f_max that makes S(f) achieve its global maximum value within the effective analysis frequency band of the power spectrum S(f); this frequency directly corresponds to the main oscillation frequency of energy fluctuations in a slow-time signal. For example, when a three-bladed fan operates at 1200 rpm, the blades generate 3 reflection intensity pulsations per revolution, with a theoretical fundamental frequency of f = (1200 / 60) ×3 = 60 Hz. In the actual power spectrum, f_max is located within the range of 60 Hz ± 2 Hz. The swaying frequency of curtains under the action of airflow is 0.5–3 Hz, corresponding to f_max in this range. The frequency of chest cavity micro-movements caused by human walking is 0.8–2.5 Hz, but its energy fluctuations are broad-spectrum rather than peaked in the power spectrum, so the amplitude of f_max is low and there are multiple peaks in the neighborhood.

[0106] S103. Extract infrared features from pyroelectric infrared data.

[0107] In this embodiment of the invention, the "infrared feature" focuses on the time-domain dynamic response characteristics of the PIR signal, rather than the static temperature value.

[0108] The rise / fall slope of the signal is determined by calculating the maximum absolute value of the voltage difference between adjacent sampling points, characterizing the transient rate of the heat source entering / leaving the field of view. The peak count is the number of local maxima exceeding a preset signal-to-noise ratio threshold (e.g., 3 times the root mean square noise) within a unit time window, reflecting the discontinuity and rhythm of the heat source's movement. The absolute energy level is defined as the square root of the sum of the squares of the signal amplitudes within this window (i.e., the L2 norm), characterizing the total intensity of thermal radiation. These three factors together constitute a 3D infrared feature vector.

[0109] Extracting infrared features from pyroelectric infrared data includes: extracting the rise / fall slope and number of peaks as signal waveform features from pyroelectric infrared data; extracting the absolute energy level of the signal from pyroelectric infrared data; and combining the signal waveform features and absolute energy level into infrared features.

[0110] The rise / fall slope of the signal is used to characterize the rate of temperature change as the heat source crosses the PIR field of view. In practice, a first-order difference operation is performed on the original PIR digital signal to obtain the amplitude change ΔV between adjacent sampling points. This change is then divided by the corresponding time interval Δt (i.e., the sampling period) to obtain the instantaneous slope sequence. Subsequently, within the effective response interval of the signal (i.e., a continuous period where the amplitude exceeds three times the root mean square value of the baseline noise), the maximum rise slope (max(dV / dt)) is extracted. + ) and the maximum descent slope (max(dV / dt)) - The slope feature is used as two independent feature components. This slope feature has a strong ability to distinguish slowly drifting disturbances (such as gradual changes in background temperature caused by heating and air conditioning) - the absolute value of the slope corresponding to such disturbances is usually less than 0.5 V / s, while the slope of the PIR response caused by normal human walking can reach 2–8 V / s.

[0111] The number of peaks reflects the complexity of the overlap between the heat source's movement path and the multi-segment field of view of the PIR. In practice, an adaptive threshold detection is first applied to the PIR signal: a dynamic threshold is set using the weighted sum of the signal mean and standard deviation within a sliding window (500 ms) (e.g., mean + 2 × standard deviation). Then, all local maxima points exceeding this threshold and lasting ≥20 ms are identified, and redundant points with an interval of less than 100 ms between adjacent peaks are removed (to prevent ringing effect miscounting). Finally, the total number of valid peaks within a unit time window is counted. When a person walks upright, the alternating entry of the torso and limbs' heat radiation centers into different Fresnel lens zones typically produces 3–7 resolvable peaks. A curtain swaying in one direction triggers only 1–2 broad peaks, while random swaying of green plants produces irregular, sparse peaks (≤2), and the airflow from a fan hardly triggers a PIR response (peak count ≈ 0). In optional embodiments, peak morphological features (such as full width at half maximum (FWHM) and kurtosis) are introduced for joint judgment, or wavelet transform modulus maxima detection is used to replace the fixed threshold method to improve the ability to resolve weak signals and overlapping peaks.

[0112] The absolute energy level of the signal is used to quantify the relative difference between the target's thermal radiation intensity and the environmental background. Specifically, this is achieved by calculating the sum of squares (Σv) of the PIR digital signal within the current time window. i =²) or the absolute value integral (∫|v(t)|dt), divided by the window length to normalize to the energy per unit time; this value is directly related to the combined effect of target surface area, body temperature and distance, and can effectively distinguish adults (typical energy level: 1.2–5.0 au), children (0.6–2.5 au), pets (0.3–1.8 au) and background disturbances (<0.1 au). It should be noted that this feature needs to be jointly calibrated with the millimeter-wave radar ranging results—when the radar feedback target distance is 2 m, if the PIR energy level is less than 0.8 au, it is likely to correspond to a small-sized interference source or a distant human body; conversely, if the distance is >4 m but the energy level is still >3.0 au, it suggests the possible presence of an abnormally strong heat source (such as an open flame).

[0113] There is a complementary and temporally coupled relationship between signal waveform characteristics and absolute energy level: the rising / falling edge slope and the number of peaks jointly characterize the dynamic pattern of heat source movement (fast / slow change, continuous / discrete), while the absolute energy level provides a static benchmark for thermal radiation intensity. The combination of the two constitutes a two-dimensional feature subspace—for example, a high slope + multiple peaks + high energy level combination points to typical human intrusion; a low slope + single peak + medium energy level points to curtain swaying; and zero slope + zero peaks + low energy level points to background temperature drift. This combination method does not rely on a specific mathematical model and can be directly input into subsequent fusion modules. It also supports linear weighting (e.g., weight coefficients 0.4:0.4:0.2) or principal component analysis (PCA) dimensionality reduction before being connected to a decision tree classifier.

[0114] S104. By fusing radar and infrared features, a fused feature is obtained.

[0115] In this embodiment of the invention, "fusion" refers to concatenating radar feature vectors and infrared feature vectors column-wise to form a single high-dimensional feature vector, i.e., the fused feature. This fusion method is early fusion, which has the advantage of preserving the integrity of the original modal features, avoiding information loss, and has extremely low computational overhead, making it suitable for MCU-level embedded processors. The dimension of the fused feature is the sum of the radar feature dimension and the infrared feature dimension, typically 15–23 dimensions.

[0116] S105. Input the fused features into the pre-trained decision tree model to obtain the human detection results.

[0117] In this embodiment of the invention, the "pre-trained decision tree model" is a CART (Classification and Regression Tree) classifier that has been trained and pruned offline. Its maximum depth is ≤8, the number of leaf nodes is ≤64, and it is implemented entirely using integer operations, without relying on floating-point units. It can complete a single inference operation on an ARM Cortex-M4 or higher architecture MCU with a latency of <10 ms. The model input is the fused feature vector obtained in step S104, and the output is a binary classification label: {human, non-human}, along with a confidence score (based on the class distribution probability of training samples within the leaf nodes). This model is constructed using supervised learning. The training data comes from real-world testing in multiple scenarios—including typical deployment environments such as indoor corridors, bedrooms, balconies, and outdoor porches—covering actions such as walking, standing, squatting, and waving, as well as interference samples such as swaying plants, fluttering curtains, rotating fans, running pets, and heating system activation / deactivation. Each class has no fewer than 500 samples, and the labels are confirmed through manual cross-validation.

[0118] The human detection method based on dual-technology sensors provided by this invention synchronously acquires, decouples, extracts, and fuses features from millimeter-wave radar and pyroelectric infrared signals, and performs real-time classification on the edge based on a decision tree model. This enables the system to intelligently determine the presence or absence of a human based on the inherent micro-motion-thermal radiation coupled fingerprint of the target, rather than an isolated single-mode threshold. The fused features imply whether the target possesses the dual physical essence of "biological movement + heat source", thus effectively eliminating interference sources that only have one aspect (such as a fan with only micro-motion and no heat source, or a heater with only heat source and no micro-motion). This reduces the false alarm rate caused by major environmental interference sources such as swaying plants, swaying curtains, and rotating fans, and improves the detection accuracy.

[0119] Figure 2 This invention provides a schematic diagram of the structure of a human body detection device based on a dual-technology sensor, as shown below. Figure 2 As shown, the human body detection device based on dual-technology sensors includes:

[0120] Data acquisition module 201 is used to acquire millimeter-wave radar data and pyroelectric infrared data within the current time window;

[0121] Radar feature extraction module 202 is used to extract radar features from the millimeter-wave radar data;

[0122] Infrared feature extraction module 203 is used to extract infrared features from the pyroelectric infrared data;

[0123] Feature fusion module 204 is used to fuse the radar features and the infrared features to obtain fused features;

[0124] The result prediction module 205 is used to input the fused features into a pre-trained decision tree model to obtain human detection results.

[0125] In some embodiments of the present invention, the radar feature extraction module 202 includes:

[0126] The micro-Doppler feature extraction submodule is used to extract spectral entropy, periodic intensity, and Doppler frequency from the millimeter-wave radar data as micro-Doppler features.

[0127] The motion trajectory feature extraction submodule is used to extract the linearity and randomness of the target trajectory from the millimeter-wave radar data as motion trajectory features;

[0128] The morphological feature extraction submodule is used to extract the density and dispersion of the target point cloud from the millimeter-wave radar data as point cloud morphological features.

[0129] The energy change feature extraction submodule is used to extract the fluctuation frequency and fluctuation amplitude of the signal energy from the millimeter-wave radar data as energy change features.

[0130] The first combination submodule is used to combine the micro-Doppler features, the motion trajectory features, the point cloud morphology features, and the energy change features into radar features.

[0131] In some embodiments of the present invention, the micro-Doppler feature extraction submodule includes:

[0132] The time-spectrum determination unit is used to perform a short-time Fourier transform on the slow-time signal of the millimeter-wave radar data to obtain the time-spectrum.

[0133] The probability distribution conversion unit is used to normalize the frequency distribution of the signal intensity at each moment in the time spectrum into a probability distribution;

[0134] An entropy calculation unit is used to calculate the entropy value of the radar signal at that moment based on the probability distribution.

[0135] The spectrum entropy calculation unit is used to calculate the average value of the radar signal entropy at all times as the spectrum entropy;

[0136] The first autocorrelation function calculation unit is used to calculate the autocorrelation function of the slow-time signal of the millimeter-wave radar data;

[0137] A periodicity intensity calculation unit is used to calculate the entropy value of the autocorrelation function in the time domain as the periodicity intensity;

[0138] The Doppler frequency determination unit is used to take the frequency corresponding to the maximum signal intensity in the time spectrum and the average value of all signal intensities as the Doppler frequency.

[0139] In some embodiments of the present invention, the motion trajectory feature extraction submodule includes:

[0140] The location sequence acquisition unit is used to obtain the location sequence of the target through a tracking algorithm;

[0141] A fitting unit is used to fit the position sequence using the least squares method to obtain a fitted straight line;

[0142] A distance calculation unit is used to calculate the distance from each point in the position sequence to the fitted straight line;

[0143] A linearity calculation unit is used to calculate the fitting error of each point in the position sequence as the linearity of the target trajectory based on the distance.

[0144] The second autocorrelation function calculation unit is used to calculate the autocorrelation function of the position sequence;

[0145] The randomness calculation unit is used to calculate the entropy value of the autocorrelation function in the time domain as randomness.

[0146] In some embodiments of the present invention, the morphological feature extraction submodule includes:

[0147] A point cloud map generation unit is used to generate a range Doppler map as a point cloud map based on the millimeter-wave radar data;

[0148] A density statistics unit is used to count the density of the target point cloud in the point cloud image;

[0149] The coordinate matrix construction unit is used to construct a coordinate matrix from the coordinates of the target point cloud in the point cloud diagram;

[0150] The covariance matrix calculation unit is used to calculate the covariance matrix of the coordinate matrix;

[0151] An eigenvalue calculation unit is used to calculate the eigenvalues ​​of the covariance matrix;

[0152] The dispersion calculation unit is used to calculate the entropy of the feature value as the dispersion of the target point cloud.

[0153] In some embodiments of the present invention, the energy change feature extraction submodule includes:

[0154] A power spectrum calculation unit is used to calculate the power spectrum of the slow-time signal of the millimeter-wave radar data;

[0155] A fluctuation amplitude calculation unit is used to calculate the coefficient of variation of the power spectrum as the fluctuation amplitude;

[0156] The fluctuation frequency calculation unit is used to take the frequency corresponding to the maximum power value in the power spectrum as the fluctuation frequency.

[0157] In some embodiments of the present invention, the infrared feature extraction module 203 includes:

[0158] The waveform feature extraction submodule is used to extract the rise / fall slope and number of peaks of the signal from the pyroelectric infrared data as signal waveform features.

[0159] An energy level extraction submodule is used to extract the absolute energy level of the signal from the pyroelectric infrared data;

[0160] The second combination submodule is used to combine the signal waveform characteristics and the absolute energy level into infrared characteristics.

[0161] The aforementioned human detection device based on dual-technology sensors can execute the human detection method based on dual-technology sensors provided in the foregoing embodiments of the present invention, and has the corresponding functional modules and beneficial effects for executing the human detection method based on dual-technology sensors.

[0162] Figure 3 This is a schematic diagram of an electronic device provided by the present invention. The electronic device is intended to represent various forms of digital computers, such as laptop computers, desktop computers, workstations, personal digital assistants, servers, blade servers, mainframe computers, and other suitable computers. The electronic device can also represent various forms of mobile devices, such as personal digital processors, cellular phones, smartphones, wearable devices (such as helmets, glasses, watches, etc.), and other similar computing devices. The components shown herein, their connections and relationships, and their functions are merely illustrative and are not intended to limit the implementation of the invention described and / or claimed herein.

[0163] like Figure 3As shown, the electronic device includes at least one processor 11 and a memory, such as a read-only memory (ROM) 12 or a random access memory (RAM) 13, communicatively connected to the at least one processor 11. The memory stores computer programs executable by the at least one processor. The processor 11 can perform various appropriate actions and processes based on the computer programs stored in the ROM 12 or loaded from storage unit 18 into the RAM 13. The RAM 13 can also store various programs and data required for the operation of the electronic device. The processor 11, ROM 12, and RAM 13 are interconnected via a bus 14. An input / output (I / O) interface 15 is also connected to the bus 14.

[0164] Multiple components in the electronic device are connected to the I / O interface 15, including: an input unit 16, such as a keyboard, mouse, etc.; an output unit 17, such as various types of displays, speakers, etc.; a storage unit 18, such as a disk, optical disk, etc.; and a communication unit 19, such as a network card, modem, wireless transceiver, etc. The communication unit 19 allows the electronic device to exchange information / data with other devices through computer networks such as the Internet and / or various telecommunications networks.

[0165] Processor 11 can be a variety of general-purpose and / or special-purpose processing components with processing and computing capabilities. Some examples of processor 11 include, but are not limited to, a central processing unit (CPU), a graphics processing unit (GPU), various special-purpose artificial intelligence (AI) computing chips, various processors running machine learning model algorithms, a digital signal processor (DSP), and any suitable processor, controller, microcontroller, etc. Processor 11 performs the various methods and processes described above, such as human detection methods based on dual-technology sensors.

[0166] In some embodiments, the dual-technology sensor-based human detection method can be implemented as a computer program tangibly contained in a computer-readable storage medium, such as storage unit 18. In some embodiments, part or all of the computer program can be loaded and / or installed on an electronic device via ROM 12 and / or communication unit 19. When the computer program is loaded into RAM 13 and executed by processor 11, one or more steps of the dual-technology sensor-based human detection method described above can be performed. Alternatively, in other embodiments, processor 11 can be configured to perform the dual-technology sensor-based human detection method by any other suitable means (e.g., by means of firmware).

[0167] Various embodiments of the systems and techniques described above herein can be implemented in digital electronic circuit systems, integrated circuit systems, field-programmable gate arrays (FPGAs), application-specific integrated circuits (ASICs), application-specific standard products (ASSPs), systems-on-a-chip (SoCs), payload-programmable logic devices (CPLDs), computer hardware, firmware, software, and / or combinations thereof. These various embodiments may include implementations in one or more computer programs that can be executed and / or interpreted on a programmable system including at least one programmable processor, which may be a dedicated or general-purpose programmable processor, capable of receiving data and instructions from a storage system, at least one input device, and at least one output device, and transmitting data and instructions to the storage system, the at least one input device, and the at least one output device.

[0168] Computer programs used to implement the methods of the present invention may be written in any combination of one or more programming languages. These computer programs may be provided to a processor of a general-purpose computer, a special-purpose computer, or other programmable data processing device, such that when executed by the processor, the computer programs cause the functions / operations specified in the flowcharts and / or block diagrams to be performed. The computer programs may be executed entirely on a machine, partially on a machine, or as a standalone software package, partially on a machine and partially on a remote machine, or entirely on a remote machine or server.

[0169] In the context of this invention, a computer-readable storage medium can be a tangible medium that may contain or store a computer program for use by or in conjunction with an instruction execution system, apparatus, or device. A computer-readable storage medium may include, but is not limited to, electronic, magnetic, optical, electromagnetic, infrared, or semiconductor systems, apparatus, or devices, or any suitable combination thereof. Alternatively, a computer-readable storage medium may be a machine-readable signal medium. More specific examples of machine-readable storage media include electrical connections based on one or more wires, portable computer disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fibers, portable compact disk read-only memory (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination thereof.

[0170] To provide interaction with a user, the systems and techniques described herein can be implemented on an electronic device having: a display device (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor) for displaying information to the user; and a keyboard and pointing device (e.g., a mouse or trackball) through which the user provides input to the electronic device. Other types of devices can also be used to provide interaction with the user; for example, feedback provided to the user can be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user can be received in any form (including sound input, voice input, or tactile input).

[0171] The systems and technologies described herein can be implemented in computing systems that include backend components (e.g., as data servers), or middleware components (e.g., application servers), or frontend components (e.g., user computers with graphical user interfaces or web browsers through which users can interact with implementations of the systems and technologies described herein), or any combination of such backend, middleware, or frontend components. The components of the system can be interconnected via digital data communication of any form or medium (e.g., communication networks). Examples of communication networks include local area networks (LANs), wide area networks (WANs), blockchain networks, and the Internet.

[0172] A computing system can include clients and servers. Clients and servers are generally located far apart and typically interact through communication networks. The client-server relationship is created by computer programs running on the respective computers and having a client-server relationship with each other. The server can be a cloud server, also known as a cloud computing server or cloud host, which is a hosting product within the cloud computing service system to address the shortcomings of traditional physical hosts and VPS services, such as high management difficulty and weak business scalability.

[0173] This invention also provides a computer program product, including a computer program that, when executed by a processor, implements the human body detection method based on dual-technology sensors as provided in any embodiment of this application.

[0174] In implementing the computer program product, computer program code for performing the operations of this invention can be written in one or more programming languages ​​or a combination thereof. Programming languages ​​include object-oriented programming languages ​​such as Java, Smalltalk, and C++, as well as conventional procedural programming languages ​​such as C or similar languages. The program code can be executed entirely on the user's computer, partially on the user's computer, as a standalone software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In cases involving remote computers, the remote computer can be connected to the user's computer via any type of network—including a local area network (LAN) or a wide area network (WAN)—or can be connected to an external computer (e.g., via the Internet using an Internet service provider).

[0175] It should be understood that the various forms of processes shown above can be used, with steps reordered, added, or deleted. For example, the steps described in this invention can be executed in parallel, sequentially, or in different orders, as long as the desired result of the technical solution of this invention can be achieved, and this is not limited herein.

[0176] The specific embodiments described above do not constitute a limitation on the scope of protection of this invention. Those skilled in the art should understand that various modifications, combinations, sub-combinations, and substitutions can be made according to design requirements and other factors. Any modifications, equivalent substitutions, and improvements made within the spirit and principles of this invention should be included within the scope of protection of this invention.

Claims

1. A human body detection method based on dual-technology sensors, characterized in that, include: Acquire millimeter-wave radar data and pyroelectric infrared data within the current time window; Extract radar features from the millimeter-wave radar data; Infrared features are extracted from the pyroelectric infrared data; The radar features and the infrared features are fused to obtain the fused features; The fused features are input into a pre-trained decision tree model to obtain human detection results.

2. The human body detection method based on a dual-technology sensor according to claim 1, characterized in that, Extracting radar features from the millimeter-wave radar data includes: The spectral entropy, periodic intensity, and Doppler frequency are extracted from the millimeter-wave radar data as micro-Doppler features. The linearity and randomness of the target trajectory are extracted from the millimeter-wave radar data as motion trajectory features; The density and dispersion of the target point cloud are extracted from the millimeter-wave radar data as point cloud morphological features. The fluctuation frequency and fluctuation amplitude of the signal energy are extracted from the millimeter-wave radar data as energy change characteristics; The micro-Doppler features, the motion trajectory features, the point cloud morphology features, and the energy change features are combined to form radar features.

3. The human body detection method based on a dual-technology sensor according to claim 2, characterized in that, The spectral entropy, periodic intensity, and Doppler frequency are extracted as micro-Doppler features from the millimeter-wave radar data, including: A short-time Fourier transform is performed on the slow-time signal of the millimeter-wave radar data to obtain the time spectrum; For each moment in the time spectrum, the frequency distribution of the signal intensity at that moment is normalized to a probability distribution; The entropy value of the radar signal at that moment is calculated based on the probability distribution. The mean value of the radar signal entropy at all times is calculated as the spectral entropy; Calculate the autocorrelation function of the slow-time signal of the millimeter-wave radar data; The entropy value of the autocorrelation function in the time domain is calculated as the periodicity intensity; The frequency corresponding to the maximum signal intensity in the time spectrum and the average value of all signal intensities are taken as the Doppler frequency.

4. The human body detection method based on a dual-technology sensor according to claim 2, characterized in that, The linearity and randomness of the target trajectory are extracted from the millimeter-wave radar data as motion trajectory features, including: The target's location sequence is obtained through a tracking algorithm; The position sequence was fitted using the least squares method to obtain a fitted straight line; Calculate the distance from each point in the position sequence to the fitted straight line; For each point in the position sequence, the fitting error of that point is calculated based on the distance as the linearity of the target trajectory; Calculate the autocorrelation function of the position sequence; The entropy value of the autocorrelation function in the time domain is calculated as randomness.

5. The human body detection method based on a dual-detection sensor according to claim 2, characterized in that, The density and dispersion of the target point cloud are extracted from the millimeter-wave radar data as point cloud morphological features, including: A range Doppler map is generated as a point cloud map based on the millimeter-wave radar data; Calculate the density of the target point cloud in the point cloud map; The coordinates of the target point cloud in the point cloud map are constructed into a coordinate matrix; Calculate the covariance matrix of the coordinate matrix; Calculate the eigenvalues ​​of the covariance matrix; The entropy of the eigenvalues ​​is calculated as the dispersion of the target point cloud.

6. The human body detection method based on a dual-technology sensor according to claim 2, characterized in that, Extracting the fluctuation frequency and amplitude of signal energy from the millimeter-wave radar data as energy change characteristics includes: Calculate the power spectrum of the slow-time signal of the millimeter-wave radar data; The coefficient of variation of the power spectrum is calculated as the fluctuation amplitude; The frequency corresponding to the maximum power value in the power spectrum is taken as the fluctuation frequency.

7. The human body detection method based on a dual-detection sensor according to any one of claims 1-6, characterized in that, Extracting infrared features from the pyroelectric infrared data includes: The rise / fall slope and number of peaks of the signal are extracted from the pyroelectric infrared data as signal waveform features. Extract the absolute energy level of the signal from the pyroelectric infrared data; The signal waveform characteristics and the absolute energy level are combined to form an infrared feature.

8. A human body detection device based on a dual-technology sensor, characterized in that, include: The data acquisition module is used to acquire millimeter-wave radar data and pyroelectric infrared data within the current time window; A radar feature extraction module is used to extract radar features from the millimeter-wave radar data; An infrared feature extraction module is used to extract infrared features from the pyroelectric infrared data; A feature fusion module is used to fuse the radar features and the infrared features to obtain fused features; The result prediction module is used to input the fused features into a pre-trained decision tree model to obtain human detection results.

9. An electronic device, characterized in that, include: One or more processors; Storage device for storing one or more programs; When the one or more programs are executed by the one or more processors, the one or more processors implement the human body detection method based on dual-technology sensors as described in any one of claims 1-7.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that, When executed by the processor, the program implements the human detection method based on dual-technology sensors as described in any one of claims 1-7.

Images