Daily scene millimeter wave perception non-human target filtering method and system

Abstract

The scheme provides a daily scene millimeter wave perception non-human target filtering method and system, and the steps of the method comprise: based on a millimeter wave radar, millimeter waves are emitted to a target scene, and echo signals are collected; a point cloud sequence of the echo signals is mapped to a high-dimensional frequency domain space to obtain a first point cloud sequence; the first point cloud sequence is input into a preset uniform motion filtering module, and an output of the uniform motion filtering module is mapped to a time domain space to obtain a second point cloud sequence; the second point cloud sequence is input into a preset periodic motion filtering module, the periodic motion filtering module is provided with a multi-head attention layer, a linear layer and a multi-layer perception machine layer, and a point cloud feature sequence is output through the multi-layer perception machine layer; and the point cloud feature sequence is input into a preset classifier to complete point cloud classification.

Description

Technical Field

[0001] This solution relates to the field of millimeter wave technology, and in particular to a method and system for filtering non-human targets using millimeter wave sensing in everyday scenarios. Background Technology

[0002] In recent years, millimeter waves (mmWave) have shown great potential in human perception. With their fine-grained sensing capabilities and privacy-preserving characteristics (i.e., no visual imaging required), millimeter wave technology is expected to play a significant role in home privacy scenarios. Especially with the increasing prevalence of smart home systems, indoor smart applications based on millimeter wave sensing (such as human presence detection and health monitoring) have received widespread attention.

[0003] However, real-world indoor environments are far from ideal (i.e., neither spacious nor bright, nor containing only human targets). Various non-human targets are prevalent in real-world home scenarios, including swaying curtains, running fans, and false targets (i.e., "Ghost artifacts") generated by signal multipath effects (i.e., signals reflecting off surfaces like metal and glass and interacting with the human body). These non-human targets coexist with human targets, and existing algorithms struggle to effectively distinguish between them, leading to frequent false alarms in downstream tasks, such as misdetecting a swaying curtain as the presence of a human. Summary of the Invention

[0004] In view of this, this embodiment of the solution provides a method and system for filtering non-human targets using millimeter-wave sensing in everyday scenarios, in order to eliminate or improve one or more defects existing in the prior art.

[0005] One aspect of this solution provides a method for filtering non-human targets in everyday millimeter-wave sensing scenarios, the steps of which include: Based on the millimeter-wave radar transmitting millimeter waves to the target scene and collecting the echo signals, the point cloud sequence of the echo signals is mapped to a high-dimensional frequency domain space to obtain the first point cloud sequence; The first point cloud sequence is input into a preset uniformity motion filtering module, and the output of the uniformity motion filtering module is mapped to the time domain space to obtain the second point cloud sequence. The second point cloud sequence is input into a preset periodic motion filtering module, which is equipped with a multi-head attention layer, a linear layer and a multi-layer perceptron layer, and outputs a point cloud feature sequence through the multi-layer perceptron layer. The point cloud feature sequence is input into a preset classifier to complete the point cloud classification.

[0006] The above-mentioned scheme has two main aspects. First, it converts time-domain features into frequency-domain representations and uses dynamically learnable weights to filter out nondeterministic spectral peaks in the low-frequency band to suppress uniform motion interference. For periodic motion interference (such as fan rotation and oscillation), the scheme divides the motion space into multiple sub-regions and applies global autocorrelation analysis to each sub-region to separate different periodic components (such as rotation and oscillation components) and suppress them separately. Second, it uses a contrastive learning framework to align the high-dimensional human kinematic features from optical sensors with the spatiotemporal features extracted from millimeter waves, thereby filtering out Ghost artifacts that do not meet human kinematic constraints. It also introduces Physics-Informed Neural Networks (PINNs) to ensure the rationality of human kinematics and guarantee accurate identification of human or non-human objects.

[0007] In some embodiments of this scheme, in the step of mapping the point cloud sequence of the echo signal to a high-dimensional frequency domain space to obtain the first point cloud sequence, the dimension of the point cloud sequence is increased by a linear layer, and then the time domain data is mapped to the frequency domain by a fast Fourier transform to obtain the first point cloud sequence.

[0008] In some embodiments of this solution, in the step of inputting the first point cloud sequence into a preset uniformity motion filtering module, the uniformity motion filtering module uses a multilayer perceptron to extract information in the frequency domain of the first point cloud sequence.

[0009] In some embodiments of this solution, in the step of inputting the second point cloud sequence into a preset periodic motion filtering module, the corresponding data of each point cloud in the second point cloud sequence is linearly transformed to obtain a query vector, a key vector, and a value vector, and periodic interference components are filtered through a linear layer and a multi-layer perceptron layer.

[0010] In some embodiments of this solution, in the step of inputting the point cloud feature sequence into a preset classifier to complete point cloud classification, the point cloud feature sequence and the point cloud data in the original point cloud sequence are dimension-unified, so that the data dimension of each point cloud in the point cloud feature sequence is the same as the data dimension of the point cloud in the original point cloud sequence. The dimension-unified point cloud feature sequence is then input into the preset classifier to complete point cloud classification.

[0011] In some embodiments of this solution, the method further includes pre-training the model. The pre-training step includes calculating a point cloud classification loss, and pre-training the model, which includes a uniformity motion filtering module, a periodic motion filtering module, and a classifier, based on the value of the point cloud classification loss. The point cloud classification loss is calculated using the following formula: in, The value represents the point cloud classification loss, where T is the number of frames in the point cloud sequence of the echo signal, and N is the number of points in each frame of the point cloud sequence of the echo signal. This represents the point cloud classification loss value for each point cloud instance. Indicates the category label of the real point cloud. This indicates the predicted point cloud category output by the classifier.

[0012] In some embodiments of this solution, the pre-training step further includes: Data is collected synchronously using a depth camera and millimeter-wave radar to obtain joint coordinates. The joint coordinates are then input into the Ghost artifact filtering module. The Ghost artifact filtering module performs sequential processing through the implicit neural representation layer and joint encoder to obtain a joint feature sequence. The human limb motion constraint loss is calculated based on the joint feature sequence. The joint feature sequence and the point cloud feature sequence are aligned, and the human motion sequence feature alignment loss is calculated based on the aligned joint feature sequence and the point cloud feature sequence.

[0013] In some implementations of this scheme, in the step of calculating the human motion sequence feature alignment loss based on the joint feature sequence and the point cloud feature sequence after feature alignment, the following formula is used to calculate the human motion sequence feature alignment loss: in, This represents the value of the human motion sequence feature alignment loss. The feature dimension is the result of aligning the joint feature sequence with the point cloud feature sequence. and Let represent the covariance matrices of the point cloud feature sequence and the joint feature sequence, respectively. This represents the Frobenius norm.

[0014] In some implementations of this solution, the following formula is used to calculate the human limb motion constraint loss in the step of calculating the human limb motion constraint loss based on joint feature sequences: in, This represents the value indicating the loss of motor restraint in human limbs. This represents the value of the joint coordinate constraint loss. This represents the value of the mean square error.A The number of frames for data acquisition by millimeter-wave radar. This represents any frame of data acquired by the millimeter-wave radar. For any key point, These are preset hyperparameters. express Frames The coordinates of the joints in the joint feature sequence express Frames Joint coordinates of key points in millimeter-wave radar data; express Frames The value of the preset torque imbalance term at the joint; express The inertia matrix of the joints, express Joint angular acceleration at the joint points Indicates preset The coupling value of centrifugal force and Coriolis force at the joint point express The key point is The joint angular velocity of the frame, Joint gravity at the joint points.

[0015] In some embodiments of this solution, the pre-training step further includes calculating the total loss based on the human limb motion constraint loss, human motion sequence feature alignment loss, and point cloud classification loss, and pre-training the model including the uniformity motion filtering module, the periodic motion filtering module, and the classifier based on the total loss, using the following formula to calculate the total loss: in, This represents the value of the total loss. This represents the value of the point cloud classification loss. This represents the value of the human motion sequence feature alignment loss. This represents the value indicating the loss of motor restraint in human limbs. and All of these are preset hyperparameters.

[0016] The second aspect of this solution also provides a millimeter-wave sensing non-human target filtering system for everyday scenarios. The system includes a computer device, which includes a processor and a memory. The memory stores computer instructions, and the processor executes the computer instructions stored in the memory. When the computer instructions are executed by the processor, the system implements the steps of the method described above.

[0017] A third aspect of this solution also provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the steps of the aforementioned millimeter-wave sensing non-human target filtering method for everyday scenarios.

[0018] Additional advantages, objectives, and features of this solution will be set forth in part in the description which follows, and will in part become apparent to those skilled in the art upon review of the description, or may be learned by practice of the solution. The objectives and other advantages of this solution will become apparent from the specific details provided in the specification and the accompanying drawings.

[0019] Those skilled in the art will understand that the purposes and advantages that can be achieved by this solution are not limited to those specifically described above, and the above and other purposes that this solution can achieve will be more clearly understood from the following detailed description. Attached Figure Description

[0020] The accompanying drawings, which are provided to further illustrate the present invention and form part of this application, do not constitute a limitation thereof.

[0021] Figure 1 This is a schematic diagram illustrating one implementation of the millimeter-wave sensing non-human target filtering method for everyday scenarios in this solution. Figure 2 This is a schematic diagram of the overall architecture of the millimeter-wave sensing non-human target filtering method for everyday scenarios in this solution; Figure 3 This is a schematic diagram of the processing architecture of the uniformity motion filtering module in this solution. Figure 4 This is a schematic diagram of the processing architecture of the periodic motion filtering module in this solution; Figure 5 This is a schematic diagram of the processing architecture of the Ghost artifact filtering module in this solution. Detailed Implementation

[0022] To make the objectives, technical solutions, and advantages of this solution clearer, the following detailed description is provided in conjunction with the implementation methods and accompanying drawings. The illustrative implementation methods and their descriptions are used to explain this solution, but are not intended to limit it.

[0023] It should also be noted that, in order to avoid obscuring the present solution with unnecessary details, only the structures and / or processing steps closely related to the solution are shown in the accompanying drawings, while other details that are not closely related to the present solution are omitted.

[0024] In millimeter-wave target detection, traditional algorithms (such as Constant False Alarm Rate (CFAR)) typically detect targets by extracting regions where signal energy exceeds an adaptive threshold. Recent research has mainly fallen into two categories: one employs a holistic modeling approach, using machine learning-driven spatiotemporal feature analysis to distinguish between human and non-human targets; the other uses semantic localization methods, extracting human signals by marking human-related regions in radar observations. However, existing methods fail to adequately consider the key kinematic differences between human and non-human interference (i.e., the human body can be considered a system composed of multiple rigid bodies, with coordinated but non-deterministic joint movements), resulting in poor performance in home environments.

[0025] To address the aforementioned issues, this solution employs a deep learning architecture to enable it to perceive the kinematic characteristics of heterogeneous targets. It utilizes two types of discriminative interference features: (1) statistical features of regular motion interference (such as the uniform motion of curtains and the periodic motion of fans); and (2) ghost artifacts caused by non-ideal reflection patterns due to multipath effects. The framework comprises three complementary modules: a frequency-domain dynamic filtering module to suppress interference caused by motion uniformity, a cross-subspace spatiotemporal autocorrelation analysis module to eliminate periodic motion interference, and a kinematic feature alignment module to suppress ghost targets generated by multipath effects.

[0026] like Figure 1 As shown, this solution proposes a millimeter-wave sensing method for filtering non-human targets in everyday scenarios. The steps of this method include: Step S100: Based on the millimeter-wave radar, a millimeter wave is transmitted to the target scene and the echo signal is collected. The point cloud sequence of the echo signal is mapped to a high-dimensional frequency domain space to obtain the first point cloud sequence. In the specific implementation process, conventional signal processing algorithms are first used to generate millimeter-wave point clouds: ① Range-FFT operation is performed on the received millimeter-wave signal to obtain the range map of the signal; ② Static filtering algorithm is used to remove static information in the environment to retain the motion characteristics of dynamic targets; ③ Capon Beamforming method is applied to generate Range-Azimuth matrix for subsequent target spatial localization; ④ CFAR (Constant False Alarm Rate) algorithm is used to process the above matrix to remove noise signals and retain target echoes; ⑤ Elevation estimation and Doppler-FFT operation are further performed on each Range-Azimuth dimension to obtain the target's elevation information and radial velocity information.

[0027] Step S200: Input the first point cloud sequence into a preset uniformity motion filtering module, and map the output of the uniformity motion filtering module to the temporal space to obtain the second point cloud sequence. Step S300: The second point cloud sequence is input into a preset periodic motion filtering module. The periodic motion filtering module is equipped with a multi-head attention layer, a linear layer and a multi-layer perceptron layer, and outputs a point cloud feature sequence through the multi-layer perceptron layer. Step S400: Input the point cloud feature sequence into a preset classifier to complete the point cloud classification.

[0028] This solution addresses two key technical challenges related to the two common non-human interferences encountered in everyday scenarios: 1. Uncertainty of Regular Motion Interference: In a home setting, moving objects typically exhibit regular trajectories, such as the swaying of curtains and the rotation of fans. Curtain motion exhibits uniformity and can be considered a quasi-rigid body, with its components maintaining relatively consistent velocities. Fan motion, on the other hand, exhibits periodicity, with its trajectory changing periodically over time and predictable through linear models. However, these regular motions also present uncertainties: the uniformity of curtains can be disturbed by wind or human intervention, leading to inconsistent speeds; the multi-degree-of-freedom periodic motion of fans (such as rotation and oscillation) can be inextricably coupled into aperiodic motion, and the temporal sequence of the periodicity of each motion component may also change. To address these issues, this solution converts time-domain features into frequency-domain representations and utilizes dynamically learnable weights to filter out indeterminate spectral peaks in the low-frequency band to suppress uniform motion interference. For periodic motion interference (such as fan rotation and oscillation), this solution divides the motion space into multiple sub-regions and applies global autocorrelation analysis within each sub-region to separate different periodic components (such as rotation and oscillation components), which are then suppressed separately.

[0029] 2. Mirroring of Ghost Artifacts: In a home environment, due to multipath propagation, a spatiotemporal mirroring phenomenon occurs between human motion and ghost artifacts, with the dynamic symmetry of the reflected signal mimicking the original motion trajectory. To address this issue, this solution first discovers that although ghost artifacts and the human body are kinematically coupled, multipath attenuation leads to incomplete or distorted motion representations. For example, ghost artifacts may only retain parts of the limbs (such as the arm), and the motion trajectory may be distorted. This phenomenon violates the kinematic laws of specular reflection, exhibiting a nonlinear deformation similar to a "funhouse mirror" effect. Therefore, this solution utilizes a contrastive learning framework to align the high-dimensional kinematic features of the human body from optical sensors with the spatiotemporal features extracted from millimeter waves, thereby filtering out ghost artifact components that do not satisfy the constraints of human kinematics. Furthermore, it introduces Physics-Informed Neural Networks (PINNs) to ensure the rationality of human kinematics.

[0030] The above-mentioned scheme has two main aspects. First, it converts time-domain features into frequency-domain representations and uses dynamically learnable weights to filter out nondeterministic spectral peaks in the low-frequency band to suppress uniform motion interference. For periodic motion interference (such as fan rotation and oscillation), the scheme divides the motion space into multiple sub-regions and applies global autocorrelation analysis to each sub-region to separate different periodic components (such as rotation and oscillation components) and suppress them separately. Second, it uses a contrastive learning framework to align the high-dimensional human kinematic features from optical sensors with the spatiotemporal features extracted from millimeter waves, thereby filtering out Ghost artifacts that do not meet human kinematic constraints. It also introduces Physics-Informed Neural Networks (PINNs) to ensure the rationality of human kinematics and guarantee accurate identification of human or non-human objects.

[0031] In some embodiments of this scheme, in the step of mapping the point cloud sequence of the echo signal to a high-dimensional frequency domain space to obtain the first point cloud sequence, the dimension of the point cloud sequence is increased by a linear layer, and then the time domain data is mapped to the frequency domain by a fast Fourier transform to obtain the first point cloud sequence.

[0032] In some embodiments of this solution, in the step of inputting the first point cloud sequence into a preset uniformity motion filtering module, the uniformity motion filtering module uses a multilayer perceptron to extract information in the frequency domain of the first point cloud sequence.

[0033] In practical implementation, the uniformity motion filtering module of this solution is used to effectively extract human point cloud features from environments containing non-human objects (such as curtains and plants) with uniform motion. To achieve this, this solution designs a frequency domain filtering process based on neural networks, which uses the spectral characteristics of motion signals to dynamically suppress non-human interference.

[0034] Specifically, such as Figure 2 As shown, the point cloud sequence of the echo signal is defined as follows: This module first maps the input point cloud to a high-dimensional frequency domain space through a linear layer. Since uniform motion interference (such as curtains and plants) exhibits significant low-frequency components in the frequency domain, traditional filtering algorithms can suppress signals with significant isolated peaks in the frequency domain. However, in real-world environments, changes in environmental conditions and time fluctuations, coupled with external non-stationary interference, make these characteristic peaks unstable. As a result, the energy of the low-frequency components is no longer concentrated at a fixed value but fluctuates within a certain range, exhibiting a diffuse distribution, making it difficult for traditional filtering algorithms to completely suppress such interference.

[0035] Therefore, the model described in this scheme uses learnable weights to weight each feature element in the frequency domain to suppress uniform motion interference. Subsequently, a multilayer perceptron (MLP) is used to extract key information of the human motion signal in the frequency domain, and the features are transformed back to the time domain via inverse Fourier transform, providing input for subsequent processing in the periodic motion filtering module. In summary, the uniform motion filtering module uses spectral information to separate motion signals. Uniformly moving objects (such as curtains and plants) typically have highly concentrated energy in a few low-frequency regions of the frequency domain, while human motion exhibits a more dispersed spectral distribution. Through training, the learnable weights dynamically adjust the contribution of each frequency component, thereby effectively attenuating interference signals after the inverse transform and retaining only the key information reflecting human motion.

[0036] In some embodiments of this solution, in the step of inputting the second point cloud sequence into a preset periodic motion filtering module, the corresponding data of each point cloud in the second point cloud sequence is linearly transformed to obtain a query vector, a key vector, and a value vector, and periodic interference components are filtered through a linear layer and a multi-layer perceptron layer.

[0037] In the specific implementation process, the periodic motion filtering module of this solution is used to effectively extract human motion features from non-human objects (such as fans and air conditioners) with periodic motion. In order to effectively filter periodic motion interference features, this module needs to solve two main technical challenges: (1) Some objects are composed of multiple independent parts. If the motion periods of each part are incommensurable, treating the object as a whole will lead to the merging into non-periodic motion; (2) The uncertainty of the duration of the period of the interfering object makes it difficult for autocorrelation analysis with fixed lag coefficients to effectively suppress interference.

[0038] The specific process is as follows: Figure 4 As shown. To mitigate the interference caused by the independent motion of different parts of an object, this scheme first performs a linear transformation on the spatial motion time series data of each object to obtain Queries (Q), Keys (K), and Values ​​(V). Then, a multi-head attention mechanism is used to project the different motion parts of the object into a high-dimensional subspace, i.e. (Q, K, and V are divided into h heads, and each head has a dimension of $) ,in (This refers to the total projection dimension). To address the uncertainty of the motion period of interfering objects, this scheme utilizes a self-attention mechanism to capture the temporal correlation between different time steps in the input sequence, i.e. Because the self-attention matrix of the periodic subspace exhibits significant horizontal or vertically spaced high-weight bands (e.g., Figure 3 (As shown in the periodic features), the model then concatenates the features of each subspace and filters out periodic interference components through linear layers and multilayer perceptron (MLP) layers, retaining only the key features of human motion.

[0039] In some embodiments of this solution, in the step of inputting the point cloud feature sequence into a preset classifier to complete point cloud classification, the point cloud feature sequence and the point cloud data in the original point cloud sequence are dimension-unified, so that the data dimension of each point cloud in the point cloud feature sequence is the same as the data dimension of the point cloud in the original point cloud sequence. The dimension-unified point cloud feature sequence is then input into the preset classifier to complete point cloud classification. The classifier can be a softmax classifier.

[0040] In some embodiments of this solution, the method further includes pre-training the model. The pre-training step includes calculating a point cloud classification loss, and pre-training the model, which includes a uniformity motion filtering module, a periodic motion filtering module, and a classifier, based on the value of the point cloud classification loss. The point cloud classification loss is calculated using the following formula: in, The value represents the point cloud classification loss, where T is the number of frames in the point cloud sequence of the echo signal, and N is the number of points in each frame of the point cloud sequence of the echo signal. This represents the point cloud classification loss value for each point cloud instance. Indicates the category label of the real point cloud. This indicates the predicted point cloud category output by the classifier.

[0041] like Figure 1 and 5 As shown, in some embodiments of this solution, the pre-training step further includes: Data is collected synchronously using a depth camera and millimeter-wave radar to obtain joint coordinates. The joint coordinates are then input into the Ghost artifact filtering module. The Ghost artifact filtering module performs sequential processing through the implicit neural representation layer and joint encoder to obtain a joint feature sequence. The human limb motion constraint loss is calculated based on the joint feature sequence. The joint feature sequence and the point cloud feature sequence are aligned, and the human motion sequence feature alignment loss is calculated based on the aligned joint feature sequence and the point cloud feature sequence.

[0042] In practical implementation, the Ghost artifact filtering module of this solution is used to extract human point cloud features from environments containing ghost artifacts. In a home environment, when a signal travels through a human body and along multiple paths to the receiver, ghost artifacts that are easily confused with real human targets are generated. Since the overall motion trend of these artifacts is consistent with that of a real human body, it is difficult to directly filter them based solely on position and trajectory information. On the other hand, ghost signals experience energy attenuation after multiple reflections in the environment, resulting in a reduction in the number of point clouds. However, in a quasi-static state (such as when a person is sitting and reading or typing), the point cloud density is also low, so it is impossible to distinguish between a real human body and a ghost solely based on the number of point clouds.

[0043] Inspired by the concept of comparative learning, its specific process is as follows: Figure 5 As shown in the figure, the upper part is the main network, which processes millimeter-wave sensing point cloud data collected from the real environment through a point cloud encoder (including a uniformity motion filtering module and a periodic motion filtering module) to obtain a point cloud motion feature sequence containing only human and ghost artifacts. The lower part is the auxiliary network, serving as a supplementary source of human limb motion features. Specifically, the Ghost artifact filtering module uses a depth camera to collect joint coordinates of human movement in the real environment and processes them through implicit neural representation (INR) and a joint encoder to generate a feature sequence containing only human motion features. Unlike the sparse and localized distribution of millimeter-wave radar point clouds, depth camera data focuses only on human joint information, providing a more comprehensive representation of human limb movement. Based on the principle of contrastive learning, the millimeter-wave sensing feature sequence is aligned with the human joint feature sequence generated by the depth camera. Since the motion characteristics of ghosts do not conform to the motion patterns of human joints, the millimeter-wave sensing features will tend to retain only attributes related to human movement, thereby effectively filtering ghost interference.

[0044] Since feature alignment is performed in a high-dimensional space, it eliminates the need to assign point clouds to specific body parts, effectively avoiding the inaccurate matching between point clouds and body parts present in previous works. Furthermore, to mitigate joint coordinate fluctuations caused by temporal synchronization differences between the two modalities (depth camera frame rate is higher than millimeter-wave radar), an INR module is employed to enhance the continuity of feature sequences across frames. Further, after feature alignment, the auxiliary network maps features to joint coordinates to complete network training. This scheme utilizes a Ghost artifact filtering module in pre-training, enabling the backbone network to learn effective ghost feature filtering capabilities during point cloud encoding.

[0045] In some implementations of this scheme, in the step of calculating the human motion sequence feature alignment loss based on the joint feature sequence and the point cloud feature sequence after feature alignment, the following formula is used to calculate the human motion sequence feature alignment loss: in, This represents the value of the human motion sequence feature alignment loss. The feature dimension is the result of aligning the joint feature sequence with the point cloud feature sequence. and Let represent the covariance matrices of the point cloud feature sequence and the joint feature sequence, respectively. This represents the Frobenius norm.

[0046] In some implementations of this solution, the following formula is used to calculate the human limb motion constraint loss in the step of calculating the human limb motion constraint loss based on joint feature sequences: in, This represents the value indicating the loss of motor restraint in human limbs. This represents the value of the joint coordinate constraint loss. This represents the value of the mean square error. A The number of frames for data acquisition by millimeter-wave radar. This represents any frame of data acquired by the millimeter-wave radar. For any key point, These are preset hyperparameters. express Frames The coordinates of the joints in the joint feature sequence express Frames Joint coordinates of key points in millimeter-wave radar data; express Frames The value of the preset torque imbalance term at the joint; express The inertia matrix of the joints, express Joint angular acceleration at the joint points Indicates preset The coupling value of centrifugal force and Coriolis force at the joint point express The key point is The joint angular velocity of the frame, Joint gravity at the joint points.

[0047] In some embodiments of this solution, the pre-training step further includes calculating the total loss based on the human limb motion constraint loss, human motion sequence feature alignment loss, and point cloud classification loss, and pre-training the model including the uniformity motion filtering module, the periodic motion filtering module, and the classifier based on the total loss, using the following formula to calculate the total loss: in, This represents the value of the total loss. This represents the value of the point cloud classification loss. This represents the value of the human motion sequence feature alignment loss. This represents the value indicating the loss of motor restraint in human limbs. and All of these are preset hyperparameters.

[0048] In summary, the core of this solution lies in combining physical information constraints with neural network-based methods to filter out interfering objects that do not conform to human motion characteristics. The overall architecture is as follows: Figure 1 As shown, its three main modules are used to filter the above three types of non-human interference: (1) Uniform Motion Interference. In home scenarios, wind or object collisions may generate non-human noise such as curtains and green plants, interfering with human recognition tasks. To address this, this solution proposes a uniformity filtering module to filter out such noise. Since the distribution, velocity, and other characteristics of such objects in the point cloud space are uniform, this module proposes to filter targets with significant characteristic peaks based on Fourier transform (FT) and filtering concepts. However, in real-world scenarios, the motion caused by wind and collisions is not constant and varies with the environment and the material of the object itself. Therefore, this solution combines neural network algorithms to design a dynamically learnable filter to adapt to uniform moving object noise in space.

[0049] (2) Periodic Motion Interference. To address highly periodic non-human motion interference in real-world scenarios, such as from fans and air conditioners, this solution proposes a periodic filtering module. Since these objects move periodically along specific trajectories, this module uses autocorrelation analysis for filtering. However, the motion trajectories and periodic time windows of different objects vary in different environments, and selecting a specific lag coefficient for autocorrelation analysis of the sequence can easily lead to false filtering. Furthermore, the independent motion of multiple parts of such objects may interfere with periodicity judgment, such as the rotation and oscillation of a fan. Therefore, this module utilizes a self-attention mechanism to establish the correlation of point cloud features in each frame of the sequence from a global perspective, and combines a multi-head mechanism to separate different motion frequency components of each part of the object to analyze the periodic motion of each part, thereby achieving more accurate filtering.

[0050] (3) Ghost images generated by multipath signals passing through the human body exhibit complexity and randomness during motion, and their trajectories are highly similar to human activities, making them a major source of confusion in existing human body recognition. However, non-ideal planar reflections and environmental complexity in everyday scenes lead to significant signal energy attenuation, causing Ghost artifacts to appear incomplete or distorted relative to the corresponding human body. Based on this finding, this solution proposes a Ghost artifact filtering module. First, it extracts the human body joint coordinates generated by a depth camera, and then uses a neural network to perform feature transformation and modal alignment on the point clouds of each object to remove Ghost artifacts that are inconsistent with human motion characteristics. In addition, this module is designed to facilitate the backbone network to learn this function step by step, eliminating the need for an auxiliary network during the inference phase.

[0051] After processing by the three filtering modules mentioned above, the point cloud sequence in the spatial domain retains only features related to human activity. By concatenating the features of the original sequence after feature mapping via a multilayer perceptron (MLP) with the filtered features, this operation is equivalent to labeling the human body in the target in a high-dimensional feature space. This enables the finally trained model to distinguish between human and non-human entities in the spatial domain. Furthermore, to address the inevitable loss and distortion of joint coordinates in home environments where only a single or a few auxiliary optical sensors can be deployed, this solution proposes a constraint based on Physical Information Neural Networks (PINNs).

[0052] This solution can accurately filter out non-human targets (including regular movements and ghost artifacts) in complex everyday scenarios, achieving a classification accuracy of over 97% in distinguishing between human and non-human targets. Furthermore, this invention has strong generalization ability, maintaining high classification accuracy in different environments, with unseen individuals, and with new motion interference. This invention can serve as a foundation to prevent performance degradation in subsequent downstream tasks (such as activity recognition and presence detection) due to the presence of non-human targets.

[0053] The beneficial effects of this plan include: 1. By analyzing the macroscopic kinematic differences between human and non-human targets, it is possible to effectively distinguish between the two in complex home environments.

[0054] 2. Based on the prior knowledge of motion differences, a corresponding neural network algorithm was designed. The neural network is divided into three non-human interference filtering modules to dynamically remove the above interference.

[0055] 3. The uniformity motion filtering module uses learnable weights to filter dynamic peaks of uniform motion in the frequency domain; the periodic motion filtering module uses multi-head self-attention to decompose periodic motion with multiple degrees of freedom into subspaces, and uses the high and low interval weights of the self-attention matrix unique to periodic motion to filter out the interference; the ghost artifact filtering module uses the distortion and missing parts of the ghost relative to the human body, and uses the contrast learning idea to align the millimeter wave sensing features with the human joint features of the depth camera to filter out ghosts that do not conform to normal limb movements.

[0056] 4. This scheme designs corresponding loss functions to address the noise filtering requirements of neural network structures: point cloud classification loss, human motion sequence feature alignment loss, and human limb motion constraints. Their respective tasks are defining the neural network's human / non-human classification task, filtering ghost artifact features, and ensuring the reasonableness of depth camera human motion features.

[0057] 5. This solution can accurately filter out non-human targets (including regular movements and ghost artifacts) in complex everyday scenarios, achieving a classification accuracy of over 97% in distinguishing between human and non-human targets. Furthermore, this invention has strong generalization ability, maintaining high classification accuracy in different environments, with unseen individuals, and with new motion interference.

[0058] This embodiment of the solution also provides a millimeter-wave sensing non-human target filtering system for everyday scenarios. The system includes a computer device, which includes a processor and a memory. The memory stores computer instructions, and the processor is used to execute the computer instructions stored in the memory. When the computer instructions are executed by the processor, the system implements the steps of the method described above.

[0059] This embodiment also provides a computer-readable storage medium storing a computer program that, when executed by a processor, implements the steps of the aforementioned millimeter-wave sensing non-human target filtering method for everyday scenarios. The computer-readable storage medium can be a tangible storage medium, such as random access memory (RAM), main memory, read-only memory (ROM), electrically programmable ROM, electrically erasable programmable ROM, registers, floppy disks, hard disks, removable storage disks, CD-ROMs, or any other form of storage medium known in the art.

[0060] Those skilled in the art will understand that the exemplary components, systems, and methods described in conjunction with the embodiments disclosed herein can be implemented in hardware, software, or a combination of both. Whether implemented in hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of this solution. When implemented in hardware, it can be, for example, electronic circuits, application-specific integrated circuits (ASICs), appropriate firmware, plug-ins, function cards, etc. When implemented in software, the elements of this solution are programs or code segments used to perform the required tasks. The programs or code segments can be stored on a machine-readable medium or transmitted over a transmission medium or communication link via data signals carried on a carrier wave.

[0061] It should be clarified that this solution is not limited to the specific configurations and processes described above and shown in the figures. For the sake of brevity, detailed descriptions of known methods are omitted here. In the above embodiments, several specific steps are described and shown as examples. However, the method process of this solution is not limited to the specific steps described and shown. Those skilled in the art can make various changes, modifications, and additions, or change the order of steps, after understanding the spirit of this solution.

[0062] In this solution, features described and / or illustrated for one implementation may be used in the same or similar manner in one or more other implementations, and / or combined with or replaced by features of other implementations.

[0063] The above description is merely a preferred embodiment of this solution and is not intended to limit the solution. For those skilled in the art, various modifications and variations can be made to this solution. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of this solution should be included within the protection scope of this solution.

Claims

1. A method for filtering non-human targets using millimeter-wave sensing in everyday scenarios, characterized in that, The steps of this method include: Based on the millimeter-wave radar transmitting millimeter waves to the target scene and collecting the echo signals, the point cloud sequence of the echo signals is mapped to a high-dimensional frequency domain space to obtain the first point cloud sequence; The first point cloud sequence is input into a preset uniformity motion filtering module, and the output of the uniformity motion filtering module is mapped to the time domain space to obtain the second point cloud sequence. The second point cloud sequence is input into a preset periodic motion filtering module, which is equipped with a multi-head attention layer, a linear layer and a multi-layer perceptron layer, and outputs a point cloud feature sequence through the multi-layer perceptron layer. The point cloud feature sequence is input into a preset classifier to complete the point cloud classification; The method further includes pre-training the model, which further includes: simultaneously acquiring data using a depth camera and millimeter-wave radar to obtain joint coordinates; inputting the joint coordinates into a Ghost artifact filtering module; sequentially processing the joint coordinates through the implicit neural representation layer and joint encoder in the Ghost artifact filtering module to obtain a joint feature sequence; and calculating the human limb motion constraint loss based on the joint feature sequence using the following formula: in, This represents the value indicating the loss of motor restraint in human limbs. This represents the value of the joint coordinate constraint loss. This represents the value of the mean square error. A The number of frames for data acquisition by millimeter-wave radar. This represents any frame of data acquired by the millimeter-wave radar. For any key point, These are preset hyperparameters. express Frames The coordinates of the joints in the joint feature sequence. express Frames Joint coordinates of key points in millimeter-wave radar data; express Frames The value of the preset torque imbalance term at the joint; express The inertia matrix of the joints, express Joint angular acceleration at the joint points Indicates preset The coupling value of centrifugal force and Coriolis force at the joint point express The key point is The joint angular velocity of the frame, express Joint gravity at the joint points; The joint feature sequence and the point cloud feature sequence are aligned, and the human motion sequence feature alignment loss is calculated based on the aligned joint feature sequence and the point cloud feature sequence.

2. The method for filtering non-human targets in everyday scenarios using millimeter-wave sensing according to claim 1, characterized in that, In the step of mapping the point cloud sequence of the echo signal to a high-dimensional frequency domain space to obtain the first point cloud sequence, the dimension of the point cloud sequence is increased by a linear layer, and then the time domain data is mapped to the frequency domain by a fast Fourier transform to obtain the first point cloud sequence.

3. The method for filtering non-human targets in everyday scenarios using millimeter-wave sensing according to claim 1, characterized in that, In the step of inputting the first point cloud sequence into the preset uniformity motion filtering module, the uniformity motion filtering module uses a multilayer perceptron to extract information in the frequency domain of the first point cloud sequence.

4. The method for filtering non-human targets in everyday scenarios using millimeter-wave sensing according to claim 1, characterized in that, In the step of inputting the second point cloud sequence into the preset periodic motion filtering module, the corresponding data of each point cloud in the second point cloud sequence is linearly transformed to obtain the query vector, key vector and value vector, and the periodic interference components are filtered through the linear layer and the multi-layer perceptron layer.

5. The method for filtering non-human targets in everyday scenarios using millimeter-wave sensing according to claim 1, characterized in that, In the step of inputting the point cloud feature sequence into a preset classifier to complete point cloud classification, the point cloud feature sequence and the point cloud data in the original point cloud sequence are dimension-unified so that the data dimension of each point cloud in the point cloud feature sequence is the same as the data dimension of the point cloud in the original point cloud sequence. The point cloud feature sequence after dimension unification is then input into the preset classifier to complete point cloud classification.

6. The method for filtering non-human targets in everyday scenarios using millimeter-wave sensing according to any one of claims 1 to 5, characterized in that, The pre-training step includes calculating the point cloud classification loss, and pre-training the model including the uniformity motion filtering module, the periodic motion filtering module, and the classifier based on the value of the point cloud classification loss. The point cloud classification loss is calculated using the following formula: in, The value represents the point cloud classification loss, where T is the number of frames in the point cloud sequence of the echo signal, and N is the number of points in each frame of the point cloud sequence of the echo signal. This represents the point cloud classification loss value for each point cloud instance. Indicates the category label of the real point cloud. This indicates the predicted point cloud category output by the classifier.

7. The method for filtering non-human targets in everyday scenarios using millimeter-wave sensing according to claim 6, characterized in that, In the step of calculating the feature alignment loss of human motion sequence based on the joint feature sequence and point cloud feature sequence after feature alignment, the following formula is used to calculate the feature alignment loss of human motion sequence: in, This represents the value of the human motion sequence feature alignment loss. The feature dimension is the result of aligning the joint feature sequence with the point cloud feature sequence. and Let represent the covariance matrices of the point cloud feature sequence and the joint feature sequence, respectively. This represents the Frobenius norm.

8. A millimeter-wave sensing non-human target filtering system for everyday scenarios, characterized in that, The system includes a computer device, which includes a processor and a memory. The memory stores computer instructions, and the processor executes the computer instructions stored in the memory. When the computer instructions are executed by the processor, the system implements the steps of the method as described in any one of claims 1 to 7.

Images