A neural network-based millimeter wave radar living body recognition method

Abstract

The application discloses a kind of millimeter wave radar living body identification methods based on neural network, comprising: obtaining package original ADC data, signal pre-processing is carried out, and three-dimensional radar point cloud is generated;Three-dimensional radar point cloud is respectively subjected to space geometry feature enhancement and dynamics mode feature enhancement, and enhanced feature is obtained;Three-dimensional radar point cloud and enhanced feature are parallelly input into multimodal heterogeneous feature decoupling network, and the deep physical property feature of target is respectively decoupled and extracted;Cross-modal interaction and adaptive weighted fusion are carried out to the deep physical property feature, and obtain fusion feature vector;Fusion feature vector is input into classifier, and the discrimination result whether the target in package is living body is output.The deep physical property feature of target is extracted by the multimodal heterogeneous feature decoupling network of the application, and adaptive depth fusion is carried out using cross-modal attention mechanism, so as to effectively suppress non-living interference under complex dynamic environment, and the accuracy of living body detection is significantly improved.

Description

Technical Field

[0001] This invention relates to the field of logistics safety detection and wireless sensing intelligent processing technology, and more specifically to a millimeter-wave radar liveness detection method based on neural networks. Background Technology

[0002] In modern logistics security inspections, non-contact detection of live targets inside opaque packages is of great significance. Existing X-ray scanning equipment suffers from problems such as bulkiness, high cost, and ionizing radiation risks, making it difficult to meet the high-throughput, low-risk logistics inspection requirements. Millimeter-wave radar, as a non-ionizing, non-contact sensing method, can penetrate common packaging materials and has been introduced into the field of package liveness detection in recent years.

[0003] However, existing detection methods based on millimeter-wave radar still have significant shortcomings:

[0004] Firstly, many methods use single-parameter detection or direct threshold judgment on the distance-Doppler graph. However, the strong static clutter generated by the outer shell can easily mask the weak echoes of the living organism. Furthermore, the false signals generated by the slight deformation of the packaging material are highly similar to the micro-movements of the organism in the time and frequency domain, making it difficult for traditional algorithms to distinguish them and easily leading to false alarms.

[0005] Secondly, some schemes construct radar point clouds for modeling, but simply splice together heterogeneous physical quantities such as spatial position, velocity and echo intensity and process them uniformly, ignoring the essential differences of each physical quantity in radar imaging mechanism. This results in a high degree of overlap between geometric structure, micro-motion characteristics and scattering characteristics in the feature space, making it difficult to form a stable criterion.

[0006] Third, most existing models are purely data-driven and lack physical prior constraints, which severely limits their generalization ability under complex conditions such as changes in the orientation of the package and mutual occlusion.

[0007] Therefore, there is an urgent need to develop a high-precision millimeter-wave radar liveness detection method that can deeply decouple and fuse multimodal heterogeneous features to improve the robustness and accuracy of liveness detection in complex environments. Summary of the Invention

[0008] In view of the above problems, this invention is proposed to provide a neural network-based millimeter-wave radar liveness detection method that overcomes or at least partially solves the above problems. By constructing a spatial geometric perception network, a temporal dynamics network, and a radial scattering structure perception network, the heterogeneous physical properties of radar point clouds are extracted in parallel and decoupled. A cross-modal dynamic interactive fusion mechanism is introduced to effectively suppress strong clutter interference and solve the problem of heterogeneous feature aliasing in complex enclosed environments, thereby achieving highly reliable non-contact identification of live targets inside non-transparent enclosed environments.

[0009] To achieve the above objectives, the present invention adopts the following technical solution: This invention provides a method for liveness detection using millimeter-wave radar based on neural networks, comprising the following steps: S1. Acquire the raw ADC data of the package based on millimeter-wave radar, perform signal preprocessing, and generate a three-dimensional radar point cloud containing the target's spatial position, radial velocity, and echo intensity; the package includes logistics security inspection objects that conceal live targets or interference objects. S2. Perform spatial geometric feature enhancement and dynamic pattern feature enhancement on the three-dimensional radar point cloud to obtain enhanced features; S3. The three-dimensional radar point cloud and enhanced features are input in parallel into a multimodal heterogeneous feature decoupling network to decouple and extract the deep physical attribute features of the target respectively; the multimodal heterogeneous feature decoupling network includes: a spatial geometry perception network, a temporal dynamics network and a radial scattering structure perception network; S4. Perform cross-modal interaction and adaptive weighted fusion on the deep physical attribute features to obtain a fused feature vector; S5. Input the fused feature vector into the classifier and output the judgment result of whether the target inside the package is a living body.

[0010] Further, step S1 specifically includes: S11. Obtain the raw ADC data of the package based on millimeter-wave radar; S12. Perform a DC bias removal operation on the original ADC data; S13. Perform differential or high-pass filtering on the continuous pulse sequence after DC removal along the slow time dimension to filter out stationary noise and retain the micro-motion signal; S14. Perform windowing processing on the filtered signal in both fast and slow time dimensions; S15. Perform range-direction and Doppler-direction fast Fourier transforms on the windowed signal sequentially to generate a range-Doppler image; S16. In the range-Doppler image, candidate targets are detected using two-dimensional cell average constant false alarm rate detection, and angle estimation is performed by combining a multi-signal classification algorithm to generate a three-dimensional radar point cloud. ; The three-dimensional radar point cloud The physical quantity is characterized as follows:

[0011] in, Indicates the first p The three-dimensional spatial location of each candidate target Indicates the first p The radial velocity of each candidate target, Indicates the first pThe slant range between the candidate target and the millimeter-wave radar Indicates the normalized number of... p The echo intensity of each candidate target.

[0012] Further, step S2 specifically includes: Using spatial geometric feature enhancement operators, the local point density, normal vector, and curvature of the three-dimensional spatial position tensor of the three-dimensional radar point cloud are calculated to generate geometric and physical property enhancement features; By utilizing the dynamic mode enhancement operator, the higher-order velocity derivative information of the radial velocity tensor of the three-dimensional radar point cloud is extracted to generate dynamic physical property enhancement features.

[0013] Furthermore, in step S3, the target extraction process of the spatial geometry perception network is as follows: Based on the three-dimensional spatial location tensor, an initial local connectivity graph is constructed using the K-nearest neighbor algorithm, and a nearest neighbor index matrix is ​​generated. Based on the normal vectors of each point in the enhanced geometric physical properties, the cosine similarity between the normal vectors of the center point and each neighboring point is calculated, and the neighboring connections with cosine similarity below the preset normal vector angle threshold are truncated to construct a graph structure that conforms to the normal vector consistency constraint. On the graph structure, the approximate geodesic distance between each connected node is calculated using the shortest path iterative algorithm, and a geodesic distance matrix is ​​generated. Based on the geodesic distance matrix, multiple different scale radius thresholds are set, and corresponding neighborhood nodes are selected at each scale for convolution operations and feature aggregation to obtain local geometric features at multiple scales. The local geometric features at multiple scales are concatenated with the normal vectors and curvatures in the geometric physical property enhancement features to form a comprehensive descriptive tensor; The attention mechanism is applied to the comprehensive description tensor to perform weighted compression of node features, and then projected to a preset dimension through a linear mapping layer to obtain a feature vector representing the geometric shape of the target space.

[0014] Furthermore, in step S3, the temporal dynamics network includes: an input unit, a micro-motion periodic frequency domain analysis unit, a velocity field orthogonal decomposition unit, and a motion coherence graph construction and temporal extraction unit; The input unit is used to receive the radial velocity tensor and the enhanced dynamic physical properties as a velocity field input; The micro-motion periodic frequency domain analysis unit is used to perform Fourier transform on the input velocity field, extract the dominant frequency of micro-motion, and obtain the respiratory cycle and muscle tremor frequency. The velocity field orthogonal decomposition unit is used to decompose the velocity field into normal components and tangential components. The motion coherence graph construction and temporal extraction unit is used to construct dynamic topological connections based on velocity consistency, capture the evolution law of the normal and tangential components in the time dimension using a recurrent neural network, extract temporal dynamic features including velocity field smoothness and transient acceleration modes, and output a temporal dynamic feature vector.

[0015] Furthermore, in step S3, the radial scattering structure perception network includes: a depth layering and occlusion modeling unit, a scattering characteristic and material extraction unit, and a radial-scattering cross-modal fusion unit; The depth layering and occlusion modeling unit is used to receive the slant range tensor of the three-dimensional radar point cloud, map it to a discrete depth layer, and generate a depth mask matrix; it is also used to construct an occlusion relationship graph based on the radar line of sight, calculate and generate an occlusion mask matrix; and output depth layering features that characterize the simulated depth occlusion relationship. The scattering characteristics and material extraction unit is used to receive the echo intensity tensor of the three-dimensional radar point cloud and perform distance compensation; and to calculate the local mean and variance of the compensated intensity tensor on the K-nearest neighbor graph to generate scattering statistical features characterizing the surface roughness and material uniformity of the target. The radial-scattering cross-modal fusion unit is used to fuse the depth layering features and scattering statistical features, combine the gradient direction information of the echo intensity, perform target location through a multi-head cross-modal attention mechanism, and output a comprehensive feature vector characterizing the target depth skeleton and scattering characteristics.

[0016] Furthermore, step S4 specifically includes: The comprehensive feature vector output by the radial scattering structure perception network is used as the query vector, the temporal dynamics feature vector output by the temporal dynamics network is used as the value vector, and the feature vector representing the spatial geometry of the target is used as the key vector; multi-head cross-attention interaction is performed. The contribution weights of the three sub-network features are dynamically generated based on the current environment, and then weighted and fused using an adaptive gating mechanism to obtain a fused feature vector.

[0017] Furthermore, in step S5, the classifier is a three-layer deep multilayer perceptron classification operator, including an input layer, a hidden layer, and an output layer; Each layer is configured with a linear mapping operator and cascaded with the LeakyReLU activation function and a batch normalization layer; The output layer executes the Log-Softmax operator to generate the class probability distribution; The classifier is trained using a joint loss function, which includes a label smoothing focus loss term, an inter-class separation loss term, and a physical constraint penalty term.

[0018] As can be seen from the above technical solution, compared with the prior art, the present invention discloses a millimeter-wave radar liveness detection method based on neural networks, which has the following beneficial effects: First, this invention constructs a spatial geometry perception network, a temporal dynamics network, and a radial scattering structure perception network to extract heterogeneous physical quantities such as spatial position, radial velocity, and echo intensity in radar point clouds in parallel and decoupled manner. This solves the problem of feature aliasing caused by simply splicing heterogeneous features in existing methods, enabling the network to focus on the target's geometry, micro-motion period, and material scattering characteristics respectively, significantly improving the discriminativeness and stability of liveness features.

[0019] Secondly, this invention introduces a physical heuristic enhancement operator in the signal preprocessing stage to extract physical properties such as local density, normal vector, curvature, velocity gradient and inter-frame acceleration, which effectively suppresses strong static clutter generated by the outer shell and reduces environmental artifact interference caused by subtle deformation of the packaging material, thereby improving the signal-to-noise ratio and physical interpretability of the input data from the source.

[0020] Third, the present invention introduces a cross-modal dynamic interaction and adaptive gating mechanism in the feature fusion stage, which enables this embodiment to automatically adjust the contribution weight of each physical mode according to the current detection environment. In the training stage, a physical constraint penalty term is introduced to ensure that the model learning results conform to the physical laws of radar, which significantly enhances the robustness and generalization ability under complex conditions such as wrapped attitude changes, mutual occlusion and low signal-to-noise ratio.

[0021] This invention achieves highly reliable detection of live targets inside packages without the need for ionizing radiation or contact with the package. It has significant advantages such as high safety, strong engineering adaptability, and good robustness, and is suitable for practical applications in modern logistics security inspection and related fields. Attached Figure Description

[0022] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on the provided drawings without creative effort.

[0023] Figure 1 This is a flowchart of a millimeter-wave radar liveness detection method based on a neural network provided in an embodiment of the present invention; Figure 2 This is a schematic diagram of a multimodal heterogeneous feature decoupling network provided in an embodiment of the present invention. Detailed Implementation

[0024] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0025] This invention discloses a method for liveness detection using millimeter-wave radar based on neural networks, referring to... Figure 1 As shown, it includes the following steps: S1. Acquire raw ADC data of the package based on millimeter-wave radar, perform signal preprocessing, and generate a three-dimensional radar point cloud containing the target's spatial position, radial velocity, and echo intensity; the package includes logistics security inspection objects that conceal live targets or interference objects. S2. Enhance the spatial geometric features and dynamic pattern features of the three-dimensional radar point cloud to obtain enhanced features; S3. Input the 3D radar point cloud and enhanced features in parallel into the multimodal heterogeneous feature decoupling network to decouple and extract the deep physical attribute features of the target respectively; the multimodal heterogeneous feature decoupling network includes: spatial geometry perception network, temporal dynamics network and radial scattering structure perception network; S4. Perform cross-modal interaction and adaptive weighted fusion on deep physical property features to obtain a fused feature vector; S5. Input the fused feature vector into the classifier and output the result of whether the target inside the package is a living body.

[0026] This embodiment is applied to parcel security inspection. It collects data using millimeter-wave radar and generates a 3D radar point cloud. A multimodal heterogeneous feature decoupling network is constructed to extract the heterogeneous physical properties of the radar point cloud in parallel. Finally, a cross-modal dynamic interactive fusion mechanism is introduced to effectively suppress strong clutter interference and solve the problem of heterogeneous feature aliasing in complex parcel environments, achieving highly reliable non-contact identification of live targets inside non-transparent parcels. This embodiment simulates a logistics security inspection scenario in a laboratory environment to verify the effectiveness of the proposed "multimodal heterogeneous feature decoupling network" in distinguishing between live targets (taking live insects as an example) and environmental artifacts (false positives). In this embodiment, positive samples (live organisms) are small live creatures, such as mealworms, ants, and cockroaches, placed inside a standard express delivery box to simulate the illegal transportation of protected insects. Negative samples are foreign objects / artifacts / empty samples, including: an empty cardboard box (containing only static background clutter), a mechanical vibration motor placed inside the box (simulating periodic vibration interference from non-biological sources), and a metal zippered bag placed inside the box (simulating highly reflective non-living foreign objects). This embodiment simulates a customs security inspection environment, collecting negative and positive packages during real logistics transportation to provide sufficient data support for the subsequent network construction.

[0027] This embodiment first establishes a data acquisition environment. The acquisition device utilizes a collaborative architecture between a millimeter-wave radar RF front-end and a dedicated high-speed acquisition card. The radar processing device in this embodiment includes the millimeter-wave radar chip AWR2243 and the data acquisition card DCA1000 physically connected to it. The AWR2243 operates in the 77GHz–81GHz millimeter-wave band and employs a wideband linear frequency modulated continuous wave (FMCW) system for signal transmission and reception. By configuring a large frequency modulation bandwidth, this radar embodiment achieves high range resolution, thereby effectively distinguishing the echo response of the outer shell from that of small internal targets under obstructed conditions.

[0028] During the data acquisition phase, the AWR2243 transmits the intermediate frequency signal output from the multi-channel receiver link directly to the DCA1000 data acquisition card via the LVDS interface in a high-speed serial manner. The DCA1000 synchronously samples the signals from each receiver channel and stores them in the raw ADC data format without on-chip compression, denoising, or feature extraction processing.

[0029] This embodiment follows step S1 to acquire the raw ADC data and perform signal preprocessing. Specifically, it includes: DC cancellation: The time mean of the acquired raw ADC data is subtracted point by point along the fast time dimension to perform DC bias cancellation operation and output the DC baseline signal to suppress zero-frequency component interference.

[0030] Static clutter suppression: Along the slow time dimension, multiple consecutive pulse signals corresponding to the same distance cell in the DC baseline signal are arranged to form a slow time sequence signal; inter-frame difference operation or high-pass filtering is performed on the slow time sequence signal to filter out stationary components with frequencies close to zero and output dynamic residual signals, thereby eliminating static clutter generated by the outer shell and preserving the micro-motion characteristics of the living body.

[0031] Windowing and weighting: Apply window functions to the dynamic residual signal for weighted smoothing in both the fast and slow time dimensions, and output a windowed modulated signal.

[0032] After completing the signal preprocessing, this embodiment maps the signal step by step from the waveform domain to a point cloud representation with a clear physical meaning.

[0033] First, a Range-Fourier Transform (RangeFFT) is performed on the windowed signal in a fast time dimension to extract the radial distance information of the target. Specifically, a Range-FFT is performed on the N ADC sampling points acquired by each receiving antenna. If the spectral energy peak is located at the k-th frequency point (RangeBin), the corresponding beat frequency is calculated as follows:

[0034] in, Where N is the sampling frequency of the data acquisition card, and N is the number of Fourier transform points.

[0035] Since the frequency of a linear frequency modulated continuous wave (FMCW) increases linearly with time, its slope... Echo lag time The resulting frequency difference is ,according to Derive the slant distance :

[0036] in, Indicates the frequency modulation bandwidth. This represents the physical duration of a single linear frequency modulated pulse (Chirp) emitted by the radar. C Represents the speed of light (3 × 10⁻⁶) 8 m / s).

[0037] Subsequently, a Doppler FFT is performed in the slow time dimension to obtain the radial velocity information of the target. Specifically, in this embodiment, the target's radial velocity information is obtained across the same range cell (RangeBin). Perform Doppler FFT on the first consecutive pulse. If the energy peak is located at the first pulse... If there are several Doppler frequencies, then the corresponding Doppler frequencies are... for:

[0038] Phase change caused by radial displacement Directly related to the Doppler frequency, it is mapped to radial velocity through the Doppler effect formula. :

[0039] in, For millimeter wave wavelength, The center frequency.

[0040] The range-Doppler map (RDM) is constructed by the above two-dimensional frequency domain transformation to characterize the energy distribution of the target inside the package at different range and velocity units.

[0041] In RDM, this embodiment introduces two-dimensional cell average constant false alarm rate (2D-CA-CFAR) detection. This operator sets up training and protection cells for each cell to be detected and adaptively calculates the detection threshold based on the statistical characteristics of the package background noise. When the energy of the cell to be detected exceeds the threshold, it is determined to be a candidate target, thereby achieving reliable detection of weak dynamic targets in complex backgrounds.

[0042] For target elements detected by CFAR, this embodiment further utilizes the phase difference between multiple receiving antenna channels and employs the MUSIC algorithm for high-resolution angle estimation. This algorithm decomposes the subspace, distinguishing between the signal subspace and the noise subspace, thereby obtaining the target's precise azimuth information.

[0043] Echo intensity (I) extraction: This is not a simple raw amplitude but a physically corrected normalized index. First, the power value of the target cell detected by CFAR on the range-Doppler diagram (RDM) is calculated. :

[0044] in, The target cell is indexed in the distance dimension. For Doppler index, for It is the complex tensor value after a two-dimensional FFT.

[0045] Because the received power in the radar equation is inversely proportional to the fourth power of the range ( ), P r This represents the received power, which is the physical power of the target echo signal actually captured by the radar receiving antenna. Indicates the radar cross-section; R This indicates the target slant range, which is the straight-line physical distance between the target and the radar antenna; This indicates a positive correlation. To ensure that similar objects exhibit consistent intensity characteristics at different distances, compensation is required.

[0046] I compensated This indicates the echo intensity after compensation.

[0047] The compensated value is then logarithmically compressed or linearly normalized to obtain the final intensity I.

[0048] Finally, this embodiment combines the target's distance, Doppler velocity, and angle estimation results for joint mapping to generate a three-dimensional radar point cloud in a Cartesian coordinate system.

[0049] In this embodiment, each radar point in the three-dimensional radar point cloud is characterized by the following physical quantities:

[0050] in, Indicates the first p The three-dimensional spatial location of each candidate target This represents the radial velocity of the p-th candidate target. This represents the slant range between the p-th candidate target and the millimeter-wave radar. This represents the echo intensity of the p-th candidate target after normalization.

[0051] In this embodiment, each radar point in the 3D radar point cloud is considered as a candidate target for primary detection. First, the number of candidate targets within the current detection period is counted. If the target is within a preset detection range, the number of effective reflection points, i.e., the number of detected point clouds, is determined. ,in If the number of points is set to a preset threshold, the current package is determined to be a negative package, meaning it does not contain targets with significant scattering characteristics. If the number of detected point clouds... However, due to low signal-to-noise ratio or scattering interference from packaging materials, if environmental disturbances (such as slight shaking of the package, metal zippers, etc.) cannot be eliminated based solely on the original parameters (such as mean RCS or average velocity) - false positives will occur. In such cases, the current point cloud set and related physical quantities will be automatically defined as a suspected target sequence, and a secondary judgment will be triggered.

[0052] Once the secondary judgment is triggered, the suspected point cloud sequence enters the multimodal heterogeneous feature decoupling network.

[0053] Before entering the multimodal heterogeneous feature decoupling network, according to step S2, the three-dimensional radar point cloud is enhanced with spatial geometric features and dynamic mode features respectively to obtain enhanced features.

[0054] Spatial Dimension Enhancement: By using spatial geometric feature enhancement operators, normal vectors, curvature, and local density are extracted to describe the geometric surface details of the target.

[0055] 3D Spatial Position Tensor Based on 3D Radar Point Cloud The surface morphology features of the target are extracted by spatial manifold analysis operators.

[0056] For local point cloud density estimation, a radius search operator is used to determine the K-neighborhood set of each point, and tensor counting operations are performed to calculate the number of points per unit volume to describe the local point cloud density. This feature can quantify the compactness of targets within a package and identify the local aggregation characteristics unique to biological tissues.

[0057] For normal vector and curvature estimation, this embodiment performs principal component analysis (PCA) on the neighborhood point set. First, the local covariance matrix is ​​calculated, and then the eigenvector corresponding to its smallest eigenvalue is extracted as the normal vector through eigenvalue decomposition. Subsequently, using eigenvalues (Ascending order) Perform arithmetic operations This allows for the acquisition of surface curvature. The sequence of operators can finely describe the geometric edges and curvature abrupt changes of the target, providing a basis for distinguishing between flat enclosing walls and irregular living surfaces.

[0058] Enhanced Dynamics: Enhancing operators through dynamic modes to compute temporal acceleration. and velocity gradient It captures nonlinear micro-motion patterns.

[0059] Based on the radial velocity tensor of a 3D radar point cloud, a higher-order derivative extraction operator is performed to capture nonlinear motion.

[0060] For velocity gradient Spatial extraction is performed, the velocity difference between center point i and its neighbor j is calculated using the difference operator, and Euclidean distance normalization is then applied. .in, and Representing the midpoints of space respectively i and points j velocity scalar; molecule The calculation is based on the absolute velocity difference between these two points; and Representing points respectively i and points j The three-dimensional spatial coordinates; Point i and points j The three-dimensional Euclidean distance between them. This operator is used to characterize the consistency of motion and shear force characteristics within the target.

[0061] For time-series acceleration a In this embodiment, a cross-frame caching mechanism is used to perform speed difference operations on points with the same index (or neighboring matching points) in two consecutive time steps. This feature can keenly capture the unique transient burst motion patterns of living targets (such as insect jumping or biological tremors) and effectively filter out background clutter that drifts at a constant speed.

[0062] In this embodiment, through the synergistic effect of the aforementioned physical heuristic operators, the preprocessing layer transforms the original, environmentally-affected low-level data into enhanced feature vectors with strong physical laws.

[0063] Then, following step S3, the deep physical attribute features of the target are decoupled and extracted respectively.

[0064] The multimodal heterogeneous feature decoupling network in this embodiment refers to... Figure 2 As shown, it includes: a spatial geometry sensing network, a temporal dynamics network, and a radial scattering structure sensing network.

[0065] The spatial geometric perception network achieves stable representation of the nonlinear geometric structure of living targets by introducing geodesic topology modeling, multi-scale geometric feature extraction, and shape-aware pooling mechanism.

[0066] The input to the spatial geometry perception network consists of a three-dimensional spatial position tensor X and geometric physical property enhancement features F. aug composition:

[0067]

[0068] X represents the three-dimensional spatial position of n points in the three-dimensional radar point cloud in each sample, in the Cartesian coordinate system; F aug Output from the physical preprocessing layer, including local point density and normal vector (n vec ), curvature (k) cur These geometric and physical properties.

[0069] For each sample's n point cloud, the K-nearest neighbor algorithm is first executed in three-dimensional Euclidean space to generate a nearest neighbor index matrix. (Take K=20) Construct an initial set of candidate local connections to provide a basis for subsequent manifold constraint screening.

[0070] Subsequently, this embodiment introduces a normal vector consistency constraint during the topology construction process. Specifically, for each center point i and its neighboring points j, the cosine similarity of the corresponding normal vectors is calculated to obtain the corresponding similarity matrix S:

[0071] Set the threshold for the angle between normal vectors The logic layer compares the similarity matrix S with the scalar threshold by calling the comparison operator in the tensor library. Perform element-by-element comparison, when The connection is forcibly truncated to complete the filtering and construct the graph structure.

[0072] After the graph structure is established, the dynamic distance assignment phase begins. A weight tensor is initialized isomorphic to the adjacency matrix and assigned the Euclidean distance. Then, an iterative operator based on the shortest path per moment is invoked to perform feature diffusion within each locally connected component. Within each iteration cycle, the tensor computation unit calculates according to this formula: The distance matrix is ​​updated according to the rules until the neighborhood weight benchmark of the path weight law of all valid connection points on the manifold surface is obtained, where This indicates the current iteration cycle number. and These represent the center point index and the target point index in the sample, respectively. This indicates the center point after filtering by normal vector consistency constraints. The effective local neighborhood node set of K. n for and The intermediate wandering nodes on the manifold surface. Indicates the period in the previous iteration. In the calculation, the slave node is obtained. To the node The cumulative shortest path weight. Indicates intermediate nodes With the target node The initial physical edge weights between the two points (i.e., the three-dimensional Euclidean distance between the two points).

[0073] After completing the geodesic topology modeling, multi-branch parallel value convergence is employed to ultimately generate an approximate geodesic distance matrix that accurately describes the physical geometric relationship of the target. This provides a physically conforming manifold structure for subsequent multi-scale spatial convolution, extracting target geometric features from different spatial scales. Specifically, a radius threshold R1=0.1 is set, and for each center point, only features that satisfy the following criteria are retained. The neighborhood nodes are convolved (conv) with weighted aggregation of the neighborhood features to output local shape features f1. This is used to capture the microscopic deformations and detailed structures of the living surface. Convolution operations are then performed with the same parameters, but with a larger radius threshold R2=0.3 and no distance constraint radius threshold R3, to obtain regional and global features f2 and f3 of the same dimension as f1. These multi-scale features are then concatenated with physical geometric properties to form a comprehensive descriptive tensor. .

[0074] After obtaining the comprehensive description tensor T, attention weights are used to perform weighted compression on the node features. The fused features are then normalized using the Softmax function and finally projected onto a 256-dimensional space using a linear mapping layer. This feature, as the final output of the spatial geometry perception network, characterizes the spatial geometry of the target.

[0075] The temporal dynamics network aims to decouple the micro-motion characteristics of a living target from radial velocity and its higher-order derivatives. This module receives input tensors containing multi-frame velocity tensors. Where T represents the number of consecutive radar frames, n f This indicates the number of sampling points in a single frame of point cloud.

[0076] The velocity tensor is enhanced by performing feature enhancement and higher-order derivative extraction operators to obtain the enhanced feature tensor. .

[0077] Enhanced features include velocity gradients and inter-frame acceleration a It is used to characterize the nonlinear and non-stationary motion characteristics of living targets.

[0078] In the implementation of the micro-motion periodic frequency domain analysis unit, a one-dimensional discrete Fourier transform operator (1D-DFT) is first performed on the time axis of the T-frame continuous point cloud to convert the time-domain velocity tensor into a complex-domain frequency tensor. , where n is the number of point clouds in the sample. Subsequently, the power spectral density is calculated using the modulus operator, and a Top-K masking operation is performed to extract the top 5 dominant frequency components. These components are then linearly mapped through a one-dimensional convolution operator (Conv1d, kernel size 1, 5 input channels, 64 output channels) to convert micro-motion frequencies such as breathing, heartbeat, or limb tremors into high-dimensional frequency domain representation vectors.

[0079] The module then moves on to the velocity field manifold learning and decomposition, specifically performing a spatial normal vector-based process on the smoothed velocity field. The orthogonal decomposition operator. The specific implementation logic is as follows: calculate the normal projection using the inner product operator. The tangential component is obtained through the vector subtraction operator. These two decoupled velocity components, along with the velocity gradient tensor, The signal is fed into a dual-branch feature extraction operator. This operator consists of two sets of parallel linear transformation layers, each containing two perceptron (MLP) layers with a LeakyReLU activation function and a batch normalization layer embedded in between. This maps the decoupled velocity components to a 128-dimensional latent space to distinguish between the target surface motion and the radial motion toward the radar.

[0080] In the motion coherence graph construction and temporal extraction module, this embodiment first constructs a dynamic topological connection based on the velocity consistency principle. For each frame of point cloud, a correlation mask matrix is ​​constructed by calculating the cosine similarity of the velocity vectors between nodes. Motion coherence edges are established only between node pairs with a similarity greater than 0.85. Based on this topology, this embodiment calls the gated cyclic unit operator (ResidualGRU) with residual structure. This operator receives the node features aggregated from the coherence graph at each time step t. The system utilizes reset and update gates to capture nonlinear instantaneous acceleration patterns. To further enhance the perception of complex motion patterns, the output sequence of the GRU is fed into a temporal-aware pooling layer. This layer performs self-attention weighted summation across time steps, ultimately outputting a 384-dimensional temporal dynamics feature vector. This vector can accurately pinpoint the instantaneous bursts or periodic vital signs of a living target within the package along the time axis, providing a core criterion for subsequent liveness detection.

[0081] The radial scattering structure perception network aims to extract the depth distribution patterns, material properties, and transient attitude features of targets inside a package by decoupling the physical dependence of distance (r) and intensity (I).

[0082] Specifically, in the depth layering and occlusion modeling unit, this embodiment first processes the input radial distance tensor... Mapping to a discretized depth coding space. By performing a linear discretization operator, continuous distance values ​​are divided into L concentric spherical depth layers (L=8), and corresponding depth mask matrices are generated. Based on this, this embodiment constructs an occlusion graph, the core operator of which is the "line-of-sight projection mask": for any point, this embodiment calculates its radial projection angle in the radar line-of-sight direction; if the angle is less than a preset threshold and Then, through Boolean assignment logic, node c is marked as the occlusion source of node d in the mask matrix. This mask is then applied to the neighborhood feature aggregation operator, enabling the network to infer the complete depth structure of the target by utilizing the spatiotemporal continuity between successive layers.

[0083] In the scattering characteristics and material extraction unit, this embodiment introduces an intensity compensation operator based on the radar equation. This operator is applied to the input echo intensity tensor. This embodiment performs the operation through a multiplication operator. Range gain compensation is applied to eliminate differences in received energy caused by varying target placement locations. The compensated intensity tensor... The data is fed into a spatial distribution statistical operator, which slides across the K-nearest neighbor graph to perform local mean and variance calculations, generating a statistical feature tensor reflecting the surface roughness and material uniformity of the target. For rigid foreign bodies with high RCS, such as metals, this tensor will exhibit high local uniformity; while for living biological tissues, it will exhibit significant intensity scintillation and non-uniform scattering characteristics due to water distribution and surface undulations.

[0084] In the radial-scattering cross-modal fusion unit, this embodiment employs a bilinear interactive convolution operator (BilinearInteractionConv). This operator performs an element-wise Hadamard product operation on the depth layering features and material scattering features, and combines it with an intensity gradient direction operator (for calculating...). The transient pose of the target is indicated by the projection in three-dimensional space. To further extract higher-order semantics, this embodiment executes a multi-head cross-modal attention operator (Cross-ModalAttention), using intensity features as a query vector to index key depth positions in distance features, thereby accurately locating the scattering center of a live target in complex encapsulation environments (such as multi-layer packaging and object occlusion), and outputting a comprehensive feature vector representing the target's depth skeleton and scattering characteristics.

[0085] Specifically, all branch features are aggregated through a feature fusion mapping layer (containing a fully connected layer and LayerNorm normalization), and global max pooling based on mask weights is performed to output a comprehensive feature vector with a dimension of 512. That is, the depth-layered features representing the simulated depth occlusion relationship output by the depth layering and occlusion modeling unit, and the scattering statistical features representing the surface roughness and material uniformity of the target generated by the scattering characteristics and material extraction unit are finally aggregated, normalized, and dimension-mapped to output a comprehensive feature vector of fixed dimensions. This vector can simultaneously represent the target's "depth skeleton" and "scattering skin," providing a highly discriminative physical prior for the subsequent classifier, significantly improving the liveness detection capability in non-transparent wrapped scenes.

[0086] Subsequently, following step S4, cross-modal interaction and adaptive weighted fusion are performed to obtain the fused feature vector.

[0087] In this embodiment, a multi-head cross-attention interaction operator is first executed. This operator will... As a query vector, respectively with and The key vector and value vector are subjected to tensor dot product. During the calculation, an interaction weight matrix is ​​generated by scaling the dot product operator and then normalized using Softmax. Specifically, the comprehensive feature vector output by the radial scattering structure-aware network is used as the query vector, the temporal dynamics feature vector output by the temporal dynamics network is used as the value vector, and the feature vector representing the target spatial geometry output by the spatial geometry-aware network is used as the key vector.

[0088] Subsequently, the system introduces a modal weighting operator based on an adaptive gating mechanism. This operator utilizes three sets of parallel perceptron units to nonlinearly compress the features of the three branches, generating a scalar of modal contribution scores. These scores are generated through a fully connected mapping layer and a sigmoid activation operator, representing the information gain of each physical mode under the current detection environment (e.g., strong occlusion or high dynamics). The final fused feature vector... Obtained by executing the weighted summation operator: ,in This refers to the tensor splicing and dimensionality reduction mapping operations after dimension alignment.

[0089] Finally, following step S5, the fused feature vector is fed into a three-layer deep multilayer perceptron classification operator.

[0090] The operator consists of an input layer (1024 dimensions), hidden layers (512 and 256 dimensions), and an output layer (class probability dimension). Each layer is configured with a linear mapping operator with a kernel size of 1, and is strictly cascaded with a LeakyReLU activation function (with a negative slope set to 0.1) and a batch normalization layer to ensure the stability of gradient propagation and mitigate the problem of neuron death. The output layer performs a Log-Softmax operator to generate the class probability distribution of the target within the packet.

[0091] To guide the model in learning discriminative features with physical consistency, this embodiment executes a multi-task joint optimization loss operator (Joint Optimization Loss) during the training phase. The total loss function... It consists of three weighted parts: first, the focus loss is applied to perform label-smoothed focal loss, and then adjusted by a factor... The algorithm employs several methods: First, it increases the penalty weight for weak signal samples with low signal-to-noise ratios. Second, it uses inter-class separation loss, which calculates the cosine similarity between the feature vectors of positive and negative samples within a batch and performs a minimization operator to force a greater Euclidean distance between "living objects" and "static foreign objects" in the feature space. Finally, it includes a physical constraint penalty term to ensure that the model's learning results conform to physical laws such as velocity field smoothness and energy conservation. Through this multi-loss collaborative optimization mechanism, this embodiment ultimately outputs accurate liveness detection results, effectively solving the problem of non-contact detection in non-transparent package security inspection scenarios.

[0092] For the positive samples in this embodiment, the dynamics mode enhancement operator in the feature enhancement module captured the weak acceleration signal generated by the insect's body tremors; in the temporal dynamics network, the frequency domain analysis unit extracted the dominant respiratory / tremor frequencies of 8-12Hz. However, when using the traditional PointNet++ method, the point cloud features are submerged by background noise when concatenated with coordinates and velocity, making it difficult for the network to extract effective micro-motion features, resulting in a high false negative rate.

[0093] For the positive and negative samples in this embodiment, the radial scattering structure perception network, which specifically processes echo intensity and depth distribution, identifies the aluminum foil packaging as a "rigid high-scattering object." Simultaneously, the spatial geometry perception network, during topology construction, uses a normal vector mask to filter out the static features of rigid foreign objects, preserving the irregular geometric manifold of living organisms. The cross-modal fusion layer ultimately determines: "A foreign object exists, but a living organism also exists." However, using the traditional PointNet++ method, the aluminum foil packaging generates a strong radar cross section, and its strong echo masks the weak point cloud of the insect. Because traditional networks treat spatial coordinates and intensity "equally," the model is often misled by the high-energy features of foreign objects, leading to misjudgments as "non-living" or failure to identify them.

[0094] For the negative sample in this embodiment, although the vibration motor triggered the initial point cloud detection level (first-level judgment), in the second-level judgment, the temporal dynamics network analysis of the velocity field found that the motion lacked the biologically specific "nonlinearity" and "transient burstiness" (such as discontinuous acceleration); at the same time, the radial scattering structure perception network analysis of material properties found that its RCS was stable and the surface was smooth, which does not conform to the "non-uniform scattering" characteristics of biological tissues. However, using the traditional PointNet++ method, the periodic vibrations generated by the vibration motor are extremely similar to biological respiration in the time-frequency domain (both are periodic signals). Because the traditional method lacks physical prior constraints, it is very easy to misclassify it as a "living organism."

[0095] This invention operates stably on existing millimeter-wave radar hardware and can handle various logistics security inspection scenarios, ranging from tiny insects to complex mixed packages. Compared to traditional point cloud processing networks, this invention successfully solves two major engineering challenges—feature aliasing and false alarms due to mechanical artifacts—through decoupling of spatial, temporal, and radial sub-networks. Even under extreme interference from foreign objects, it maintains extremely low false alarm and false negative rates, demonstrating its high reliability in practical logistics security inspections.

[0096] Based on the same inventive concept, this invention also provides a millimeter-wave radar liveness detection device based on neural networks. Since the principle of solving the problem by these devices and clients is similar to that of the aforementioned millimeter-wave radar liveness detection method based on neural networks, the implementation of the device and client can refer to the implementation of the aforementioned method, and the repeated parts will not be described again.

[0097] The various embodiments in this specification are described in a progressive manner, with each embodiment focusing on the differences from other embodiments. The same or similar parts between the various embodiments can be referred to each other.

[0098] The above description of the disclosed embodiments enables those skilled in the art to make or use the invention. Various modifications to these embodiments will be readily apparent to those skilled in the art, and the general principles defined herein may be implemented in other embodiments without departing from the spirit or scope of the invention. Therefore, the invention is not to be limited to the embodiments shown herein, but is to be accorded the widest scope consistent with the principles and novel features disclosed herein.

Claims

1. A method for liveness detection using millimeter-wave radar based on neural networks, characterized in that, Includes the following steps: S1. Acquire the raw ADC data of the package based on millimeter-wave radar, perform signal preprocessing, and generate a three-dimensional radar point cloud containing the target's spatial position, radial velocity, and echo intensity; the package includes logistics security inspection objects that conceal live targets or interference objects. S2. Perform spatial geometric feature enhancement and dynamic pattern feature enhancement on the three-dimensional radar point cloud to obtain enhanced features; S3. The three-dimensional radar point cloud and enhanced features are input in parallel into a multimodal heterogeneous feature decoupling network to decouple and extract the deep physical attribute features of the target respectively; the multimodal heterogeneous feature decoupling network includes: a spatial geometry perception network, a temporal dynamics network and a radial scattering structure perception network; The radial scattering structure perception network includes: a depth layering and occlusion modeling unit, a scattering characteristic and material extraction unit, and a radial-scattering cross-modal fusion unit; The depth layering and occlusion modeling unit is used to receive the slant range tensor of the three-dimensional radar point cloud, map it to a discrete depth layer, and generate a depth mask matrix; it is also used to construct an occlusion relationship graph based on the radar line of sight, calculate and generate an occlusion mask matrix; and output depth layering features that characterize the simulated depth occlusion relationship. The scattering characteristics and material extraction unit is used to receive the echo intensity tensor of the three-dimensional radar point cloud and perform distance compensation; and to calculate the local mean and variance of the compensated intensity tensor on the K-nearest neighbor graph to generate scattering statistical features characterizing the surface roughness and material uniformity of the target. The radial-scattering cross-modal fusion unit is used to fuse the depth layering features and scattering statistical features, combine the gradient direction information of the echo intensity, perform target location through a multi-head cross-modal attention mechanism, and output a comprehensive feature vector characterizing the target depth skeleton and scattering characteristics. S4. Perform cross-modal interaction and adaptive weighted fusion on the deep physical attribute features to obtain a fused feature vector; S5. Input the fused feature vector into the classifier and output the judgment result of whether the target inside the package is a living body.

2. The method as described in claim 1, characterized in that, Step S1 specifically includes: S11. Obtain the raw ADC data of the package based on millimeter-wave radar; S12. Perform a DC bias removal operation on the original ADC data; S13. Perform differential or high-pass filtering on the continuous pulse sequence after DC removal along the slow time dimension to filter out stationary noise and retain the micro-motion signal; S14. Perform windowing processing on the filtered signal in both fast and slow time dimensions; S15. Perform range-direction and Doppler-direction fast Fourier transforms on the windowed signal sequentially to generate a range-Doppler image; S16. In the range-Doppler image, candidate targets are detected using two-dimensional cell average constant false alarm rate detection, and angle estimation is performed by combining a multi-signal classification algorithm to generate a three-dimensional radar point cloud. ; The three-dimensional radar point cloud The physical quantity is characterized as follows: in, Indicates the first p The three-dimensional spatial location of each candidate target Indicates the first p The radial velocity of each candidate target, Indicates the first p The slant range between the candidate target and the millimeter-wave radar Indicates the normalized number of... p The echo intensity of each candidate target.

3. The method as described in claim 2, characterized in that, Step S2 specifically includes: Using spatial geometric feature enhancement operators, the local point density, normal vector, and curvature of the three-dimensional spatial position tensor of the three-dimensional radar point cloud are calculated to generate geometric and physical property enhancement features; By utilizing the dynamic mode enhancement operator, the higher-order velocity derivative information of the radial velocity tensor of the three-dimensional radar point cloud is extracted to generate dynamic physical property enhancement features.

4. The method as described in claim 3, characterized in that, In step S3, the target extraction process of the spatial geometry perception network is as follows: Based on the three-dimensional spatial location tensor, an initial local connectivity graph is constructed using the K-nearest neighbor algorithm, and a nearest neighbor index matrix is ​​generated. Based on the normal vectors of each point in the enhanced geometric physical properties, the cosine similarity between the normal vectors of the center point and each neighboring point is calculated, and the neighboring connections with cosine similarity below the preset normal vector angle threshold are truncated to construct a graph structure that conforms to the normal vector consistency constraint. On the graph structure, the approximate geodesic distance between each connected node is calculated using the shortest path iterative algorithm, and a geodesic distance matrix is ​​generated. Based on the geodesic distance matrix, multiple different scale radius thresholds are set, and corresponding neighborhood nodes are selected at each scale for convolution operations and feature aggregation to obtain local geometric features at multiple scales. The local geometric features at multiple scales are concatenated with the normal vectors and curvatures in the geometric physical property enhancement features to form a comprehensive descriptive tensor; The attention mechanism is applied to the comprehensive description tensor to perform weighted compression of node features, and then projected to a preset dimension through a linear mapping layer to obtain a feature vector representing the geometric shape of the target space.

5. The method as described in claim 4, characterized in that, In step S3, the temporal dynamics network includes: an input unit, a micro-motion periodicity frequency domain analysis unit, a velocity field orthogonal decomposition unit, and a motion coherence graph construction and temporal extraction unit; The input unit is used to receive the radial velocity tensor and the enhanced dynamic physical properties as a velocity field input; The micro-motion periodic frequency domain analysis unit is used to perform Fourier transform on the input velocity field, extract the dominant frequency of micro-motion, and obtain the respiratory cycle and muscle tremor frequency. The velocity field orthogonal decomposition unit is used to decompose the velocity field into normal components and tangential components. The motion coherence graph construction and temporal extraction unit is used to construct dynamic topological connections based on velocity consistency, capture the evolution law of the normal and tangential components in the time dimension using a recurrent neural network, extract temporal dynamic features including velocity field smoothness and transient acceleration modes, and output a temporal dynamic feature vector.

6. The method as described in claim 5, characterized in that, Step S4 specifically includes: The comprehensive feature vector output by the radial scattering structure perception network is used as the query vector, the temporal dynamics feature vector output by the temporal dynamics network is used as the value vector, and the feature vector representing the spatial geometry of the target is used as the key vector; multi-head cross-attention interaction is performed. The contribution weights of the three sub-network features are dynamically generated based on the current environment, and then weighted and fused using an adaptive gating mechanism to obtain a fused feature vector.

7. The method as described in claim 6, characterized in that, In step S5, the classifier is a three-layer deep multilayer perceptron classification operator, including an input layer, a hidden layer and an output layer; Each layer is configured with a linear mapping operator and cascaded with the LeakyReLU activation function and a batch normalization layer; The output layer executes the Log-Softmax operator to generate the class probability distribution; The classifier is trained using a joint loss function, which includes a label smoothing focus loss term, an inter-class separation loss term, and a physical constraint penalty term.

Images