A Method and System for Early Warning of Abnormal Behavior of Dormitory Personnel Based on Millimeter-Wave Radar

Abstract

This invention discloses a method and system for early warning of abnormal behavior of dormitory personnel based on millimeter-wave radar. It combines three-dimensional spatial semantic modeling to divide the area into beds, desktops, and floors, and introduces a time gating mechanism to filter normal sleep and accurately determine abnormal prolonged lying-down states. By extracting the trajectory curvature probability distribution entropy value and round-trip repetition degree within specific semantic regions, it effectively overcomes the limitations of existing technologies in objectively quantifying and accurately identifying latent behavioral patterns such as student anxiety in real-life scenarios. A multi-source evidence fusion framework is constructed, which dynamically weights radar motion intensity evidence and acoustic energy evidence by calculating the spatial consistency between the radar target and the audio source location. Furthermore, by utilizing synthesis rules that include conflict handling, it effectively suppresses false alarms, greatly improving the accuracy and reliability of fall detection and violent conflict identification in complex practical applications.

Description

Technical Field

[0001] This invention relates to the field of intelligent health monitoring technology, specifically to a method and system for early warning of abnormal behavior of dormitory personnel based on millimeter-wave radar. Background Technology

[0002] With the advancement of smart campus construction, the safety management and health care of student dormitories are receiving increasing attention. Traditional dormitory management relies on manual room checks, which is inefficient and has blind spots in supervision. Existing technologies such as video surveillance have fatal flaws in infringing on privacy, while infrared sensors suffer from limited functionality and a tendency to generate false alarms. In recent years, non-contact millimeter-wave radar has been introduced into this field. However, in the complex environment of dormitories, with their confined spaces and dense obstructions, traditional radar signal processing and multi-target tracking algorithms are prone to problems such as missing target features, loss of multi-target tracking, false associations, and frequent changes in target IDs, making it difficult to provide stable and high-precision spatiotemporal trajectories. Furthermore, single sensors have limitations in recognizing complex interactive behaviors. How to accurately distinguish between "playing around" and "violent conflict," "normal sleep" and "abnormal prolonged lying down," and how to achieve 24 / 7 security and health care while protecting privacy are current pain points facing the technology. Existing non-contact monitoring solutions mostly focus on surface-level action recognition and often lack quantitative analysis models for students' deep mental health. Furthermore, when judging complex emergencies, they are easily affected by environmental multipath or non-target micro-movements, resulting in a high false alarm rate. They also lack spatial alignment and effective mathematical fusion processing mechanisms between multi-source sensors.

[0003] Therefore, designing a dormitory resident abnormal behavior early warning method and system that can both strictly protect student privacy and effectively overcome the target tracking difficulties in complex and obstructed environments, thereby upgrading from routine behavior monitoring to in-depth physical and mental health care and precise emergency safety response, has become a priority. Summary of the Invention

[0004] The purpose of this invention is to provide a method and system for early warning of abnormal behavior of dormitory personnel based on millimeter-wave radar, which effectively solves the technical problems existing in the background art.

[0005] To achieve the above objectives, the present invention provides the following technical solution.

[0006] A method for early warning of abnormal behavior of dormitory personnel based on millimeter-wave radar includes the following steps: S1. Construct a multimodal sensing environment based on the dormitory environment, set the physical parameters of millimeter-wave radar and microphone array, and simultaneously collect raw millimeter-wave radar intermediate frequency data and environmental audio data. S2. Receive the collected data and perform preprocessing, including: performing signal processing on the raw millimeter-wave radar intermediate frequency data to generate sparse point clouds, constructing a spatiotemporal correlation model based on graph topology, using the message passing mechanism of graph neural networks to achieve stable tracking of multiple targets under dense occlusion, and outputting the target trajectory coordinates; at the same time, extracting environmental audio data and using phase-weighted cross-correlation analysis to calculate the target sound source azimuth angle, and achieving spatial alignment between the radar coordinate system and the acoustic coordinate system; S3. Based on the physical structure of the dormitory, a three-dimensional spatial semantic model is established, and the monitoring space is divided into the bed activity area, the desktop area and the ground activity area. The target trajectory coordinates are mapped to the three-dimensional spatial semantic model to generate the target's regional semantic state and height level features. S4. Based on the logical judgment of the target's spatiotemporal trajectory, regional semantic state, and sound source characteristics, health care analysis and safety event early warning analysis are executed simultaneously and in parallel to achieve continuous tracking of multiple targets within a specific area.

[0007] Preferably, the construction of the spatiotemporal correlation model based on graph topology in step S2 includes: using a uniform velocity model to predict the three-dimensional position and velocity of the historical trajectory in the previous frame in the current frame to obtain a set of predicted trajectory points; mapping the set of radar detection points in the current frame and the set of predicted trajectory points to a set of nodes in the graph; and based on the set spatial correlation threshold and velocity correlation threshold, mapping the node pairs that satisfy the spatiotemporal proximity condition to an edge set, thereby constructing a spatiotemporal correlation model.

[0008] Preferably, the specific steps of multi-target stable tracking in step S2 are as follows: S202. Extract the original feature vector containing three-dimensional position, radial velocity and signal-to-noise ratio based on the collected data, and map it to a high-dimensional feature space through linear embedding and nonlinear activation function to generate the initial high-dimensional feature representation of the node. S203. The unnormalized attention weights between nodes are calculated by feature concatenation and activation function with leakage factor. The attention coefficients are obtained after normalization. The representation vector of the central node is updated by aggregating the weighted features of the neighboring nodes. After passing through multiple layers of network, the final node features that fuse global spatiotemporal context information are output to complete the state representation of the target in the occlusion environment. S204. Construct edge feature vectors by combining the spatial distance difference, velocity difference, and final node features between the detection point and the predicted trajectory point; calculate the association probability of the edge features to generate an association matrix, and convert the association probability into matching cost; use the bipartite graph matching algorithm to solve for the optimal allocation scheme that minimizes the total global matching cost; then introduce a smoothing filtering mechanism to perform weighted fusion of the predicted trajectory state of the successfully matched point with the current detection point state, suppress millimeter-wave radar observation noise, update the historical trajectory state, and output the final smooth and stable trajectory set.

[0009] Preferably, in step S3, establishing a three-dimensional spatial semantic model includes: Define the three-dimensional coordinate system of the dormitory space as ,in For the width direction of the dormitory For the length direction, Define a height threshold in the vertical direction. That is, the upper limit of the activity area on the bed, This refers to the upper limit of the ground activity area, which, combined with the planar coordinate range, divides the region into three semantic regions: ; in , , The planar coordinate ranges for the bed, desktop, and floor are determined based on the actual dormitory layout within a specific area.

[0010] Preferably, in step S3, the content of mapping the target trajectory coordinates to the three-dimensional spatial semantic model includes: Define the height threshold and the range of planar coordinates, and mark the target trajectory coordinates output in step S2 as follows. The target trajectory coordinates are mapped to the established 3D spatial semantic model in real time to generate the target's regional semantic state. ,in High-level features ,in This is used to create a bed activity area. With ground activity area .

[0011] Preferably, in step S4, the health care analysis includes identifying anxiety behaviors and abnormal prolonged lying down, and triggering abnormal behavior warnings by quantifying the disorder of the trajectory and the dwell time in a specific semantic region; The identification of the anxiety behavior includes collecting data on the target's activity area on the ground. The trajectory sequence is used to calculate the probability distribution entropy of the trajectory curvature and the frequency of trajectory repetition. If both the entropy and the repetition frequency exceed the preset threshold, it is determined to be an anxious pacing state. The identification of abnormal prolonged bed rest includes: setting a preset normal sleep period as a statistical exemption interval; mapping the activity area to the bed. The system monitors targets and only accumulates statistics on dwell time outside the exemption range. If the accumulated dwell time exceeds the preset health threshold and the target micro-motion energy extracted based on the radar micro-Doppler spectrum remains below the preset threshold, it is determined to be an abnormal prolonged lying state.

[0012] Preferably, in step S4, the safety incident warning includes identifying fall events and violent conflicts, and triggering abnormal behavior warnings by analyzing the abrupt change characteristics of the target height level and the spatial consistency and intensity characteristics of the radar-sound source; The identification of the fall event includes determining it as a fall event if the target continuously meets the following conditions: rapid change in height within a preset time window, vertical speed exceeding a set threshold during descent, and micro-motion energy remaining below a static threshold for more than a preset confirmation time after the change. The violent conflict identification includes constructing a multi-source evidence fusion framework, extracting millimeter-wave radar motion intensity evidence and acoustic energy evidence, dynamically weighting the spatial consistency by calculating the deviation value between the radar target and the audio source location, and finally using synthesis rules that include conflict handling to make a fusion decision.

[0013] A dormitory occupant abnormal behavior early warning system based on millimeter-wave radar includes a memory, a processor, and a computer program stored in the memory and executable on the processor. When the processor executes the program, it implements the method described above.

[0014] Compared with the prior art, the present invention has the following beneficial effects: (1) This invention adopts a "non-semantic feature" multimodal acquisition mode, which only acquires the three-dimensional spatiotemporal information of radar point clouds and the decibel and azimuth data of environmental audio, completely abandoning the traditional video and voice semantic recording. Under the premise of strictly adhering to privacy compliance, it breaks through the application barrier of all-weather monitoring in highly private spaces such as dormitories.

[0015] (2) In response to the complex scenario of small dormitory space and overlapping occlusion of people, this invention constructs a spatiotemporal association model based on graph topology. It utilizes the attention mechanism of graph neural network and message passing to aggregate spatiotemporal context features across frames to complete the feature loss of the target in the occlusion state. Combined with the bipartite graph optimal matching algorithm, it realizes stable and continuous tracking of multi-target IDs in complex environments.

[0016] (3) This invention combines three-dimensional spatial semantic modeling to divide the bed, desktop and ground, introduces time gating mechanism to filter normal sleep and accurately judge abnormal prolonged lying state; by extracting the trajectory curvature probability distribution entropy value and round-trip repetition degree in specific semantic areas, it effectively overcomes the limitations of existing technology in objectively quantifying and accurately identifying hidden behavioral patterns such as student anxiety in life scenarios, realizes the leap from surface action monitoring to deep physical and mental health care, and provides a new non-contact psychological health perception method based on physical feature quantification.

[0017] (4) This invention constructs a multi-source evidence fusion framework. By calculating the spatial consistency between the radar target and the audio source, the radar motion intensity evidence and acoustic energy evidence are dynamically weighted. By using synthesis rules that include conflict handling, false alarms are effectively suppressed, greatly improving the accuracy and reliability of fall detection and violent conflict identification in practical and complex applications, and avoiding the problem of single sensors being susceptible to environmental multipath or non-target micro-motion interference. Attached Figure Description

[0018] Figure 1 is a schematic diagram of the overall method flow of the present invention; Figure 2 is a schematic diagram of the multimodal data processing and anomaly early warning logic flow of the present invention; Figure 3 shows the simulation results of the spatiotemporal correlation tracking algorithm based on graph neural network of the present invention; Figure 4 shows the simulation results of the anxiety behavior detection algorithm of the present invention; Figure 5 shows the simulation results of the fall event recognition algorithm of the present invention; Figure 6 shows the simulation results of the abnormal prolonged lying down detection algorithm of the present invention; Figure 7 shows the simulation results of the multimodal fusion algorithm for violent conflict identification of the present invention. Detailed Implementation

[0019] The present invention will be further described below with reference to the accompanying drawings and specific embodiments.

[0020] In the description of this invention, unless otherwise stated, "a plurality of" means two or more; the terms "upper," "lower," "left," "right," "inner," "outer," "front end," "rear end," "head," "tail," etc., indicate the orientation or positional relationship based on the orientation or positional relationship shown in the accompanying drawings, and are only for the convenience of describing the invention and simplifying the description, and do not indicate or imply that the device or element referred to must have a specific orientation, or be constructed and operated in a specific orientation, and therefore should not be construed as a limitation of the invention. Furthermore, the terms "first," "second," "third," etc., are used for descriptive purposes only and should not be construed as indicating or implying relative importance.

[0021] In the description of this invention, it should be noted that, unless otherwise explicitly specified and limited, the terms "connected" and "linked" should be interpreted broadly. For example, they can refer to a fixed connection, a detachable connection, or an integral connection; they can refer to a mechanical connection or an electrical connection; they can refer to a direct connection or an indirect connection through an intermediate medium. Those skilled in the art can understand the specific meaning of the above terms in this invention based on the specific circumstances.

[0022] The dormitory occupant abnormal behavior early warning system based on millimeter-wave radar includes a memory, a processor, and a computer program stored in the memory and capable of running on the processor. When the processor executes the program, it implements a method for early warning of abnormal behavior of dormitory occupants.

[0023] like Figures 1 to 2 As shown, the method for early warning of abnormal behavior of dormitory personnel based on millimeter-wave radar includes the following steps: S1. Construct a multimodal sensing environment based on the dormitory environment, set the physical parameters of millimeter-wave radar and microphone array, and simultaneously collect raw millimeter-wave radar intermediate frequency data and environmental audio data. In this embodiment, to verify the effectiveness of the present invention, the following multimodal hardware deployment and physical parameters were set as the basis for data generation in the simulation environment: the simulation uses a 60GHz frequency modulated continuous wave (FMCW) millimeter-wave radar with a 3-transmit 4-receive (3Tx-4Rx) antenna array, the sweep bandwidth is set to 4GHz to ensure high range resolution, and the frame rate is set to 10Hz; the audio acquisition simulation uses a 4-element ring microphone array, the element spacing is set to 5cm, and the sampling rate is set to 16kHz; the radar and microphone arrays are both integrated and deployed in the center of the dormitory ceiling as the origin reference of the global coordinate system; Based on the simulation experiment parameters, in this embodiment, the physical dimensions of the dormitory are set to 6.0m long, 3.6m wide, and 2.8m high. The internal layout adopts the standard "bunk bed with desk underneath" structure in existing technologies, and a three-dimensional spatial coordinate system is established. A unified timestamp logic ensures strict alignment of radar frames and audio frames on the time axis, providing a unified time reference for subsequent data fusion.

[0024] S2. Receive and preprocess the acquired data, including: performing conventional signal preprocessing on the raw millimeter-wave radar intermediate frequency data such as denoising, three-dimensional fast Fourier transform, and constant false alarm rate (CFAR) detection; generating a sparse point cloud set of targets; extracting real-time tracking data of targets and personnel based on the sparse point cloud set; and setting thresholds through CFAR detection. ,when When a point is identified as a valid target detection point, the final result is generated. Set of radar detection points for a frame ,in The number of detection points in the current frame is [number], and each detection point is constructed as a state vector of a uniform dimension. This vector contains three-dimensional position features, radial velocity features, and signal-to-noise ratio features, which are used to characterize the target's trajectory and micro-motion state in space; A set of predicted trajectory points is obtained by predicting the three-dimensional position and velocity of the historical trajectory in the previous frame in the current frame using a uniform velocity model. The set of radar detection points in the current frame and the set of predicted trajectory points are mapped to a set of nodes in the graph. Based on the set spatial association threshold and velocity association threshold, the node pairs that meet the spatiotemporal proximity condition are mapped to an edge set, thereby constructing a spatiotemporal association model based on graph topology.

[0025] Specifically, the construction of the spatiotemporal correlation model based on graph topology includes: using a uniform velocity model to predict the three-dimensional position and velocity of the historical trajectory from the previous frame in the current frame to obtain a set of predicted trajectory points; mapping the set of radar detection points in the current frame and the set of predicted trajectory points to a set of nodes in the graph; based on set spatial correlation thresholds and velocity correlation thresholds, mapping node pairs that satisfy the spatiotemporal proximity condition to an edge set, thereby constructing a bipartite graph candidate correlation model. The expression of the spatiotemporal correlation model is: ,in, For the first The candidate association graph generated from the frame serves as the graph topology output and is used for subsequent node feature embedding and graph neural network message passing. The process of building the spatiotemporal association model also includes the following: Let the first The set of radar detection points is ,in The current frame contains the number of detection points. Each detection point's state vector includes its 3D position, radial velocity, and signal-to-noise ratio. The set of historical trajectories from the previous frame is... ,in This represents the number of historical trajectories.

[0026] The expression for the state of each historical trajectory is: ; in, For the first The historical trajectory in the first The state vector of a frame. For including three-dimensional coordinates The position vector, It is the velocity vector; Predicting the first using a uniform velocity model The expression for predicting the trajectory position in the frame trajectory state is: ; in, For the first The historical trajectory in the first Predicting the 3D position of a frame. This is the position vector of the previous frame. The velocity vector of the previous frame. The time interval between adjacent frames; The expression for trajectory velocity prediction is: ; in, For the first Frame prediction speed; Map the current frame detection point set and the previous frame predicted trajectory point set to a node set in the graph. Furthermore, the possible spatiotemporal relationships between nodes are mapped to edge sets, and the expression for establishing the candidate edge set of the bipartite graph is as follows: ; in, Let be the set of edges. This represents the edge connecting the detection point and the predicted trajectory point. The three-dimensional position of the detection point in the current frame. The radial velocity of the detection point, For the set spatial association threshold, The set speed-related threshold; Stable multi-target tracking under dense occlusion conditions is achieved using the message passing mechanism of graph neural networks. This addresses the problem of target loss and misassociation caused by confined dormitory spaces and frequent occlusion by people. The specific steps for stable multi-target tracking are as follows: S202. Extract the original feature vector containing three-dimensional position, radial velocity, and signal-to-noise ratio based on the collected data, unify the node representation, and construct an original feature vector of uniform dimension for all nodes. Then, by using linear embedding and nonlinear activation functions, it is mapped to a high-dimensional feature space to generate the initial high-dimensional feature representation of the node. The expression for the initial high-dimensional feature mapping of the node is: ; in, For nodes The initial high-dimensional features, i.e., the graph neural network's first... Layer representation, It is a ReLU nonlinear activation function. The learnable weight matrix for the feature embedding layer. This is the bias term for the feature embedding layer.

[0027] S203. The unnormalized attention weights between nodes are calculated by feature concatenation and activation function with leakage factor. The attention coefficients are obtained after normalization. The representation vector of the central node is updated by aggregating the weighted features of the neighboring nodes. After passing through multiple layers of network, the final node features that fuse global spatiotemporal context information are output to complete the state representation of the target in the occlusion environment. In this embodiment, the graph neural network message passing update is implemented to address the feature loss and cross-frame correlation of the target under occlusion, introducing a graph attention message passing mechanism. The expression for calculating the unnormalized attention weights is: ; in, For nodes with neighboring nodes The unnormalized attention weights between them For nonlinear activation functions with leakage factors, This is the learnable parameter vector in the attention mechanism. For the shared linear transformation weight matrix, and They are nodes With nodes In the graph neural network Feature vectors in the layer Indicates feature concatenation operation; The attention weights are then normalized, and the expression for calculating the attention coefficient is as follows: ; in, For nodes with neighboring nodes Normalized attention coefficients between them For nodes The set of neighboring nodes, The node index in the neighborhood set; Based on the attention coefficient, the representation vector of the center node is updated by aggregating the weighted features of neighboring nodes. The expression for node feature update is: ; in, For the first The updated node feature representation after layer update The feature transformation weight matrix is ​​used in the message passing process. Non-linear activation functions; stacked After passing through the layers of the network, the output is the final node feature representation that incorporates global spatiotemporal context information. This is used to simulate and complete the state representation of the target in an occluded environment.

[0028] S204. Construct an edge feature vector by combining the spatial distance difference and velocity difference between the detection point and the predicted trajectory point, as well as the final node features. The expression for calculating the spatial distance difference is: ; in, For testing points With predicted trajectory points The difference in spatial distance between them To detect the three-dimensional position of the point, To predict the three-dimensional position of the trajectory point, express Norm calculation; The expression for calculating the speed difference is:

[0029] in, For testing points With predicted trajectory points The radial velocity difference between them For the speed of the detection point, To predict the velocity of trajectory points; Combining the above features, we construct edge feature vectors. The expression for constructing edge feature vectors is as follows: ; in, Candidate edges to be constructed The edge feature vectors, For testing points The final node features, For trajectory points The final node features, Indicates feature concatenation operation; The association probability of edge features is calculated to generate an association matrix, and the association probability is converted into a matching cost value. The expression for calculating the association probability of candidate edges is as follows: ; in, For testing points With trajectory The probability of association belonging to the same target. Here is the weight matrix of the fully connected classification network. For bias terms, The sigmoid activation function is expressed as follows: ; Converting the association probability into a cost value, the expression for constructing the elements of the matching cost matrix is ​​as follows: ; in, To match the cost matrix of the th Line number The value of the list; The optimal allocation of target IDs is achieved using a bipartite graph matching algorithm. The expression for finding the optimal match is: ; in, To find the optimal allocation scheme that minimizes the total global matching cost, Indicates the detection point The assigned trajectory index number; The final association relationship is generated based on the optimal allocation scheme. The expression for the final association relationship is: ; in, For the current number The optimal set of frame associations. For the detection point index, For testing points The assigned trajectory index number; Subsequently, a smoothing filter mechanism is introduced to perform weighted fusion of the predicted trajectory state of the successfully matched track with the current detection point state, suppress millimeter-wave radar observation noise, update the historical trajectory state, and output the final smooth and stable trajectory set.

[0030] Based on the relationships generated above, the historical trajectory status is updated. The expression for the trajectory status update output is: ; in, This is the final set of stable trajectories after the update. For trajectory state update function, This is a collection of historical trajectories from the previous frame. The system iterates its state based on this update process, thereby achieving stable multi-target tracking in densely occluded environments. Because millimeter-wave radar point clouds inherently contain spatial observation noise, directly using the detection point state to cover the trajectory state will cause jagged, jerky fluctuations in the tracking trajectory. Therefore, the trajectory state update function... It includes an internal kinematic smoothing filter mechanism. Specifically, it addresses the optimal correlation scheme. Detection points that were successfully matched With historical trajectory This invention utilizes a preset smoothing factor to perform a weighted fusion update of the kinematic state predicted by the uniform velocity model and the current measured state. The expression for the position smoothing update is: ; The expression for smooth speed update is: ; in, and These represent the smoothed 3D position and velocity of the trajectory after filtering and updating. and The predicted 3D position and velocity are obtained based on the spatiotemporal correlation model; and For the current frame detection point The actual measurement of three-dimensional position and velocity; These are the position smoothing filter coefficients. For the velocity smoothing filter coefficients, and ; This invention completes the entire state iteration of the target by using the above-mentioned smooth update mechanism, supplemented by momentum updates of high-dimensional features of historical trajectories.

[0031] The process involves extracting environmental audio data and calculating the target sound source azimuth using phase-weighted cross-correlation analysis to achieve spatial alignment between the radar coordinate system and the acoustic coordinate system, outputting the target trajectory coordinates. Specifically, this includes: filtering and denoising the audio signal, extracting acoustic features, calculating the sound source azimuth using phase-weighted cross-correlation analysis, and achieving spatial alignment with the radar coordinate system; firstly, the audio signal... Bandpass filtering is performed, and the Mel-frequency cepstral coefficients are extracted as acoustic features. The filtered signal is Next, the time difference of arrival between the microphone pairs is calculated using the generalized cross-correlation-phase transform algorithm. The expression for the cross-correlation function is: ; in , These are the frequency domain representations of the two microphone signals, respectively. for conjugate, For time delay variables; Determine the optimal latency by searching for peak values. Then, based on the microphone array geometric spacing... and speed of sound At room temperature The expression for calculating the azimuth angle of the sound source is: ; Finally, spatial alignment transformation is performed using a coordinate transformation matrix. azimuth in acoustic coordinate system Mapped to the radar coordinate system, we get To ensure spatial consistency between the radar target and the sound source location, the alignment error is less than [missing information]. .

[0032] S3. Based on the physical structure of the dormitory, a three-dimensional spatial semantic model is established, and the monitoring space is divided into bed activity area, desktop area and ground activity area. The target trajectory coordinates are mapped to the three-dimensional spatial semantic model to generate the target's regional semantic state and height level features.

[0033] Establishing a three-dimensional spatial semantic model includes: Define the three-dimensional coordinate system of the dormitory space as ,in For the width direction of the dormitory For the length direction, Define a height threshold in the vertical direction. That is, the upper limit of the activity area on the bed, This refers to the upper limit of the ground activity area, which, combined with the planar coordinate range, divides the region into three semantic regions: ; in , , The planar coordinate ranges for the bed, desktop, and floor are determined based on the actual dormitory layout within a specific area. Target state mapping involves mapping the target trajectory coordinates output from step S2. The target trajectory coordinates are mapped to the established 3D spatial semantic model in real time, generating the target's regional semantic state. ,in High-level features ,in This is used to create a bed activity area. With ground activity area This provides contextual basis for subsequent behavior recognition.

[0034] S4. Based on the logical judgment of the target's spatiotemporal trajectory, regional semantic state, and sound source characteristics, health care analysis and safety event early warning analysis are executed simultaneously and in parallel to achieve continuous tracking of multiple targets within a specific area.

[0035] Health care analysis includes identifying anxiety behaviors and abnormal prolonged bed rest, and triggering health warnings by quantifying the disorder of trajectories and the duration of residence in specific semantic regions; Identifying anxious behaviors includes collecting data on the target's activity area on the ground. The trajectory sequence is used to calculate the probability distribution entropy of the trajectory curvature and the frequency of trajectory repetition. If both the entropy and repetition frequency exceed a preset threshold, it is determined to be an anxious pacing state, which specifically includes the following: Anxiety behavior recognition identifies anxious pacing behavior by quantifying the disorder and repetition frequency of a target's trajectory in a ground activity area, introducing trajectory curvature entropy and round-trip repetition as quantitative indicators; firstly, data is collected on the target within a preset time window. Trajectory sequence within Next, calculate the trajectory points. The expression for the local curvature at that point is: ; And divide the curvature value into discrete intervals The statistical curvature falls on the 1st The probability of each interval , The number of trajectory points within this interval is given; then the trajectory disorder is quantized, and the trajectory curvature entropy is calculated. The expression is:

[0036] And calculate the round-trip repeatability of the trajectory. Set a round-trip threshold The number of round trips within the statistical time window The expression for calculating R is:

[0037] like set up is the entropy threshold and set up If the repetition rate is set at times / min, it is determined to be an anxious pacing state, triggering an abnormal behavior warning.

[0038] The identification of abnormal prolonged bed rest includes: setting a preset normal sleep period as a statistical exemption interval; and mapping the activity area to the bed. The target is monitored, and only the dwell time outside the exemption range is accumulated and counted. If the accumulated dwell time exceeds the preset health threshold, and the target micro-motion energy extracted based on the radar micro-Doppler spectrum is continuously lower than the preset threshold (i.e. there are no significant large-amplitude movement characteristics), it is judged as an abnormal prolonged lying state.

[0039] Abnormal prolonged bed rest is identified by introducing a time gating mechanism, setting a preset normal sleep period as a statistical exemption interval; and mapping the activity area to the bed... The system monitors the target and only accumulates statistics on the dwell time outside the exemption range. If the accumulated dwell time exceeds a preset health threshold and the target does not exhibit significant movement during this period, it is determined to be an abnormal prolonged lying state or a physiological abnormality. Specifically, the normal sleep period is defined as the exemption range. Non-exempt period ;Mapped to the activity area of ​​the bed The goal is to accumulate the length of stay during non-exempt periods: ; Extracting the target's micro-kinetic energy characteristics Based on radar micro-Doppler spectrum calculation, a micro-motion energy threshold is set. ; As a health threshold, if and If there is no significant movement during the period, it is judged as abnormal prolonged lying down or an abnormal physiological state, triggering an abnormal behavior warning.

[0040] Safety incident early warning includes identifying fall incidents and violent conflicts, and triggering safety alarms by analyzing the abrupt changes in the target height level and the spatial consistency and intensity characteristics of radar-sound sources; The identification of fall events includes determining a fall event if the target continuously meets the following conditions: rapid change in height within a preset time window, vertical speed exceeding a set threshold during descent, and micro-motion energy remaining below a static threshold for more than a preset confirmation time after the change. Fall event recognition is achieved by real-time monitoring of the target's centroid three-dimensional height. and vertical downward velocity component Based on radar point cloud velocity feature extraction, and setting a time window for height abrupt changes. Fall speed threshold Static confirmation time Micro-motion energy threshold ; If a target continuously satisfies the following three temporal characteristics, it is determined to be a fall event: First, a sudden change in height, i.e. The first is a rapid descent from the bed height level to the ground height level; the second is abnormal speed, i.e., during the descent... Thirdly, the state is static, that is, after a rapid change, the target is in Microkinetic energy within the region And the duration exceeds Once the above conditions are met, a security alarm will be triggered and simultaneously pushed to the early warning system downloaded and used by dormitory managers and security personnel.

[0041] Violent conflict identification involves constructing a multi-source evidence fusion framework, extracting millimeter-wave radar motion intensity evidence and acoustic energy evidence, dynamically weighting spatial consistency by calculating the deviation values ​​between the radar target and the audio source location, and finally using synthesis rules that include conflict handling for fusion decision.

[0042] First, we define the multi-source evidence fusion framework. ,in Indicating violent conflict, This indicates normal activity. Indicates an uncertain state; Next, the Basic Probability Assignment (BPA) function is constructed for radar evidence. Velocity Dispersion Based on Radar Point Cloud Groups The expression for constructing evidence of exercise intensity is: ; in The average Doppler velocity; Radar evidence The expression is: ; in The parameter of the Sigmoid function , This is the threshold for velocity dispersion. Regarding audio evidence Based on the decibel value of ambient sound The acoustic energy evidence is constructed as follows: ; in The parameter of the Sigmoid function , The decibel threshold; Then, spatial consistency weighting is performed to calculate the deviation between the radar target azimuth and the audio source azimuth. Set spatial alignment threshold ;like This will increase the fusion weight. , ;like Then reduce the fusion weight. , Increase the conflict factor; finally, use Dempster's synthesis rule to weight the evidence. To integrate, the conflict coefficient The expression is: ; Trust level after integration The expression is: ; Set alarm threshold ,like If the event is deemed a violent conflict, an abnormal behavior warning will be triggered; otherwise, it will be considered a normal activity or an uncertain state, and no abnormal behavior warning will be triggered.

[0043] This invention verifies the multi-target tracking model and the multi-task parallel monitoring and anomaly logic judgment module through system simulation, with a simulation step size of [missing information]. Set to 0.1s.

[0044] To address the issues of multi-target tracking loss and frequent ID jumps in dormitory environments characterized by limited space and dense obstruction, this embodiment first conducts simulation verification of a spatiotemporal correlation model tracking algorithm based on graph topology.

[0045] The simulation simulates a complex scenario where two targets intersect and occlude within a confined area. The simulation results are as follows: Figure 3 As shown, Figure 3 (a) Tracking results of the spatiotemporal correlation model proposed in this invention; combined with Figure 3 (b) Analysis shows that when a target experiences spatial intersection and occlusion, entering the occlusion zone marked by the dashed line in the figure, the traditional algorithm causes trajectory breakage due to the reduction in the target's radar cross-section and feature loss. Furthermore, target identity mismatch occurs after the occlusion ends. Throughout the process, there were two severe target ID jumps caused by the allocation of new IDs after trajectory interruption. Figure 3 As shown in (a), this invention constructs a spatiotemporal correlation model and introduces the message passing mechanism of a graph neural network (GNN). It utilizes an attention mechanism to aggregate weighted features across frames, successfully completing the missing features of the target under occlusion. Figure 3 (a) The trajectory shows that even during cross-occlusion, the present invention can still maintain the smoothness and continuity of the trajectory. By using the bipartite graph matching algorithm to solve the global optimal allocation, the number of target ID jumps throughout the process is reduced to 0. The comparative results fully demonstrate that the algorithm of the present invention can effectively overcome feature loss in complex occlusion environments, provide high-precision and highly stable spatiotemporal trajectories, and lay a reliable data foundation for subsequent abnormal behavior analysis.

[0046] For anxiety behavior detection, the total sampling time was set to 60 seconds, with 0-15 seconds representing a smooth straight walk, followed by high-frequency shaking accompanied by lateral swaying to simulate anxious pacing. Simulation results are as follows: Figure 4 As shown, this invention extracts trajectory sequences from active areas on the ground and calculates the local curvature probability distribution. Figure 4 (a) The trajectory of the target was clearly reconstructed in three-dimensional space; such as Figure 4 As shown in (b), the trajectory curvature entropy after 15 seconds Rapidly surpassing the preset entropy threshold And it can reach up to about 3.0 bits; such as Figure 4 As shown in (c), the trajectory round-trip repetition rate within this period is... It also shows a step-like increase, breaking through the repetition threshold. times / min. Compared to normal walking, this invention quantifies the disorder and repetition frequency of the trajectory, requiring simultaneous satisfaction of... and The dual criteria accurately identified anxious behaviors, effectively eliminating false alarms caused by routine daily activities.

[0047] For abnormal prolonged bed rest detection, the simulation was set to a monitoring duration of 6 hours, a micro-motion energy threshold of 1.5, and a health threshold for prolonged bed rest alarm time. The time was set to 4.0 hours, and the current sleep duration was assumed to be outside the normal sleep exemption range. The simulation results are as follows: Figure 6 As shown, Figure 6(a) Visually demonstrates that the target is mapped to the bed semantic region and is in a static state; such as Figure 6 As shown in (b) the monitoring of the target's vital signs, the radar micro-motion energy distribution characteristics remained stable below the micro-motion energy threshold of 1.5 for most of the time; based on the time gating mechanism, Figure 6 (c) The time accumulation decision model accumulates dwell times below a threshold, and when the accumulated dwell time... When the time exceeds 4.0 hours, this invention accurately determines that the target is in an abnormal prolonged lying state or a physiological abnormal state, triggering an abnormal behavior warning output.

[0048] For fall detection, the total monitoring time is set to 80 seconds, with the target falling at the 10th second. Thresholds for bed height and fall speed are also set. Static energy threshold The simulation results are as follows: Figure 5 As shown, Figure 5 (a) A three-dimensional trajectory with height information was reconstructed; combined with Figure 5 (b) Analysis: The target's altitude at 10s A sudden change occurred, with the height rapidly dropping from 1.7m to 0.3m, fulfilling the conditions for a rapid descent at a specific height level, along with a synchronized downward vertical velocity. A rapid mutation occurred and crossed A velocity threshold of m / s satisfies the conditions for velocity anomalies; such as Figure 5 As shown in (c), after the high-speed abrupt change occurs, the present invention enters the state static verification stage, and the micro-motion energy of the target rapidly drops to the static energy threshold represented by the green line in the figure. The following conditions must be met, and the temperature must remain below this threshold for an extended period beyond the confirmation time. This invention significantly improves the reliability of fall detection in complex spaces by continuously verifying three temporal characteristics: abrupt changes in height, abnormal velocity, and the cessation of micro-motion energy after a fall.

[0049] For the identification of violent conflicts, this embodiment adopts a multi-source evidence fusion framework and sets a radar velocity dispersion threshold. The audio decibel threshold is set to 80 dB, and the alarm fusion trust threshold is set to 0.8. Simulation results are as follows: Figure 7 As shown, Figure 7 (a) and Figure 7 (b) This section shows the changes in input feature 1 (radar motion intensity evidence) and input feature 2 (acoustic energy evidence). Normal activity occurred 50 seconds prior, followed by a violent collision with the target 50 seconds later; for example... Figure 7 (c) As shown in the BPA output, the basic probability assignment values ​​of a single sensor all exhibit severe oscillations when a conflict occurs. Using a single-mode hard decision method can easily lead to missed or false alarms. Figure 7As shown in (d), this invention uses spatial consistency weighting and Dempster synthesis rules that include conflict handling for fusion, resulting in the DS evidence fusion result. After the conflict occurs, the value rapidly increases and converges to a high level, remaining stable above the fusion threshold of 0.8. The measured extreme value in the figure reaches above 0.9, which fully verifies the significant advantages of the multimodal fusion algorithm in suppressing heterogeneous interference and improving the robustness of malicious event judgment.

[0050] This invention completely solves the pain point of privacy infringement from the data source, achieving secure perception with zero privacy leakage. It adopts a "non-semantic feature" multimodal acquisition mode, acquiring only the three-dimensional spatiotemporal information of millimeter-wave radar point clouds and the decibel and azimuth data of environmental audio, completely abandoning traditional video footage and voice semantic recording. Under the premise of strictly adhering to privacy compliance, it breaks through the application barriers of all-weather monitoring in highly private spaces such as dormitories.

[0051] This invention effectively overcomes the challenges of target tracking loss and ID jumps in densely occluded environments. For complex scenarios such as small dormitory spaces and overlapping occlusion of people, this invention constructs a spatiotemporal association model based on graph topology. It utilizes the attention mechanism and message passing of graph neural networks to aggregate spatiotemporal context features across frames, thus completing the feature loss of targets in occluded states. Combined with a bipartite graph optimal matching algorithm, it achieves stable and continuous tracking of multiple target IDs in complex environments.

[0052] This invention combines three-dimensional spatial semantic modeling to divide the bed, desktop, and floor, and introduces a time gating mechanism to filter normal sleep and accurately determine abnormal prolonged lying down states. By extracting the trajectory curvature probability distribution entropy value and round-trip repetition degree in specific semantic regions, it effectively overcomes the limitations of existing technologies in objectively quantifying and accurately identifying latent behavioral patterns such as student anxiety in real-life scenarios.

[0053] To address the issue that single sensors are susceptible to interference from environmental multipath or non-target micro-motions, this invention constructs a multi-source evidence fusion framework. By calculating the spatial consistency between the radar target and the audio source's orientation, it dynamically weights the radar motion intensity evidence and acoustic energy evidence. Furthermore, by utilizing synthesis rules that include conflict handling, it effectively suppresses false alarms and greatly improves the accuracy and reliability of fall detection and violent conflict recognition in complex practical applications.

[0054] The above embodiments are merely preferred embodiments of the present invention and should not be construed as limiting the scope of the present invention. Therefore, any modifications, equivalent changes, improvements, etc., made in accordance with the claims of the present invention shall still fall within the scope of the present invention.

Claims

1. A method for early warning of abnormal behavior of dormitory personnel based on millimeter-wave radar, characterized in that, Includes the following steps: S1. Construct a multimodal sensing environment based on the dormitory environment, set the physical parameters of millimeter-wave radar and microphone array, and simultaneously collect raw millimeter-wave radar intermediate frequency data and environmental audio data. S2. Receive the collected data and perform preprocessing, including: performing signal processing on the raw millimeter-wave radar intermediate frequency data to generate sparse point clouds, constructing a spatiotemporal correlation model based on graph topology, using the message passing mechanism of graph neural networks to achieve stable tracking of multiple targets under dense occlusion, and outputting the target trajectory coordinates; at the same time, extracting environmental audio data and using phase-weighted cross-correlation analysis to calculate the target sound source azimuth angle, and achieving spatial alignment between the radar coordinate system and the acoustic coordinate system; S3. Based on the physical structure of the dormitory, a three-dimensional spatial semantic model is established, and the monitoring space is divided into the bed activity area, the desktop area and the ground activity area. The target trajectory coordinates are mapped to the three-dimensional spatial semantic model to generate the target's regional semantic state and height level features. S4. Based on the logical judgment of the target's spatiotemporal trajectory, regional semantic state, and sound source characteristics, health care analysis and safety event early warning analysis are executed simultaneously and in parallel to achieve continuous tracking of multiple targets within a specific area.

2. The method for early warning of abnormal behavior of dormitory personnel based on millimeter-wave radar according to claim 1, characterized in that, The construction of the spatiotemporal correlation model based on graph topology in step S2 includes: using a uniform velocity model to predict the three-dimensional position and velocity of the historical trajectory in the previous frame in the current frame to obtain a set of predicted trajectory points; mapping the set of radar detection points in the current frame and the set of predicted trajectory points to a set of nodes in the graph; based on the set spatial correlation threshold and velocity correlation threshold, mapping the node pairs that satisfy the spatiotemporal proximity condition to an edge set, thereby constructing the spatiotemporal correlation model.

3. The method for early warning of abnormal behavior of dormitory personnel based on millimeter-wave radar according to claim 2, characterized in that, In step S2, the specific steps of multi-target stable tracking are as follows: S202. Extract the original feature vector containing three-dimensional position, radial velocity and signal-to-noise ratio based on the collected data, and map it to a high-dimensional feature space through linear embedding and nonlinear activation function to generate the initial high-dimensional feature representation of the node. S203. The unnormalized attention weights between nodes are calculated by feature concatenation and activation function with leakage factor. The attention coefficients are obtained after normalization. The representation vector of the central node is updated by aggregating the weighted features of the neighboring nodes. After passing through multiple layers of network, the final node features that fuse global spatiotemporal context information are output to complete the state representation of the target in the occlusion environment. S204. Construct an edge feature vector by combining the spatial distance difference, velocity difference, and final node features between the detection point and the predicted trajectory point; calculate the association probability of the edge features to generate an association matrix, and convert the association probability into a matching cost; use a bipartite graph matching algorithm to solve for the optimal allocation scheme that minimizes the total global matching cost. Subsequently, a smoothing filter mechanism is introduced to perform weighted fusion of the predicted trajectory state of the successfully matched track with the current detection point state, suppress millimeter-wave radar observation noise, update the historical trajectory state, and output the final smooth and stable trajectory set.

4. The method for early warning of abnormal behavior of dormitory personnel based on millimeter-wave radar according to claim 3, characterized in that, In step S3, establishing the three-dimensional spatial semantic model includes: Define the three-dimensional coordinate system of the dormitory space as ,in For the width direction of the dormitory For the length direction, Define a height threshold in the vertical direction. That is, the upper limit of the activity area on the bed, This refers to the upper limit of the ground activity area, which, combined with the planar coordinate range, divides the region into three semantic regions: ； in , , The planar coordinate ranges for the bed, desktop, and floor are determined based on the actual dormitory layout within a specific area.

5. The method for early warning of abnormal behavior of dormitory personnel based on millimeter-wave radar according to claim 4, characterized in that, In step S3, the mapping of the target trajectory coordinates to the three-dimensional spatial semantic model includes: Define the height threshold and the range of planar coordinates, and mark the target trajectory coordinates output in step S2 as follows. The target trajectory coordinates are mapped to the established 3D spatial semantic model in real time to generate the target's regional semantic state. ,in High-level features ,in This is used to create a bed activity area. With ground activity area .

6. The method for early warning of abnormal behavior of dormitory personnel based on millimeter-wave radar according to claim 5, characterized in that, In step S4, the health care analysis includes identifying anxiety behaviors and abnormal prolonged lying down, and triggering abnormal behavior warnings by quantifying the disorder of the trajectory and the dwell time in specific semantic regions. The identification of the anxiety behavior includes collecting data on the target's activity area on the ground. The trajectory sequence is used to calculate the probability distribution entropy of the trajectory curvature and the frequency of trajectory repetition. If both the entropy and the repetition frequency exceed the preset threshold, it is determined to be an anxious pacing state. The identification of abnormal prolonged bed rest includes: setting a preset normal sleep period as a statistical exemption interval; mapping the activity area to the bed. The system monitors targets and only accumulates statistics on dwell time outside the exemption range. If the accumulated dwell time exceeds the preset health threshold and the target micro-motion energy extracted based on the radar micro-Doppler spectrum remains below the preset threshold, it is determined to be an abnormal prolonged lying state.

7. The method for early warning of abnormal behavior of dormitory personnel based on millimeter-wave radar according to claim 6, characterized in that, In step S4, the safety incident warning includes identifying fall events and violent conflicts, and triggering abnormal behavior warnings by analyzing the abrupt change characteristics of the target height level and the spatial consistency and intensity characteristics of the radar-sound source. The identification of the fall event includes determining it as a fall event if the target continuously meets the following conditions: rapid change in height within a preset time window, vertical speed exceeding a set threshold during descent, and micro-motion energy remaining below a static threshold for more than a preset confirmation time after the change. The violent conflict identification includes constructing a multi-source evidence fusion framework, extracting millimeter-wave radar motion intensity evidence and acoustic energy evidence, dynamically weighting the spatial consistency by calculating the deviation value between the radar target and the audio source location, and finally using synthesis rules that include conflict handling to make a fusion decision.

8. A dormitory occupant abnormal behavior early warning system based on millimeter-wave radar, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that, When the processor executes the program, it implements the method described in any one of claims 1 to 7.

Images