A multi-radar data splicing human activity state analysis system and method

Abstract

The application belongs to the technical field of human activity monitoring and health management, and discloses a multi-radar data splicing human activity state analysis system and method. The system comprises a plurality of millimeter wave radar units for collecting point cloud data in a monitoring area; a unified clock and phase compensation unit for time and phase correction of each radar output; a raw point cloud joint registration unit for performing point cloud registration under a unified coordinate system in combination with geometric and Doppler constraints to obtain a high-density point cloud set; a posture recognition unit for converting the point cloud set into a range-Doppler heat map and inputting the range-Doppler heat map into a convolutional neural network and a long short-term memory network to output a human posture category; and a health analysis and alarm unit for establishing a multi-scale work and rest regularity model based on a posture sequence and triggering a multi-level alarm process when detecting long-term abnormalities in individual work and rest. The application is suitable for home care, smart elderly care and medical auxiliary monitoring scenes.

Description

Technical Field

[0001] This invention belongs to the field of human activity monitoring and health management technology, specifically a human activity status analysis system and method that uses multi-radar data stitching. Background Technology

[0002] With the accelerating aging of the population and the increasing demand for home-based health management, how to monitor the daily activities of the elderly and chronically ill patients in real time without relying on wearable devices and video surveillance has become an important issue in the fields of smart healthcare and smart elderly care. Millimeter-wave radar, due to its advantages of being non-contact, penetrating partial obstructions, and not infringing on image privacy, is gradually being introduced into applications for human activity recognition and fall detection. Compared with traditional camera-based methods, millimeter-wave radar can operate stably in low-light or nighttime environments, making it more suitable for practical applications in homes and medical settings.

[0003] Most common human activity monitoring methods currently rely on a single millimeter-wave radar. These methods typically generate range-Doppler heatmaps by processing echo signals, and then combine them with feature extraction or classification algorithms to identify human postures. However, due to the limited coverage of a single radar and the insufficient number of point clouds collected, problems such as data sparsity and decreased recognition accuracy can easily occur when there are obstructions or multiple targets in the monitoring area. At the same time, a single radar is affected by clock drift and phase instability during long-term operation, resulting in a lack of a unified benchmark for data at different time periods, which in turn reduces the reliability of posture recognition.

[0004] To improve coverage and recognition rate, some solutions attempt to deploy multiple radars and perform data fusion at the result level. However, these methods often only perform simple merging at the target trajectory level, lacking in-depth stitching of the original point cloud data and failing to fully utilize the information gain brought by multi-radar collaboration. In addition, existing registration methods are mostly based on geometric constraints and fail to combine the radar-specific Doppler velocity information. In dynamic scenes, there may be situations where the geometric position is aligned but the velocity characteristics are inconsistent, thus affecting the accuracy of attitude judgment. Therefore, existing multi-radar methods often struggle to form a high-density point cloud set that maintains uniformity in both spatial and dynamic features, which has become a significant bottleneck limiting the performance of multi-radar collaborative monitoring.

[0005] In terms of health monitoring, existing systems mostly rely on daily thresholds, such as triggering an alarm when the daily activity level is lower than a preset level. This approach is more sensitive to occasional behavior, which can easily lead to false alarms and is difficult to reflect an individual's long-term routine. For scenarios that require continuous monitoring, a single or short-term abnormality is insufficient to support scientific health management and medical assistance. Summary of the Invention

[0006] The purpose of this invention is to provide a human activity state analysis system and method based on multi-radar data stitching, so as to solve the problems mentioned in the background art.

[0007] To achieve the above objectives, the present invention provides the following technical solution: a human activity state analysis system based on multi-radar data stitching, the system comprising: multiple FMCW+MIMO millimeter-wave radars, each equipped with an antenna array of 24 transmit and 22 receive, capable of collecting four-dimensional point cloud data of no less than one hundred points in a single frame output;

[0008] Unified clock and phase compensation unit; used to distribute reference clocks among multiple radars and correct frequency offset, delay and phase drift in real time to ensure consistency of point clouds across devices in time and phase;

[0009] The original point cloud joint registration module is used to perform registration on the corrected point cloud in a unified coordinate system. During the registration process, the geometric consistency constraint and Doppler coherence consistency constraint of the spatial overlapping region are satisfied at the same time. After the matching and correction are completed, a unified high-density point cloud set is generated. This set is consistent in spatial morphology and dynamic features, providing input data for subsequent attitude recognition.

[0010] The posture recognition module is used to convert the high-density point cloud set into a time-ordered distance-Doppler heat map and input it into a convolutional neural network and a long short-term memory network. The convolutional neural network is used to extract spatial features, and the long short-term memory network is used to model temporal features. The combination of the two can take into account both spatial feature extraction and short-term dynamic capture, thereby improving the recognition accuracy of rapid abnormal states such as falls.

[0011] The health analysis and alarm module is used to calculate movement trajectory, average walking speed and dwell time based on posture recognition results, generate daily, weekly and monthly routine models, and trigger an alarm process in sequence when the statistical indicators deviate from the individual's historical average by ±20% or exceed twice the historical standard deviation for three consecutive days. This process includes detection confirmation, emergency call, secondary alarm and two-way voice communication.

[0012] Preferably, four millimeter-wave radar units are deployed within a residential room (approximately 5m × 6m). Each unit employs an FMCW (Frequency Modulated Continuous Wave) system and a MIMO (Multiple Transmitters Multiple Receivers) architecture, with an array configuration of 24 transmitting elements and 22 receiving elements. The installation positions are calibrated to ensure that the overlapping monitoring area covers the human activity space. Each radar collects at least 100 points of four-dimensional point cloud per frame, and records each point individually. ),in In three-dimensional coordinates, The radial velocity is a lower limit derived from experimental calibration, ensuring that the overlapping area has at least 50 pairs of valid corresponding points during ICP registration, thus avoiding geometric constraint degradation.

[0013] Distance resolution: In the formula: The minimum resolvable distance; For the speed of light ( ); For the transmission bandwidth (taken as 2GHz),

[0014] This formula determines the point cloud density; the larger the bandwidth, the denser the point cloud distribution.

[0015] Relationship between radial velocity and Doppler frequency shift: In the formula: Radial velocity; The operating wavelength (77GHz millimeter wave corresponds to approximately 3.9mm); For Doppler frequency shift;

[0016] Ensure that each point is accompanied by velocity semantics, providing input for subsequent coherent consistency constraints;

[0017] Existing technologies often do not set a clear minimum number of points for point cloud acquisition. This invention uses a 24-transmit 22-receive array plus ≥100 points per frame as the registration input condition to ensure the stability of subsequent algorithms.

[0018] Preferably, the unified clock and phase compensation unit includes a reference clock distribution module, an electromagnetic link round-trip delay measurement module, and a carrier frequency deviation compensation module. Each module works in concert to perform time delay, frequency deviation, and phase correction on the output signals of multiple millimeter-wave radars, so that the point cloud data collected by the multiple radars are consistent in terms of time reference and phase reference.

[0019] The processing order is as follows:

[0020] Clock distribution: All radars share the same reference clock source;

[0021] Delay measurement: Send probe pulses to obtain round-trip time. Deducting processing delay One-way delay: In the formula: This is the one-way propagation delay; Round-trip time; To address system latency;

[0022] Compensating for propagation differences between different radars;

[0023] Frequency Offset and Phase Correction: Estimating Carrier Frequency Offset and initial phase difference Correction formula: In the formula: For sampling points; For sampling after correction; The sampling interval; For sampling index; For frequency offset; Phase difference; The imaginary unit;

[0024] Ensure that different radar sampling point clouds have a unified time and phase reference;

[0025] Unlike common methods that only synchronize timestamps, this invention performs dual time and phase corrections before registration to ensure the coherence of point cloud fusion.

[0026] Preferably, when performing registration, the original point cloud joint registration unit performs coordinate matching on point cloud data collected by multiple millimeter-wave radars in the spatially overlapping area, and performs Doppler frequency and phase correction on the velocity information of the point cloud to ensure that the stitched point cloud data maintains consistency in spatial morphology and dynamic motion characteristics, thereby forming a high-density point cloud set that can be processed by the attitude recognition unit.

[0027] First, candidate point pairs are extracted in the spatially overlapping region using the Iterative Closest Point (ICP) algorithm. Subsequently, geometric consistency and Doppler consistency constraints are introduced to form a joint optimization objective function:

[0028] Registration objective function

[0029] ,

[0030] In the formula: Source radar point; Target radar point; : Three-dimensional rotation matrix; Translation vector; Doppler frequency shift at corresponding points between the two radars; Geometric and Doppler weights;

[0031] By minimizing Obtain the optimal registration parameters The results are used to verify phase continuity, eliminate outlier point pairs, and finally output a high-density point cloud set, which is unified under the same coordinate system and time reference to provide input for attitude recognition.

[0032] Preferably, the posture recognition unit receives a high-density point cloud set output by the original point cloud joint registration unit, converts it into a distance-Doppler heat map arranged in time order, and inputs the heat map into a convolutional neural network to extract spatial features and a long short-term memory network to model temporal features, thereby outputting human posture categories such as standing, walking, falling, sitting and lying down, providing input data for the health analysis and alarm unit;

[0033] Point cloud data is transformed using a 2D Fourier transform to generate a distance-Doppler heatmap: ,

[0034] In the formula: : No. Frame heatmap; Complex sampled signal; : Number of snapshots; : Pulse count; Distance index; Speed ​​index;

[0035] CNN extracts spatial features ;

[0036] LSTM modeling time characteristics: ;

[0037] Classify and output attitude labels ,

[0038] In the formula: Output features for CNN; This is the hidden state of the LSTM; The result is the pose recognition result;

[0039] CNN captures spatial patterns, LSTM captures temporal changes, and the combination is used to identify dynamic activities such as falls.

[0040] Preferably, the health analysis and alarm module receives human posture category data output by the posture recognition unit, performs statistical processing on the data to obtain movement trajectory, average walking speed and dwell time, and establishes a daily, weekly and monthly work-rest pattern model based on this. When any statistical indicator in the model deviates from the individual's historical mean by ±20% within three consecutive days, or exceeds twice the individual's historical standard deviation, the health analysis and alarm unit sequentially executes the alarm process of detection confirmation, emergency call, secondary alarm and two-way voice communication.

[0041] With attitude sequence and location information For input:

[0042] Movement trajectory: ,

[0043] Average walking speed: , ,

[0044] In the formula: Position in a unified coordinate system; : Number of valid frames on the day; : Step speed at time t; Doppler frequency shift at time t; Millimeter wave wavelength; Frame time interval;

[0045] In the startup window Establish individual baselines within 30 days:

[0046] , ,

[0047] in For the first Daily index values;

[0048] The judgment rule is: if the daily indicator satisfy: ,

[0049] If the alarm occurs for three consecutive days, an alarm will be triggered. The alarm process is divided into: detection and confirmation, calling for help, secondary alarm, and two-way voice communication. The duration of each stage is given in the example, for example, confirmation wait is 60 seconds, and the interval for secondary alarm is 120 seconds.

[0050] A method for analyzing human activity status by stitching together multi-radar data is disclosed. This method is based on the aforementioned system, and its specific steps are as follows:

[0051] Radar acquisition and reference correction: Multiple millimeter-wave radars are deployed in the monitoring area. Each radar acquires no less than one hundred points of point cloud data in a single frame. The output data is corrected through reference clock distribution and phase compensation steps to ensure consistency in time and phase.

[0052] Point cloud registration and consistency correction: Joint registration is performed on the corrected point cloud in a unified coordinate system. The registration satisfies both geometric consistency and Doppler coherence consistency to generate a high-density point cloud set. The joint registration uses the iterative nearest point method to match the point clouds in the overlapping region and corrects the result by combining the Doppler spectral peak shift and phase continuity.

[0053] Posture recognition and daily routine modeling: The high-density point cloud set is converted into a time-ordered distance-Doppler heat map and input into a convolutional neural network and a long short-term memory network for recognition. The convolutional neural network is used for spatial feature extraction, and the long short-term memory network is used for temporal feature modeling, outputting human posture categories. Based on the posture sequence output by the posture recognition unit, daily, weekly, and monthly daily routine models are established. When the index is detected to deviate from the individual's historical mean by ±20% or more than twice the standard deviation for three consecutive days, a multi-level alarm process including detection confirmation, emergency call, and secondary alarm is triggered.

[0054] Preferably, the specific steps of radar acquisition and reference correction are as follows:

[0055] Four millimeter-wave radar units were deployed within the monitoring area. Each unit adopted a 24-transmitter, 22-receiver array structure and operated in the 77 GHz band. During a single frame acquisition, each radar output a four-dimensional point cloud of at least 100 points, with data fields including three-dimensional coordinates. radial velocity Signal-to-noise ratio and timestamp;

[0056] After data acquisition is complete, a unified clock reference is sent to each radar via the reference clock distribution module to ensure consistent sampling time. Subsequently, the round-trip delay is obtained using link round-trip messages. Deducting equipment processing delay The one-way propagation delay is obtained. This is used to correct time deviations across radars;

[0057] In the frequency and phase correction stage, the carrier frequency offset of each radar output is estimated. Phase difference from the initial phase and combined with sampling interval Phase compensation is performed on the echo signal to obtain the corrected point cloud data. The relevant compensation formula has been disclosed in the system embodiment and is applied directly here.

[0058] After the above processing, the point clouds of all radars have a unified time and phase reference, providing input for subsequent registration;

[0059] Unlike existing solutions that only synchronize timestamps, this solution completes both time and phase correction during the acquisition phase, avoiding splicing errors caused by phase drift in the subsequent registration process.

[0060] Preferably,

[0061] The specific steps of point cloud registration and consistency correction are as follows:

[0062] The input is a multi-radar point cloud sequence that has been corrected for time and phase. First, in the spatially overlapping regions, candidate point pairs are extracted using the Iterative Closest Point (ICP) method. );

[0063] Subsequently, during the registration process, both geometric consistency constraints and Doppler coherence consistency constraints are applied simultaneously. The geometric part ensures the alignment of points in spatial location; the Doppler part ensures the consistency of velocity information of corresponding points. This joint optimization model, in the system embodiment, has been implemented with an objective function... Since the form is publicly available, this model is directly used for solving the problem, and the rotation matrix is ​​output. With translation vector Complete the alignment of radar point clouds in a unified coordinate system;

[0064] During the iteration process, the Doppler spectrum peak shift and phase continuity conditions are also introduced to eliminate point pairs that do not meet the velocity and phase consistency requirements, thereby further improving the registration accuracy and finally obtaining a high-density point cloud set as the input for the attitude recognition step.

[0065] Traditional ICP algorithms rely solely on geometric features, making them prone to mismatches in fast-moving scenarios. This approach incorporates velocity and phase information into the registration constraints, significantly enhancing stability in dynamic scenes.

[0066] Preferably,

[0067] The specific steps of posture recognition and daily routine modeling are as follows:

[0068] Upon receiving a high-density point cloud set in a unified coordinate system, a time-ordered sequence of distance-Doppler heatmaps is first generated using a two-dimensional Fourier transform. ;

[0069] The heatmap is then fed into a convolutional neural network (CNN) to extract spatial features. Then input the data into a Long Short-Term Memory (LSTM) network to model the time series features and obtain the hidden states. Finally, the pose label is output through the classifier. The categories include standing, walking, falling, sitting, and lying down.

[0070] In acquiring attitude sequences Subsequently, the method further establishes an integrated work-rest pattern model at three time scales: daily, weekly, and monthly. Taking the daily scale as an example, it calculates the length of the movement trajectory. Average walking speed and length of stay And based on historical window periods Forming individual baseline means with standard deviation If a certain indicator meets the requirements for three consecutive days If this occurs, an alarm process is triggered, including detection confirmation, calling for help and secondary alarm, and establishing a two-way voice call if necessary;

[0071] Existing technologies mostly rely on single radar heat maps for identification, and alarm judgment is based on a single-day threshold, which is easily affected by occasional behavior. This solution, based on a unified heat map input from multiple radars, combines CNN+LSTM spatiotemporal modeling and a multi-day consistency judgment mechanism, which significantly reduces the false alarm rate and enables long-term health monitoring.

[0072] The beneficial effects of this invention are as follows:

[0073] 1. This invention deploys multiple millimeter-wave radar units in the monitoring area to achieve a preset density of point cloud data collected per frame. A unified clock distribution and phase compensation process is then introduced to ensure consistent temporal and phase outputs across all radars. Subsequently, a joint registration method combining geometric and Doppler constraints is employed in the point cloud stitching stage. Point pair relationships are established using an iterative nearest-point method, and the results are corrected by considering spectral peak shift and phase continuity. This process ultimately forms a unified high-density point cloud set that maintains consistency in spatial location and dynamic velocity information, effectively avoiding mismatches caused by cross-device drift and providing reliable and more consistent data input for subsequent attitude recognition steps.

[0074] 2. This invention utilizes the aforementioned high-density point cloud set as input in the pose recognition stage, transforming it into a time-sequentially arranged distance-Doppler heatmap, and establishing a recognition model composed of a convolutional neural network and a long short-term memory network. The convolutional network is used to extract spatial features from the heatmap, while the long short-term memory network is used to characterize the dynamic changes between consecutive frames. The combination of the two can balance the recognition accuracy of static poses and dynamic actions. With the input of the high-density point cloud set, the model can still maintain stable output when there are rapid pose changes or local occlusion, avoiding misjudgments caused by sparse radar point clouds or too many noise points, and achieving accurate classification of various poses such as standing, walking, falling, sitting, and lying down.

[0075] 3. This invention, through the sequence output by posture recognition in the health analysis stage, statistically analyzes the length of daily movement trajectories, average walking speed, and dwell time in specific areas. Based on this, it establishes a multi-scale daily, weekly, and monthly work-rest pattern model. Since the posture recognition results come from a unified high-density point cloud set, its statistical indicators have higher consistency and reliability. When modeling work-rest patterns, the system uses individualized historical mean and standard deviation as references. When a certain indicator deviates from the baseline range for three consecutive days, it sequentially triggers detection confirmation, emergency call, and secondary alarm, and establishes a voice call function when necessary. Through this multi-day statistical approach combined with individual benchmarks, the system can effectively distinguish between occasional abnormalities and persistent abnormalities, reduce the probability of false alarms, more realistically reflect long-term health status, and meet the continuous monitoring needs of home care and elderly care scenarios. Attached Figure Description

[0076] Figure 1 This is a structural diagram of the human activity state analysis system based on multi-radar data stitching of the present invention;

[0077] Figure 2 This is a flowchart of the human activity state analysis method based on multi-radar data stitching of the present invention. Detailed Implementation

[0078] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0079] like Figures 1 to 2 As shown, this embodiment of the invention provides a human activity state analysis system based on multi-radar data stitching. The system includes: multiple FMCW+MIMO millimeter-wave radars, each equipped with an antenna array of 24 transmitters and 22 receivers, capable of collecting four-dimensional point cloud data of no less than 100 points in a single frame output.

[0080] Unified clock and phase compensation unit; used to distribute reference clocks among multiple radars and correct frequency offset, delay and phase drift in real time to ensure consistency of point clouds across devices in time and phase;

[0081] The original point cloud joint registration module is used to perform registration on the corrected point cloud in a unified coordinate system. The registration satisfies both the geometric consistency and Doppler coherence consistency of the spatially overlapping regions, thereby generating a high-density point cloud set.

[0082] The posture recognition module is used to convert the high-density point cloud set into a time-ordered distance-Doppler heat map and input it into a convolutional neural network and a long short-term memory network. The convolutional neural network is used to extract spatial features, and the long short-term memory network is used to model temporal features. The combination of the two can take into account both spatial feature extraction and short-term dynamic capture, thereby improving the recognition accuracy of rapid abnormal states such as falls.

[0083] The health analysis and alarm module is used to calculate movement trajectory, average walking speed and dwell time based on posture recognition results, generate daily, weekly and monthly routine models, and trigger an alarm process in sequence when the statistical indicators deviate from the individual's historical average by ±20% or exceed twice the historical standard deviation for three consecutive days. This process includes detection confirmation, emergency call, secondary alarm and two-way voice communication.

[0084] Four millimeter-wave radar units were deployed within a residential room (approximately 5m x 6m). Each unit employed an FMCW (Frequency Modulated Continuous Wave) system and a MIMO (Multiple Transmitters Multiple Receivers) architecture, with an array of 24 transmitting elements and 22 receiving elements. The installation locations were calibrated to ensure overlapping monitoring areas covered the human activity space. Each radar collected at least 100 points of four-dimensional point cloud data per frame, recording data at each point. ),in In three-dimensional coordinates, The radial velocity is a lower limit derived from experimental calibration, ensuring that the overlapping area has at least 50 pairs of valid corresponding points during ICP registration, thus avoiding geometric constraint degradation.

[0085] Distance resolution: In the formula: The minimum resolvable distance; For the speed of light ( ); For the transmission bandwidth (taken as 2GHz),

[0086] This formula determines the point cloud density; the larger the bandwidth, the denser the point cloud distribution.

[0087] Relationship between radial velocity and Doppler frequency shift: In the formula: Radial velocity; The operating wavelength (77GHz millimeter wave corresponds to approximately 3.9mm); For Doppler frequency shift;

[0088] Ensure that each point is accompanied by velocity semantics, providing input for subsequent coherent consistency constraints;

[0089] Existing technologies often do not set a clear minimum number of points for point cloud acquisition. This solution uses a 24-transmitter, 22-receiver array plus ≥100 points per frame as the registration input condition to ensure the stability of subsequent algorithms.

[0090] The unified clock and phase compensation unit includes a reference clock distribution module, an electromagnetic link round-trip delay measurement module, and a carrier frequency deviation compensation module. Each module works together to perform time delay, frequency deviation, and phase correction on the output signals of multiple millimeter-wave radars, so that the point cloud data collected by the multiple radars are consistent in terms of time reference and phase reference.

[0091] The processing order is as follows:

[0092] Clock distribution: All radars share the same reference clock source;

[0093] Delay measurement: Send probe pulses to obtain round-trip time. Deducting processing delay One-way delay: In the formula: This is the one-way propagation delay; Round-trip time; To address system latency;

[0094] Compensating for propagation differences between different radars;

[0095] Frequency Offset and Phase Correction: Estimating Carrier Frequency Offset and initial phase difference Correction formula: In the formula: For sampling points; For sampling after correction; The sampling interval; For sampling index; For frequency offset; Phase difference; The imaginary unit;

[0096] Ensure that different radar sampling point clouds have a unified time and phase reference;

[0097] Unlike common methods that only synchronize timestamps, this solution performs dual time and phase corrections before registration to ensure the coherence of point cloud fusion.

[0098] The original point cloud joint registration unit performs coordinate matching on point cloud data collected by multiple millimeter-wave radars in the spatially overlapping area during registration, and performs Doppler frequency and phase correction on the velocity information of the point cloud to ensure that the spliced ​​point cloud data maintains consistency in spatial morphology and dynamic motion characteristics, thereby forming a high-density point cloud set that can be processed by the attitude recognition unit.

[0099] First, candidate point pairs are extracted in the spatially overlapping region using the Iterative Closest Point (ICP) algorithm. Subsequently, geometric consistency and Doppler consistency constraints are introduced to form a joint optimization objective function:

[0100] Registration objective function

[0101] ,

[0102] In the formula: Source radar point; Target radar point; : Three-dimensional rotation matrix; Translation vector; Doppler frequency shift at corresponding points between the two radars; Geometric and Doppler weights;

[0103] By minimizing Obtain the optimal registration parameters The results are used to verify phase continuity, eliminate outlier point pairs, and finally output a high-density point cloud set, which is unified under the same coordinate system and time reference to provide input for attitude recognition.

[0104] The posture recognition unit receives a high-density point cloud set output by the original point cloud joint registration unit, converts it into a distance-Doppler heat map arranged in time order, and inputs the heat map into a convolutional neural network to extract spatial features and a long short-term memory network to model temporal features, thereby outputting human posture categories such as standing, walking, falling, sitting and lying down, providing input data for the health analysis and alarm unit.

[0105] Point cloud data is transformed using a 2D Fourier transform to generate a distance-Doppler heatmap: ,

[0106] In the formula: : No. Frame heatmap; Complex sampled signal; : Number of snapshots; : Pulse count; Distance index; Speed ​​index;

[0107] CNN extracts spatial features ;

[0108] LSTM modeling time characteristics: ;

[0109] Classify and output attitude labels ,

[0110] In the formula: Output features for CNN; This is the hidden state of the LSTM; The result is the pose recognition result;

[0111] CNN captures spatial patterns, LSTM captures temporal changes, and the combination is used to identify dynamic activities such as falls.

[0112] The health analysis and alarm module receives human posture category data output by the posture recognition unit, performs statistical processing on the data to obtain movement trajectory, average walking speed and dwell time, and establishes a daily, weekly and monthly work-rest pattern model based on this. When any statistical indicator in the model deviates from the individual's historical mean by ±20% or exceeds twice the individual's historical standard deviation within three consecutive days, the health analysis and alarm unit sequentially executes the alarm process of detection confirmation, emergency call, secondary alarm and two-way voice communication.

[0113] With attitude sequence and location information For input:

[0114] Movement trajectory: ,

[0115] Average walking speed: , ,

[0116] In the formula: Position in a unified coordinate system; : Number of valid frames on the day; : Step speed at time t; Doppler frequency shift at time t; Millimeter wave wavelength; Frame time interval;

[0117] In the startup window Establish individual baselines within 30 days:

[0118] , ,

[0119] in For the first Daily index values;

[0120] The judgment rule is: if the daily indicator satisfy: ,

[0121] If the alarm occurs for three consecutive days, an alarm will be triggered. The alarm process is divided into: detection and confirmation, calling for help, secondary alarm, and two-way voice communication. The duration of each stage is given in the example, for example, confirmation wait is 60 seconds, and the interval for secondary alarm is 120 seconds.

[0122] A method for analyzing human activity status by stitching together multi-radar data is disclosed. This method is based on the aforementioned system, and its specific steps are as follows:

[0123] Radar acquisition and reference correction: Multiple millimeter-wave radars are deployed in the monitoring area. Each radar acquires no less than one hundred points of point cloud data in a single frame. The output data is corrected through reference clock distribution and phase compensation steps to ensure consistency in time and phase.

[0124] Point cloud registration and consistency correction: Joint registration is performed on the corrected point cloud in a unified coordinate system. The registration satisfies both geometric consistency and Doppler coherence consistency to generate a high-density point cloud set. The joint registration uses the iterative nearest point method to match the point clouds in the overlapping region and corrects the result by combining the Doppler spectral peak shift and phase continuity.

[0125] Posture recognition and daily routine modeling: The high-density point cloud set is converted into a time-ordered distance-Doppler heat map and input into a convolutional neural network and a long short-term memory network for recognition. The convolutional neural network is used for spatial feature extraction, and the long short-term memory network is used for temporal feature modeling, outputting human posture categories. Based on the posture sequence, daily, weekly, and monthly daily routine models are established. When the index is detected to deviate from the individual's historical mean by ±20% or more than twice the standard deviation for three consecutive days, a multi-level alarm process including detection confirmation, emergency call, and secondary alarm is triggered.

[0126] The specific steps of radar acquisition and benchmark correction are as follows:

[0127] Four millimeter-wave radar units were deployed within the monitoring area. Each unit adopted a 24-transmitter, 22-receiver array structure and operated in the 77 GHz band. During a single frame acquisition, each radar output a four-dimensional point cloud of at least 100 points, with data fields including three-dimensional coordinates. radial velocity Signal-to-noise ratio and timestamp;

[0128] After data acquisition is complete, a unified clock reference is sent to each radar via the reference clock distribution module to ensure consistent sampling time. Subsequently, the round-trip delay is obtained using link round-trip messages. Deducting equipment processing delay The one-way propagation delay is obtained. This is used to correct time deviations across radars;

[0129] In the frequency and phase correction stage, the carrier frequency offset of each radar output is estimated. Phase difference from the initial phase and combined with sampling interval Phase compensation is performed on the echo signal to obtain the corrected point cloud data. The relevant compensation formula has been disclosed in the system embodiment and is applied directly here.

[0130] After the above processing, the point clouds of all radars have a unified time and phase reference, providing input for subsequent registration;

[0131] Unlike existing solutions that only synchronize timestamps, this solution completes both time and phase correction during the acquisition phase, avoiding splicing errors caused by phase drift in the subsequent registration process.

[0132] The specific steps of point cloud registration and consistency correction are as follows:

[0133] The input is a multi-radar point cloud sequence that has been corrected for time and phase. First, in the spatially overlapping regions, candidate point pairs are extracted using the Iterative Closest Point (ICP) method. );

[0134] Subsequently, during the registration process, both geometric consistency constraints and Doppler coherence consistency constraints are applied simultaneously.

[0135] The geometric part ensures the alignment of points in space; the Doppler part ensures the consistency of velocity information of corresponding points. This joint optimization model has been implemented in the system embodiment with an objective function. Since the form is publicly available, this model is directly used for solving the problem, and the rotation matrix is ​​output. With translation vector Complete the alignment of radar point clouds in a unified coordinate system;

[0136] During the iteration process, the Doppler spectrum peak shift and phase continuity conditions are also introduced to eliminate point pairs that do not meet the velocity and phase consistency requirements, thereby further improving the registration accuracy and finally obtaining a high-density point cloud set as the input for the attitude recognition step.

[0137] Traditional ICP algorithms rely solely on geometric features, making them prone to mismatches in fast-moving scenarios. This approach incorporates velocity and phase information into the registration constraints, significantly enhancing stability in dynamic scenes.

[0138] The specific steps of posture recognition and daily routine modeling are as follows:

[0139] Upon receiving a high-density point cloud set in a unified coordinate system, a time-ordered sequence of distance-Doppler heatmaps is first generated using a two-dimensional Fourier transform. ;

[0140] The heatmap is then fed into a convolutional neural network (CNN) to extract spatial features. Then input the data into a Long Short-Term Memory (LSTM) network to model the time series features and obtain the hidden states. Finally, the pose label is output through the classifier. The categories include standing, walking, falling, sitting, and lying down.

[0141] In acquiring attitude sequences Subsequently, the method further establishes an integrated work-rest pattern model at three time scales: daily, weekly, and monthly. Taking the daily scale as an example, it calculates the length of the movement trajectory. Average walking speed and length of stay And based on historical window periods Forming individual baseline means with standard deviation If a certain indicator meets the requirements for three consecutive days If this occurs, an alarm process is triggered, including detection confirmation, calling for help and secondary alarm, and establishing a two-way voice call if necessary;

[0142] Existing technologies mostly rely on single radar heat maps for identification, and alarm judgment is based on a single-day threshold, which is easily affected by occasional behavior. This solution, based on a unified heat map input from multiple radars, combines CNN+LSTM spatiotemporal modeling and a multi-day consistency judgment mechanism, which significantly reduces the false alarm rate and enables long-term health monitoring.

[0143] It should be noted that, in this document, relational terms such as "first" and "second" are used only to distinguish one entity or operation from another, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Furthermore, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such process, method, article, or apparatus.

[0144] Although embodiments of the invention have been shown and described, it will be understood by those skilled in the art that various changes, modifications, substitutions and alterations can be made to these embodiments without departing from the principles and spirit of the invention, the scope of which is defined by the appended claims and their equivalents.

Claims

1. A human activity state analysis system based on multi-radar data stitching, characterized in that: The system includes: multiple FMCW+MIMO millimeter-wave radars, each equipped with an antenna array of 24 transmit and 22 receive, capable of acquiring no less than 100 points of four-dimensional point cloud data in a single frame output; Unified clock and phase compensation unit; used to distribute reference clocks among multiple radars and to correct frequency offset, delay and phase drift in real time; The original point cloud joint registration module is used to perform registration on the corrected point cloud in a unified coordinate system. During the registration process, the geometric consistency constraint and Doppler coherence consistency constraint of the spatial overlapping region are satisfied at the same time. After the matching and correction are completed, a unified high-density point cloud set is generated. This set is consistent in spatial morphology and dynamic features, providing input data for subsequent attitude recognition. The pose recognition module is used to convert the high-density point cloud set into a time-ordered distance-Doppler heat map and input it into a convolutional neural network and a long short-term memory network. The convolutional neural network is used to extract spatial features, and the long short-term memory network is used to model temporal features. The combination of the two can take into account both spatial feature extraction and short-term dynamic capture. The health analysis and alarm module is used to calculate movement trajectory, average walking speed and dwell time based on posture recognition results, generate daily, weekly and monthly routine models, and trigger an alarm process in sequence when the statistical indicators deviate from the individual's historical average by ±20% or exceed twice the historical standard deviation for three consecutive days. This process includes detection confirmation, emergency call, secondary alarm and two-way voice communication.

2. The human activity state analysis system based on multi-radar data stitching according to claim 1, characterized in that: The multiple millimeter-wave radars all adopt FMCW frequency-modulated continuous wave and MIMO multiple transmit multiple receive system, and are equipped with an antenna array of 24 transmit and 22 receive, which can form at least 100 points of four-dimensional point cloud data in a single frame output, thereby providing stable and high-density input data for subsequent clock synchronization, phase compensation and point cloud stitching.

3. The human activity state analysis system based on multi-radar data stitching according to claim 2, characterized in that: The unified clock and phase compensation unit includes a reference clock distribution module, an electromagnetic link round-trip delay measurement module, and a carrier frequency deviation compensation module. These modules work together to correct the time delay, frequency deviation, and phase of the output signals of multiple millimeter-wave radars, thereby ensuring that the point cloud data collected by the multiple radars are consistent in terms of time and phase references.

4. The human activity state analysis system based on multi-radar data stitching according to claim 3, characterized in that: The original point cloud joint registration module is used to register the point cloud data after correction by a unified clock and phase compensation unit in a unified coordinate system. During the registration process, the geometric consistency constraint and Doppler coherence consistency constraint of the spatially overlapping region are satisfied simultaneously. After completing point pair matching and result correction, a unified high-density point cloud set is generated. The high-density point cloud set is consistent in spatial position and dynamic velocity characteristics and serves as the input data for the attitude recognition unit.

5. A human activity state analysis system based on multi-radar data stitching according to claim 4, characterized in that: The posture recognition unit receives a high-density point cloud set output by the original point cloud joint registration unit, converts it into a distance-Doppler heat map arranged in time sequence, and inputs the heat map into a convolutional neural network to extract spatial features and a long short-term memory network to model temporal features, thereby outputting human posture categories such as standing, walking, falling, sitting, and lying down, providing input data for the health analysis and alarm unit.

6. The human activity state analysis system based on multi-radar data stitching according to claim 5, characterized in that: The health analysis and alarm module receives human posture category data output by the posture recognition unit, performs statistical processing on the data to obtain movement trajectory, average walking speed, and dwell time, and establishes a daily, weekly, and monthly work-rest pattern model based on this. When any statistical indicator in the model deviates from the individual's historical mean by ±20% or exceeds twice the individual's historical standard deviation within three consecutive days, the health analysis and alarm unit sequentially executes the alarm process of detection confirmation, emergency call, secondary alarm, and two-way voice communication.

7. A method for analyzing human activity status by stitching together multi-radar data, characterized in that: The method for analyzing human activity status by stitching together multi-radar data is based on the system described in claim 6, and the specific steps of the method are as follows: Radar acquisition and reference correction: Multiple millimeter-wave radars are deployed in the monitoring area. Each radar acquires no less than one hundred points of point cloud data in a single frame. The output data is corrected through reference clock distribution and phase compensation steps to ensure consistency in time and phase. Point cloud registration and consistency correction: Joint registration is performed on the corrected point cloud in a unified coordinate system. The registration satisfies both geometric consistency and Doppler coherence consistency to generate a high-density point cloud set. The joint registration uses the iterative nearest point method to match the point clouds in the overlapping region and corrects the result by combining the Doppler spectral peak shift and phase continuity. Posture recognition and daily routine modeling: The high-density point cloud set is converted into a time-ordered distance-Doppler heat map and input into a convolutional neural network and a long short-term memory network for recognition. The convolutional neural network is used for spatial feature extraction, and the long short-term memory network is used for temporal feature modeling, outputting human posture categories. Based on the posture sequence output by the posture recognition unit, daily, weekly, and monthly daily routine models are established. When the index is detected to deviate from the individual's historical mean by ±20% or more than twice the standard deviation for three consecutive days, a multi-level alarm process including detection confirmation, emergency call, and secondary alarm is triggered.

8. The method for analyzing human activity status by stitching together multi-radar data according to claim 7, characterized in that: The specific steps of radar data acquisition and benchmark correction are as follows: Multiple millimeter-wave radars are deployed in the monitoring area. Each millimeter-wave radar outputs four-dimensional point cloud data of at least one hundred points in a single frame acquisition. The output of each radar is unified in time reference by the reference clock distribution, and the frequency offset, time delay and phase drift are corrected by the phase compensation step to ensure that the point cloud data of the multiple radars are under the same reference system in terms of time reference and phase reference.

9. The method for analyzing human activity status by stitching together multi-radar data according to claim 8, characterized in that: The specific steps of point cloud registration and consistency correction are as follows: Joint registration is performed on the reference-corrected point cloud data under a unified coordinate system. The joint registration process requires that the geometric consistency constraint and the Doppler coherence consistency constraint of the spatially overlapping region be satisfied at the same time. The overlapping region point cloud is matched by the iterative nearest point method. The matching result is corrected by combining the Doppler spectral peak shift and phase continuity to obtain a high-density point cloud data set for use in the subsequent attitude recognition step.

10. A method for analyzing human activity status by stitching together multi-radar data according to claim 9, characterized in that: The specific steps of posture recognition and daily routine modeling are as follows: The high-density point cloud set output by point cloud registration is converted into a distance-Doppler heatmap arranged in time sequence. The heatmap is then input into a convolutional neural network to extract spatial features, and then into a long short-term memory network to model temporal features, thereby outputting human posture categories. These human posture categories form a human posture sequence output by the posture recognition unit. Based on this posture sequence, a daily routine model is established at three time scales: daily, weekly, and monthly. When any statistical indicator in the model deviates from the individual's historical mean by ±20% or exceeds twice the individual's historical standard deviation within three consecutive days, a multi-level alarm process of detection confirmation, distress call, and secondary alarm is executed.

Images