Human body activity state analysis system and method based on multi-radar data splicing

Abstract

The invention belongs to the technical field of human body activity monitoring and health management, and discloses a multi-radar data spliced human body activity state analysis system and method, and the system comprises a plurality of millimeter wave radar units which are used for collecting point cloud data in a monitoring region; the unified clock and phase compensation unit is used for performing time and phase correction on each radar output; the original point cloud joint registration unit is used for executing point cloud registration in a unified coordinate system in combination with geometric and Doppler constraints to obtain a high-density point cloud set; the posture recognition unit is used for converting the point cloud set into a distance-Doppler heat map, inputting the distance-Doppler heat map into a convolutional neural network and a long-short-term memory network, and outputting a human body posture category; and the health analysis and alarm unit is used for establishing a multi-scale work and rest rule model based on the posture sequence and triggering a multi-stage alarm process when long-term abnormity of individual work and rest is detected. The method is suitable for scenes such as home care, smart old-age care and medical auxiliary monitoring.

Description

TECHNICAL FIELD

[0001] The application belongs to the technical field of human activity monitoring and health management, and specifically relates to a human activity state analysis system and method based on multi-radar data splicing. BACKGROUND

[0002] With the acceleration of population aging and the increasing demand for home health management, how to monitor the daily activities of the elderly and long-term patients in real time without relying on wearable devices and video monitoring has become an important issue in the field of smart medical care and smart aging. Millimeter wave radar is gradually introduced into human activity recognition and fall detection applications due to its advantages of non-contact, penetration of partial obstruction, and no involvement of image privacy. Compared with traditional camera-based methods, millimeter wave radar can operate stably in low-light or night environments, which is more in line with the actual application needs of homes and medical facilities.

[0003] Most of the current common human activity monitoring methods rely on a single millimeter wave radar. Such methods usually generate a range-Doppler heat map by processing the echo signal, and then recognize human poses by combining feature extraction or classification algorithms. However, due to the limited coverage of a single radar, the number of point clouds collected is insufficient, and when there are obstructions or multi-target scenarios in the monitoring area, problems such as data sparsity and decreased recognition accuracy may occur. At the same time, a single radar may be affected by clock drift and phase instability during long-term operation, resulting in a lack of unified reference for data in different time periods, which further reduces the reliability of pose recognition.

[0004] To improve coverage and recognition rate, some solutions attempt to use multiple radar deployments and perform data fusion at the result level, but such methods often only perform simple merging at the target trajectory level, lack deep splicing of original point cloud data, and fail to fully utilize the information gain brought by multi-radar collaboration. In addition, existing registration methods are mostly based on geometric constraints and fail to incorporate Doppler velocity information specific to radars, which may result in inconsistent velocity characteristics despite geometric position alignment in dynamic scenarios, thereby affecting the accuracy of pose judgment. Therefore, existing multi-radar methods often struggle to form a high-density point cloud set that maintains uniformity in both space and dynamic characteristics, which is an important bottleneck limiting the performance of multi-radar collaborative monitoring.

[0005] In terms of health monitoring, existing systems mostly rely on single-day thresholds, such as triggering an alarm when daily activity is below a preset level. This approach is sensitive to occasional behaviors and prone to false positives, making it difficult to reflect individual long-term work habits. For scenarios requiring continuous monitoring, a single or short-time anomaly is insufficient to support scientific health management and medical assistance judgment. SUMMARY

[0006] The present application aims to provide a multi-radar data splicing human activity state analysis system and method to solve the problems in the background art.

[0007] To achieve the above-mentioned purpose, the present application provides the following technical scheme: a multi-radar data splicing human activity state analysis system, comprising: a plurality of FMCW+MIMO millimeter wave radars, each having a twenty-four-transmit-twenty-two-receive antenna array, capable of collecting not less than one hundred points of four-dimensional point cloud data in a single frame output; a unified clock and phase compensation unit for distributing reference clocks among the plurality of radars and correcting frequency offset, time delay and phase drift in real time to ensure the consistency of cross-device point clouds in time and phase; a raw point cloud joint registration module for performing registration on the corrected point clouds in a unified coordinate system, the registration simultaneously satisfying the geometric consistency constraint and the Doppler coherence consistency constraint of the spatial overlap area during processing, and generating a unified high-density point cloud set after matching and correction, which maintains consistency in spatial form and dynamic characteristics, providing input data for subsequent posture recognition; a posture recognition module for converting the high-density point cloud set into a time-ordered range-Doppler heat map and inputting it into a convolutional neural network and a long short-term memory network, the convolutional neural network being used to extract spatial features, the long short-term memory network being used to model time features, both of which can take into account spatial feature extraction and short-time dynamic capture, thereby improving the recognition accuracy of rapid abnormal states such as falls; a health analysis and alarm module for calculating motion trajectory, average step speed and residence time, generating daily, weekly and monthly work and rest regularity models, and triggering the alarm process of detection confirmation, help, secondary alarm and two-way voice call in sequence when detecting that the statistical indicators deviate from the individual historical mean ± 20% range or exceed twice the historical standard deviation for three consecutive days.

[0008] Preferably, four millimeter wave radar units are arranged in a residential room (about 5m x 6m), each unit using FMCW (frequency-modulated continuous wave) system and MIMO (multiple transmit and multiple receive) architecture, with an array specification of 24 transmitting elements and 22 receiving elements, and the installation position is calibrated to ensure that the overlapping monitoring area covers the human activity space; each radar collects not less than 100 points of four-dimensional point cloud in a single frame, with a single point record , wherein is a three-dimensional coordinate, is a radial velocity, and the lower limit value is obtained from experimental calibration to ensure that there are at least 50 pairs of effective corresponding points in the overlapping area during ICP registration, avoiding degradation of geometric constraints; the distance resolution is: , wherein: is the minimum resolvable distance; c is the speed of light ( ) ; B is the transmit bandwidth (take 2GHz), The formula determines the point cloud density, the larger the bandwidth, the denser the point cloud distribution; Radial velocity and Doppler shift relationship: , wherein: is the radial velocity; is the operating wavelength (77GHz millimeter wave corresponds to about 3.9mm) ; is the Doppler shift; Ensure that each point is accompanied by a speed semantics, which provides input for subsequent coherence constraints; The prior art does not set a clear lower limit for point cloud collection, and the present application sets 24 22 receiving arrays + single frame ≥100 points as the registration input condition, which ensures the stability of the subsequent algorithm.

[0009] Preferably, the unified clock and phase compensation unit comprises a reference clock distribution module, an electromagnetic link round-trip delay measurement module, and a carrier frequency offset compensation module, each module works together to correct the delay, frequency offset and phase of the output signal of multiple millimeter wave radars, so that the point cloud data collected by the multiple radars remains consistent in time reference and phase reference; The processing sequence is as follows: Clock distribution: all radars share the same reference clock source; Delay measurement: send a probe pulse to get the round-trip delay Subtract the processing delay One-way delay: , wherein: is the one-way propagation delay; is the round-trip time; is the system processing delay; Compensate for the propagation difference between different radars; Frequency offset and phase correction: estimate the carrier frequency offset And the initial phase difference Correction formula: , wherein: is the sampling point; is the corrected sample; is the sampling interval; is the sampling index; is the frequency offset; is the phase difference; is the imaginary unit; Ensure that the point clouds sampled by different radars have a unified time and phase reference; Unlike the common method of only synchronizing the timestamp, the present application implements time and phase double correction before registration, ensuring the coherence of point cloud fusion.

[0010] Preferably, the original point cloud joint registration unit, when performing registration, coordinates the point cloud data collected by multiple millimeter wave radars in the spatial overlap area, and corrects the velocity information of the point cloud in Doppler frequency and phase, to ensure that the spliced point cloud data remains consistent in spatial form and dynamic motion characteristics, thereby forming a high-density point cloud set for the attitude recognition unit to process; First, candidate point pairs are extracted in the spatial overlap area by the iterative closest point (ICP) algorithm ), and then geometric consistency and Doppler consistency constraints are introduced to form a joint optimization objective function: Registration objective function , In the formula: : source radar point; : target radar point; : three-dimensional rotation matrix; : translation vector; : Doppler shift of corresponding points of two radars; : geometric and Doppler weight; By minimizing , the optimal registration parameters are obtained, and the phase continuity is checked in the result, and the abnormal point pairs are removed, and finally a high-density point cloud set is output, which is unified in the same coordinate system and time reference, and provides input for attitude recognition.

[0011] Preferably, the attitude recognition unit receives the high-density point cloud set output by the original point cloud joint registration unit, converts it into a time-sequentially arranged range-Doppler heat map, and inputs the heat map into a convolutional neural network in turn to extract spatial features and a long short-term memory network to model time features, thereby outputting human posture categories such as standing, walking, falling, sitting, and lying, and providing input data for the health analysis and alarm unit; The point cloud data is subjected to two-dimensional Fourier transform to generate a range-Doppler heat map: , In the formula: : the first frame heat map; : complex sampling signal; : number of snapshots; : number of pulses; : distance index; : velocity index; CNN extracts spatial features ; LSTM models time features: ; Classifies and outputs posture labels , wherein: is the CNN output feature; is the LSTM hidden state; is the pose recognition result; CNN captures spatial patterns, LSTM captures temporal changes, combined to recognize dynamic activities such as falls.

[0012] Preferably, the health analysis and alarm module receives human body pose category data output by the pose recognition unit, statistically processes the data to obtain motion trajectory, average step speed and stay time, and establishes a work and rest regularity model of three time scales of day, week and month on this basis; when any statistical index in the model deviates from the individual historical mean ± 20% range for three consecutive days, or more than twice the individual historical standard deviation, the health analysis and alarm unit executes the alarm process of detection confirmation, help, secondary alarm and two-way voice call in turn; with pose sequence and position information as input: motion trajectory: , average step speed: , , wherein: : position in the unified coordinate system; : valid frame number of the day; : step speed at time t; : Doppler shift at time t; : millimeter wave wavelength; : frame time interval; in the starting window establish individual baseline within 30 days (take 30 days): , , wherein is the index value of the th day; The determination rule is: if the daily index satisfies: , and appears for three consecutive days, an alarm is triggered, and the alarm process is divided into: detection confirmation, help, secondary alarm, two-way voice call, the time length of each stage is given in the embodiment, for example, confirmation waiting for 60s, secondary alarm interval 120s.

[0013] A multi-radar data splicing human activity state analysis method, which is based on the above system, and the specific steps of the method are as follows: Radar acquisition and reference correction: Multiple millimeter-wave radars are deployed in the monitoring area. Each radar acquires no less than one hundred points of point cloud data in a single frame. The output data is corrected through reference clock distribution and phase compensation steps to ensure consistency in time and phase. Point cloud registration and consistency correction: Joint registration is performed on the corrected point cloud in a unified coordinate system. The registration satisfies both geometric consistency and Doppler coherence consistency to generate a high-density point cloud set. The joint registration uses the iterative nearest point method to match the point clouds in the overlapping region and corrects the result by combining the Doppler spectral peak shift and phase continuity. Posture recognition and daily routine modeling: The high-density point cloud set is converted into a time-ordered distance-Doppler heat map and input into a convolutional neural network and a long short-term memory network for recognition. The convolutional neural network is used for spatial feature extraction, and the long short-term memory network is used for temporal feature modeling, outputting human posture categories. Based on the posture sequence output by the posture recognition unit, daily, weekly, and monthly daily routine models are established. When the index is detected to deviate from the individual's historical mean by ±20% or more than twice the standard deviation for three consecutive days, a multi-level alarm process including detection confirmation, emergency call, and secondary alarm is triggered.

[0014] Preferably, the specific steps of radar acquisition and reference correction are as follows: Four millimeter-wave radar units were deployed within the monitoring area. Each unit adopted a 24-transmitter, 22-receiver array structure and operated in the 77 GHz band. During a single frame acquisition, each radar output a four-dimensional point cloud of at least 100 points, with data fields including three-dimensional coordinates. radial velocity Signal-to-noise ratio and timestamp; After data acquisition is complete, a unified clock reference is sent to each radar via the reference clock distribution module to ensure consistent sampling time. Subsequently, the round-trip delay is obtained using link round-trip messages. After deducting equipment processing delay The one-way propagation delay is obtained. This is used to correct time deviations across radars; In the frequency and phase correction stage, the carrier frequency offset of each radar output is estimated. Phase difference from the initial phase and combined with sampling interval Phase compensation is performed on the echo signal to obtain the corrected point cloud data. The relevant compensation formula has been disclosed in the system embodiment and is applied directly here. After the above processing, the point clouds of all radars have a unified time and phase reference, providing input for subsequent registration; Unlike existing solutions that only synchronize timestamps, this solution completes both time and phase correction during the acquisition phase, avoiding splicing errors caused by phase drift in the subsequent registration process.

[0015] Preferably, The specific steps of point cloud registration and consistency correction are as follows: The input is a multi-radar point cloud sequence that has been corrected for time and phase. First, in the spatially overlapping regions, candidate point pairs are extracted using the Iterative Closest Point (ICP) method. ); Subsequently, during the registration process, both geometric consistency constraints and Doppler coherence consistency constraints are applied simultaneously. The geometric part ensures the alignment of points in spatial location; the Doppler part ensures the consistency of velocity information of corresponding points. This joint optimization model, in the system embodiment, has been implemented with an objective function... Since the form is publicly available, this model is directly used for solving the problem, and the rotation matrix is ​​output. With translation vector Complete the alignment of radar point clouds in a unified coordinate system; During the iteration process, the Doppler spectrum peak shift and phase continuity conditions are also introduced to eliminate point pairs that do not meet the velocity and phase consistency requirements, thereby further improving the registration accuracy and finally obtaining a high-density point cloud set as the input for the attitude recognition step. Traditional ICP algorithms rely solely on geometric features, making them prone to mismatches in fast-moving scenarios. This approach incorporates velocity and phase information into the registration constraints, significantly enhancing stability in dynamic scenes.

[0016] Preferably, The specific steps of posture recognition and daily routine modeling are as follows: Upon receiving a high-density point cloud set in a unified coordinate system, a time-ordered sequence of distance-Doppler heatmaps is first generated using a two-dimensional Fourier transform. ; The heatmap is then fed into a convolutional neural network (CNN) to extract spatial features. Then input the data into a Long Short-Term Memory (LSTM) network to model the time series features and obtain the hidden states. Finally, the pose label is output through the classifier. The categories include standing, walking, falling, sitting, and lying down. In acquiring attitude sequences Subsequently, the method further establishes an integrated work-rest pattern model at three time scales: daily, weekly, and monthly. Taking the daily scale as an example, it calculates the length of the movement trajectory. Average walking speed and length of stay And based on historical window periods Forming individual baseline means with standard deviation If a certain index meets for three consecutive days, the alarm process is triggered, including detection confirmation, help and secondary alarm, and if necessary, two-way voice call is established; The prior art relies on single radar heat map for recognition, and alarm judgment is based on single-day threshold, which is easily disturbed by occasional behavior, and the scheme significantly reduces the false alarm rate based on the input of multiple radar unified heat maps, combined with CNN+LSTM spatiotemporal modeling and multi-day consistency determination mechanism, and realizes long-term health monitoring.

[0017] The beneficial effects of the present application are as follows: 1、The present application arranges multiple millimeter wave radar units in the monitoring area, so that the number of single-frame acquisition point clouds reaches the preset density, and on this basis, introduces a unified clock distribution and phase compensation link to ensure the consistency of the output of each radar in time and phase, then adopts a joint registration method combining geometric constraint and Doppler constraint in the point cloud splicing link, and uses the iterative closest point method to establish point pair relationship, and then combines spectrum peak displacement and phase continuity to correct the result, the processing process finally forms a unified high-density point cloud set, which maintains consistency in spatial position and dynamic speed information, effectively avoids mismatch caused by cross-device drift, and provides reliable and more consistent data input for the subsequent posture recognition step.

[0018] 2、The present application uses the above high-density point cloud set as input in the posture recognition link, converts it into a distance-Doppler heat map arranged in time sequence, and establishes an identification model composed of a convolutional neural network and a long short-term memory network, the convolutional network is used to extract spatial features in the heat map, and the long short-term memory network is used to describe the dynamic change between consecutive frames, the combination of the two can take into account the recognition accuracy of static posture and dynamic action, with the input of high-density point cloud set, the model can still maintain stable output when the posture changes quickly or is partially blocked, avoiding misjudgment caused by single radar point cloud sparseness or too many noise points, and realizing accurate classification of standing, walking, falling, sitting and lying and other postures.

[0019] 3、The application outputs a sequence based on posture recognition in the health analysis link, and counts the length of daily motion trajectory, the average walking speed and the time length of staying in a specific area, and establishes a daily, weekly and monthly multi-scale work and rest regularity model, since the posture recognition result comes from the unified high-density point cloud set, the statistical indicators have higher consistency and reliability, in the work and rest modeling, the system uses individual historical mean and standard deviation as a reference, when a certain indicator deviates from the baseline range for three consecutive days, detection confirmation, help and secondary alarm are triggered in turn, and voice call function is established when necessary, through the multi-day statistics combined with individual benchmark, the system can effectively distinguish between incidental abnormalities and persistent abnormalities, reduce the false alarm probability, more truly reflect the long-term health status, and meet the continuous monitoring needs of home care and old-age care scenes. BRIEF DESCRIPTION OF DRAWINGS

[0020] Fig. 1 A multi-radar data splicing human activity state analysis system structure diagram of the application; Fig. 2 A multi-radar data splicing human activity state analysis method flow chart of the application. DETAILED DESCRIPTION

[0021] The technical solutions in the embodiments of the application will be clearly and completely described below with reference to the drawings in the embodiments of the application. Obviously, the described embodiments are only part of the embodiments of the application, not all the embodiments. Based on the embodiments in the application, all other embodiments obtained by those skilled in the art without creative labor are within the scope of protection of the application.

[0022] As shown in Figs. 1-2 The embodiments of the application provide a multi-radar data splicing human activity state analysis system, which comprises: a plurality of FMCW+MIMO millimeter wave radars, each having a twenty-four-transmit-twenty-two-receive antenna array, capable of collecting four-dimensional point cloud data of not less than one hundred points in a single frame output; A unified clock and phase compensation unit is used to distribute reference clocks among the plurality of radars, and to correct frequency offset, time delay and phase drift in real time, so as to ensure the consistency of cross-device point clouds in time and phase; An original point cloud joint registration module is used to perform registration on the corrected point clouds in a unified coordinate system, and the registration satisfies the geometric consistency and Doppler coherence consistency of the spatial overlapping area, thereby generating a high-density point cloud set; The posture recognition module is configured to convert the high-density point cloud set into a time-ordered distance-Doppler heat map, and input the heat map into a convolutional neural network and a long short-term memory network. The convolutional neural network is configured to extract spatial features, and the long short-term memory network is configured to model time features. The combination of the two can take into account spatial feature extraction and short-time dynamic capture, thereby improving the recognition accuracy of rapid abnormal states such as falls. The health analysis and alarm module is configured to calculate a motion trajectory, an average walking speed, and a stay time length according to the posture recognition result, generate a daily, weekly, and monthly work and rest regularity model, and trigger, in sequence, a detection confirmation, a help-seeking, a secondary alarm, and a two-way voice call alarm process when detecting that the statistical indicators deviate from the individual historical mean ± 20% range or exceed twice the historical standard deviation for three consecutive days.

[0023] In the residential room (about 5m x 6m), four millimeter wave radar units are arranged. Each unit adopts a FMCW (frequency-modulated continuous wave) system and a MIMO (multiple-input multiple-output) architecture, with an array specification of 24 transmitting array elements and 22 receiving array elements. The installation position is calibrated to ensure that the overlapping monitoring area covers the human activity space. Each radar unit collects a four-dimensional point cloud with not less than 100 points per frame. The single-point record is , where is a three-dimensional coordinate, is a radial velocity. The lower limit value is obtained from experimental calibration, which ensures that there are at least 50 pairs of effective corresponding points in the overlapping area during ICP registration, thereby avoiding the degeneration of geometric constraints. The distance resolution is , where is the minimum resolvable distance; is the speed of light ( ); is the transmission bandwidth (2GHz is taken as an example), The formula determines the point cloud density. The greater the bandwidth, the denser the point cloud distribution. The relationship between the radial velocity and the Doppler shift is , where is the radial velocity; is the operating wavelength (about 3.9mm for a 77GHz millimeter wave); is the Doppler shift; ensures that each point is associated with a velocity semantic, thereby providing an input for subsequent coherent consistency constraints; Most existing technologies do not set a clear lower limit for point cloud collection. In this scheme, the 24-transmitting 22-receiving array + single frame ≥ 100 points are used as the registration input condition, thereby ensuring the stability of subsequent algorithms.

[0024] The unified clock and phase compensation unit comprises a reference clock distribution module, an electromagnetic link round-trip delay measurement module, and a carrier frequency offset compensation module, each module works cooperatively to correct the output signal of the plurality of millimeter wave radars in time delay, frequency offset and phase, so that the point cloud data collected by the plurality of radars is consistent in time reference and phase reference; The processing sequence is as follows: Clock distribution: all radars share the same reference clock source; Time delay measurement: send a probe pulse to obtain the round-trip time delay Subtract the processing delay One-way time delay: Wherein: is the one-way propagation time delay; is the round-trip time; is the system processing delay; Compensate for the propagation difference between different radars; Frequency offset and phase correction: estimate the carrier frequency offset And the initial phase difference Correction formula: Wherein: is the sampling point; is the corrected sampling; is the sampling interval; is the sampling index; is the frequency offset; is the phase difference; is the imaginary unit; Ensure that the point clouds of different radars have unified time and phase reference; Unlike the common method of synchronizing time stamp only, the present scheme implements time and phase double correction before registration, ensuring the coherence of point cloud fusion.

[0025] The original point cloud joint registration unit, when performing registration, matches the coordinates of the point cloud data collected by the plurality of millimeter wave radars in the spatial overlap area, and corrects the Doppler frequency and phase of the speed information of the point cloud, to ensure that the spliced point cloud data is consistent in spatial form and dynamic motion characteristics, thereby forming a high-density point cloud set for the attitude recognition unit to process; First, extract the candidate point pair (P ) in the spatial overlap area by the iterative closest point (ICP) algorithm, then introduce geometric consistency and Doppler consistency constraints to form a joint optimization objective function: Registration objective function , Wherein: : source radar point; : target radar point; : 3D rotation matrix; : translation vector; : Doppler shift of corresponding points of two radars; : geometry and Doppler weight; By minimizing Get the optimal registration parameters , and check the phase continuity in the results, eliminate abnormal point pairs, and finally output a high-density point cloud set, unified in the same coordinate system and time reference, to provide input for pose recognition.

[0026] Among them, the pose recognition unit receives the high-density point cloud set output by the original point cloud joint registration unit, converts it into a time-ordered distance-Doppler heat map, and inputs the heat map into a convolutional neural network in turn to extract spatial features and a long short-term memory network to model time features, thereby outputting human posture categories such as standing, walking, falling, sitting, and lying, to provide input data for the health analysis and alarm unit; Point cloud data is transformed into a distance-Doppler heat map by two-dimensional Fourier transform: , In the formula: : the first frame heat map; : complex sampling signal; : number of snapshots; : number of pulses; : distance index; : velocity index; CNN extracts spatial features ; LSTM models time features: ; Classify and output posture labels , In the formula: is the CNN output feature; is the LSTM hidden state; is the pose recognition result; CNN captures spatial patterns, LSTM captures temporal changes, and combines to recognize dynamic activities such as falling.

[0027] Among them, the health analysis and alarm module receives human posture category data output by the pose recognition unit, performs statistical processing on the data to obtain motion trajectory, average step speed, and dwell time, and on this basis, establishes a work and rest regularity model on three time scales of day, week, and month. When any statistical indicator in the model deviates from the individual historical mean ± 20% range for three consecutive days, or exceeds twice the individual historical standard deviation, the health analysis and alarm unit executes the alarm process of detection confirmation, help, secondary alarm, and two-way voice call in turn. with pose sequence and position information as input: motion trajectory: , average step speed: , , wherein: : position in unified coordinate system; : valid frame number per day; : step speed at time t; : Doppler shift at time t; : millimeter wave wavelength; : frame time interval; in the start window establish individual baseline within 30 days (take 30 days): , , wherein is the index value of the th day; The determination rule is: if the daily index satisfies: , and appears for three consecutive days, an alarm is triggered, and the alarm process is divided into: detection confirmation, help, secondary alarm, and two-way voice call. The time length of each stage is given in the embodiment, for example, confirmation waiting for 60s, secondary alarm interval for 120s.

[0028] A multi-radar data splicing human activity state analysis method, which is based on the above system, and the specific steps of the method are as follows: Radar acquisition and reference correction: multiple millimeter wave radars are arranged in the monitoring area, each radar has no less than one hundred points in single-frame collected point cloud data, and the output data is corrected through reference clock distribution and phase compensation steps to keep consistent in time and phase; Point cloud registration and consistency correction: the corrected point cloud is executed joint registration under the unified coordinate system, which satisfies geometric consistency and Doppler coherence consistency at the same time, to generate high-density point cloud set, wherein the joint registration adopts iterative closest point method to match the overlapping area point cloud, and combines Doppler spectrum peak displacement and phase continuity to correct the result; Posture recognition and daily routine modeling: The high-density point cloud set is converted into a time-ordered distance-Doppler heat map and input into a convolutional neural network and a long short-term memory network for recognition. The convolutional neural network is used for spatial feature extraction, and the long short-term memory network is used for temporal feature modeling, outputting human posture categories. Based on the posture sequence, daily, weekly, and monthly daily routine models are established. When the index is detected to deviate from the individual's historical mean by ±20% or more than twice the standard deviation for three consecutive days, a multi-level alarm process including detection confirmation, emergency call, and secondary alarm is triggered.

[0029] The specific steps of radar acquisition and benchmark correction are as follows: Four millimeter-wave radar units were deployed within the monitoring area. Each unit adopted a 24-transmitter, 22-receiver array structure and operated in the 77 GHz band. During a single frame acquisition, each radar output a four-dimensional point cloud of at least 100 points, with data fields including three-dimensional coordinates. radial velocity Signal-to-noise ratio and timestamp; After data acquisition is complete, a unified clock reference is sent to each radar via the reference clock distribution module to ensure consistent sampling time. Subsequently, the round-trip delay is obtained using link round-trip messages. Deducting equipment processing delay The one-way propagation delay is obtained. This is used to correct time deviations across radars; In the frequency and phase correction stage, the carrier frequency offset of each radar output is estimated. Phase difference from the initial phase and combined with sampling interval Phase compensation is performed on the echo signal to obtain the corrected point cloud data. The relevant compensation formula has been disclosed in the system embodiment and is applied directly here. After the above processing, the point clouds of all radars have a unified time and phase reference, providing input for subsequent registration; Unlike existing solutions that only synchronize timestamps, this solution completes both time and phase correction during the acquisition phase, avoiding splicing errors caused by phase drift in the subsequent registration process.

[0030] The specific steps of point cloud registration and consistency correction are as follows: The input is a multi-radar point cloud sequence that has been corrected for time and phase. First, in the spatially overlapping regions, candidate point pairs are extracted using the Iterative Closest Point (ICP) method. ); Subsequently, during the registration process, both geometric consistency constraints and Doppler coherence consistency constraints are applied simultaneously. The geometric part ensures the alignment of points in spatial position; the Doppler part ensures the velocity information of corresponding points is consistent, the joint optimization model has been disclosed in the form of objective function in the system embodiment, the model is directly used for solving here, and the rotation matrix and the translation vector are output, so as to complete the alignment of cross-radar point clouds in a unified coordinate system; In the iteration process, the Doppler spectrum peak displacement and phase continuity condition are also introduced, and the point pairs that do not satisfy the velocity and phase consistency are removed, so as to further improve the registration accuracy, and finally obtain a high-density point cloud set as the input of the posture recognition step; The traditional ICP algorithm only uses geometric features, which is easy to misalign in fast motion scenes, and the scheme introduces velocity and phase information into registration constraints, which significantly enhances the stability in dynamic scenes.

[0031] Among them, the specific steps of posture recognition and activity modeling are as follows: Receive the high-density point cloud set in the unified coordinate system, first, generate the distance-Doppler heat map sequence arranged in time sequence through two-dimensional Fourier transform ; Then, input the heat map into a convolutional neural network (CNN) to extract spatial features ; then input the long short-term memory network (LSTM) to model the time sequence features to obtain the hidden state , and finally output the posture label through the classifier , including standing, walking, falling, sitting and lying, etc.; After obtaining the posture sequence , the method further establishes a three-dimensional activity model at the day, week and month time scales. Taking the day scale as an example, the motion trajectory length , average step speed and residence time are calculated, and based on the historical window period , the individual baseline mean value and standard deviation are formed. If a certain index meets for three consecutive days, the alarm process is triggered, including detection confirmation, help and secondary alarm, and two-way voice communication is established if necessary; The prior art relies on single radar heat map for recognition, and the alarm judgment is based on single-day threshold, which is easy to be disturbed by occasional behavior. Based on the unified heat map input of multiple radars, combined with CNN+LSTM spatiotemporal modeling and multi-day consistency determination mechanism, the false alarm rate is significantly reduced, and long-term health monitoring is realized.

[0032] It is to be understood that the terminology used herein is for the purpose of describing particular embodiments only and is not intended to be limiting; it is not intended to exclude myriad other embodiments of the present application that other inventors can develop based on the same general inventive concepts embodied by the described embodiments. That is, although the present application is described in terms of particular embodiments and illustrative figures, it should be apparent that the scope of the present application is not limited to these specific embodiments.

[0033] While the embodiments of the application have been shown and described herein, it will be understood by those skilled in the art that many changes, modifications, substitutions and alterations to these embodiments can be made without departing from the principles and spirits of the application, the scope of which is defined by the appended claims and their equivalents.

Claims

1. A multi-radar data stitching human activity state analysis system, characterized by: The system comprises: a plurality of FMCW+MIMO millimeter wave radars, each having a twenty-four-transmit-twenty-two-receive antenna array, capable of collecting four-dimensional point cloud data of not less than one hundred points in a single frame output; A unified clock and phase compensation unit is used to distribute reference clocks among the plurality of radars and to correct frequency offset, time delay and phase drift in real time. An original point cloud joint registration module is used to perform registration on the corrected point cloud in a unified coordinate system, which simultaneously satisfies the geometric consistency constraint and Doppler coherence consistency constraint of the spatial overlapping area in the processing process, and generates a unified high-density point cloud set after matching and correction, which maintains consistency in spatial form and dynamic characteristics, providing input data for subsequent posture recognition. A posture recognition module is used to convert the high-density point cloud set into a time-ordered distance-Doppler heat map, and input it into a convolutional neural network and a long short-term memory network, the convolutional neural network is used to extract spatial features, and the long short-term memory network is used to model time features, both of which can take into account spatial feature extraction and short-time dynamic capture. A health analysis and alarm module is used to calculate the motion trajectory, average step speed and residence time, generate a daily, weekly and monthly work and rest model, and trigger the alarm process of detection confirmation, help, secondary alarm and two-way voice call in sequence when detecting that the statistical indicators deviate from the individual historical mean ± 20% range or exceed twice the historical standard deviation for three consecutive days.

2. A multi-radar data stitched human activity state analysis system according to claim 1, characterized in that: The plurality of millimeter wave radars all adopt FMCW frequency-modulated continuous wave and MIMO multiple-transmit-multiple-receive systems, and are configured with a twenty-four-transmit-twenty-two-receive antenna array, which can form at least one hundred points of four-dimensional point cloud data in a single frame output, thereby providing stable and high-density input data for subsequent clock synchronization, phase compensation and point cloud stitching.

3. A multi-radar data stitched human activity state analysis system according to claim 2, characterized in that: The unified clock and phase compensation unit includes a reference clock distribution module, an electromagnetic link round-trip time delay measurement module and a carrier frequency offset compensation module, which work together to correct the time delay, frequency offset and phase of the output signals of the plurality of millimeter wave radars, so that the point cloud data collected by the plurality of radars remains consistent in time reference and phase reference.

4. A multi-radar data stitched human activity state analysis system according to claim 3, characterized in that: The original point cloud joint registration module is used to register the point cloud data corrected by the unified clock and phase compensation unit in a unified coordinate system, which simultaneously satisfies the geometric consistency constraint and Doppler coherence consistency constraint of the spatial overlapping area in the processing process, and generates a unified high-density point cloud set after matching and correction, which maintains consistency in spatial position and dynamic speed characteristics as input data for the posture recognition unit.

5. A multi-radar data stitched human activity state analysis system according to claim 4, characterized in that: The posture recognition unit receives the high-density point cloud set output by the original point cloud joint registration unit, converts it into a time-ordered distance-Doppler heat map, and inputs the heat map into a convolutional neural network to extract spatial features and a long short-term memory network to model time features, thereby outputting human posture categories such as standing, walking, falling, sitting and lying, providing input data for the health analysis and alarm unit.

6. A multi-radar data stitched human activity state analysis system according to claim 5, characterized in that: The health analysis and alarm module receives human posture category data output by the posture recognition unit, statistically processes the data to obtain a motion trajectory, an average walking speed and a stay time, and establishes a work and rest regularity model of three time scales of day, week and month on this basis; when any statistical index in the model deviates from the individual historical mean ± 20% range for three consecutive days, or exceeds twice the individual historical standard deviation, the health analysis and alarm unit executes the alarm process of detection confirmation, help, secondary alarm and two-way voice call in turn.

7. A multi-radar data stitching human activity state analysis method, characterized by: The multi-radar data splicing human activity state analysis method is based on the system of claim 6, and the specific steps of the method are as follows: Radar acquisition and reference correction: multiple millimeter wave radars are arranged in the monitoring area, each radar outputs at least one hundred points of four-dimensional point cloud data in a single frame acquisition, and the output data is corrected through reference clock distribution and phase compensation steps to ensure that the point cloud data of the multiple radars is in the same reference system in terms of time reference and phase reference; Point cloud registration and consistency correction: the corrected point cloud is registered in a unified coordinate system, and the registration satisfies the geometric consistency and Doppler coherence consistency of the overlapping area at the same time to generate a high-density point cloud set, wherein the iterative closest point method is used to match the point clouds in the overlapping area, and the matching results are corrected according to the Doppler spectrum peak displacement and phase continuity; Posture recognition and work and rest modeling: the high-density point cloud set is converted into a time-ordered distance-Doppler heat map and input into a convolutional neural network and a long short-term memory network for recognition, the convolutional neural network is used for spatial feature extraction, the long short-term memory network is used for time feature modeling, and the human posture category is output; a daily, weekly and monthly work and rest regularity model is established according to the posture sequence output by the posture recognition unit, and when it is detected that the index deviates from the individual historical mean ± 20% range or exceeds twice the standard deviation for three consecutive days, a multi-level alarm process including detection confirmation, help and secondary alarm is triggered.

8. The method of claim 7, wherein: The specific steps of the radar acquisition and reference correction are as follows: Multiple millimeter wave radars are arranged in the monitoring area, each millimeter wave radar outputs at least one hundred points of four-dimensional point cloud data in a single frame acquisition; the output of each radar is unified in time reference through reference clock distribution, and the frequency offset, time delay and phase drift are corrected through the phase compensation step to ensure that the point cloud data of the multiple radars is in the same reference system in terms of time reference and phase reference.

9. The method of claim 8, wherein: The specific steps of the point cloud registration and consistency correction are as follows: The reference-corrected point cloud data is registered in a unified coordinate system, the joint registration requires that the geometric consistency constraint and the Doppler coherence consistency constraint of the overlapping area are satisfied at the same time during the processing, the overlapping area point clouds are matched through the iterative closest point method, and the matching results are corrected according to the Doppler spectrum peak displacement and phase continuity to obtain a high-density point cloud data set for subsequent posture recognition steps.

10. The method of claim 9, wherein: The specific steps of the posture recognition and work and rest modeling are as follows: The high-density point cloud set output by point cloud registration is converted into a time-sequentially arranged range-Doppler heat map, and the heat map is input into a convolutional neural network to extract spatial features, and then input into a long short-term memory network to model time features, thereby outputting a human body posture category, which forms a human body posture sequence output by a posture recognition unit, and a work and rest regularity model is established based on the posture sequence in three time scales of day, week and month, and when any statistical indicator in the model deviates from the individual historical mean ± 20% range or exceeds twice the individual historical standard deviation within three consecutive days, a multi-level alarm process of detection confirmation, distress and secondary alarm is performed.

Images