Human perception data processing method and device, electronic equipment and storage medium

Abstract

The application provides a human perception data processing method and device, electronic equipment and storage medium, and relates to the technical field of Internet of Things.The method comprises the following steps: feature extraction is performed on original perception data on a local device side to obtain structured features containing micro-motion features and spatial form distribution, and multi-modal fusion processing is performed on two types of features of continuous multiple frames to output human posture perception results, so that the original perception data is completely processed without leaving the local device, and privacy protection of the original perception data is realized; meanwhile, through comprehensive determination by fusing local micro-motion information and overall spatial form information, the perception accuracy and robustness of abnormal postures such as falling are enhanced; in addition, since only the structured features instead of the original perception data are used as the data basis for subsequent processing, the burden of data transmission and processing is greatly reduced, and the data processing efficiency and response speed of the system are improved.

Description

Technical Field

[0001] This invention relates to the field of Internet of Things (IoT) technology, and in particular to a human body perception data processing method, a human body perception data processing device, an electronic device, and a computer-readable storage medium. Background Technology

[0002] With the rapid development of smart home products, non-contact human body sensing and status monitoring technologies are showing increasingly urgent demand in application scenarios such as smart elderly care and home security. In typical smart monitoring applications, users expect indoor sensing devices to detect the presence of people, identify behavioral activities, and provide real-time monitoring and alarms for abnormal events (such as falls) without affecting the normal life of the monitored individual or requiring their active cooperation. This ensures both user safety and privacy protection. In the process of human body sensing and status monitoring, related technologies include: optical vision-based solutions using cameras and other optical sensors combined with image recognition algorithms to achieve human body detection and behavior analysis; wearable device-based solutions using motion sensors built into devices such as wristbands and watches to detect changes in human posture; and Wi-Fi channel state information-based solutions using commercial Wi-Fi devices to extract channel state information and infer human activity by analyzing amplitude and phase changes during signal propagation.

[0003] However, in the process of human body perception and status monitoring, on the one hand, the scheme based on a single type of feature or a fixed threshold is difficult to accurately distinguish between falling and similar actions such as squatting or lying down in a real home environment, resulting in frequent false alarms and insufficient accuracy of human posture perception; on the other hand, regardless of the method used, there is a risk that the original perception data may be intercepted, misused or reused, and the security of the data cannot be guaranteed. Summary of the Invention

[0004] This invention provides a data processing method, apparatus, electronic device, and computer-readable storage medium for human body perception, in order to solve or partially solve the technical problems of the inability to guarantee the security of raw perception data during human body perception and the insufficient accuracy of human posture perception.

[0005] This invention discloses a data processing method for human perception, comprising: Acquire raw sensor data; On the local device side, feature extraction is performed on the raw sensing data to obtain corresponding structured features. The structured features include a first type of features that reflect the micro-motion features of the sensed object and a second type of features that reflect the morphological distribution of the sensed object in space. Multimodal feature fusion processing is performed based on the first type of features and the second type of features from multiple consecutive frames to output the corresponding human posture perception results.

[0006] In some feasible implementations, the step of performing multimodal feature fusion processing based on the first type of features and the second type of features from multiple consecutive frames to output the corresponding human pose perception result includes: The first type of features and the second type of features from multiple consecutive frames are input into a temporal neural network model for multimodal feature fusion, and the probability distribution corresponding to each behavior category is output, including the fall category; If the probability of the fall category is greater than or equal to the first preset threshold, then the original perception data of the current frame is used as the starting frame, the distribution of the behavior category output by the temporal neural network model in the original perception data of the subsequent preset number of frames is statistically analyzed, and the target centroid height corresponding to each frame in the subsequent preset number of frames is obtained. Based on the distribution of the behavior categories and the centroid height of the target, a confidence verification process is performed, and the corresponding human posture perception result is output.

[0007] In some feasible implementations, the structured features further include target motion trajectory features. The step of inputting the first type of features and the second type of features from multiple consecutive frames into a temporal neural network model for multimodal feature fusion, and outputting the probability distribution corresponding to each behavior category, includes: The first type of features, the second type of features, and the target motion trajectory features from multiple consecutive frames are spliced ​​together in chronological order to obtain the corresponding two-dimensional tensor. The two-dimensional tensor is input into the temporal neural network model, which extracts temporal dependencies through convolutional or recurrent layers, and outputs the probability distribution of each behavior category corresponding to the temporal dependencies through fully connected layers and softmax layers.

[0008] In some feasible implementations, the confidence verification process based on the behavior category distribution and the target centroid height, and the output of the corresponding human posture perception result, includes: If the proportion of frames in the behavior category distribution within the subsequent preset number of frames that are determined to be of the fall category is greater than or equal to the second preset threshold, and the change in the target centroid height within the subsequent preset number of frames satisfies the preset fall determination condition, then it is determined that the perceived object is in a fall posture.

[0009] In some feasible implementations, after determining that the sensed object is in a falling posture, the method further includes: Generate alarm information for the sensed object; The alarm information is pushed to a preset terminal device via a communication module; The alarm information includes at least one of the following: event timestamp, event type, and location identifier.

[0010] In some feasible implementations, the raw sensing data is collected by a sensing module, and after determining that the sensed object is in a falling posture, the method further includes: The sensing module is controlled to continuously sense the object being sensed at a target acquisition frequency.

[0011] In some feasible implementations, acquiring the raw sensor data includes: Acquire the echo signal collected by the sensing module; The echo signal is transformed to obtain the corresponding range-Doppler spectrum; Valid points are extracted from the distance-Doppler spectrum to generate the corresponding raw point cloud data; The original point cloud data is clustered, and point clouds belonging to the same physical target are merged into a single perceived object. The perceived object is then tracked and filtered to output the corresponding original perception data.

[0012] In some feasible implementations, the step of extracting features from the raw sensory data to obtain corresponding structured features includes: The original point cloud data in the original sensing data is converted from polar coordinates to Cartesian space coordinates to obtain the corresponding target point cloud data. Extracting multi-dimensional features from the target point cloud data after coordinate transformation; The multi-dimensional features of multiple consecutive frames are constructed into a corresponding temporal feature sequence according to a preset sliding window; The time-series feature sequence is normalized to obtain structured features with fixed dimensions.

[0013] In some feasible implementations, the multi-dimensional features also include target motion trajectory features. Extracting multi-dimensional features from the target point cloud data after coordinate transformation includes: Extract the three-dimensional spatial position of the target centroid from the target point cloud data after coordinate transformation; Obtain the temporal position sequence of the target centroid; The displacement velocity, acceleration, first statistic of the displacement velocity, and second statistic of the acceleration of the target centroid are obtained by performing a difference operation on the time-series position sequence. The three-dimensional spatial position, the displacement velocity, the acceleration, the first statistic, and the second statistic are used as features of the target motion trajectory.

[0014] In some feasible implementations, the extraction of multi-dimensional features from the target point cloud data after coordinate transformation includes: Static background filtering is performed on the target point cloud data after coordinate transformation; Extract the micro-Doppler spectrum of the target echo from the target point cloud data after background filtering; Calculate at least one of the peak frequency, spectral width, spectral entropy, and energy concentration of the microDoppler spectrum; The peak frequency, the spectral width, the spectral entropy, and the energy concentration are taken as the first type of feature.

[0015] In some feasible implementations, the extraction of multi-dimensional features from the target point cloud data after coordinate transformation includes: Extract the vertical extension of the target point cloud from the target point cloud data after coordinate transformation; Extract the horizontal projected area of ​​the target point cloud from the target point cloud data after coordinate transformation; Principal component analysis is performed on the target point cloud data after coordinate transformation to obtain the variance distribution of the target point cloud in three orthogonal directions, and the variance distribution is used as the shape principal component of the target point cloud. The vertical extension, the horizontal projected area, and at least one of the principal components of the shape are designated as the second type of feature.

[0016] In some feasible implementations, acquiring the raw sensor data includes: In response to power-on initialization, the control sensing module collects raw sensing data at the first acquisition frequency; If an active target is detected in the target area to which the sensing module belongs based on the original sensing data, and the duration reaches a first preset duration, then the sensing module is controlled to switch from the first acquisition frequency to the second acquisition frequency to acquire the original sensing data at the second acquisition frequency. The second sampling frequency is greater than the first sampling frequency.

[0017] In some feasible implementations, after acquiring the raw sensing data at the second acquisition frequency, the method further includes: If, based on the original sensing data, it is detected that there are no active targets in the target area to which the sensing module belongs, and the duration reaches the second preset duration, then the sensing module is controlled to switch to the first acquisition frequency to acquire the original sensing data at the first acquisition frequency.

[0018] In some feasible implementations, the method further includes: The human posture perception results and / or the structured features are transmitted to the communication module as data transmitted externally by the communication module.

[0019] In some feasible implementations, the raw sensing data is collected by a sensing module, and the method further includes: Obtain the channel state parameters of the communication module; If the channel state parameters exceed the preset range, the sensing module is controlled to perform at least one of the following operations: adjusting the waveform parameters of the transmitted signal, reducing the transmission power, and adjusting the receiver gain.

[0020] In some feasible implementations, the method further includes: If the channel state parameters recover to within the preset range, then control the sensing module to restore the working parameter configuration before the adjustment operation was performed.

[0021] This invention also discloses a human body perception data processing device, comprising: The acquisition module is used to acquire raw sensor data; The feature extraction module is used to extract features from the original sensing data on the local device side to obtain corresponding structured features. The structured features include a first type of features that reflect the micro-motion features of the sensed object and a second type of features that reflect the morphological distribution of the sensed object in space. The perception processing module is used to perform multimodal feature fusion processing based on the first type of features and the second type of features from multiple consecutive frames, and output the corresponding human posture perception results.

[0022] In some feasible implementations, the perception processing module is specifically used for: The first type of features and the second type of features from multiple consecutive frames are input into a temporal neural network model for multimodal feature fusion, and the probability distribution corresponding to each behavior category is output, including the fall category; If the probability of the fall category is greater than or equal to the first preset threshold, then the original perception data of the current frame is used as the starting frame, the distribution of the behavior category output by the temporal neural network model in the original perception data of the subsequent preset number of frames is statistically analyzed, and the target centroid height corresponding to each frame in the subsequent preset number of frames is obtained. Based on the distribution of the behavior categories and the centroid height of the target, a confidence verification process is performed, and the corresponding human posture perception result is output.

[0023] In some feasible implementations, the structured features also include target motion trajectory features, and the perception processing module is specifically used for: The first type of features, the second type of features, and the target motion trajectory features from multiple consecutive frames are spliced ​​together in chronological order to obtain the corresponding two-dimensional tensor. The two-dimensional tensor is input into the temporal neural network model, which extracts temporal dependencies through convolutional or recurrent layers, and outputs the probability distribution of each behavior category corresponding to the temporal dependencies through fully connected layers and softmax layers.

[0024] In some feasible implementations, the perception processing module is specifically used for: If the proportion of frames in the behavior category distribution within the subsequent preset number of frames that are determined to be of the fall category is greater than or equal to the second preset threshold, and the change in the target centroid height within the subsequent preset number of frames satisfies the preset fall determination condition, then it is determined that the perceived object is in a fall posture.

[0025] In some feasible implementations, the apparatus further includes: An alarm module is used to generate alarm information for the sensed object after the sensing processing module determines that the sensed object is in a falling posture; A communication module is used to push the alarm information to a preset terminal device; The alarm information includes at least one of the following: event timestamp, event type, and location identifier.

[0026] In some feasible implementations, the raw sensing data is collected by a sensing module, and the device further includes: The perception control module is used to control the perception module to continuously perceive the perceived object at a target acquisition frequency after the perception processing module determines that the perceived object is in a falling posture.

[0027] In some feasible implementations, the acquisition module is specifically used for: Acquire the echo signal collected by the sensing module; The echo signal is transformed to obtain the corresponding range-Doppler spectrum; Valid points are extracted from the distance-Doppler spectrum to generate the corresponding raw point cloud data; The original point cloud data is clustered, and point clouds belonging to the same physical target are merged into a single perceived object. The perceived object is then tracked and filtered to output the corresponding original perception data.

[0028] In some feasible implementations, the feature extraction module is specifically used for: The original point cloud data in the original sensing data is converted from polar coordinates to Cartesian space coordinates to obtain the corresponding target point cloud data. Extracting multi-dimensional features from the target point cloud data after coordinate transformation; The multi-dimensional features of multiple consecutive frames are constructed into a corresponding temporal feature sequence according to a preset sliding window; The time-series feature sequence is normalized to obtain structured features with fixed dimensions.

[0029] In some feasible implementations, the multi-dimensional features also include target motion trajectory features, and the feature extraction module is specifically used for: Extract the three-dimensional spatial position of the target centroid from the target point cloud data after coordinate transformation; Obtain the temporal position sequence of the target centroid; The displacement velocity, acceleration, first statistic of the displacement velocity, and second statistic of the acceleration of the target centroid are obtained by performing a difference operation on the time-series position sequence. The three-dimensional spatial position, the displacement velocity, the acceleration, the first statistic, and the second statistic are used as features of the target motion trajectory.

[0030] In some feasible implementations, the feature extraction module is specifically used for: Static background filtering is performed on the target point cloud data after coordinate transformation; Extract the micro-Doppler spectrum of the target echo from the target point cloud data after background filtering; Calculate at least one of the peak frequency, spectral width, spectral entropy, and energy concentration of the microDoppler spectrum; The peak frequency, the spectral width, the spectral entropy, and the energy concentration are taken as the first type of feature.

[0031] In some feasible implementations, the feature extraction module is specifically used for: Extract the vertical extension of the target point cloud from the target point cloud data after coordinate transformation; Extract the horizontal projected area of ​​the target point cloud from the target point cloud data after coordinate transformation; Principal component analysis is performed on the target point cloud data after coordinate transformation to obtain the variance distribution of the target point cloud in three orthogonal directions, and the variance distribution is used as the shape principal component of the target point cloud. The vertical extension, the horizontal projected area, and at least one of the principal components of the shape are designated as the second type of feature.

[0032] In some feasible implementations, the acquisition module is specifically used for: In response to power-on initialization, the control sensing module collects raw sensing data at the first acquisition frequency; If an active target is detected in the target area to which the sensing module belongs based on the original sensing data, and the duration reaches a first preset duration, then the sensing module is controlled to switch from the first acquisition frequency to the second acquisition frequency to acquire the original sensing data at the second acquisition frequency. The second sampling frequency is greater than the first sampling frequency.

[0033] In some feasible implementations, the acquisition module is further used for: After acquiring raw sensing data at the second acquisition frequency, if the raw sensing data indicates that there is no active target in the target area to which the sensing module belongs, and the duration reaches the second preset duration, then the sensing module is controlled to switch to the first acquisition frequency to acquire raw sensing data at the first acquisition frequency.

[0034] In some feasible implementations, the apparatus further includes: The transmission module is used to transmit the human posture perception results and / or the structured features to the communication module as data transmitted externally by the communication module.

[0035] In some feasible implementations, the raw sensing data is collected by a sensing module, and the device further includes: The channel monitoring module is used to acquire the channel status parameters of the communication module; An anti-interference control module is used to control the sensing module to perform at least one of the following operations if the channel state parameters exceed a preset range: adjusting the waveform parameters of the transmitted signal, reducing the transmission power, and adjusting the receiver gain.

[0036] In some feasible implementations, the anti-interference control module is further used for: If the channel state parameters recover to within the preset range, then control the sensing module to restore the working parameter configuration before the adjustment operation was performed.

[0037] This invention also discloses an electronic device, including a processor, a communication interface, a memory, and a communication bus, wherein the processor, the communication interface, and the memory communicate with each other through the communication bus; The memory is used to store computer programs; When the processor executes a program stored in the memory, it implements the method described in the embodiments of the present invention.

[0038] This invention also discloses a computer-readable storage medium storing instructions that, when executed by one or more processors, cause the processors to perform the methods described in this invention.

[0039] The embodiments of the present invention have the following advantages: In this embodiment of the invention, raw sensing data is acquired, and then features are extracted from the raw sensing data on the local device side to obtain structured features that reflect the micro-motion features and spatial morphological distribution of the sensed object. Then, multimodal feature fusion processing is performed based on the first and second types of features in multiple consecutive frames to output the corresponding human posture perception result. Thus, by completing feature extraction on the local device side, the raw sensing data does not need to leave the local device, achieving privacy protection for the raw sensing data. This not only isolates the raw sensing data of the sensed object from the external transmission link at the physical level, but also, by extracting structured features containing micro-motion features and spatial morphological distribution features and combining them with multimodal feature fusion processing, human posture perception can make a comprehensive judgment based on the local micro-motion information and the overall spatial morphological information of the sensed object. This enhances the accuracy and robustness of perceiving abnormal postures such as falls. At the same time, since only structured features, rather than raw sensing data, are used as the data basis for subsequent processing, the burden of data transmission and processing is greatly reduced, and the data processing efficiency and response speed of the system are improved. Attached Figure Description

[0040] Figure 1 This is a flowchart of the steps of a human perception data processing method provided in an embodiment of the present invention; Figure 2 This is a schematic diagram of the system control process provided in an embodiment of the present invention; Figure 3 This is a structural block diagram of a human body perception data processing device provided in an embodiment of the present invention. Detailed Implementation

[0041] To make the above-mentioned objects, features and advantages of the present invention more apparent and understandable, the present invention will be further described in detail below with reference to the accompanying drawings and specific embodiments.

[0042] It should be noted that the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for analysis, data stored, data displayed, etc.) involved in this invention are all information and data authorized by the user or fully authorized by all parties. Furthermore, the collection, use and processing of related data must comply with the relevant laws, regulations and standards of the relevant countries and regions, and corresponding operation entry points are provided for users to choose to authorize or refuse.

[0043] As examples, in smart monitoring scenarios such as smart elderly care and home security, it is often necessary to deploy sensing devices in the indoor environment to achieve real-time monitoring of the presence of people, identification of behavioral activities, and abnormal events (such as falls) without affecting the normal life of the monitored individuals or requiring their active cooperation. However, existing technologies have the following drawbacks in achieving these functions: On the one hand, solutions based on single-type features or fixed thresholds are difficult to accurately distinguish between falls and similar actions such as squatting or lying down in a real home environment, resulting in frequent false alarms and insufficient accuracy in human posture perception; on the other hand, regardless of whether solutions based on optical vision, wearable devices, or Wi-Fi channel status information are used, the raw sensing data is at risk of being intercepted, misused, or reused, and data security cannot be guaranteed.

[0044] In this embodiment of the invention, a core mechanism is established based on extracting features from the raw sensing data on the local device side to obtain structured features containing micro-motion features and spatial morphological distribution features, and determining human posture through multimodal feature fusion processing. When the processing unit on the local device side receives the raw sensing data collected by the sensing module, the processing unit can extract features from the raw sensing data on the local device side to obtain structured features containing a first type of feature reflecting the micro-motion features of the sensed object and a second type of feature reflecting the spatial morphological distribution of the sensed object. Multimodal feature fusion processing is then performed based on the first type of feature and the second type of feature in multiple consecutive frames to output the corresponding human posture perception result. This achieves accurate perception of human posture without the raw sensing data leaving the local device, ensuring data security and improving the accuracy of posture perception.

[0045] Furthermore, in this embodiment of the invention, the raw sensing data is discarded or stored in a local cache after feature extraction on the local device side without being transmitted externally. Only the structured features or human posture perception results are output externally through the communication module. This achieves physical isolation of the raw sensing data at the data transmission level, not only eliminating the possibility of interception or misuse of sensing data in the transmission link from the source, but also significantly reducing communication bandwidth usage and transmission power consumption because the amount of transmitted data is much smaller than the raw sensing data. At the same time, by introducing a dynamic frame rate control mechanism, the sensing module runs at a first acquisition frequency to execute a simplified signal processing flow in the idle state, and automatically switches to a second acquisition frequency to execute a complete signal processing chain after detecting an active target. This achieves adaptive optimization of the system's average power consumption while ensuring high-precision sensing. Moreover, by introducing a sensing-communication collaborative anti-interference mechanism, the communication module can detect signals and then perform signal processing. When the status parameters exceed the preset range, the system sends an adjustment instruction to the sensing module. The sensing module then performs at least one operation: adjusting the waveform parameters of the transmitted signal, reducing the transmission power, and adjusting the receiver gain. This proactively adjusts the sensing parameters to avoid interference when the communication environment deteriorates, ensuring the stability of sensing performance in complex electromagnetic environments and reducing potential mutual interference between the sensing module and the communication module. Furthermore, by introducing a confidence verification mechanism based on the probability distribution output by the temporal neural network model, the system does not immediately confirm when the fall category probability exceeds a first preset threshold. Instead, it comprehensively judges the distribution of behavior categories and the temporal changes in the target centroid height within a preset number of subsequent frames. This effectively filters false alarms caused by single-frame false detections, improving the accuracy of fall detection and enhancing user trust and reliance on the system. This comprehensively improves the overall performance and practicality of human posture perception in intelligent monitoring scenarios.

[0046] Reference Figure 1 The diagram illustrates a flowchart of a human perception data processing method provided in an embodiment of the present invention, which may specifically include the following steps: Step 101: Obtain raw sensor data; In this embodiment of the invention, it can be applied to an intelligent monitoring system (hereinafter referred to as the system). The intelligent monitoring system may include a sensing module, an edge AI processing unit, and a communication module. The sensing module is deployed in an indoor environment (such as a living room, bedroom, or bathroom) to collect raw sensing data of the monitored area. The communication module is used to establish a communication link with an external network (home router, cloud server, or mobile terminal). The edge AI processing unit can be integrated into the processor of the sensing module or exist as an independent processor. It is connected to both the sensing module and the communication module to process the raw sensing data collected by the sensing module and transmit the processing results externally through the communication module.

[0047] Optionally, in the following embodiments, the edge AI processing unit is integrated into the processor of the sensing module for illustrative purposes. It is understood that the present invention is not limited thereto.

[0048] The sensing module, which can be a microwave radar sensing module, internally includes a radio frequency (RF) transceiver unit, an analog-to-digital converter (ADC), and a first processor unit. The RF transceiver unit transmits frequency-modulated continuous wave signals and receives echo signals. The ADC converts the received analog intermediate frequency (IF) signals into digital signals. The first processor unit executes underlying signal processing algorithms. An edge AI processing unit runs a lightweight artificial intelligence model, performing feature extraction and classification reasoning on the structured data output by the radar sensing module. The communication module, including a second processor unit and an RF transceiver unit, establishes a communication link with an external network. The communication module connects to the sensing module via a high-speed serial interface.

[0049] The raw sensing data can be the initial data generated by the sensing module after detecting the environment, without feature extraction. Optionally, in the implementation of the microwave radar sensing module, the raw sensing data can be represented as raw point cloud data, containing information such as the three-dimensional spatial position, radial velocity, and signal strength of multiple target points. In addition, the raw sensing data is rich in environmental spatial features and behavioral details of the monitored objects, which can serve as the basic data source for subsequent feature extraction and attitude perception.

[0050] In some feasible implementations, during the acquisition of raw sensing data, the echo signal collected by the sensing module can be acquired, and then the echo signal can be transformed to obtain the corresponding range-Doppler spectrum. Then, effective points can be extracted from the range-Doppler spectrum to generate the corresponding raw point cloud data. The raw point cloud data can then be clustered to merge point clouds belonging to the same physical target into a single sensed object. The sensed object can then be tracked and filtered to output the corresponding raw sensing data.

[0051] In its implementation, the sensing module can transmit a frequency-modulated continuous wave signal and receive the echo signal. It then performs analog-to-digital conversion on the received echo signal to obtain a digital intermediate frequency (IF) signal. The first processor unit within the sensing module can perform range-dimensional Fast Fourier Transform (FFT) and Doppler-dimensional FFT on the digital IF signal to generate a range-Doppler spectrum. Optionally, the range-Doppler spectrum can be a two-dimensional matrix, where the horizontal axis represents the range element and the vertical axis represents the Doppler frequency element. The value of each element reflects the echo energy intensity at the corresponding distance and velocity.

[0052] Next, the sensing module can use an adaptive threshold detection algorithm to extract valid target points from the range-Doppler spectrum and generate raw point cloud data. One example of this adaptive threshold detection algorithm is a constant false alarm rate (CFAR) detection algorithm, which can be implemented as follows: For each unit to be detected in the range-Doppler spectrum, a reference window is selected centered on that unit. The reference window includes a guard unit and a reference unit. The guard unit is adjacent to the unit to be detected, and its data is not included in the calculation of background statistics to prevent the target signal from leaking into the reference window and affecting background estimation. The reference unit is located outside the guard unit and is used to estimate the background noise level. By calculating the background statistics (e.g., mean or sort value) of the data in the reference unit, the background statistics are multiplied by a preset threshold factor to obtain the adaptive detection threshold. If the energy value of the unit to be detected exceeds the adaptive detection threshold, the unit is determined to be a valid target point. After traversing all units to be detected, peak aggregation processing is performed on multiple adjacent units that are determined to be valid target points. That is, adjacent valid points belonging to the same local peak region are merged, and the aggregated valid target points are output as points in the raw point cloud data.

[0053] Subsequently, the original point cloud data is clustered based on spatial proximity, grouping point clouds belonging to the same physical target into a single detection object. One example of a clustering algorithm is a density-based spatial clustering algorithm, which groups spatially adjacent point clouds into the same cluster, thereby merging multiple reflection points belonging to the same human target into a single detection object.

[0054] Furthermore, tracking filtering can be applied to each detected object to smooth its trajectory and predict its motion state. Optionally, the tracking filtering algorithm can be Kalman filtering. Kalman filtering establishes a motion state model of the detected object (including state variables such as position and velocity) and recursively estimates the trajectory of the detected object using two steps: prediction and update. Specifically, in the prediction stage, the state at the current moment can be predicted based on the state estimate from the previous moment; in the update stage, the observed value at the current moment can be weighted and fused with the predicted value to obtain the optimal state estimate. Through Kalman filtering, trajectory jitter caused by noise can be smoothed, and a reasonable estimate of the detected object's position can still be maintained even when the detected object is temporarily occluded.

[0055] After the above processing, the perception module can output structured perception data containing information such as target spatial location (e.g., three-dimensional Cartesian coordinates), radial velocity (e.g., the radial movement speed of the target relative to the radar), and point cloud distribution, which is then provided as raw perception data to the edge AI processing unit.

[0056] In some examples, it is assumed that an elderly person is located in the center of a living room, and the radar signal emitted by the sensing module is reflected back to the receiving antenna after being reflected by the person. The first processor unit performs a range-Doppler transform on the echo signal, resulting in multiple high-energy cells in the range-Doppler spectrum corresponding to the distance and velocity of the person. Accordingly, the constant false alarm rate (CFAR) detection algorithm extracts approximately 20 to 50 valid target points from the spectrum by calculating the background statistics of a reference window and multiplying them by a threshold factor (e.g., a threshold factor of 15, meaning cells with a signal-to-noise ratio exceeding 15 dB are considered target points). Then, based on spatial proximity clustering, these point clouds are grouped into a single detection object (i.e., the elderly person). Furthermore, Kalman filtering can smooth the centroid position and radial velocity of the detected object. For example, when the elderly person walks slowly, the change in centroid position between adjacent frames is smoothed by the filtering algorithm, outputting a stable position estimate (e.g., the displacement between two consecutive frames is smoothed from 0.15 meters to 0.12 meters, removing random jitter introduced by measurement noise). Finally, the perception module outputs structured data containing the elderly person's three-dimensional spatial position, radial velocity, and point cloud distribution information as raw perception data to the edge AI processing unit.

[0057] Furthermore, in some feasible implementations, the sampling frequency of the sensing module can be dynamically adjusted according to the environmental conditions to achieve an adaptive balance between power consumption and sensing performance. Optionally, in response to the system's power-on initialization, the sensing module can be controlled to collect raw sensing data at a first sampling frequency. At the first sampling frequency, if an active target is detected in the target area to which the sensing module belongs, and the duration reaches a first preset duration, the sensing module is controlled to switch from the first sampling frequency to a second sampling frequency to collect raw sensing data at the second sampling frequency. The second sampling frequency is greater than the first sampling frequency. Further, at the second sampling frequency, if no active target is detected in the target area to which the sensing module belongs, and the duration reaches a second preset duration, the sensing module is controlled to switch back to the first sampling frequency to collect raw sensing data at the first sampling frequency.

[0058] In the specific implementation, after the system is powered on and initialized, the sensing module operates at the first acquisition frequency (i.e., low frame rate mode). At the first acquisition frequency, the sensing module only performs a simplified signal processing procedure, such as performing only distance-dimensional fast Fourier transform and constant false alarm rate detection, mainly detecting whether there is a moving target in the monitoring area, without performing complete point cloud clustering and tracking filtering, in order to reduce computational power consumption. At the same time, the edge AI processing unit can comprehensively judge the detection results of multiple consecutive frames. If the judgment result is "there is an active target in the monitoring area" and the duration reaches a first preset duration (e.g., active targets are detected for more than 2 seconds), then the sensing module is controlled to switch to the second acquisition frequency (i.e., high frame rate mode).

[0059] At the second acquisition frequency, the sensing module executes the complete signal processing chain (including range-Doppler transformation, constant false alarm rate detection, point cloud clustering, and Kalman filter tracking) at a higher frame rate. The edge AI processing unit initiates full-function feature extraction and classification inference to achieve precise recognition of the behavior of the sensed object. For example, the second acquisition frequency can be 15 frames / second to 30 frames / second, while the first acquisition frequency can be 1 frame / second to 5 frames / second. Through the above dynamic frame rate control mechanism, the system operates at a low frame rate in idle state to reduce power consumption, and automatically switches to a high frame rate for precise feature extraction after detecting personnel entering, thereby optimizing the average power consumption of the system while ensuring perception accuracy.

[0060] Furthermore, after switching to the second acquisition frequency, if the edge AI processing unit determines that there is no active target in the monitoring area based on the original perception data, and the duration of this state reaches the second preset duration (e.g., no active target is detected for more than 10 seconds), then the perception module is controlled to revert from the second acquisition frequency to the first acquisition frequency to further reduce system power consumption.

[0061] Furthermore, in other feasible implementations, the system may also include a sensing-communication collaborative anti-interference mechanism. Optionally, the communication module can continuously monitor the status parameters of the current communication channel during operation. These status parameters may include at least one of channel occupancy (e.g., the proportion of time the channel is currently occupied), signal strength (e.g., Received Signal Strength Indicator (RSSI) value), and signal-to-noise ratio (e.g., the ratio of signal power to noise power). When the channel occupancy exceeds a preset threshold (e.g., channel occupancy exceeds 70%) or the signal-to-noise ratio falls below a preset threshold (e.g., signal-to-noise ratio is below 10dB), it indicates that there is strong interference in the current communication environment, and the communication module sends an adjustment instruction to the sensing module through the control signaling channel.

[0062] Accordingly, the sensing module can perform one or more of the following adaptive adjustment operations based on the adjustment instructions: adjusting the waveform parameters of the transmitted signal (e.g., adjusting at least one of the frequency modulation slope and starting frequency of the frequency-modulated continuous wave signal to avoid the interfered frequency band), reducing the transmit power (e.g., reducing the transmit power from 10dBm to 7dBm to reduce potential electromagnetic interference with the communication module), and adjusting the receiver gain setting (e.g., increasing the receiver gain from 30dB to 36dB to enhance the ability to suppress interference). When the communication module detects that the channel state has returned to normal (e.g., the channel occupancy rate drops below 50% and the signal-to-noise ratio recovers to above 15dB), it notifies the sensing module via control signaling to restore the operating parameter configuration before the adjustment operation was performed.

[0063] In some scenarios, such as multiple devices simultaneously transmitting large amounts of data in a home network (e.g., a smart TV playing 4K video or a mobile phone downloading a large file), the 2.4GHz band channel occupancy rate reaches 85%, and the signal-to-noise ratio drops to 8dB. Upon detecting this anomaly, the communication module sends an adjustment instruction to the sensing module via the control signaling channel. Based on this instruction, the sensing module adjusts the starting frequency of its FM continuous wave signal from 24.0GHz to 24.2GHz (offsetting by 200MHz to avoid interference bands) and reduces the transmit power from 10dBm to 7dBm. After video playback and file download are completed, the communication module detects that the channel occupancy rate has recovered to 30% and the signal-to-noise ratio has recovered to 18dB. It then sends a recovery instruction to the sensing module, which restores the starting frequency to 24.0GHz and the transmit power to 10dBm.

[0064] Step 102: On the local device side, feature extraction is performed on the original sensing data to obtain corresponding structured features. The structured features include a first type of feature for reflecting the micro-motion features of the sensed object and a second type of feature for reflecting the morphological distribution of the sensed object in space. In this embodiment of the invention, the raw sensing data collected by the sensing module can undergo a corresponding feature extraction process through an edge AI processing unit. During the feature extraction process, the raw sensing data does not leave the local device but is completed on the local device side, thereby ensuring data security at the architectural level.

[0065] Structured features are sets of feature vectors with fixed dimensions and structured organization extracted from raw sensory data. Compared to raw sensory data, structured features significantly reduce the amount of data (e.g., to less than one percent of the original data volume) and the original data cannot be irreversibly recovered. Therefore, they reduce the burden of subsequent data transmission and processing while preserving the key information required for posture perception while protecting privacy.

[0066] Optionally, the structured features may include a first type of feature and a second type of feature. The first type of feature can be used to reflect the micro-motion characteristics of the perceived object, while the second type of feature can be used to reflect the spatial morphological distribution of the perceived object. Micro-motion characteristics can be signal features generated by small-amplitude movements of the perceived object or its body parts (such as limbs, torso, etc.) beyond overall displacement, which can be captured by the micro-Doppler effect in radar echoes. Furthermore, micro-motion characteristics can reflect subtle movement information such as limb swinging and torso tremors, and are a key basis for distinguishing falling from similar actions such as squatting or lying down. Correspondingly, the second type of feature, namely spatial morphological distribution characteristics, can be the geometric shape and distribution characteristics of the perceived object in three-dimensional space, including its vertical extension (i.e., height distribution), horizontal projection range, and overall geometric features of its shape. These features can reflect the posture (e.g., standing, sitting, lying down) and stability of the perceived object.

[0067] In some feasible implementations, the process of extracting structured features can first convert the original point cloud data in the original perceptual data from polar coordinates to Cartesian space coordinates to obtain the corresponding target point cloud data. Then, multi-dimensional features are extracted from the target point cloud data after coordinate transformation. Then, the multi-dimensional features of multiple consecutive frames are constructed into a corresponding temporal feature sequence according to a preset sliding window. The temporal feature sequence is then normalized to obtain structured features with fixed dimensions.

[0068] In practical implementation, the raw point cloud data in the original sensing data can first be converted from polar coordinates to Cartesian space coordinates to obtain target point cloud data. The raw point cloud data can be represented in polar coordinates, where each point contains three components: radial distance, azimuth angle, and elevation angle, representing the target's distance, horizontal angle, and vertical angle relative to the radar, respectively. Through coordinate transformation, the polar coordinates are converted to three-dimensional Cartesian space coordinates, i.e., (x, y, z), where x and y represent the position on the horizontal plane, and z represents the altitude. The purpose of coordinate transformation is to convert the data from the sensor coordinate system to a unified Cartesian space coordinate system, facilitating subsequent geometric feature extraction and alignment between multiple frames of data. Next, multi-dimensional features can be extracted from the coordinate-transformed target point cloud data. These multi-dimensional features can include first-type features and second-type features. Furthermore, multi-dimensional features can also include target motion trajectory features, etc., which are not limited in this invention.

[0069] In the process of extracting the first type of features, static background filtering can be performed on the target point cloud data after coordinate transformation to eliminate clutter interference generated by stationary objects in the environment (such as walls, furniture, floors, etc.), and only the dynamic point cloud corresponding to the perceived object can be retained. Then, the micro-Doppler spectrum of the target echo can be extracted from the target point cloud data after background filtering. Then, at least one of the peak frequency, spectral width, spectral entropy and energy concentration of the micro-Doppler spectrum can be calculated. Finally, at least one of the peak frequency, spectral width, spectral entropy and energy concentration is used as the first type of feature.

[0070] In practical implementation, the micro-Doppler spectrum can be the spectral distribution obtained after time-frequency analysis of radar echo signals. When the body parts of the sensed object (such as limbs and torso) exhibit minute movements relative to the overall center of mass (e.g., arm swinging, torso trembling), these minute movements generate modulation sidebands at the main Doppler frequency in the radar echo, forming a micro-Doppler spectrum. The peak frequency corresponds to the frequency value at the point of maximum energy in the micro-Doppler spectrum, reflecting the main velocity component of the minute movement; the spectral width represents the broadening of the micro-Doppler spectrum along the frequency axis, reflecting the dispersion of the minute movement velocity; the spectral entropy measures the disorder of the energy distribution in the micro-Doppler spectrum, reflecting the complexity of the minute movement mode; and the energy concentration reflects the degree of energy concentration near the peak frequency in the micro-Doppler spectrum.

[0071] The above parameters can be used to characterize the micro-motion patterns of the perceived object from different dimensions. For example, the violent swinging of the limbs during a fall will produce micro-Doppler features with a sharp increase in spectral width and spectral entropy, while the squatting action will show features with a slow change in spectral width and a basically stable spectral entropy.

[0072] Accordingly, in the process of extracting the second type of features, the vertical extension of the target point cloud can be extracted from the coordinate-transformed target point cloud data (for example, the difference between the maximum and minimum height values ​​in the target point cloud reflects the height of the perceived object and can be used to determine whether the posture is standing, sitting, or lying down; when the perceived object is standing, the vertical extension is approximately 1.5 to 1.8 meters, when sitting, it is approximately 0.7 to 0.9 meters, and when lying down, it is approximately 0.3 to 0.5 meters); the horizontal projection area of ​​the target point cloud can be extracted (for example, the area covered by the projection of the target point cloud onto a horizontal ground plane reflects the horizontal occupancy of the perceived object; when lying down, the horizontal projection area is significantly larger than when standing); principal component analysis is performed on the target point cloud data to obtain the variance distribution of the target point cloud in three orthogonal directions. This variance distribution is used as the shape principal component of the target point cloud to determine whether the point cloud is linear, planar, or spherical, thereby assisting in determining the posture type. At least one of the vertical extension, horizontal projection area, and shape principal component is used as the second type of feature.

[0073] Furthermore, after extracting the first and second types of features, a temporal feature sequence can be constructed from the first and second types of features across multiple consecutive frames using a preset sliding window. The window length of the sliding window can be a preset number of frames (e.g., 20 frames), and the sliding step size can be one frame. Specifically, at each frame, the system organizes the feature vectors of the current frame and the previous N-1 frames (N being the window length) into a feature sequence. This sequence contains the first and second types of features from each of the N frames. Through the sliding window mechanism, the feature vectors of each frame are organized chronologically into a feature sequence containing temporal information, providing input for subsequent processing by the temporal neural network.

[0074] Finally, the time-series feature sequences are normalized to obtain structured features with fixed dimensions. Normalization methods can include Z-score normalization or Min-Max normalization. For example, using Z-score normalization, for each feature dimension, its mean and standard deviation in the time-series feature sequence are calculated. Then, the mean is subtracted from each feature value, and the result is divided by the standard deviation, ensuring that each feature dimension has zero mean and unit variance. The purpose of normalization is to eliminate dimensional differences between different features, bringing them to the same order of magnitude, which facilitates processing by subsequent neural network models.

[0075] The structured features obtained through the above processing have fixed dimensions (e.g., 256 or 512 dimensions), and their data volume is much smaller than that of the original point cloud data. Moreover, the original data cannot be recovered irreversibly, thus ensuring both transmission efficiency and privacy protection.

[0076] In some scenarios, such as an elderly person slowly squatting down from a standing position to a sitting position, the sensing module continuously collects raw sensing data at a second acquisition frequency (e.g., 20 frames / second). The edge AI processing unit extracts features from each frame of data: In the first type of feature (micro-Doppler features), the spectral width of the micro-Doppler spectrum changes from about 30Hz when standing normally to about 25Hz during the squatting process, and the spectral entropy changes slowly from 0.55 to 0.50, with a relatively gentle change range; In the second type of feature (spatial distribution morphology features), the vertical extension gradually decreases from about 1.6 meters to about 0.8 meters (the descent process lasts about 2 seconds), and the horizontal projection area increases from about 0.2 square meters to about 0.4 square meters. The shape principal component analysis results show that the point cloud gradually transitions from "linear" (standing posture, with the variance in one direction significantly greater than the other two directions) to "planar" (sitting posture, with the variance in two directions significantly greater than the third direction). The aforementioned features are organized into a temporal feature sequence by the edge AI processing unit using a sliding window of 20 frames. After normalization, a fixed-dimensional structured feature is obtained, which is then used by the subsequent temporal neural network model for pose determination. The structured feature has a feature dimension of 256, and the data volume is approximately 0.5% of the original point cloud data volume (calculated based on approximately 500 point cloud points per frame, each point containing three-dimensional coordinates and velocity information (4 floating-point numbers, each floating-point number being 4 bytes), the original point cloud data volume is approximately 500×4×4=8000 bytes, and the structured feature is 256×4=1024 bytes), achieving efficient data compression and privacy protection.

[0077] Optionally, during the extraction of multi-dimensional features, target motion trajectory features can also be extracted. Specifically, the edge AI processing unit can extract the three-dimensional spatial position of the target centroid from the target point cloud data after coordinate transformation (i.e., the centroid coordinates of the target point cloud in three-dimensional space; for example, for a target cluster containing approximately 50 point cloud points, its centroid position is the arithmetic mean of the coordinates of all point cloud points); then, it obtains the temporal position sequence of the target centroid in multiple consecutive frames, that is, it records the three-dimensional coordinates of the target centroid in each frame in frame order, forming a position sequence {p1, p2, p3, ..., p n}, where p k =(x k ,y k ,z k ) represents the three-dimensional spatial position of the target centroid in the k-th frame; then, the displacement velocity and acceleration of the target centroid are obtained by differential operation using this temporal position sequence, as well as the first statistic of displacement velocity and the second statistic of acceleration.

[0078] The differential operation can be implemented as follows: for the time interval Δt between two adjacent frames (for example, when the frame rate is 20 frames / second, Δt = 0.05 seconds), the displacement velocity vector v k =(pk -p_{k-1}) / Δt, that is, the velocity vector is obtained by dividing the difference between the centroid positions of two adjacent frames by the inter-frame time interval; the acceleration vector a k =(v k -v_{k-1}) / Δt, that is, obtaining the acceleration vector by dividing the difference between the velocity vectors of two adjacent frames by the inter-frame time interval. The first statistic can include the mean of the displacement velocity (e.g., for the sequence {v1,v2,...,v...}). n The first statistic can be calculated as at least one of the following: arithmetic mean, variance (e.g., the dispersion of velocity values ​​relative to the mean), and extreme values ​​(e.g., maximum and minimum velocities); the second statistic can include at least one of the mean, variance, and extreme values ​​of acceleration. Finally, the three-dimensional spatial position, displacement velocity, acceleration, the first statistic, and the second statistic can be used as features of the target motion trajectory.

[0079] Among these features, the characteristics of the target's motion trajectory can reflect the overall motion pattern of the perceived object. For example, a fall event exhibits a rapid descent followed by a stop on its trajectory, meaning the center of mass height decreases rapidly in a short period (vertical velocity changes rapidly from approximately 0 m / s to approximately -1.5 m / s, and vertical acceleration reaches approximately -2.0 m / s²), followed by the center of mass remaining stationary at a low position (velocity approaches 0 m / s, and acceleration approaches 0 m / s²). Walking exhibits a periodic and regular motion, with the center of mass height fluctuating periodically by a small amplitude (approximately ±0.02 m), the horizontal velocity remaining stable (approximately 0.5 m / s to 1.0 m / s), and the acceleration fluctuating slightly near zero. Running exhibits a motion pattern of increasing speed (horizontal velocity can reach over 2.0 m / s); and when stationary, it exhibits a motion pattern of unchanged position (velocity approaches 0 m / s, and acceleration approaches 0 m / s²).

[0080] By introducing target motion trajectory features, the temporal neural network model can make comprehensive judgments based on three dimensions: macroscopic motion (overall displacement and velocity changes), microscopic motion (limb micro-movements), and spatial morphology (geometric distribution) of the perceived object, further improving the accuracy and robustness of posture perception.

[0081] Accordingly, when constructing a temporal feature sequence from the multi-dimensional features of multiple consecutive frames according to a preset sliding window, the multi-dimensional features include a first type of feature, a second type of feature, and the aforementioned target motion trajectory features. Specifically, for each frame, the edge AI processing unit organizes the target motion trajectory features corresponding to that frame, along with the first and second types of features, into a multi-dimensional feature vector for that frame. Then, it constructs a temporal feature sequence from the multi-dimensional feature vectors of multiple consecutive frames according to the sliding window, so that each frame in the temporal feature sequence contains three complementary types of information: target motion trajectory features, micro-motion features, and spatial distribution morphology features.

[0082] In subsequent processing, the edge AI processing unit can concatenate the first-type features, second-type features, and target motion trajectory features from multiple consecutive frames into a two-dimensional tensor input temporal neural network model. The three types of features describe the state of the perceived object from different dimensions: the target motion trajectory features reflect the overall displacement path, velocity, and acceleration; the first-type features (micro-motion features) capture detailed information such as limb swings and trunk tremors; and the second-type features (spatial distribution morphology features) describe the spatial geometric characteristics of the posture. Through multimodal feature fusion, the system can comprehensively utilize these three complementary types of information for posture determination, further improving the accuracy and robustness of perception.

[0083] In some scenarios, such as when an elderly person slowly squats down from a standing position to a sitting position, the sensing module continuously collects raw sensing data at a second acquisition frequency (e.g., 20 frames / second). The edge AI processing unit extracts features from each frame of data: In the target motion trajectory features, the target centroid height gradually decreases from about 1.6 meters to about 0.8 meters, the vertical velocity stabilizes at about -0.3 m / s (the change is gradual and there are no drastic jumps), and the vertical acceleration fluctuates slightly around -0.2 m / s². In the first type of feature (micro-motion features), the spectral width of the micro-Doppler spectrum changes from about 30 Hz when standing normally to about 25 Hz during the squatting process, and the spectral entropy changes slowly from 0.55 to 0.50, with a relatively gradual change. In the second type of feature (spatial distribution morphology features), the vertical extension gradually decreases from about 1.6 meters to about 0.8 meters (the descent process lasts about 2 seconds), the horizontal projection area increases from about 0.2 square meters to about 0.4 square meters, and the shape principal component analysis results show that the point cloud gradually transitions from "linear" (standing posture, the variance in one direction is significantly greater than the other two directions) to "area" (sitting posture, the variance in two directions is significantly greater than the third direction). The aforementioned features are organized into a temporal feature sequence by the edge AI processing unit using a sliding window of 20 frames. After normalization, a fixed-dimensional structured feature is obtained, which is then used by the subsequent temporal neural network model for pose determination. The structured feature has a feature dimension of 256, and the data volume is approximately 0.5% of the original point cloud data volume (calculated with approximately 500 point cloud points per frame, each point containing 4 floating-point numbers for 3D coordinates and velocity information, and each floating-point number being 4 bytes, the original point cloud data volume is approximately 500×4×4=8000 bytes, and the structured feature is 256×4=1024 bytes), achieving efficient data compression and privacy protection.

[0084] In other scenarios, such as when an elderly person suddenly falls while walking, the sensing module continuously collects raw sensing data at a second acquisition frequency (e.g., 20 frames / second). The edge AI processing unit extracts features from each frame of data: In the target motion trajectory features, the target's center of mass height drops sharply from about 1.6 meters to about 0.4 meters in about 1 second, the vertical velocity changes rapidly from about 0 m / s to about -1.5 m / s, and the vertical acceleration reaches about -2.0 m / s² (exceeding the typical value of gravitational acceleration), showing a clear non-periodic rapid descent motion pattern; In the first type of feature (micro-motion features), the spectral width of the micro-Doppler spectrum expands sharply from about 50 Hz during normal walking to about 150 Hz, and the spectral entropy rises rapidly from 0.6 to 0.9, reflecting the micro-Doppler modulation generated by violent limb swinging; In the second type of feature (spatial distribution morphology features), the vertical extension decreases from about 1.6 meters to about 0.35 meters, the horizontal projection area increases from about 0.2 square meters to about 0.8 square meters, and the principal component of the shape rapidly changes from "linear" (standing posture) to "planar" (lying posture). The three types of features mentioned above are organized into a temporal feature sequence by the edge AI processing unit using a sliding window of 20 frames. After normalization, fixed-dimensional structured features are obtained for subsequent posture determination by the temporal neural network model. Based on the input of the three types of features from the above consecutive multiple frames, the temporal neural network model outputs a probability of 0.85 for the fall category, which exceeds the first preset threshold of 0.7. The system then enters the confidence verification stage. This scenario illustrates that by introducing target motion trajectory features and combining micro-motion features and spatial distribution morphology features, the system can comprehensively identify the multi-dimensional abnormal pattern of "macroscopic rapid descent + drastic micro-motion changes + abrupt spatial morphology changes" during a fall, effectively distinguishing falls from similar actions such as squatting.

[0085] Step 103: Perform multimodal feature fusion processing based on the first type of features and the second type of features from multiple consecutive frames, and output the corresponding human posture perception result.

[0086] In this embodiment of the invention, the edge AI processing unit can perform multimodal feature fusion processing based on first-type and second-type features from multiple consecutive frames to output the corresponding human posture perception result. Multimodal feature fusion processing refers to fusing multiple types of features reflecting different types of information (such as first-type features reflecting micro-motion details and second-type features reflecting spatial morphology) at the model level, fully utilizing the complementary information of various features to improve the accuracy and robustness of posture determination. The human posture perception result can be the final determination of the current posture of the perceived object, which may at least include a determination of whether the perceived object is in a falling posture.

[0087] In some feasible implementations, during multimodal feature fusion and posture perception, first-class and second-class features from multiple consecutive frames are input into a temporal neural network model for multimodal feature fusion. The model outputs a probability distribution corresponding to each behavior category, including the fall category. By obtaining the probability corresponding to the fall category, if the probability of the fall category is greater than or equal to a first preset threshold, the original perception data of the current frame is used as the starting frame. The distribution of behavior categories output by the temporal neural network model within the original perception data of the subsequent preset number of frames is statistically analyzed. The target centroid height corresponding to each frame in the subsequent preset number of frames is obtained, and confidence verification is performed based on the behavior category distribution and the target centroid height, outputting the corresponding human posture perception result.

[0088] In practical implementation, the first and second type features of multiple consecutive frames can be concatenated into a two-dimensional tensor in chronological order. For example, for an input containing 20 frames, each with a feature dimension of 256, the concatenation results in a 20×256 two-dimensional tensor. In this tensor, the rows correspond to the time dimension (i.e., frame order), and the columns correspond to the feature dimension (i.e., the values ​​of each type of feature). Through this concatenation method, each row of the feature vector contains the first and second type features of that frame, and the frames are arranged chronologically, preserving complete temporal information.

[0089] Furthermore, this two-dimensional tensor can be input into a pre-trained temporal neural network model. The temporal neural network model can employ a temporal convolutional network (TCN) or a long short-term memory network (LSTM), whose core function is to extract temporal dependencies from the input feature sequence. TCNs capture long-distance temporal dependencies through causal convolution and dilated convolution, offering advantages such as strong parallel computing power and fast training speed. LSTMs, on the other hand, control the flow of information in the time dimension through gating mechanisms (input gate, forget gate, output gate), effectively capturing long-term dependencies and avoiding the gradient vanishing problem. The temporal neural network model extracts temporal dependencies through convolutional or recurrent layers and outputs the probability distributions corresponding to each behavior category via fully connected layers and softmax layers. The fully connected layers map the high-dimensional temporal features extracted by the temporal neural network to output values ​​equal to the number of behavior categories, and the softmax layers convert these output values ​​into probability distributions, ensuring that the probability values ​​for each category are between 0 and 1, with a sum of 1.

[0090] Furthermore, the behavior categories can include at least normal activity, falling, and lying still. Accordingly, the probability distribution reflects the likelihood that the perceived object is currently in each behavior state. For example, the output {normal activity: 0.05, falling: 0.85, lying still: 0.10} indicates that the probability of the current frame being classified as falling is 85%.

[0091] In other feasible implementations, when the structured features also include target motion trajectory features, the first-class features, second-class features, and target motion trajectory features from multiple consecutive frames can be concatenated into a two-dimensional tensor input temporal neural network model. The three types of features describe the state of the perceived object from different dimensions: the target motion trajectory features reflect the overall displacement path, velocity, and acceleration; the first-class features (micro-motion features) capture detailed information such as limb swings and trunk tremors; and the second-class features (spatial distribution morphology features) describe the spatial geometric characteristics of the posture. By introducing target motion trajectory features, the temporal neural network model can simultaneously make comprehensive judgments based on the macroscopic motion (overall displacement and velocity changes), micro-motions (limb micro-motions), and spatial morphology (geometric distribution) of the perceived object, further improving the accuracy and robustness of posture perception.

[0092] In some examples, the temporal neural network model can be a temporal convolutional network (TCN), which contains four dilated convolutional layers with dilation rates of 1, 2, 4, and 8, respectively. Each layer has a kernel size of 3 and 64 channels. The input is a 20×256 two-dimensional tensor, and the output, after global average pooling and fully connected layers, yields the probability distribution for each behavior category. During training, the model uses radar perception data labeled with behavior categories as training samples, employing a cross-entropy loss function and a stochastic gradient descent optimizer for parameter learning, ensuring that the output probability distribution accurately reflects the true behavior category.

[0093] Building upon this foundation, the system can also introduce a confidence verification mechanism to reduce the false alarm rate for the critical event of a fall. Specifically, when the probability of the fall category output by the sequential neural network model is greater than or equal to a first preset threshold (e.g., the first preset threshold is 0.7, and its specific value can be adjusted according to the accuracy and false alarm rate requirements of the actual scenario; for example, it can be set to 0.6 in scenarios requiring higher sensitivity and 0.8 in scenarios requiring higher reliability), the system does not immediately confirm the fall event, but instead takes the current frame as the starting frame and enters the confidence verification stage.

[0094] During the confidence verification phase, the system performs the following operations: It statistically analyzes the distribution of behavior categories output by the temporal neural network model within a subsequent preset number of frames (e.g., 20 frames, the specific value of which can be set according to the distribution of action duration in the actual scene; for example, 15 frames for scenes with short fall durations and 30 frames for scenes with long action durations). Simultaneously, it obtains the target centroid height for each frame within the subsequent preset number of frames. The target centroid height is derived from the height coordinate (z-coordinate) of the target centroid's three-dimensional spatial position extracted from the target point cloud data after coordinate transformation. This height value reflects the vertical position of the perceived object.

[0095] In the specific implementation, the system can make a comprehensive judgment based on the distribution of behavior categories and the height of the target centroid: if the proportion of frames in the behavior category distribution within the subsequent preset number of frames is greater than or equal to the second preset threshold (for example, the second preset threshold is 50%, and its specific value can be adjusted according to the confidence requirements in the actual scenario, for example, it can be set to 60% in the scenario that requires higher confidence, and to 40% in the scenario that requires faster response), and the change of the target centroid height in the subsequent preset number of frames meets the preset fall judgment conditions, then it is determined that the perceived object is in a fall posture.

[0096] The preset fall determination conditions can specifically include two stages: (1) The target centroid height falls within a third preset duration (corresponding to the first N1 frames in the subsequent preset number of frames, for example, the third preset duration is 0.25 seconds to 0.5 seconds, corresponding to 5 to 10 frames, and its specific value can be set according to the typical duration of human fall action, for example, it can be set to 0.5 seconds for a scenario where the fall speed of a weak elderly person is relatively slow, and to 0.25 seconds for a scenario where the fall speed of a young person is relatively fast) and the height drop exceeds the first preset height threshold (for example, the first preset height threshold is 0.8 meters, and its specific value can be set according to the height of the perceived object and the scenario, for example, it can be set to 1.0 meters for a taller perceived object, and to 0.6 meters for a shorter perceived object), this condition corresponds to the fall (1) The kinematic characteristics of a person falling rapidly from a standing height when a fall occurs; (2) During the subsequent fourth preset duration (corresponding to the last N2 frames in the subsequent preset number of frames, for example, the fourth preset duration is 0.5 seconds to 1.0 seconds, corresponding to 10 to 20 frames, and its specific value can be set according to the duration of the still state after the fall, for example, it can be set to 0.5 seconds for a scenario where the person gets up quickly after falling, and to 1.0 seconds for a scenario where the person remains still after falling), the target centroid height is continuously lower than the second preset height threshold (for example, the second preset height threshold is 0.6 meters, and its specific value can be set according to the typical height of the human body when sitting or lying down, for example, it can be set to 0.7 meters for a sofa sitting posture, and to 0.5 meters for a ground lying posture), this condition corresponds to the spatial characteristics of the human body in a lying still state after a fall. The above two stage conditions together constitute the preset fall judgment conditions, and the accuracy of the fall event judgment is ensured by simultaneously satisfying the kinematic descent characteristics and the static spatial characteristics.

[0097] It should be noted that the specific values ​​of the first and second preset height thresholds can be personalized according to the individual characteristics (such as height) of the perceived object. For example, for a taller perceived object, the center of gravity height when standing is approximately 1.0 to 1.1 meters, and when lying down is approximately 0.3 to 0.4 meters. Therefore, the first preset height threshold can be set to 0.7 to 0.8 meters, and the second preset height threshold can be set to 0.4 to 0.5 meters. For a shorter perceived object, the center of gravity height when standing is approximately 0.8 to 0.9 meters, and when lying down is approximately 0.25 to 0.35 meters. Therefore, the first preset height threshold can be set to 0.5 to 0.6 meters, and the second preset height threshold can be set to 0.3 to 0.4 meters, etc. This invention does not impose any limitations on these settings.

[0098] The confidence verification mechanism described above effectively reduces the probability of false alarms in a single frame by introducing temporal information from multiple consecutive frames and the auxiliary judgment of the target centroid height (for example, test results in a laboratory environment show that the false alarm rate is reduced from about 5 times / day to about 1 time / day after the confidence verification mechanism is introduced), thereby improving the reliability of fall detection.

[0099] After determining that the object being sensed is in a fallen posture, the system can generate an alarm message for that object and push it to a preset terminal device (such as a guardian's mobile phone, community service center, etc.) via the communication module. The alarm message includes at least one of the following: an event timestamp (e.g., "2026-06-29 14:35:22"), an event type (e.g., "fall event"), and a location identifier (e.g., "living room - sofa area"). Simultaneously, after determining that the object is in a fallen posture, the system can also control the sensing module to continuously sense the object at a target acquisition frequency (e.g., a second acquisition frequency, such as 20 frames / second) to monitor changes in the human body's state after the fall (e.g., whether it recovers to stand), until the behavior category output by the temporal neural network model is standing and the duration exceeds a preset duration (e.g., 5 seconds).

[0100] Furthermore, in some feasible implementations, the system can also transmit human posture perception results and / or structured features to the communication module as data transmitted externally by the communication module, while the original perception data is not transmitted externally through the communication module. Through this mechanism, the system only outputs structured data after feature extraction and compression or the final perception judgment result during external transmission, while the original perception data permanently resides on the local device side, thereby achieving the privacy protection goal of "data usable but not visible".

[0101] In some scenarios, such as when an elderly person accidentally falls in their living room, the sensing module continuously collects raw sensing data at a second acquisition frequency (e.g., 20 frames / second). The edge AI processing unit extracts structured features from the raw point cloud data. Among these features, the micro-Doppler spectral width of the first type (micro-motion features) expands from about 50Hz during normal walking to about 150Hz during the fall, and the spectral entropy increases from 0.6 to 0.9, reflecting the micro-Doppler modulation caused by violent limb swinging. The vertical extension of the second type (spatial distribution morphology features) decreases from about 1.6 meters to about 0.4 meters in about 1 second, and the horizontal projected area increases from about 0.2 square meters to about 0.8 square meters. The principal component of the shape changes from "linear" (standing posture) to "planar" (lying posture). At the same time, the target motion trajectory features show that the vertical velocity of the target's center of mass changes from about 0 m / s to about -1.2 m / s (i.e., the downward falling velocity), and the acceleration reaches about -1.5 m / s².

[0102] Next, the temporal neural network model, based on the three types of feature inputs from the aforementioned consecutive frames, outputs a probability of 0.85 for the fall category, exceeding the first preset threshold of 0.7. The system then enters the confidence verification phase, using the current frame as the starting frame, and statistically analyzes the distribution of behavioral categories over the next 20 frames (approximately 1 second). 15 frames are identified as fall categories (75% of frames, exceeding the second preset threshold of 50%), and the target's centroid height decreases from approximately 1.6 meters to approximately 0.4 meters in the first 8 frames (a decrease of approximately 1.2 meters, exceeding the first preset height threshold of 0.8 meters), and remains below 0.5 meters for the subsequent 12 frames (below the second preset height threshold of 0.6 meters). Based on these criteria, the system confirms that the perceived object is in a fall posture, generates an alarm message containing an event timestamp ("2026-06-29 14:35:22"), event type ("fall event"), and location identifier ("living room - sofa area"), and pushes the alarm to the preset guardian's mobile phone via the communication module. Meanwhile, the system control sensing module continues to sense at a collection frequency of 20 frames per second. After about 8 seconds, the behavior category output by the temporal neural network model changes to "standing" and lasts for more than 5 seconds. The system continues to monitor the elderly person.

[0103] In other scenarios, such as when the perceived object normally squats from a standing position to a sitting position (e.g., sitting on a sofa), the perception module collects raw perception data at a second acquisition frequency (e.g., 20 frames / second). Among the structured features extracted by the edge AI processing unit, the micro-Doppler spectral width of the first type of feature (micro-motion features) changes from approximately 50Hz during normal walking to approximately 35Hz during squatting, and the spectral entropy changes from 0.6 to 0.55, with a relatively gradual change. The vertical extension of the second type of feature (spatial distribution morphology features) decreases from approximately 1.6 meters to approximately 0.7 meters within approximately 2.5 seconds (a decrease of 0.9 meters, not exceeding the first preset height threshold of 0.8 meters), and the horizontal projected area increases from approximately 0.2 square meters to approximately 0.45 square meters, with the principal component of the shape changing from "linear" to "area-like." The temporal neural network model outputs a fall category probability of only 0.15, lower than the first preset threshold of 0.7, so the system does not trigger confidence verification and does not push alarm information. This scenario illustrates that this solution, through multimodal feature fusion and confidence verification mechanisms, can effectively distinguish between falling and similar actions such as squatting, thus avoiding false alarms.

[0104] In other scenarios, such as when the perceived object is lying down and resting on a bed in a bedroom, the sensing module collects raw sensing data at a first acquisition frequency (e.g., 2 frames / second). Among the structured features extracted by the edge AI processing unit, the first type of feature (micro-motion feature) has a micro-Doppler spectral width of approximately 15Hz, a spectral entropy of approximately 0.35, and high energy concentration, reflecting that the perceived object is in a quiet state with only slight chest rises and falls caused by breathing. The second type of feature (spatial distribution morphology feature) has a vertical extension of approximately 0.35 meters (indicating that the perceived object is lying down), a horizontal projected area of ​​approximately 1.0 square meters, and a planar shape as its principal component. The temporal neural network model outputs a probability of 0.92 for the lying still category, 0.06 for the normal activity category, and 0.02 for the fall category. Since the fall category probability is far below the first preset threshold of 0.7, the system does not trigger confidence verification and does not push alarm information. This scenario demonstrates that this solution can accurately identify a lying still state, avoiding misjudging normal resting as a fall event.

[0105] It should be noted that the embodiments of the present invention include, but are not limited to, the examples described above. It is understood that those skilled in the art can make further settings according to actual needs under the guidance of the ideas in the embodiments of the present invention, and the present invention does not limit such settings.

[0106] In this embodiment of the invention, raw sensing data is acquired, and then features are extracted from the raw sensing data on the local device side to obtain structured features that reflect the micro-motion features and spatial morphological distribution of the sensed object. Then, multimodal feature fusion processing is performed based on the first and second types of features in multiple consecutive frames to output the corresponding human posture perception result. Thus, by completing feature extraction on the local device side, the raw sensing data does not need to leave the local device, achieving privacy protection for the raw sensing data. This not only isolates the raw sensing data of the sensed object from the external transmission link at the physical level, but also, by extracting structured features containing micro-motion features and spatial morphological distribution features and combining them with multimodal feature fusion processing, human posture perception can make a comprehensive judgment based on the local micro-motion information and the overall spatial morphological information of the sensed object. This enhances the accuracy and robustness of perceiving abnormal postures such as falls. At the same time, since only structured features, rather than raw sensing data, are used as the data basis for subsequent processing, the burden of data transmission and processing is greatly reduced, and the data processing efficiency and response speed of the system are improved.

[0107] To enable those skilled in the art to better understand the technical solutions in the embodiments of the present invention, the following examples are provided for illustrative purposes: As some examples, see Figure 2 The diagram illustrates a flow chart of system control provided in an embodiment of the present invention, involving data processing of human perception, specifically: I. System State Control and Radar Sensing Signal Processing Stage After the system is powered on and initialized, the system status control module first controls the sensing module to operate in low frame rate mode. In low frame rate mode, the sensing module only performs a simplified signal processing procedure, namely, performing range-Doppler fast Fourier transform (FFT) on the radar echo data and using the constant false alarm rate (CFAR) detection algorithm to extract effective target points from the range-Doppler spectrum. It mainly monitors whether there are active targets in the area, without performing full-function processing such as complete point cloud clustering, tracking filtering, and feature extraction, in order to reduce system power consumption.

[0108] In its current operating mode, the perception module outputs the point cloud data or detection results obtained from each frame to the edge AI processing unit. The edge AI processing unit comprehensively judges the detection results of multiple consecutive frames to determine whether there is an active target within the monitored area. When an active target is detected, the system starts a "cumulative active frame" counter; when the cumulative number of frames in which an active target is detected reaches a preset condition (e.g., an active target is detected in N consecutive frames), the system controls the perception module to switch from low frame rate mode to high frame rate mode.

[0109] After switching to high frame rate mode, the perception module executes a complete signal processing chain. The perception module performs range-Doppler FFT on the radar echo data to generate a range-Doppler spectrum, uses the CFAR detection algorithm to extract valid target points to generate raw point cloud data, clusters the point cloud based on spatial proximity to group point clouds belonging to the same physical target into a single detection object, and performs Kalman filtering on the detection object to smooth its motion trajectory, outputting structured perception data containing target spatial position, radial velocity, and point cloud distribution information.

[0110] II. End-side characteristic distillation stage In high frame rate mode, the edge AI processing unit performs complete feature extraction processing on the structured perception data output by the perception module. First, the original point cloud data is converted from polar coordinates to Cartesian space coordinates. Multi-dimensional features are then extracted from the transformed target point cloud data, including target motion trajectory features, micro-motion features (micro-Doppler spectrum features), and spatial distribution morphology features. Specifically, the target motion trajectory features are obtained by extracting the temporal position sequence of the target's centroid and performing differential operations to obtain displacement velocity, acceleration, and corresponding statistics. Micro-motion features are obtained by extracting the micro-Doppler spectrum from the point cloud data after filtering out the static background and calculating its peak frequency, spectral width, spectral entropy, and energy concentration. Spatial distribution morphology features are obtained by extracting the vertical extension, horizontal projected area, and principal components of the shape of the target point cloud.

[0111] The edge AI processing unit constructs a temporal feature sequence from the aforementioned multi-dimensional features across multiple consecutive frames using a pre-defined sliding window. After normalization, it obtains structured features with fixed dimensions. This structured feature vector compresses the original point cloud data by more than 30 times, and the original data can be irreversibly recovered. The edge AI processing unit uses the generated structured feature vector as the output of edge feature distillation for subsequent AI inference.

[0112] III. AI Reasoning and Decision-Making Stage The edge AI processing unit inputs structured feature vectors from multiple consecutive frames into a temporal neural network model for multimodal feature fusion inference. The temporal neural network model extracts temporal dependencies through convolutional or recurrent layers and outputs the probability distribution of each behavior category via fully connected layers and softmax layers. Behavior categories include normal activity, falling, and lying still.

[0113] The edge AI processing unit makes decisions based on the probability distribution output by the temporal neural network model: When the probability of a fall is greater than or equal to a first preset threshold, the system triggers a preliminary alarm and enters the state machine confirmation phase. During this phase, the system statistically analyzes the distribution of behavior categories output by the temporal neural network model within a preset number of frames, and uses this information in conjunction with the temporal changes in the target centroid height for auxiliary judgment. If the proportion of frames with the fall category exceeds a second preset threshold within the preset number of frames, and the change in the target centroid height within these frames meets the preset fall judgment conditions (i.e., the target centroid height drops rapidly in a short period and then remains at a low level), then it is confirmed as a valid fall event, an "fall alarm" is output, and alarm information is pushed to preset terminal devices via the wireless communication module.

[0114] When the probability of the fall category does not reach the first preset threshold, the system outputs the corresponding normal state (such as normal activity, lying still, etc.) based on the maximum probability category output by the temporal neural network model.

[0115] IV. Dynamic Frame Rate Control Stage During operation in high frame rate mode, the edge AI processing unit continuously monitors the target area for the presence of active targets based on the perception results. When no active target is detected in the target area, the system starts a "continuous inactivity" timer. When the cumulative duration of no active target reaches a second preset duration (e.g., T consecutive seconds of inactivity), the system controls the perception module to fall back from high frame rate mode to low frame rate mode to reduce system power consumption. In low frame rate mode, the perception module only performs presence detection, while the edge AI processing unit does not start full-function feature extraction and AI inference.

[0116] V. Wireless Communication and Cooperative Anti-interference Phase The structured features or human posture perception results output by the edge AI processing unit are encrypted and transmitted to the cloud or terminal device via the wireless communication module. During operation, the wireless communication module continuously monitors the status parameters of the current communication channel.

[0117] When the wireless communication module detects strong interference signals or channel occupancy exceeding a preset threshold, it sends a control command to the sensing module via control signaling. The sensing module then performs adaptive adjustments based on this command, including at least one of the following: adjusting waveform parameters of the transmitted signal (such as frequency modulation slope and starting frequency), reducing transmit power, and adjusting receiver gain. This aims to avoid interfering frequency bands, reduce potential interference with the communication module, and improve interference suppression capabilities. Once the wireless communication module detects that the channel status has returned to normal, it notifies the sensing module via control signaling to revert to its original operating parameter configuration, restoring normal communication.

[0118] It should be noted that, for the sake of simplicity, the method embodiments are all described as a series of actions. However, those skilled in the art should understand that the embodiments of the present invention are not limited to the described order of actions, because according to the embodiments of the present invention, some steps can be performed in other orders or simultaneously. Furthermore, those skilled in the art should also understand that the embodiments described in the specification are preferred embodiments, and the actions involved are not necessarily essential to the embodiments of the present invention.

[0119] Reference Figure 3 The diagram illustrates a structural block diagram of a human body perception data processing device provided in an embodiment of the present invention, which may specifically include the following modules: The acquisition module 301 is used to acquire raw sensor data; The feature extraction module 302 is used to extract features from the original sensing data on the local device side to obtain corresponding structured features. The structured features include a first type of features that reflect the micro-motion features of the sensed object and a second type of features that reflect the morphological distribution of the sensed object in space. The perception processing module 303 is used to perform multimodal feature fusion processing based on the first type of features and the second type of features in multiple consecutive frames, and output the corresponding human posture perception result.

[0120] In some feasible implementations, the sensing processing module 303 is specifically used for: The first type of features and the second type of features from multiple consecutive frames are input into a temporal neural network model for multimodal feature fusion, and the probability distribution corresponding to each behavior category is output, including the fall category; If the probability of the fall category is greater than or equal to the first preset threshold, then the original perception data of the current frame is used as the starting frame, the distribution of the behavior category output by the temporal neural network model in the original perception data of the subsequent preset number of frames is statistically analyzed, and the target centroid height corresponding to each frame in the subsequent preset number of frames is obtained. Based on the distribution of the behavior categories and the centroid height of the target, a confidence verification process is performed, and the corresponding human posture perception result is output.

[0121] In some feasible implementations, the structured features also include target motion trajectory features, and the perception processing module 303 is specifically used for: The first type of features, the second type of features, and the target motion trajectory features from multiple consecutive frames are spliced ​​together in chronological order to obtain the corresponding two-dimensional tensor. The two-dimensional tensor is input into the temporal neural network model, which extracts temporal dependencies through convolutional or recurrent layers, and outputs the probability distribution of each behavior category corresponding to the temporal dependencies through fully connected layers and softmax layers.

[0122] In some feasible implementations, the sensing processing module 303 is specifically used for: If the proportion of frames in the behavior category distribution within the subsequent preset number of frames that are determined to be of the fall category is greater than or equal to the second preset threshold, and the change in the target centroid height within the subsequent preset number of frames satisfies the preset fall determination condition, then it is determined that the perceived object is in a fall posture.

[0123] In some feasible implementations, the apparatus further includes: An alarm module is used to generate alarm information for the sensed object after the sensing processing module 303 determines that the sensed object is in a falling posture; A communication module is used to push the alarm information to a preset terminal device; The alarm information includes at least one of the following: event timestamp, event type, and location identifier.

[0124] In some feasible implementations, the raw sensing data is collected by a sensing module, and the device further includes: The sensing control module is used to control the sensing module to continuously sense the object being sensed at a target acquisition frequency after the sensing processing module 303 determines that the object being sensed is in a falling posture.

[0125] In some feasible implementations, the acquisition module 301 is specifically used for: Acquire the echo signal collected by the sensing module; The echo signal is transformed to obtain the corresponding range-Doppler spectrum; Valid points are extracted from the distance-Doppler spectrum to generate the corresponding raw point cloud data; The original point cloud data is clustered, and point clouds belonging to the same physical target are merged into a single perceived object. The perceived object is then tracked and filtered to output the corresponding original perception data.

[0126] In some feasible implementations, the feature extraction module 302 is specifically used for: The original point cloud data in the original sensing data is converted from polar coordinates to Cartesian space coordinates to obtain the corresponding target point cloud data. Extracting multi-dimensional features from the target point cloud data after coordinate transformation; The multi-dimensional features of multiple consecutive frames are constructed into a corresponding temporal feature sequence according to a preset sliding window; The time-series feature sequence is normalized to obtain structured features with fixed dimensions.

[0127] In some feasible implementations, the multi-dimensional features also include target motion trajectory features, and the feature extraction module 302 is specifically used for: Extract the three-dimensional spatial position of the target centroid from the target point cloud data after coordinate transformation; Obtain the temporal position sequence of the target centroid; The displacement velocity, acceleration, first statistic of the displacement velocity, and second statistic of the acceleration of the target centroid are obtained by performing a difference operation on the time-series position sequence. The three-dimensional spatial position, the displacement velocity, the acceleration, the first statistic, and the second statistic are used as features of the target motion trajectory.

[0128] In some feasible implementations, the feature extraction module 302 is specifically used for: Static background filtering is performed on the target point cloud data after coordinate transformation; Extract the micro-Doppler spectrum of the target echo from the target point cloud data after background filtering; Calculate at least one of the peak frequency, spectral width, spectral entropy, and energy concentration of the microDoppler spectrum; The peak frequency, the spectral width, the spectral entropy, and the energy concentration are taken as the first type of feature.

[0129] In some feasible implementations, the feature extraction module 302 is specifically used for: Extract the vertical extension of the target point cloud from the target point cloud data after coordinate transformation; Extract the horizontal projected area of ​​the target point cloud from the target point cloud data after coordinate transformation; Principal component analysis is performed on the target point cloud data after coordinate transformation to obtain the variance distribution of the target point cloud in three orthogonal directions, and the variance distribution is used as the shape principal component of the target point cloud. The vertical extension, the horizontal projected area, and at least one of the principal components of the shape are designated as the second type of feature.

[0130] In some feasible implementations, the acquisition module 301 is specifically used for: In response to power-on initialization, the control sensing module collects raw sensing data at the first acquisition frequency; If an active target is detected in the target area to which the sensing module belongs based on the original sensing data, and the duration reaches a first preset duration, then the sensing module is controlled to switch from the first acquisition frequency to the second acquisition frequency to acquire the original sensing data at the second acquisition frequency. The second sampling frequency is greater than the first sampling frequency.

[0131] In some feasible implementations, the acquisition module 301 is further configured to: After acquiring raw sensing data at the second acquisition frequency, if the raw sensing data indicates that there is no active target in the target area to which the sensing module belongs, and the duration reaches the second preset duration, then the sensing module is controlled to switch to the first acquisition frequency to acquire raw sensing data at the first acquisition frequency.

[0132] In some feasible implementations, the apparatus further includes: The transmission module is used to transmit the human posture perception results and / or the structured features to the communication module as data transmitted externally by the communication module.

[0133] In some feasible implementations, the raw sensing data is collected by a sensing module, and the device further includes: The channel monitoring module is used to acquire the channel status parameters of the communication module; An anti-interference control module is used to control the sensing module to perform at least one of the following operations if the channel state parameters exceed a preset range: adjusting the waveform parameters of the transmitted signal, reducing the transmission power, and adjusting the receiver gain.

[0134] In some feasible implementations, the anti-interference control module is further used for: If the channel state parameters recover to within the preset range, then control the sensing module to restore the working parameter configuration before the adjustment operation was performed.

[0135] As the device embodiment is basically similar to the method embodiment, the description is relatively simple, and relevant parts can be found in the description of the method embodiment.

[0136] In addition, this invention also provides an electronic device, including: a processor, a memory, and a computer program stored in the memory and executable on the processor. When the computer program is executed by the processor, it implements the various processes of the above-described human perception data processing method embodiments and achieves the same technical effect. To avoid repetition, it will not be described again here.

[0137] This invention also provides a readable storage medium storing a computer program. When executed by a processor, the computer program implements the various processes of the above-described human perception data processing method embodiments and achieves the same technical effects. To avoid repetition, it will not be described again here. The readable storage medium may be a read-only memory (ROM), a random access memory (RAM), a magnetic disk, or an optical disk, etc.

[0138] The various embodiments in this specification are described in a progressive manner, with each embodiment focusing on the differences from other embodiments. The same or similar parts between the various embodiments can be referred to each other.

[0139] Those skilled in the art will understand that embodiments of the present invention can be provided as methods, apparatus, or computer program products. Therefore, embodiments of the present invention can take the form of entirely hardware embodiments, entirely software embodiments, or embodiments combining software and hardware aspects. Furthermore, embodiments of the present invention can take the form of computer program products implemented on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, EEPROM, Flash, and eMMC, etc.) containing computer-usable program code.

[0140] Embodiments of the present invention are described with reference to flowchart illustrations and / or block diagrams of methods, terminal devices (systems), and computer program products according to embodiments of the invention. It will be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, special-purpose computer, embedded processor, or other programmable data processing terminal device to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing terminal device, generate instructions for implementing the flowchart illustrations and / or block diagrams. Figure 1 One or more processes and / or boxes Figure 1 A device that provides the functions specified in one or more boxes.

[0141] These computer program instructions may also be stored in a computer-readable storage medium that can direct a computer or other programmable data processing terminal device to operate in a particular manner, such that the instructions stored in the computer-readable storage medium produce an article of manufacture including instruction means, which are implemented in a process Figure 1 One or more processes and / or boxes Figure 1 The function specified in one or more boxes.

[0142] These computer program instructions can also be loaded onto a computer or other programmable data processing terminal equipment, causing a series of operational steps to be performed on the computer or other programmable terminal equipment to produce a computer-implemented process, thereby providing instructions that execute on the computer or other programmable terminal equipment for implementing the process. Figure 1 One or more processes and / or boxes Figure 1 The steps of the function specified in one or more boxes.

[0143] Although preferred embodiments of the present invention have been described, those skilled in the art, upon learning the basic inventive concept, can make other changes and modifications to these embodiments. Therefore, the appended claims are intended to be interpreted as including the preferred embodiments as well as all changes and modifications falling within the scope of the embodiments of the present invention.

[0144] Finally, it should be noted that in this document, relational terms such as "first" and "second" are used only to distinguish one entity or operation from another, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Furthermore, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or terminal device that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or terminal device. Without further limitations, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or terminal device that includes said element.

[0145] The present invention has provided a detailed description of a human body perception data processing method and a human body perception data processing device. Specific examples have been used to illustrate the principles and implementation methods of the present invention. The description of the above embodiments is only for the purpose of helping to understand the method and core ideas of the present invention. At the same time, for those skilled in the art, there will be changes in the specific implementation methods and application scope based on the ideas of the present invention. Therefore, the content of this specification should not be construed as a limitation of the present invention.

Claims

1. A data processing method for human perception, characterized in that, include: Acquire raw sensor data; On the local device side, feature extraction is performed on the raw sensing data to obtain corresponding structured features. The structured features include a first type of features that reflect the micro-motion features of the sensed object and a second type of features that reflect the morphological distribution of the sensed object in space. Multimodal feature fusion processing is performed based on the first type of features and the second type of features from multiple consecutive frames to output the corresponding human posture perception results.

2. The method according to claim 1, characterized in that, The step of performing multimodal feature fusion processing based on the first type of features and the second type of features from multiple consecutive frames, and outputting the corresponding human pose perception result, includes: The first type of features and the second type of features from multiple consecutive frames are input into a temporal neural network model for multimodal feature fusion, and the probability distribution corresponding to each behavior category is output, including the fall category; If the probability of the fall category is greater than or equal to the first preset threshold, then the original perception data of the current frame is used as the starting frame, the distribution of the behavior category output by the temporal neural network model in the original perception data of the subsequent preset number of frames is statistically analyzed, and the target centroid height corresponding to each frame in the subsequent preset number of frames is obtained. Based on the distribution of the behavior categories and the centroid height of the target, a confidence verification process is performed, and the corresponding human posture perception result is output.

3. The method according to claim 2, characterized in that, The structured features also include target motion trajectory features. The process of inputting the first type of features and the second type of features from multiple consecutive frames into a temporal neural network model for multimodal feature fusion, and outputting the probability distribution corresponding to each behavior category, includes: The first type of features, the second type of features, and the target motion trajectory features from multiple consecutive frames are spliced ​​together in chronological order to obtain the corresponding two-dimensional tensor. The two-dimensional tensor is input into the temporal neural network model, which extracts temporal dependencies through convolutional or recurrent layers, and outputs the probability distribution of each behavior category corresponding to the temporal dependencies through fully connected layers and softmax layers.

4. The method according to claim 2, characterized in that, The confidence verification process based on the behavior category distribution and the target centroid height, and the output of the corresponding human posture perception result, includes: If the proportion of frames in the behavior category distribution within the subsequent preset number of frames that are determined to be of the fall category is greater than or equal to the second preset threshold, and the change in the target centroid height within the subsequent preset number of frames satisfies the preset fall determination condition, then it is determined that the perceived object is in a fall posture.

5. The method according to claim 4, characterized in that, After determining that the sensed object is in a falling posture, the method further includes: Generate alarm information for the sensed object; The alarm information is pushed to a preset terminal device via a communication module; The alarm information includes at least one of the following: event timestamp, event type, and location identifier.

6. The method according to claim 4, characterized in that, The original sensing data is collected by the sensing module. After determining that the sensed object is in a falling posture, the method further includes: The sensing module is controlled to continuously sense the object being sensed at a target acquisition frequency.

7. The method according to claim 1, characterized in that, The acquisition of raw sensory data includes: Acquire the echo signal collected by the sensing module; The echo signal is transformed to obtain the corresponding range-Doppler spectrum; Valid points are extracted from the distance-Doppler spectrum to generate the corresponding raw point cloud data; The original point cloud data is clustered, and point clouds belonging to the same physical target are merged into a single perceived object. The perceived object is then tracked and filtered to output the corresponding original perception data.

8. The method according to claim 7, characterized in that, The step of extracting features from the original perceived data to obtain corresponding structured features includes: The original point cloud data in the original sensing data is converted from polar coordinates to Cartesian space coordinates to obtain the corresponding target point cloud data. Extracting multi-dimensional features from the target point cloud data after coordinate transformation; The multi-dimensional features of multiple consecutive frames are constructed into a corresponding temporal feature sequence according to a preset sliding window; The time-series feature sequence is normalized to obtain structured features with fixed dimensions.

9. The method according to claim 8, characterized in that, The multi-dimensional features also include target motion trajectory features. Extracting multi-dimensional features from the target point cloud data after coordinate transformation includes: Extract the three-dimensional spatial position of the target centroid from the target point cloud data after coordinate transformation; Obtain the temporal position sequence of the target centroid; The displacement velocity, acceleration, first statistic of the displacement velocity, and second statistic of the acceleration of the target centroid are obtained by performing a difference operation on the time-series position sequence. The three-dimensional spatial position, the displacement velocity, the acceleration, the first statistic, and the second statistic are used as features of the target motion trajectory.

10. The method according to claim 8, characterized in that, The extraction of multi-dimensional features from the target point cloud data after coordinate transformation includes: Static background filtering is performed on the target point cloud data after coordinate transformation; Extract the micro-Doppler spectrum of the target echo from the target point cloud data after background filtering; Calculate at least one of the peak frequency, spectral width, spectral entropy, and energy concentration of the microDoppler spectrum; At least one of the peak frequency, the spectral width, the spectral entropy, and the energy concentration is used as the first type of feature.

11. The method according to claim 8, characterized in that, The extraction of multi-dimensional features from the target point cloud data after coordinate transformation includes: Extract the vertical extension of the target point cloud from the target point cloud data after coordinate transformation; Extract the horizontal projected area of ​​the target point cloud from the target point cloud data after coordinate transformation; Principal component analysis is performed on the target point cloud data after coordinate transformation to obtain the variance distribution of the target point cloud in three orthogonal directions, and the variance distribution is used as the shape principal component of the target point cloud. The vertical extension, the horizontal projected area, and at least one of the principal components of the shape are designated as the second type of feature.

12. The method according to claim 1, characterized in that, The acquisition of raw sensory data includes: In response to power-on initialization, the control sensing module collects raw sensing data at the first acquisition frequency; If an active target is detected in the target area to which the sensing module belongs based on the original sensing data, and the duration reaches a first preset duration, then the sensing module is controlled to switch from the first acquisition frequency to the second acquisition frequency to acquire the original sensing data at the second acquisition frequency. The second sampling frequency is greater than the first sampling frequency.

13. The method according to claim 12, characterized in that, After acquiring the raw sensing data at the second acquisition frequency, the method further includes: If, based on the original sensing data, it is detected that there are no active targets in the target area to which the sensing module belongs, and the duration reaches the second preset duration, then the sensing module is controlled to switch to the first acquisition frequency to acquire the original sensing data at the first acquisition frequency.

14. The method according to claim 1, characterized in that, Also includes: The human posture perception results and / or the structured features are transmitted to the communication module as data transmitted externally by the communication module.

15. The method according to claim 14, characterized in that, The raw sensing data is collected by the sensing module, and the method further includes: Obtain the channel state parameters of the communication module; If the channel state parameters exceed the preset range, the sensing module is controlled to perform at least one of the following operations: adjusting the waveform parameters of the transmitted signal, reducing the transmission power, and adjusting the receiver gain.

16. The method according to claim 15, characterized in that, Also includes: If the channel state parameters recover to within the preset range, then control the sensing module to restore the working parameter configuration before the adjustment operation was performed.

17. A data processing device for human body perception, characterized in that, include: The acquisition module is used to acquire raw sensor data; The feature extraction module is used to extract features from the original sensing data on the local device side to obtain corresponding structured features. The structured features include a first type of features that reflect the micro-motion features of the sensed object and a second type of features that reflect the morphological distribution of the sensed object in space. The perception processing module is used to perform multimodal feature fusion processing based on the first type of features and the second type of features from multiple consecutive frames, and output the corresponding human posture perception results.

18. An electronic device, characterized in that, It includes a processor, a communication interface, a memory, and a communication bus, wherein the processor, the communication interface, and the memory communicate with each other through the communication bus; The memory is used to store computer programs; When the processor executes a program stored in the memory, it implements the method as described in any one of claims 1-16.

19. A computer-readable storage medium having instructions stored thereon that, when executed by one or more processors, cause the processors to perform the method as described in any one of claims 1-16.

Images