Multi-modal health monitoring method and device, radar, electronic equipment and storage medium

Abstract

The invention provides a multi-mode health monitoring method and device, a radar, electronic equipment and a storage medium. The method comprises the following steps: transmitting a linear frequency modulation signal and receiving an echo signal through a frequency-modulated continuous wave radar, wherein the frequency-modulated continuous wave radar is arranged in an area near a bed; clustering analysis is carried out on the echo signals, bed-related behavior states are dynamically judged according to a clustering result, and the behavior states comprise stable bed occupation, continuous empty bed and object movement; switching among a plurality of monitoring modes according to the behavior state and the switching condition of the finite-state machine, wherein the plurality of monitoring modes comprise a bed occupation monitoring mode, a vital sign monitoring mode and a tumble monitoring mode; wherein the switching condition of the finite-state machine comprises switching to a vital sign monitoring mode when the behavior state is stable bed occupation, switching to a fall monitoring mode when the behavior state is object movement, and switching to a bed occupation monitoring mode when the behavior state is continuous empty bed, and the multi-mode dynamic monitoring requirement in a complex scene is met.

Description

TECHNICAL FIELD

[0001] The present disclosure belongs to the technical field of auxiliary medical devices, and particularly relates to a multi-modal health monitoring method and device, radar, electronic device and storage medium. BACKGROUND

[0002] In recent years, health monitoring technology has been widely applied in human body monitoring, smart elderly care, public security and the like. In related technologies, off-bed monitoring is achieved by means of a pressure sensing mattress or an infrared bedside strip, fall detection is achieved by means of posture recognition through an image or a depth camera, and vital sign monitoring is achieved by means of a wearable blood oxygen / heart rate device or a patch sensor. In order to simultaneously achieve multi-modal health monitoring functions, multiple monitoring devices need to be deployed at the same time, resulting in low hardware resource utilization and poor algorithm collaboration. Moreover, alarm points of the multiple monitoring devices are scattered, and staff need to check alarm information of different devices respectively, increasing manual work burden. In addition, camera type monitoring devices may fail due to insufficient light or obstruction and have the risk of privacy leakage. Therefore, the way of simultaneously deploying multiple monitoring devices to achieve multi-modal health monitoring functions obviously cannot meet the multi-modal dynamic monitoring demand in complex scenarios. SUMMARY

[0003] In view of the above problems, the present disclosure provides a multi-modal health monitoring method, device, radar, electronic device and storage medium, aiming to realize dynamic switching between bed occupancy monitoring, vital sign monitoring and fall detection monitoring multi-modal health monitoring functions through a single frequency-modulated continuous wave radar, and meet the multi-modal dynamic monitoring demand in complex scenarios.

[0004] According to a first aspect of the present disclosure, a multi-modal health monitoring method is provided, comprising:

[0005] emitting a linear frequency-modulated signal by a frequency-modulated continuous wave radar and receiving a return signal, the frequency-modulated continuous wave radar being arranged in a region near a bed;

[0006] performing cluster analysis on the return signal, and dynamically determining a behavior state related to the bed according to a cluster result, the behavior state including bed stable occupancy, continuous empty bed and object movement;

[0007] switching between a plurality of monitoring modes according to the behavior state and a switching condition of a finite state machine, the plurality of monitoring modes including a bed occupancy monitoring mode, a vital sign monitoring mode and a fall detection monitoring mode;

[0008] wherein the switching condition of the finite state machine includes switching to the vital sign monitoring mode when the behavior state is the bed stable occupancy, switching to the fall detection monitoring mode when the behavior state is the object movement, and switching to the bed occupancy monitoring mode when the behavior state is the continuous empty bed.

[0009] Optionally, the echo signals are subjected to cluster analysis, and the bed-related behavior state is dynamically determined according to the cluster result, comprising:

[0010] The echo signals are subjected to spatial clustering based on a distance-angle heat map, and a cluster stability and / or a cluster area of the bed area are calculated;

[0011] If the cluster stability of the bed area is greater than a first preset threshold and / or a duration that the cluster area of the bed area is within a preset range reaches a second preset threshold, it is determined that the behavior state is bed stable occupancy;

[0012] If the cluster stability of the bed area is less than or equal to the first preset threshold and / or a cluster in the distance-angle heat map has a rapid movement feature and / or a new cluster appears in the bed surrounding area in the distance-angle heat map, it is determined that the behavior state is object movement;

[0013] If the cluster intensity of the bed area in the distance-angle heat map is lower than a third preset threshold and / or a duration that the bed area has no other cluster reaches a fourth preset threshold, it is determined that the behavior state is continuous empty bed.

[0014] Optionally, the vital sign monitoring mode comprises:

[0015] A region with the maximum standard deviation in multiple frames of the distance-angle heat map is determined as a chest cavity region;

[0016] The clusters corresponding to the chest cavity region in multiple frames of the distance-angle heat map are subjected to phase unwrapping processing, and chest cavity micro-motion phase information is extracted;

[0017] After a first difference operation is performed on the chest cavity micro-motion phase information, a band-pass filter is used to extract a respiratory signal;

[0018] After a second difference operation is performed on the respiratory signal, a band-pass filter is used to extract a heartbeat signal;

[0019] Respiratory spectrum and respiratory frequency peak value, and heartbeat spectrum and heartbeat frequency peak value are determined through spectrum analysis.

[0020] Optionally, the fall monitoring mode comprises:

[0021] Micro-Doppler features or four-dimensional radar cubic data are used for analysis;

[0022] When a pitch angle height mutation exceeding a fifth preset threshold is detected and / or a deep learning model output determines that a fall event occurs, it is determined that a fall event is detected, and a fall alarm is triggered.

[0023] Optionally, the switching condition of the finite state machine further comprises:

[0024] switching to the fall monitoring mode when detecting that the stability of the cluster existing in the distance-angle heat map is lower than a sixth preset threshold in the vital sign monitoring mode;

[0025] switching to the bed occupancy monitoring mode when a continuous running time exceeds a seventh preset threshold and no fall event is detected in the fall monitoring mode.

[0026] Optionally, the multi-modal health monitoring method further comprises:

[0027] establishing a personalized behavior pattern baseline of the target based on historical monitoring data;

[0028] adapting the first preset threshold to the seventh preset threshold according to the personalized behavior pattern baseline;

[0029] wherein the personalized behavior pattern baseline comprises daily activity regularity, vital sign fluctuation characteristics and spatial moving trajectory characteristics of the target object.

[0030] According to a second aspect of the embodiments of the present disclosure, a multi-modal health monitoring device is provided, comprising:

[0031] an echo signal receiving module configured to transmit a linear frequency modulation signal and receive an echo signal by a frequency-modulated continuous wave radar, the frequency-modulated continuous wave radar being arranged in a bed vicinity area;

[0032] a cluster analysis module configured to perform cluster analysis on the echo signal, and dynamically determine a behavior state related to the bed according to a cluster result, the behavior state comprising bed stable occupancy, continuous empty bed and object moving;

[0033] a monitoring mode switching module configured to switch among a plurality of monitoring modes according to the behavior state and a switching condition of a finite state machine, the plurality of monitoring modes comprising a bed occupancy monitoring mode, a vital sign monitoring mode and a fall monitoring mode;

[0034] wherein the switching condition of the finite state machine comprises switching to the vital sign monitoring mode when the behavior state is the bed stable occupancy, switching to the fall monitoring mode when the behavior state is the object moving, and switching to the bed occupancy monitoring mode when the behavior state is the continuous empty bed.

[0035] According to a third aspect of the embodiments of the present disclosure, a radar arranged in a bed vicinity area is provided, comprising:

[0036] an antenna array;

[0037] A radar signal generation module for generating a linear frequency modulated signal, emitting the linear frequency modulated signal through the antenna array, and processing a received echo signal received by the antenna array;

[0038] A processing unit comprising the multi-modal health monitoring device as described above, switching between a plurality of monitoring modes according to steps of the method as described above and switching conditions of the bed-related behavior state and the finite state machine, the plurality of monitoring modes comprising a bed occupancy monitoring mode, a vital sign monitoring mode and a fall monitoring mode.

[0039] According to a fourth aspect of the present disclosure, an electronic device is provided, comprising a processor, a memory and a program stored on the memory and executable on the processor, the program, when executed by the processor, implementing steps of the method according to any one of the above.

[0040] According to a fifth aspect of the present disclosure, a storage medium is provided, the storage medium storing a computer program or instructions, the computer program or instructions, when executed by a processor, implementing steps of the method according to any one of the above.

[0041] The present disclosure brings the following beneficial effects:

[0042] The multi-modal health monitoring method provided by the present disclosure transmits a linear frequency modulated signal and receives an echo signal through a frequency modulated continuous wave radar arranged in a region near a bed, performs clustering analysis on the echo signal, dynamically determines a bed-related behavior state according to a clustering result, the behavior state comprises a bed stable occupancy, a continuous empty bed and object movement, switches between a plurality of monitoring modes according to the behavior state and switching conditions of a finite state machine, the plurality of monitoring modes comprises a bed occupancy monitoring mode, a vital sign monitoring mode and a fall monitoring mode, wherein the switching conditions of the finite state machine comprise switching to the vital sign monitoring mode when the behavior state is the bed stable occupancy, switching to the fall monitoring mode when the behavior state is the object movement, and switching to the bed occupancy monitoring mode when the behavior state is the continuous empty bed. In this way, through the combination of a single frequency modulated continuous wave radar and a finite state machine, dynamic integration and seamless switching of multi-modal health monitoring functions are realized. Through a unified hardware platform and an adaptive algorithm framework, not only is the device deployment cost reduced, but also user privacy is protected without visual data acquisition characteristics. At the same time, through the hierarchical logic judgment mechanism of the finite state machine, the bed occupancy monitoring, vital sign monitoring and fall monitoring are cooperatively operated in a complex scene, data conflicts and false triggering between multiple sensor systems are avoided, and problems such as high hardware cost, great privacy controversy and complex maintenance caused by simultaneous deployment of multiple monitoring devices in related technologies are effectively solved.

[0043] Other features and advantages of the present disclosure will be set forth in the descriptions that follow, and in part will be apparent from the description or can be learned by practice of the present disclosure. The purposes and other advantages of the present disclosure will be realized and attained by the structures particularly pointed out in the description and the appended drawings.

[0044] To make the above-mentioned purposes, features and advantages of the present disclosure more obvious and easy to understand, the following preferred embodiments are specifically described below, and the accompanying drawings are referred to. BRIEF DESCRIPTION OF DRAWINGS

[0045] The above-mentioned and other purposes, features and advantages of the present disclosure will be more apparent from the following description of the embodiments of the present disclosure with reference to the accompanying drawings, in which:

[0046] Figure 1A An application architecture diagram of a multi-modal health monitoring method according to an embodiment of the present disclosure is provided;

[0047] Figure 1B An application architecture diagram of a multi-modal health monitoring method according to another embodiment of the present disclosure is provided;

[0048] Figure 2 A flowchart of a multi-modal health monitoring method according to an embodiment of the present disclosure is provided;

[0049] Figure 3 A schematic diagram of a state machine for multi-modal health monitoring according to an embodiment of the present disclosure is provided;

[0050] Figure 4 A structural schematic diagram of a multi-modal health monitoring device according to an embodiment of the present disclosure is provided;

[0051] Figure 5 A structural schematic diagram of an electronic device according to an embodiment of the present disclosure is provided. DETAILED DESCRIPTION

[0052] Various embodiments of the present disclosure will be described in detail below with reference to the accompanying drawings. In each of the drawings, the same elements are denoted by the same or similar reference numerals to indicate the same or similar elements. Each part in the drawings is not drawn to scale for the sake of clarity.

[0053] The following terms are used herein:

[0054] Frequency Modulated Continuous Wave (FMCW) is a technology that uses frequency-modulated continuous wave signals for target detection and measurement. Its core is that the frequency of the transmitted signal changes over time, usually linearly modulated. FMCW technology is widely used in radar systems, which measures the distance, distance change rate (radial velocity), azimuth, height, etc. of the target to the electromagnetic wave emission point by analyzing the frequency difference between the transmitted signal and the echo signal.

[0055] Multiple Input Multiple Output (MIMO) refers to the configuration of multiple transmitting and receiving antenna arrays of the radar, which can achieve higher angular resolution.

[0056] Chirp is a term in communication technology related to coded pulse technology, which refers to the encoding of pulses, and the carrier frequency increases linearly within the pulse duration (i.e. a signal whose frequency changes (increases or decreases) with time). When the pulse is converted to audio, it will emit a sound that sounds like a bird's chirp, hence the name "chirp".

[0057] Figure 1A The application architecture diagram of the multi-modal health monitoring method according to one embodiment of the present disclosure is provided. As shown in Figure 1A The application architecture 100 provided by the embodiment of the present disclosure includes an FMCW radar 110 and a bed 120. The FMCW radar 110 can be arranged in the area near the bed 120, for example, directly above the bed head, wall-mounted on the bed side, or ceiling downward angle installation, etc. Its radiation direction covers the bed area and the surrounding activity space, and the present disclosure does not limit this.

[0058] Figure 1B The application architecture diagram of the multi-modal health monitoring method according to another embodiment of the present disclosure is provided. As shown in Figure 1B The application architecture 100 provided by the embodiment of the present disclosure includes an FMCW radar 110 and a bed 120. The FMCW radar 110 includes an antenna array 101, a radar signal generation module 102, and a processing unit 103. The antenna array 101 is configured in a MIMO architecture, which can form a virtual array aperture through a 3-transmit-4-receive or 4-transmit-4-receive antenna combination, supporting distance-angle two-dimensional resolution capability. The processing unit 103 can include a multi-modal health monitoring device 104. It should be noted that in the embodiments of the present disclosure, the module division is only an exemplary description for clearly explaining the technical solutions and function implementation of the present disclosure. Those skilled in the art should understand that the division of the modules can be adjusted and optimized according to the actual application scene and technical requirements, without departing from the protection scope of the present disclosure.

[0059] Again refer toFigure 1B In some embodiments, the radar signal generation module 102 is configured to generate a linear frequency modulation signal, and transmit the linear frequency modulation signal through the antenna array 101. The linear frequency modulation signal is composed of one or more bursts, and each burst includes a plurality of chirp signals. After the linear frequency modulation signal is transmitted, a return signal is formed after being reflected by one or more targets, and the return signal is received by a receiving antenna. The radar signal generation module 102 is further configured to sample and transform the return signal received by the antenna array 101, and transmit the processed return signal to the processing unit 103. In some embodiments, the multi-modal health monitoring device 104 performs clustering analysis on the return signal received by the FMCW radar 110 according to the multi-modal health monitoring method of an embodiment of the present disclosure, dynamically determines the behavior state related to the bed 120 according to the clustering result, and switches between a plurality of health monitoring modes including a bed occupancy monitoring mode, a vital sign monitoring mode, and a fall monitoring mode according to the behavior state related to the bed 120 and the switching condition of the finite state machine.

[0060] It should be noted that in the embodiments of the present disclosure, the dynamic integration and seamless switching of the multi-modal monitoring function are realized by only combining a single FMCW radar 110 with a finite state machine. The algorithm module in the multi-modal health monitoring device 104 using the FMCW radar 110 realizes a unified hardware platform and an adaptive algorithm framework, which not only reduces the device deployment cost, but also protects user privacy by not using visual data acquisition features. At the same time, the hierarchical logic judgment mechanism of the finite state machine realizes the cooperative operation of bed occupancy monitoring, vital sign monitoring, and fall monitoring in complex scenarios, avoids data conflicts and false triggers between multi-sensor systems, and effectively solves the problems of high hardware cost, privacy controversy, and complex maintenance caused by the simultaneous deployment of multiple monitoring devices in related technologies.

[0061] Those skilled in the art should understand that the algorithm module in the multi-modal health monitoring device 104 can be extended and optimized according to actual application scenarios and technical requirements without departing from the protection scope of the present disclosure. Only one FMCW radar 110 and its corresponding algorithm need to be maintained, and once there is a new function or algorithm update, the device can be quickly upgraded through OTA, reducing after-sales and maintenance costs.

[0062] Since the specific process of multi-modal health monitoring will be described in detail below, it will not be described here.

[0063] Figure 2 A flowchart of a multi-modal health monitoring method according to an embodiment of the present disclosure is shown. The multi-modal health monitoring method in the embodiments of the present disclosure is applied to the multi-modal health monitoring device 104. As shown in FIG. 6, the multi-modal health monitoring method includes: Figure 2 ​

[0064] In step S210, a chirp signal is transmitted and a return signal is received by a frequency-modulated continuous wave radar arranged in a vicinity of a bed area.

[0065] In some embodiments, the radar signal generation module 102 generates a chirp signal, and the chirp signal is transmitted by the antenna array 101 to the bed area 120 and the surrounding activity space. After the chirp signal is emitted, the chirp signal is reflected by one or more targets to form a return signal, which is received by the receiving antenna. The radar signal generation module 102 is also used to sample and transform the return signal received by the antenna array 101, and generate a four-dimensional radar cube containing distance, speed, horizontal angle and pitch angle parameters, to provide raw data support for subsequent multi-modal health monitoring. The radar signal generation module 102 transmits the four-dimensional radar cube to the processing unit 103.

[0066] In step S220, a clustering analysis is performed on the return signal, and a bed-related behavior state is dynamically determined according to the clustering result, the behavior state including bed stable occupancy, continuous empty bed and object movement.

[0067] In some embodiments, the processing unit 103 performs radar field of view range scanning every preset period (e.g., 5 minutes), and by generating a range-angle heatmap from the four-dimensional radar cube data of the received echo signals, a spatial clustering algorithm (such as DBSCAN) can be used to group the target point cloud, and a dynamic environment model of the bed area is established. By comparing the preset bed 120 area (i.e., the region of interest, abbreviated as ROI in English) with the surrounding environment baseline data in real time, the model can effectively distinguish between target objects and fixed interference sources (such as fans, etc.), providing a basis for subsequent mode switching. In some embodiments, the processing unit 103 calculates the cluster stability and cluster area of the bed 120 area. The cluster stability index is evaluated by the position offset, intensity fluctuation value, and angle consistency of the cluster between consecutive frames; a deep learning module (such as CNN or LSTM) can also be used for regression or classification judgment on the heatmap. The cluster area index is obtained from the spatial distribution characteristics on the range-angle heatmap. In some embodiments, if the cluster stability of the bed 120 area is greater than a first preset threshold (e.g., 80%) and / or the duration of the cluster area of the bed 120 area within a preset range (e.g., 0.5 to 2.0 square meters) reaches a second preset threshold, the behavior state is determined to be stable bed occupancy. If the cluster stability of the bed 120 area is less than or equal to the first preset threshold (e.g., 80%) and / or there are rapid movement features (e.g., the moving speed of the cluster exceeds a preset kinetic energy threshold) in the cluster in the range-angle heatmap and / or a new cluster appears in the area around the bed 120 in the range-angle heatmap, the behavior state is determined to be object movement. If the cluster intensity of the bed 120 area in the range-angle heatmap is lower than a third preset threshold and / or the duration of no other cluster in the bed 120 area reaches a fourth preset threshold, the behavior state is determined to be a continuous empty bed.

[0068] In step S230, switching between a plurality of monitoring modes according to the behavior state and the switching condition of the finite state machine, the plurality of monitoring modes including a bed occupancy monitoring mode, a vital sign monitoring mode, and a fall monitoring mode; wherein the switching condition of the finite state machine includes switching to the vital sign monitoring mode when the behavior state is stable bed occupancy, switching to the fall monitoring mode when the behavior state is object movement, and switching to the bed occupancy monitoring mode when the behavior state is a continuous empty bed.

[0069] In some embodiments, the bed 120 related behavior state and the switching condition of the finite state machine are used to switch among multiple monitoring modes, including bed occupancy monitoring mode, vital sign monitoring mode, and fall monitoring mode. When the bed 120 related behavior state is determined as stable bed occupancy, the processing unit 103 switches to the vital sign monitoring mode. In the vital sign monitoring mode, the chest micro-motion signal is extracted by phase unwrapping technique: the region with the largest standard deviation in the distance-angle heat map of consecutive multiple frames is determined as the chest region, and the chest region data in the distance-angle heat map of consecutive multiple frames is processed by a sliding window (e.g., 20 frames) and phase unwrapping to eliminate the 2π phase folding effect, thereby extracting the chest micro-motion phase information. Then, the first difference operation is performed on the phase unwrapped chest micro-motion phase information due to the slow variability of respiration to eliminate the trend bias, and the respiratory signal is extracted by a band-pass filter (e.g., passband is 0.1-0.5 Hz). Further, the second difference operation is performed on the respiratory signal due to the small heartbeat signal and the influence of the respiratory signal (harmonic component), and the heartbeat signal is obtained after being processed by a band-pass filter (e.g., passband is 0.8-2 Hz). Fast Fourier Transform (FFT) is performed on the obtained respiratory signal and heartbeat signal respectively, and the respiratory spectrum and respiratory frequency peak value, and the heartbeat spectrum and heartbeat frequency peak value are determined by spectrum analysis. In some embodiments, in the case of stable bed occupancy and the processing unit 103 activating the vital sign monitoring mode, the processing unit 103 can simultaneously maintain the execution of the bed occupancy monitoring mode and the vital sign monitoring mode.

[0070] In some embodiments, when the behavior state associated with bed 120 is determined to be object movement, processing unit 103 automatically switches to fall monitoring mode. In fall monitoring mode, micro-Doppler signatures or 4D radar cube data are used to analyze the target's motion trajectory. For example, the 4D radar cube calculates the pitch angle altitude change rate (Δθ / Δt) and the kinetic energy index (ΔR² + Δv²). When a sudden change in pitch angle altitude exceeding a fifth preset threshold (e.g., 0.8 m / s) and an abnormal kinetic energy index (i.e., outside a safe range) is detected, a fall event is detected, triggering a fall alarm. A deep learning model can also be used to determine whether a fall event has occurred to improve accuracy and reduce the probability of false alarms. In some embodiments, when the behavior state associated with bed 120 is determined to be continuously vacant, processing unit 103 automatically switches to bed occupancy monitoring mode. In some embodiments, when the stability of a cluster in the distance-angle heat map is detected to be below a sixth preset threshold in vital sign monitoring mode, processing unit 103 switches to fall monitoring mode. In some embodiments, when the continuous operation time in the fall monitoring mode exceeds a seventh preset threshold and no fall event is detected, the processing unit 103 switches to the bed occupancy monitoring mode.

[0071] It should be noted that in some embodiments, a personalized behavioral pattern baseline is established for the target based on historical monitoring data. This personalized behavioral pattern baseline includes the target's daily activity patterns, vital sign fluctuation characteristics, and spatial movement trajectory characteristics. Based on this personalized behavioral pattern baseline, the first through seventh preset thresholds are adaptively adjusted, enabling the FMCW radar 110 to dynamically adapt to the physiological characteristics and behavioral habits of different users. For example, the phase difference window length is adjusted to account for differences in breathing rate, or the cluster stability calculation threshold is modified based on the range of daily activities. This establishes a personalized behavioral pattern database, and based on this database, threshold parameters are adjusted and optimized. This significantly improves monitoring accuracy and scenario adaptability, while also avoiding the risk of misjudgment associated with traditional fixed thresholds due to individual differences or environmental changes.

[0072] Figure 3 FIG. 1 is a schematic diagram of a state machine for multimodal health monitoring according to an embodiment of the present disclosure. Figure 3As shown, in some embodiments, processing unit 103 is in bed occupancy monitoring mode. In bed occupancy monitoring mode, it scans the radar field of view at predetermined intervals, spatially clusters echo signals based on the distance-angle heat map, and calculates cluster stability and / or cluster area in the area of ​​bed 120. If the behavior state associated with bed 120 is determined to be stable occupancy, processing unit 103 switches to vital sign monitoring mode. If the behavior state associated with bed 120 is determined to be object movement, processing unit 103 switches to fall monitoring mode. If the behavior state associated with bed 120 is determined to be continuously vacant, processing unit 103 switches to bed occupancy monitoring mode. In some embodiments, when the stability of a cluster detected in the distance-angle heat map is lower than a sixth preset threshold in the vital sign monitoring mode, processing unit 103 switches to fall monitoring mode. In some embodiments, when the continuous operation time in fall monitoring mode exceeds a seventh preset threshold and no fall event is detected, processing unit 103 switches to bed occupancy monitoring mode.

[0073] It should be noted that the dynamic mode switching mechanism based on the finite state machine realizes the organic integration of multimodal health monitoring functions through multi-level logical judgment, so that different monitoring modes can maintain functional independence and data reliability when sharing the same hardware platform, and solves the problem of false triggering caused by parameter conflicts between multi-sensor systems.

[0074] Figure 4 FIG. 1 shows a schematic diagram of the structure of a multimodal health monitoring device according to an embodiment of the present disclosure. Figure 4 The multimodal health monitoring device 400 shown includes an echo signal receiving module 410 , a cluster analysis module 420 , and a monitoring mode switching module 430 .

[0075] The echo signal receiving module 410 is used to transmit a linear frequency modulation signal and receive an echo signal through a frequency modulation continuous wave radar, wherein the frequency modulation continuous wave radar is set in an area near the bed.

[0076] The cluster analysis module 420 is configured to perform cluster analysis on the echo signals and dynamically determine the bed-related behavior status based on the clustering results. The behavior status includes stable bed occupancy, continuous empty bed, and object movement.

[0077] The monitoring mode switching module 430 is used to switch between multiple monitoring modes according to the behavior state and the switching conditions of the finite state machine. The multiple monitoring modes include bed occupancy monitoring mode, vital signs monitoring mode and fall monitoring mode; wherein the switching conditions of the finite state machine include switching to vital signs monitoring mode when the behavior state is stable bed occupancy, switching to fall monitoring mode when the behavior state is object movement, and switching to bed occupancy monitoring mode when the behavior state is continuous empty bed.

[0078] Since the specific process of the multi-modal health monitoring has been described in detail above, it will not be repeated here.

[0079] The embodiments of the present disclosure further provide an electronic device, which comprises a memory 520, a processor 510 and a program stored in the memory 520 and executable on the processor 510, wherein the program is executed by the processor 510 to implement the processes of each embodiment of the above method and achieve the same technical effects. To avoid repetition, it will not be repeated here. Figure 5 As shown, the electronic device comprises a memory 520, a processor 510 and a program stored in the memory 520 and executable on the processor 510, wherein the program is executed by the processor 510 to implement the processes of each embodiment of the above method and achieve the same technical effects. To avoid repetition, it will not be repeated here.

[0080] Those skilled in the art can understand that all or part of the steps of the various methods of the above embodiments can be completed by instructions or control relevant hardware by instructions, and the instructions can be stored in a computer readable storage medium and loaded and executed by a processor. For this purpose, the embodiments of the present disclosure further provide a storage medium having a computer program or instructions stored thereon, wherein the computer program or instructions are executed by the processor to implement the processes of each embodiment of the above method.

[0081] Since the instructions stored in the storage medium can execute the steps in the method provided by the embodiments of the present disclosure, the beneficial effects that can be achieved by the method provided by the embodiments of the present disclosure can be achieved, which are described in detail in the above embodiments and will not be repeated here. The specific implementation of each operation can refer to the above embodiments, which will not be repeated here.

[0082] In summary, according to the embodiments of the present disclosure, a linear frequency modulation signal is transmitted by the frequency modulation continuous wave radar arranged in the area near the bed and a return signal is received, the return signal is subjected to cluster analysis, the bed-related behavior state is dynamically determined according to the cluster result, the behavior state includes bed stable occupation, continuous empty bed and object movement, the multiple monitoring modes are switched according to the behavior state and the switching condition of the finite state machine, the multiple monitoring modes include bed occupation monitoring mode, vital sign monitoring mode and fall monitoring mode, wherein the switching condition of the finite state machine includes switching to the vital sign monitoring mode when the behavior state is the bed stable occupation, switching to the fall monitoring mode when the behavior state is the object movement, and switching to the bed occupation monitoring mode when the behavior state is the continuous empty bed. In this way, by combining the single frequency modulation continuous wave radar with the finite state machine, the dynamic integration and seamless switching of the multi-modal health monitoring function are realized, by using the unified hardware platform and the adaptive algorithm framework, not only the device deployment cost is reduced, but also the user privacy is protected by the non-vision data acquisition characteristics, and by using the hierarchical logic judgment mechanism of the finite state machine, the cooperative operation of the bed occupation monitoring, the vital sign monitoring and the fall monitoring is realized in the complex scene, the data conflict and the false triggering between the multiple sensor systems are avoided, and the problems of high hardware cost, great privacy controversy and complex maintenance caused by the simultaneous deployment of multiple monitoring devices in the related art are effectively solved.

[0083] Finally, it should be noted that: obviously, the above embodiments are only examples for clearly illustrating the present disclosure, and are not limitations on the embodiments. Based on the above description, other different forms of changes or variations can be made by those of ordinary skill in the art. Here, all the embodiments cannot be exhausted, and the obvious changes or variations derived therefrom are still within the protection scope of the present disclosure.

Claims

1. A multimodal health monitoring method, comprising: Transmitting a linear frequency modulation signal and receiving an echo signal by a frequency modulation continuous wave radar, wherein the frequency modulation continuous wave radar is arranged in an area near the bed; performing cluster analysis on the echo signals, and dynamically determining the behavior state related to the bed based on the clustering results, wherein the behavior state includes stable bed occupancy, continuous empty bed, and object movement; Switching between multiple monitoring modes according to the behavioral state and the switching conditions of the finite state machine, the multiple monitoring modes including a bed occupancy monitoring mode, a vital sign monitoring mode, and a fall monitoring mode; Among them, the switching conditions of the finite state machine include switching to vital signs monitoring mode when the behavior state is stable bed occupancy, switching to fall monitoring mode when the behavior state is object movement, and switching to bed occupancy monitoring mode when the behavior state is continuous empty bed.

2. The multimodal health monitoring method according to claim 1, wherein: The performing cluster analysis on the echo signal and dynamically determining the bed-related behavior state according to the clustering result includes: Performing spatial clustering on the echo signals based on the distance-angle heat map to calculate the cluster stability and / or cluster area of ​​the bed area; If the cluster stability of the bed area is greater than a first preset threshold and / or the duration of the cluster area of ​​the bed area being within a preset range reaches a second preset threshold, then the behavior state is determined to be stable bed occupancy; If the cluster stability of the bed area is less than or equal to the first preset threshold and / or the cluster in the distance-angle heat map has a fast movement feature and / or a new cluster appears in the area around the bed in the distance-angle heat map, then the behavior state is determined to be object movement; If the cluster intensity of the bed area in the distance-angle heat map is lower than a third preset threshold and / or the duration without other clusters in the bed area reaches a fourth preset threshold, the behavior state is determined to be a continuous empty bed.

3. The multimodal health monitoring method according to claim 2, wherein: The vital signs monitoring modes include: Determine the area with the largest standard deviation in the distance-angle heat map of the multiple frames as the chest area; Performing phase unwrapping processing on the clusters corresponding to the chest area in the multiple frames of the distance-angle heat map to extract the chest micro-motion phase information; After performing a first differential operation on the chest micro-motion phase information, a band-pass filter is used to extract the respiratory signal; After performing a second differential operation on the respiratory signal, a bandpass filter is used to extract the heartbeat signal; The respiratory spectrum and respiratory rate peak, as well as the heartbeat spectrum and heartbeat rate peak are determined through spectrum analysis.

4. The multimodal health monitoring method according to claim 3, wherein: The fall monitoring mode includes: Analysis using micro-Doppler signatures or 4D radar cube data; When a sudden change in the pitch angle is detected to exceed a fifth preset threshold and / or a fall event is determined to have occurred using the output of a deep learning model, it is determined that a fall event has been detected and a fall alarm is triggered.

5. The multimodal health monitoring method according to claim 4, wherein: The switching conditions of the finite state machine also include: When it is detected in the vital sign monitoring mode that the stability of the cluster in the distance-angle heat map is lower than a sixth preset threshold, switching to the fall monitoring mode; When the continuous operation time in the fall monitoring mode exceeds a seventh preset threshold and no fall event is detected, the mode is switched to the bed occupancy monitoring mode.

6. The multimodal health monitoring method according to claim 5, wherein: The multimodal health monitoring method further includes: Establish a personalized behavioral pattern baseline for the target based on historical monitoring data; Adaptively adjusting the first preset threshold to the seventh preset threshold according to the personalized behavior pattern baseline; The personalized behavior pattern baseline includes the target object's daily activity patterns, vital sign fluctuation characteristics, and spatial movement trajectory characteristics.

7. A multimodal health monitoring device comprising: an echo signal receiving module, configured to transmit a linear frequency modulation signal and receive an echo signal via a frequency modulation continuous wave radar, wherein the frequency modulation continuous wave radar is arranged in an area near the bed; a cluster analysis module, configured to perform cluster analysis on the echo signals and dynamically determine the behavior status related to the bed based on the clustering results, wherein the behavior status includes stable bed occupancy, continuous empty bed, and object movement; a monitoring mode switching module, configured to switch between a plurality of monitoring modes according to the behavioral state and a switching condition of the finite state machine, the plurality of monitoring modes including a bed occupancy monitoring mode, a vital sign monitoring mode, and a fall monitoring mode; Among them, the switching conditions of the finite state machine include switching to vital signs monitoring mode when the behavior state is stable bed occupancy, switching to fall monitoring mode when the behavior state is object movement, and switching to bed occupancy monitoring mode when the behavior state is continuous empty bed.

8. A radar, installed in the vicinity of a bed, comprising: Antenna arrays; a radar signal generation module, configured to generate a linear frequency modulation signal, transmit the linear frequency modulation signal through the antenna array, and perform signal processing on the echo signal received by the antenna array; A processing unit, comprising the multimodal health monitoring device according to claim 7, switching between multiple monitoring modes according to the steps of the method according to any one of claims 1 to 6 and the switching conditions of the finite state machine based on the bed-related behavioral state and the finite state machine, the multiple monitoring modes including a bed occupancy monitoring mode, a vital sign monitoring mode and a fall monitoring mode.

9. An electronic device comprising: A processor, a memory, and a program stored in the memory and executable on the processor, wherein the program implements the steps of the method according to any one of claims 1 to 6 when executed by the processor.

10. A storage medium having a computer program or instructions stored thereon, wherein the computer program or instructions, when executed by a processor, implements the steps of the method according to any one of claims 1 to 6.

Images