Multi-millimeter wave frequency band fusion room personnel state monitoring method and system

Abstract

The invention relates to the technical field of millimeter wave communication, and discloses a multi-millimeter wave frequency band fusion room personnel state monitoring method and system, and the method comprises the steps: carrying out the compensation of a collected multi-frequency band millimeter wave signal based on a self-adaptive cross-frequency band different-source compensation mechanism; performing feature extraction and unified fusion representation on the compensated multi-band millimeter wave signal to obtain a band fusion feature, and identifying a personnel state by using a personnel state identification model; and carrying out risk state monitoring and risk early warning on the personnel state time sequence in the room. According to the method, the multi-band millimeter wave signals are collected, the significant representation features of different millimeter wave bands are extracted, personnel state recognition is carried out based on the multi-dimensional significant representation features, and high-robustness recognition of multiple behavior states such as static, moving and violent change of personnel in a room in a complex environment is achieved; and timely early warning and automatic response to a high-risk state are realized by constructing a state sequence and a risk judgment mechanism.

Description

Technical Field

[0001] This invention relates to the field of millimeter-wave communication technology, and in particular to a method and system for monitoring the status of people in a room by multi-millimeter-wave frequency band fusion. Background Technology

[0002] With the increasing demands for precision, real-time performance, and non-contact methods in applications such as smart homes, intelligent security, healthcare, and industrial operations, personnel status monitoring technology based on millimeter-wave radar has gradually become a research hotspot. Millimeter-wave radar possesses advantages such as strong anti-interference capabilities, high penetration, and good environmental adaptability, enabling the detection of human movement characteristics under conditions of darkness, obstruction, and non-cooperation. Especially in scenarios with high privacy and high reliability requirements, millimeter-wave radar exhibits significant advantages over traditional image sensors and is widely used in fields such as fall detection, abnormal activity recognition, and vital sign monitoring.

[0003] In existing research, some solutions attempt to comprehensively assess personnel status by combining multimodal perception methods. For example, patent CN113573237B proposes a method, system, and terminal for personnel status monitoring based on facial authentication and location awareness. This solution uses a status monitoring app on the terminal for registration and facial authentication, combined with electronic fences and mobile device location awareness, to monitor personnel behavior in isolated situations. It also resets the timer based on facial recognition failure or abnormal location information for status management in epidemic prevention scenarios. While this method is practical in terms of identity binding and location verification, it relies on visual and terminal interaction and lacks support for non-cooperative, non-contact status perception, making it unsuitable for scenarios where the target is out of sight, in poor lighting, or with complex occlusion.

[0004] Furthermore, traditional personnel monitoring methods typically rely on devices such as cameras and infrared sensors. However, these methods often suffer from problems such as privacy breaches, susceptibility to lighting conditions, and low accuracy, making them unsuitable for personnel status detection in public environments. Traditional single-band millimeter-wave radar monitoring systems, due to their fixed frequency band, experience a significant drop in accuracy when faced with issues such as target obstruction, reflection attenuation, or multipath interference.

[0005] To address this issue, this invention proposes a multi-scale millimeter-wave frequency band fusion method and system for monitoring the status of people in a room. By combining the advantages of millimeter-wave information from multiple different frequency bands through multi-scale millimeter-wave frequency band fusion, the shortcomings of a single frequency band can be compensated for, thereby improving the accuracy and robustness of personnel status monitoring. Summary of the Invention

[0006] This invention provides a multi-millimeter-wave band fusion method and system for monitoring the status of people in a room. By deploying multi-band radar sensing units, it introduces the complementary characteristics of different frequency bands in terms of spatial penetration and accuracy, and improves the overall data integrity and quality based on a cross-band heterogeneous source compensation mechanism, achieving highly robust monitoring of dynamically obstructed environments. A frequency band-adaptive salient feature extraction method is designed, such as extracting micro-motion energy spectrum in the high-frequency band, velocity spread and echo intensity distribution in the mid-frequency band, and signal-to-noise ratio and energy statistics in the low-frequency band, improving the specificity and recognition resolution of frequency band sensing. Since single-band status recognition is easily affected by noise, obstruction, or motion blur, resulting in unstable output and an inability to continuously reflect real behavioral changes, this application significantly improves the accuracy and anti-interference capability of recognition by uniformly fusing multi-band salient features and inputting them into the personnel status recognition model. Simultaneously, by combining the status output time sequence, it achieves dynamic tracking of status and risk perception.

[0007] To achieve the above objectives, the present invention provides a method for monitoring the status of occupants in a room using multi-millimeter-wave band fusion, comprising the following steps:

[0008] S1: Deploy millimeter-wave radar sensing units that support different millimeter-wave frequency bands in the target room, use the millimeter-wave radar sensing units to collect multi-band millimeter-wave signals in the room in real time, and compensate the collected multi-band millimeter-wave signals based on an adaptive cross-band heterogeneous source compensation mechanism to obtain compensated multi-band millimeter-wave signals.

[0009] S2: Extract features from the compensated multi-band millimeter wave signal to obtain significant representation features of different millimeter wave frequency bands;

[0010] S3: Perform unified fusion representation of the salient representation features of different millimeter wave frequency bands to obtain frequency band fusion features. Utilize the personnel status recognition model to receive the frequency band fusion features and output the personnel status in the room.

[0011] S4: Sort the room status of personnel output by the personnel status recognition model according to the acquisition time of the associated multi-band millimeter wave signal to obtain the room status time sequence. Monitor the risk status of the room status time sequence. If a high-risk status is detected, an early warning will be automatically triggered.

[0012] As a further improvement of the present invention:

[0013] Optionally, millimeter-wave radar sensing units supporting different millimeter-wave frequency bands are deployed in the target room to collect multi-band millimeter-wave signals in the room in real time, including:

[0014] The millimeter-wave sensing unit consists of a signal transmitting device, a signal receiving antenna, a mixing module, and a signal processing module;

[0015] The signal transmitting device in the millimeter-wave radar sensing unit transmits continuous frequency-modulated signals into the room and receives the echo signals reflected after encountering a target. The frequency-modulated signal is in the form of electromagnetic waves. The mixing module performs frequency mixing processing on the frequency-modulated signal and the echo signal, and extracts the beat frequency signal to form an echo sequence.

[0016] The signal processing module performs a one-dimensional fast Fourier transform on the echo sequence to generate a range spectrum. The range spectrum represents the echo signal intensity at different range intervals in the signal transmission direction of the millimeter-wave radar sensing unit. The range interval is the maximum ranging of the millimeter-wave radar sensing unit divided into multiple range intervals.

[0017] For the range spectrum associated with multiple consecutive frequency-modulated signals, a second fast Fourier transform is performed on the time series to obtain the range-Doppler matrix. The range-Doppler matrix represents the echo signal intensity in different range intervals and different radial velocities in the signal transmission direction of the millimeter-wave radar sensing unit, and is used to identify moving targets and the speed of the moving targets.

[0018] An angle estimation algorithm is used to estimate the phase difference of multiple receiving antennas at the same distance in the range spectrum to obtain a range-angle matrix. The range-angle matrix represents the echo signal intensity at different distances and azimuth angles in the signal transmission direction of the millimeter-wave radar sensing unit, and is used to reflect the angular distribution of targets in the room.

[0019] The echo sequence, range spectrum, range-Doppler matrix, and range-angle matrix are used as millimeter-wave signals collected by the millimeter-wave radar sensing unit, and the millimeter-wave signals collected by the millimeter-wave radar sensing units of different millimeter-wave frequency bands are used to form a multi-band millimeter-wave signal.

[0020] Optionally, the acquired multi-band millimeter-wave signals are compensated based on an adaptive cross-band heterogeneous source compensation mechanism to obtain compensated multi-band millimeter-wave signals, including:

[0021] Acquire the collected multi-band millimeter wave signals and extract the millimeter wave signals corresponding to different millimeter wave frequency bands;

[0022] The quality of the millimeter-wave signal is assessed, and the signal quality is determined. If the signal quality of the millimeter-wave signal is lower than a preset quality threshold, the signal is considered to be of lower quality. The millimeter-wave signal is then labeled as the millimeter-wave signal to be compensated, and neighboring millimeter-wave signals are adaptively extracted. A cross-band heterogeneous source compensation mechanism is used to compensate the millimeter-wave signal. The neighboring millimeter-wave signal is the one among the multi-band millimeter-wave signals that is closest to the frequency band of the millimeter-wave signal to be compensated and whose signal quality is higher than a preset quality threshold. Millimeter-wave signals;

[0023] The compensation formula for the millimeter-wave signal L to be compensated is:

[0024]

[0025] in, This represents the compensation result of the millimeter-wave signal L to be compensated, where L is the signal to be compensated. * φ represents the neighboring millimeter-wave signal of the millimeter-wave signal L to be compensated. norm (L * ;L) indicates that the adjacent millimeter-wave signal L * The scale in the range spectrum is mapped to the scale range of the millimeter-wave signal L to be compensated, where the scale includes the number of range intervals in the range spectrum, the sequence length of the echo sequence, the matrix specifications of the range-Doppler matrix and the range-angle matrix, and the matrix specifications include the number of matrix rows and the number of matrix columns.

[0026] MLP(·) represents a multilayer perceptron;

[0027] exp(·) denotes an exponential function with base to the natural constant, f(L), f(L) * The numbers L1 and L2 represent the millimeter-wave signal L1 to be compensated and the adjacent millimeter-wave signal L2, respectively. * The corresponding millimeter wave frequency band, This represents the millimeter-wave signal L to be compensated and the adjacent millimeter-wave signal L. * Frequency band differences between them.

[0028] Optionally, feature extraction is performed on the compensated multi-band millimeter-wave signal to obtain salient representation features of different millimeter-wave frequency bands, including:

[0029] The millimeter wave signals corresponding to different millimeter wave frequency bands in the compensated multi-band millimeter wave signal are extracted. According to the size of the millimeter wave frequency band, the millimeter wave frequency band is divided into high frequency band, mid frequency band and low frequency band. Adaptive feature extraction is adopted for the high frequency band, mid frequency band and low frequency band millimeter wave signals to obtain the significant representation features of different millimeter wave frequency bands.

[0030] The method for extracting the adaptability features of the high-frequency millimeter-wave signal is as follows:

[0031] The energy of any distance interval in the range-Doppler matrix of a millimeter-wave signal within the range of micro-motion velocity is calculated and used as the micro-motion energy spectrum of the high-frequency millimeter-wave signal. The micro-motion energy spectrum is then used as a significant characteristic of the high-frequency millimeter-wave band.

[0032] The method for extracting the adaptability features of the mid-frequency millimeter-wave signal is as follows:

[0033] The extracted echo signal intensity in the range spectrum is higher than the preset range echo intensity. The range intervals are defined, and the mean of the velocity spread in the range-Doppler matrix across all extracted range intervals is calculated as the motion monitoring feature of the mid-frequency millimeter-wave signal.

[0034] The echo signal intensity distribution at different azimuth angles is extracted from the range-angle matrix. The echo signal intensity distribution is the difference between the maximum echo signal intensity at different azimuth angles and the mean echo signal intensity in the range-angle matrix. The standard deviation of the calculated standard deviation is used to represent the motion monitoring characteristics and the echo signal intensity distribution as significant features of the mid-frequency millimeter wave band.

[0035] The method for extracting the adaptability features of the low-frequency millimeter-wave signal is as follows:

[0036] The significant characteristics of the low-frequency millimeter-wave band include the normalized signal-to-noise ratio of the millimeter-wave signal and the mean of the echo sequence.

[0037] Optionally, salient representation features of different millimeter-wave frequency bands are uniformly fused to obtain frequency band fusion features, including:

[0038] Calculate the mean salient representation features of all millimeter-wave frequency bands in the high-frequency band as the high-frequency band component;

[0039] Calculate the mean salient representation features of all millimeter-wave frequency bands in the mid-frequency band as the mid-frequency component;

[0040] Calculate the mean salient representation features of all millimeter-wave frequency bands in the low-frequency band as the low-frequency component;

[0041] The high-frequency, mid-frequency, and low-frequency components are spliced ​​together to form the frequency band fusion feature.

[0042] Optionally, the frequency band fusion features of the personnel status recognition model are received, and the status of personnel in the room is output, including:

[0043] The personnel state recognition model includes a feature alignment layer, a frequency band weighted fusion layer, and a state recognition layer. The feature alignment layer uses a linear transformation to map high-frequency, mid-frequency, and low-frequency components to a unified dimension. The frequency band weighted fusion layer is used to calculate the attention weights of the high-frequency, mid-frequency, and low-frequency components after unifying the dimension, and performs weighted processing on the high-frequency, mid-frequency, and low-frequency components after unifying the dimension to obtain frequency band weighted fusion features.

[0044] Attention weights for high-frequency, mid-frequency, and low-frequency components are generated based on the normalized signal-to-noise ratio mean.

[0045]

[0046] Where exp(·) represents an exponential function with the natural constant as the base, γ represents the attention control parameter, and γ is set to 0.2; ω1, ω2, and ω3 represent the attention weights of the high-frequency component, mid-frequency component, and low-frequency component, respectively.

[0047] The state recognition layer is in the form of a lightweight multilayer perceptron, used to receive frequency band weighted fusion features and output the predicted probability distribution of personnel state. The personnel state category with the highest predicted probability is selected as the personnel state in the room. The predicted probability distribution of personnel state is a vector composed of the predicted probabilities of different personnel state categories. The personnel state categories include stationary, slow walking, fast moving, drastic changes, and multiple people moving.

[0048] Optionally, the room occupant status output by the occupant status recognition model is sorted according to the acquisition time of the associated multi-band millimeter-wave signal to obtain a time-series sequence of room occupant status, including:

[0049] The length of the time sequence of the status of people in the room is Len, where the time sequence of the status of people in the room acquired at time t using multi-band millimeter wave signals is:

[0050] H t =[H(t-Len+1),H(t-Len+2),...,H(t-1),H(t)];

[0051] Among them, H t Let H(t-Len+1), H(t-Len+2), H(t-1), and H(t) represent the room status of people associated with the multi-band millimeter-wave signals collected at time t, respectively. Time t represents the collection time of any multi-band millimeter-wave signal.

[0052] Optionally, the system monitors the time sequence of the status of people in the room for risk assessment. If a high-risk status is detected, an alert is automatically triggered, including:

[0053] Set risk weight coefficients for different personnel status categories, and replace the risk weight coefficients with the interrelated personnel statuses in the time series of personnel status in the room to form a time series of risk weight coefficients;

[0054] The weighted average risk value of the time series of risk weight coefficients is calculated. If the weighted average risk value is higher than the preset risk threshold, it indicates that a high-risk state has been detected and an early warning is automatically triggered.

[0055] The time sequence of the status of people in the room H t The corresponding time series sequence of risk weight coefficients is ht The time series sequence h of the risk weight coefficients t The weighted average risk value is:

[0056]

[0057] g j =exp[-60β·(tj)];

[0058] Among them, S t The time series sequence h represents the risk weight coefficients. t The weighted average risk value, h(j) represents the risk weight coefficient associated with the state H(j) of the people in the room at time j, j∈[t-Len+1,t];

[0059] g j represents the weighting coefficient of the risk weighting coefficient h(j), 60 represents the time interval in seconds between adjacent moments, and β represents the time decay factor, set to 0.05;

[0060] The preset risk threshold is set to 0.75. The higher the weighted average risk value, the more likely the people in the room are to be in a state of sudden, violent, uncontrollable or abnormal activity, and there is a higher probability of dangerous events or abnormal behaviors in the environment.

[0061] To address the aforementioned problems, the present invention also provides a multi-millimeter-wave band fusion room occupant status monitoring system, which includes a data acquisition device, a feature extraction module, and a status monitoring module.

[0062] The data acquisition device is used to deploy millimeter-wave radar sensing units supporting different millimeter-wave frequency bands in the target room, and to collect multi-band millimeter-wave signals in the room in real time using the millimeter-wave radar sensing units. The collected multi-band millimeter-wave signals are then compensated based on an adaptive cross-band heterogeneous source compensation mechanism to obtain compensated multi-band millimeter-wave signals.

[0063] The feature extraction module is used to extract features from the compensated multi-band millimeter wave signal to obtain significant representation features of different millimeter wave frequency bands;

[0064] The status monitoring module is used to perform unified fusion representation of the significant representation features of different millimeter wave frequency bands to obtain frequency band fusion features. The personnel status recognition model receives the frequency band fusion features and outputs the status of personnel in the room. The status of personnel in the room output by the personnel status recognition model is sorted according to the acquisition time of the associated multi-frequency band millimeter wave signals to obtain the time sequence of personnel status in the room. The risk status is monitored on the time sequence of personnel status in the room. If a high-risk status is detected, an early warning is automatically triggered.

[0065] This is to achieve a multi-millimeter-wave band fusion method for monitoring the status of people in a room, as described above.

[0066] To address the above problems, the present invention also provides an electronic device, the electronic device comprising:

[0067] Memory, storing at least one instruction;

[0068] Communication interfaces enable communication between electronic devices; and

[0069] The processor executes the instructions stored in the memory to implement the multi-millimeter-wave band fusion method for monitoring the status of people in a room, as described above.

[0070] To address the aforementioned problems, the present invention also provides a computer-readable storage medium storing at least one instruction, which is executed by a processor in an electronic device to implement the aforementioned multi-millimeter-wave band fusion method for monitoring the status of people in a room.

[0071] Compared with existing technologies, this invention proposes a method and system for monitoring the status of people in a room using multi-millimeter-wave frequency band fusion. This technology has the following beneficial effects:

[0072] First, this application introduces a frequency band division mechanism, dividing multi-band millimeter-wave signals into high-frequency, mid-frequency, and low-frequency bands based on the differences in the physical characteristics of each band. Adaptive feature extraction strategies are then employed for each band, significantly improving the ability to capture effective behavioral information in each band. Furthermore, high-frequency millimeter-wave signals possess higher range and velocity resolution, making them suitable for micro-motion detection. Therefore, extracting the energy of the micro-motion velocity range in their range-Doppler matrix as the micro-motion energy spectrum helps identify fine-grained dynamics such as breathing and gestures. Mid-frequency millimeter-wave signals achieve a balance between dynamic perception and angular resolution. Extracting the velocity expansion and azimuth reflection intensity distribution of target areas with prominent echo intensity effectively improves the accuracy of perceiving the number, structure, and direction of moving targets. Low-frequency millimeter-wave signals possess stronger penetration and stability, making them suitable for preliminary judgments of target presence. Extracting their signal-to-noise ratio and echo sequence mean enhances the ability to model the existence and perceived credibility of targets in obstructed areas. By extracting the aforementioned salient representation features, this invention achieves differentiated interpretation and structured summarization of multi-band information, avoiding scale mismatch and expression redundancy problems in the information fusion process, improving the robustness, interpretability, and real-time perception capability of multi-band fusion modeling for complex personnel state changes, and has good engineering deployability and practical application value.

[0073] Meanwhile, this application introduces a time decay weight function in the time aggregation process of risk weight coefficients. It constructs an exponential time weight by using the time interval between each time in the time series of risk weight coefficients and the current time. Compared with the traditional averaging method, this mechanism can highlight the dominant role of the risk weight coefficient at the current time on the risk, while avoiding the serious interference of historical risk weight coefficients on the risk assessment results. This improves the timeliness and accuracy of risk response, and is especially suitable for real-time response scenarios such as fall prediction and multi-state transition judgment. Attached Figure Description

[0074] Figure 1 This is a flowchart illustrating a multi-millimeter-wave band fusion method for monitoring the status of people in a room, as provided in an embodiment of the present invention.

[0075] The realization of the objective, functional features and advantages of the present invention will be further explained in conjunction with the embodiments and with reference to the accompanying drawings. Detailed Implementation

[0076] It should be understood that the specific embodiments described herein are merely illustrative of the invention and are not intended to limit the invention.

[0077] This application provides a method for monitoring the status of occupants in a room using multi-millimeter-band band fusion. The executing entity of this method includes, but is not limited to, at least one electronic device configured to execute the method provided in this application, such as a server or a terminal. In other words, the method can be executed by software or hardware installed on a terminal device or a server device, and the software may be a blockchain platform. The server includes, but is not limited to, a single server, a server cluster, a cloud server, or a cloud server cluster.

[0078] Reference Figure 1 Embodiment 1 of the present invention is as follows:

[0079] S1: Deploy millimeter-wave radar sensing units that support different millimeter-wave frequency bands in the target room, use the millimeter-wave radar sensing units to collect multi-band millimeter-wave signals in the room in real time, and compensate the collected multi-band millimeter-wave signals based on an adaptive cross-band heterogeneous source compensation mechanism to obtain compensated multi-band millimeter-wave signals.

[0080] Millimeter-wave radar sensing units supporting different millimeter-wave frequency bands are deployed in the target room. These units are used to collect multi-band millimeter-wave signals in the room in real time, including:

[0081] Specifically, multiple millimeter-wave radar sensing units operate in the 24GHz, 60GHz, and 77GHz frequency bands, respectively, and are installed in locations within the room with complementary viewing angles. The main control module uniformly schedules the acquisition timing to achieve the perception and compensation fusion of multi-frequency band and full-space personnel status information. Among them, the 24GHz millimeter-wave radar sensing unit is deployed at a high position in the corner for global perspective coverage, the 60GHz millimeter-wave radar sensing unit is deployed in key activity areas to achieve mesoscale attitude recognition, and the 77GHz millimeter-wave radar sensing unit is used for near-field micro-movement and physiological state perception.

[0082] It should be noted that the wavelengths corresponding to the 24GHz, 60GHz, and 77GHz frequency bands decrease sequentially, and the penetration ability of the associated signals also decreases sequentially, but the resolution increases sequentially, enabling higher-precision behavior monitoring. By deploying multi-band millimeter-wave radar sensing units in the room, joint coverage and complementary acquisition of space and frequency bands can be achieved. Specifically, each millimeter-wave radar sensing unit faces different directions to cover the entire room, achieving complementary viewing angles. 24GHz is used for full-field coverage, while 60GHz / 77GHz is used to locate key areas, achieving frequency band complementarity.

[0083] In this embodiment of the application, taking a bedroom as an example, a 24GHz millimeter-wave radar sensing unit is deployed at a high position in the corner of the bedroom and diagonally towards the center of the bedroom; a 60GHz millimeter-wave radar sensing unit is deployed on the wall of the bedroom directly opposite the bed and horizontally facing the bed to perform posture perception and turning trajectory perception; a 77GHz millimeter-wave radar sensing unit is deployed in the center of the ceiling and vertically downward to perform high-resolution analysis of breathing, heartbeat, hand movements, etc.

[0084] The millimeter-wave sensing unit consists of a signal transmitting device, a signal receiving antenna, a mixing module, and a signal processing module;

[0085] The signal transmitting device in the millimeter-wave radar sensing unit transmits a continuous frequency-modulated signal into the room and receives the echo signal reflected after encountering a target. The frequency-modulated signal is in the form of an electromagnetic wave. The mixing module performs frequency-mixing processing on the frequency-modulated signal and the echo signal, and extracts the beat frequency signal to form an echo sequence. In this embodiment, the continuous frequency-modulated signal is a linear frequency-modulated continuous wave.

[0086] The signal processing module performs a one-dimensional fast Fourier transform on the echo sequence to generate a range spectrum. The range spectrum represents the echo signal intensity at different range intervals in the signal transmission direction of the millimeter-wave radar sensing unit. The range interval is the maximum ranging of the millimeter-wave radar sensing unit divided into multiple range intervals.

[0087] For the range spectrum associated with multiple consecutive frequency-modulated signals, a second fast Fourier transform is performed on the time series to obtain the range-Doppler matrix. The range-Doppler matrix represents the echo signal intensity at different range intervals and different radial velocities in the signal transmission direction of the millimeter-wave radar sensing unit, and is used to identify moving targets and the speed of the moving targets. Specifically, the radial velocity is the radial velocity relative to the signal transmission direction, where a positive radial velocity indicates that the target is far away from the millimeter-wave radar sensing unit, and a negative radial velocity indicates that the target is close to the millimeter-wave radar sensing unit.

[0088] An angle estimation algorithm is used to estimate the phase difference of multiple receiving antennas at the same distance in the range spectrum to obtain a range-angle matrix. The range-angle matrix represents the echo signal intensity at different distances and azimuth angles in the signal transmission direction of the millimeter-wave radar sensing unit, and is used to reflect the angular distribution of targets in the room.

[0089] The echo sequence, range spectrum, range-Doppler matrix, and range-angle matrix are used as the millimeter-wave signals acquired by the millimeter-wave radar sensing unit. Millimeter-wave signals acquired by millimeter-wave radar sensing units in different millimeter-wave frequency bands are combined to form a multi-band millimeter-wave signal. The representation of the multi-band millimeter-wave signal is as follows:

[0090]

[0091] Among them, L t L represents the multi-band millimeter-wave signal acquired at time t, where time t represents the acquisition time of any multi-band millimeter-wave signal. t (i) represents the multi-band millimeter wave signal L t The millimeter-wave signal collected by the millimeter-wave radar sensing unit in the i-th millimeter-wave frequency band, where K represents the number of millimeter-wave frequency bands, and the frequency band size of the first to the Kth millimeter-wave frequency bands increases sequentially.

[0092] These represent millimeter-wave signals L, respectively. t (i) includes the echo sequence, range spectrum, range-Doppler matrix, and range-angle matrix.

[0093] As a preferred embodiment of this application, if the range spectrum shows a significant echo signal intensity detected at 3 meters, the range-Doppler matrix shows a significant echo signal intensity detected at 3 meters and a velocity of -0.6 meters per second, and the range-angle matrix shows a significant echo signal intensity detected at 3 meters and in the -45° direction, then it indicates that the target is 3 meters to the left front of the millimeter-wave radar sensing unit and is approaching the position of the millimeter-wave radar sensing unit.

[0094] The acquired multi-band millimeter-wave signals are compensated based on an adaptive cross-band heterogeneous source compensation mechanism, resulting in compensated multi-band millimeter-wave signals, including:

[0095] Acquire the collected multi-band millimeter wave signals and extract the millimeter wave signals corresponding to different millimeter wave frequency bands;

[0096] The quality of the millimeter-wave signal is assessed, and the signal quality is determined. If the signal quality of the millimeter-wave signal is lower than a preset quality threshold, the signal is considered to be of lower quality. The millimeter-wave signal is then labeled as the millimeter-wave signal to be compensated, and neighboring millimeter-wave signals are adaptively extracted. A cross-band heterogeneous source compensation mechanism is used to compensate the millimeter-wave signal. The neighboring millimeter-wave signal is the one among the multi-band millimeter-wave signals that is closest to the frequency band of the millimeter-wave signal to be compensated and whose signal quality is higher than a preset quality threshold. Millimeter-wave signals;

[0097] The signal quality assessment method for the millimeter-wave signal is as follows:

[0098] Q(L t (i))=SNR(L t (i))+ρ(L t (i))+Var(L t (i));

[0099]

[0100] Among them, Q(L) t (i) represents the millimeter-wave signal L t (i) signal quality, SNR(L) t (i) represents the millimeter-wave signal L t (i) normalized signal-to-noise ratio, ρ(L) t (i) represents the millimeter-wave signal L t (i) the effective distance interval proportion, Var(L) t (i) represents the millimeter-wave signal L t (i) Normalized echo distribution uniformity;

[0101] Represents the distance-Doppler matrix The maximum echo signal strength in Represents the distance-Doppler matrix The average echo signal intensity is lower than a preset echo signal intensity threshold (e.g., 10 dB). Represents the distance-Doppler matrix The standard deviation of echo signal intensity when the echo signal intensity is lower than a preset echo signal intensity threshold (e.g., 10 dB). This indicates the preset maximum signal-to-noise ratio (e.g., 40 dB);

[0102] Representing the distance spectrum The mid-echo signal strength is greater than the preset distance echo strength The number of distance intervals (e.g., 15 dB). Representing the distance spectrum The total number of mid-distance intervals;

[0103] Represents echo sequence The standard deviation of the sequence, This represents the preset maximum sequence standard deviation (e.g., 10), where the echo sequence consists of discrete sampling points, and the unit is volts.

[0104] It should be noted that the normalized signal-to-noise ratio (SNR) characterizes whether the target stands out from the background, reflecting the overall clarity of the signal and the presence of a strong target. The higher the SNR, the clearer the millimeter-wave signal. The effective range interval ratio measures whether the target area is dominant in terms of energy, filtering out situations where the background noise intensity is high but the target is weak. The higher the value, the stronger the target and the clearer the boundary. The normalized echo distribution uniformity characterizes whether the target is stably present, suppressing flickering or time-varying interference signals. The higher the value, the more stable and reliable the millimeter-wave signal.

[0105] The compensation formula for the millimeter-wave signal L to be compensated is:

[0106]

[0107] in, This represents the compensation result of the millimeter-wave signal L to be compensated, where L is the signal to be compensated. * φ represents the neighboring millimeter-wave signal of the millimeter-wave signal L to be compensated. norm (L * ;L) indicates that the adjacent millimeter-wave signal L * The scale in the range spectrum is mapped to the scale range of the millimeter-wave signal L to be compensated, where the scale includes the number of range intervals in the range spectrum, the sequence length of the echo sequence, the matrix specifications of the range-Doppler matrix and the range-angle matrix, and the matrix specifications include the number of matrix rows and the number of matrix columns.

[0108] MLP(·) represents a multilayer perceptron;

[0109] exp(·) denotes an exponential function with base to the natural constant, f(L), f(L) * The numbers L1 and L2 represent the millimeter-wave signal L1 to be compensated and the adjacent millimeter-wave signal L2, respectively. * The corresponding millimeter wave frequency band, This represents the millimeter-wave signal L to be compensated and the adjacent millimeter-wave signal L. *Frequency band differences between them.

[0110] Specifically, addressing the issue of signal attenuation or target loss in single-band millimeter-wave signals under dynamic obstruction, multipath interference, and non-line-of-sight conditions, this application introduces a quality assessment function. This function integrates three-dimensional indicators—normalized signal-to-noise ratio, effective range interval ratio, and normalized echo distribution uniformity—to establish a unified mechanism for judging the reliability of millimeter-wave signals. This enables dynamic evaluation of the sampling quality of millimeter-wave signals in different frequency bands, significantly enhancing the ability to identify abnormal signals and effectively mitigating the risks of false detection or missed detection. Furthermore, addressing the information silo phenomenon in multi-band millimeter-wave signals during spectrum scheduling or channel switching, a cross-band compensation mechanism is proposed. When the signal quality of a certain frequency band is insufficient, signals from neighboring frequency bands can be automatically invoked for scale alignment and completion. A frequency band difference term is introduced, ensuring that signals with closer frequency bands have higher alignment reliability. This achieves adaptive fusion control based on spectral similarity, significantly improving the stability and physical consistency of the compensated signal.

[0111] S2: Extract features from the compensated multi-band millimeter wave signal to obtain significant representation features of different millimeter wave frequency bands.

[0112] Feature extraction was performed on the compensated multi-band millimeter-wave signal to obtain salient representation features of different millimeter-wave frequency bands, including:

[0113] The millimeter wave signals corresponding to different millimeter wave frequency bands in the compensated multi-band millimeter wave signal are extracted. According to the size of the millimeter wave frequency band, the millimeter wave frequency band is divided into high frequency band, mid frequency band and low frequency band. Adaptive feature extraction is adopted for the high frequency band, mid frequency band and low frequency band millimeter wave signals to obtain the significant representation features of different millimeter wave frequency bands. Among them, the significant representation features are the dominant features of the millimeter wave frequency band.

[0114] The method for extracting the adaptability features of the high-frequency millimeter-wave signal is as follows:

[0115] The energy within the micro-motion velocity range of any distance interval in the range-Doppler matrix of a millimeter-wave signal is calculated and used as the micro-motion energy spectrum of the high-frequency millimeter-wave signal. This micro-motion energy spectrum is then used as a significant characteristic representing the high-frequency millimeter-wave band. Specifically, the millimeter-wave signal L... t The calculation method for the micro-motion energy spectrum of (i) is as follows:

[0116]

[0117] in, Indicates millimeter wave signal L t (i) micro-motion energy spectrum Represents the distance-Doppler matrix The echo signal intensity R at the r-th distance interval and at the radial velocity v i This represents the number of distance intervals in the i-th millimeter-wave frequency band, and [-δ,δ] represents the micro-motion speed range. δ is set to 2 meters per second.

[0118] The method for extracting the adaptability features of the mid-frequency millimeter-wave signal is as follows:

[0119] The extracted echo signal intensity in the range spectrum is higher than the preset range echo intensity. The range intervals are defined, and the mean of the velocity spread in the range-Doppler matrix across all extracted range intervals is calculated as the motion monitoring feature of the mid-frequency millimeter-wave signal.

[0120] The speed expansion is the difference between the maximum speed and the minimum speed within a distance range that are higher than a preset echo signal strength threshold. When there are multiple moving objects in a distance range (such as two people walking one in front of the other), they will exhibit different or even opposite speeds, causing the speed expansion to increase. This is used to determine whether there are multiple moving individuals in the room, or whether a person has exhibited complex movements (such as part of the body being still and part being moving).

[0121] The echo signal intensity distribution at different azimuth angles is extracted from the range-angle matrix. The echo signal intensity distribution is the difference between the maximum echo signal intensity at different azimuth angles and the mean echo signal intensity in the range-angle matrix. The standard deviation of the calculated standard deviation is used to represent the motion monitoring characteristics and the echo signal intensity distribution as significant features of the mid-frequency millimeter wave band.

[0122] The method for extracting the adaptability features of the low-frequency millimeter-wave signal is as follows:

[0123] The significant characteristics representing the low-frequency millimeter-wave band include the normalized signal-to-noise ratio of the millimeter-wave signal and the mean of the echo sequence. It should be noted that the millimeter-wave signal L... t The signal-to-noise ratio of (i) is in Represents the distance-Doppler matrix The maximum echo signal strength in Represents the distance-Doppler matrix The average echo signal intensity is lower than a preset echo signal intensity threshold (e.g., 10 dB). Represents the distance-Doppler matrix The standard deviation of echo signal intensity when the echo signal intensity is lower than a preset echo signal intensity threshold (e.g., 10 dB).

[0124] S3: Perform unified fusion representation of the salient representation features of different millimeter wave frequency bands to obtain frequency band fusion features. Utilize the personnel status recognition model to receive the frequency band fusion features and output the personnel status in the room.

[0125] The salient representation features of different millimeter-wave frequency bands are uniformly fused to obtain frequency band fusion features, including:

[0126] Calculate the mean salient representation features of all millimeter-wave frequency bands in the high-frequency band as the high-frequency band component;

[0127] Calculate the mean salient representation features of all millimeter-wave frequency bands in the mid-frequency band as the mid-frequency component;

[0128] Calculate the mean salient representation features of all millimeter-wave frequency bands in the low-frequency band as the low-frequency component;

[0129] The high-frequency, mid-frequency, and low-frequency components are spliced ​​together to form the frequency band fusion feature.

[0130] The system utilizes a personnel status recognition model to receive frequency band fusion features and outputs the status of people in the room, including:

[0131] The personnel state recognition model includes a feature alignment layer, a frequency band weighted fusion layer, and a state recognition layer. The feature alignment layer uses a linear transformation to map high-frequency, mid-frequency, and low-frequency components to a unified dimension. The frequency band weighted fusion layer is used to calculate the attention weights of the high-frequency, mid-frequency, and low-frequency components after unifying the dimension, and performs weighted processing on the high-frequency, mid-frequency, and low-frequency components after unifying the dimension to obtain frequency band weighted fusion features.

[0132] It should be noted that the normalized signal-to-noise ratio (SNR) of millimeter-wave signals in all millimeter-wave bands at high frequencies is obtained as α1, the normalized SNR of millimeter-wave signals in all millimeter-wave bands at mid frequencies is obtained as α2, and the normalized SNR of millimeter-wave signals in all millimeter-wave bands at low frequencies is obtained as α3.

[0133] Attention weights for high-frequency, mid-frequency, and low-frequency components are generated based on the normalized signal-to-noise ratio mean.

[0134]

[0135] Where exp(·) represents an exponential function with the natural constant as the base, γ represents the attention control parameter, and γ is set to 0.2; ω1, ω2, and ω3 represent the attention weights of the high-frequency component, mid-frequency component, and low-frequency component, respectively.

[0136] The state recognition layer is a lightweight multilayer perceptron used to receive frequency band weighted fusion features and output a predicted probability distribution of personnel states. The personnel state category with the highest predicted probability is selected as the personnel state in the room. The predicted probability distribution of personnel states is a vector composed of the predicted probabilities of different personnel state categories, including stationary, slow walking, fast movement, drastic changes, and multi-person activity. It should be noted that the lightweight multilayer perceptron consists of two ReLU activation functions and one Softmax activation function.

[0137] Specifically, the characteristics corresponding to stillness are low high-frequency micro-motion energy spectrum and small changes in mid-frequency echo signal intensity distribution; the characteristics corresponding to slow walking are high mid-frequency velocity extension and low high-frequency micro-motion energy spectrum; the characteristics corresponding to rapid movement are high mid-frequency velocity extension and high high-frequency micro-motion energy spectrum; the characteristics corresponding to drastic changes are high high-frequency micro-motion energy spectrum and low low-frequency normalized signal-to-noise ratio; and the characteristics corresponding to multi-person activity are high mid-frequency velocity extension and high low-frequency echo signal intensity distribution.

[0138] It should be explained in detail that "still" means that people in the room maintain a stable posture for a long time without significant changes in position, with minimal trunk and limb movements; "slow walking" means that people in the room walk in a small amplitude and at a low speed, accompanied by slight body or gait movements; "rapid movement" means that people in the room move at a relatively high speed, such as running, walking rapidly, or making emergency evasive maneuvers; "drastic change" means that people in the room undergo strong changes in posture or position in a short period of time, such as falling, sitting down, or lying down suddenly; and "multiple activities" means that there are two or more people in the room moving simultaneously, with multiple spatial moving targets or dynamic angular targets.

[0139] S4: Sort the room status of personnel output by the personnel status recognition model according to the acquisition time of the associated multi-band millimeter wave signal to obtain the room status time sequence. Monitor the risk status of the room status time sequence. If a high-risk status is detected, an early warning will be automatically triggered.

[0140] The room occupant status output by the occupant status recognition model is sorted according to the acquisition time of the associated multi-band millimeter wave signals to obtain a time series sequence of room occupant status, including:

[0141] The length of the time sequence of the status of people in the room is Len, where the time sequence of the status of people in the room acquired at time t using multi-band millimeter wave signals is:

[0142] H t =[H(t-Len+1),H(t-Len+2),...,H(t-1),H(t)];

[0143] Among them, H t This represents the time sequence of the room's personnel status associated with the multi-band millimeter-wave signals collected at time t. H(t-Len+1), H(t-Len+2), H(t-1), and H(t) respectively represent the room's personnel status associated with the multi-band millimeter-wave signals collected at times t-Len+1, t-Len+2, t-1, and t, where time t represents the acquisition time of any multi-band millimeter-wave signal. Specifically, the time interval between adjacent times is 1 minute. It should be noted that if steps S1 to S3 are used to monitor personnel status using multi-band millimeter-wave signals, the obtained room's personnel status is correlated with the collected multi-band millimeter-wave signals.

[0144] The system monitors the temporal sequence of the status of people in the room to assess risk. If a high-risk status is detected, an alert is automatically triggered, including:

[0145] Set risk weight coefficients for different personnel status categories, and replace the risk weight coefficients with the interrelated personnel statuses in the time series of personnel status in the room to form a risk weight coefficient time series; wherein the risk weight coefficients for the first to fifth personnel status categories are 0.1, 0.2, 0.5, 0.9, and 0.6 respectively, and the first to fifth personnel status categories are stationary, walking slowly, moving quickly, undergoing drastic changes, and multiple people active.

[0146] The weighted average risk value of the time series of risk weight coefficients is calculated. If the weighted average risk value is higher than the preset risk threshold, it indicates that a high-risk state has been detected and an early warning is automatically triggered.

[0147] In this embodiment of the application, the time sequence H of the status of the people in the room t The corresponding time series sequence of risk weight coefficients is h t The time series sequence h of the risk weight coefficients t The weighted average risk value is:

[0148]

[0149] g j =exp[-60β·(tj)];

[0150] Among them, S t The time series sequence h represents the risk weight coefficients. t The weighted average risk value, h(j) represents the risk weight coefficient associated with the state H(j) of the people in the room at time j, j∈[t-Len+1,t];

[0151] g jrepresents the weighting coefficient of the risk weighting coefficient h(j), 60 represents the time interval in seconds between adjacent moments, and β represents the time decay factor, set to 0.05;

[0152] The preset risk threshold is set to 0.75; the higher the weighted average risk value, the more likely the people in the room are to be in a state of sudden, violent, uncontrollable or abnormal activity, and there is a higher probability of dangerous events or abnormal behavior in the environment.

[0153] Specifically, this application introduces a time decay weight function in the time aggregation process of risk weight coefficients. It constructs an exponential time weight by using the time interval between each time in the time series of risk weight coefficients and the current time. Compared with the traditional averaging method, this mechanism can highlight the dominant role of the risk weight coefficient at the current time in risk, while avoiding serious interference from historical risk weight coefficients in risk assessment results. This improves the timeliness and accuracy of risk response, and is especially suitable for real-time response scenarios such as fall prediction and multi-state transition judgment.

[0154] Example 2:

[0155] A multi-millimeter-wave band fusion room occupant status monitoring system includes a data acquisition device, a feature extraction module, and a status monitoring module.

[0156] The data acquisition device is used to deploy millimeter-wave radar sensing units supporting different millimeter-wave frequency bands in the target room, and to collect multi-band millimeter-wave signals in the room in real time using the millimeter-wave radar sensing units. The collected multi-band millimeter-wave signals are then compensated based on an adaptive cross-band heterogeneous source compensation mechanism to obtain compensated multi-band millimeter-wave signals.

[0157] The feature extraction module is used to extract features from the compensated multi-band millimeter wave signal to obtain significant representation features of different millimeter wave frequency bands;

[0158] The status monitoring module is used to perform unified fusion representation of the significant representation features of different millimeter wave frequency bands to obtain frequency band fusion features. The personnel status recognition model receives the frequency band fusion features and outputs the status of personnel in the room. The status of personnel in the room output by the personnel status recognition model is sorted according to the acquisition time of the associated multi-frequency band millimeter wave signals to obtain the time sequence of personnel status in the room. The risk status is monitored on the time sequence of personnel status in the room. If a high-risk status is detected, an early warning is automatically triggered.

[0159] This is to achieve a multi-millimeter-wave band fusion method for monitoring the status of people in a room, as described in Example 1.

[0160] It should be understood that the embodiments described are for illustrative purposes only and are not limited to this structure in the scope of the patent application.

[0161] It should be noted that the sequence numbers of the above embodiments of the present invention are merely for descriptive purposes and do not represent the superiority or inferiority of the embodiments. Furthermore, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, apparatus, article, or method that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, apparatus, article, or method. Without further limitations, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, apparatus, article, or method that includes that element.

[0162] Through the above description of the embodiments, those skilled in the art can clearly understand that the methods of the above embodiments can be implemented by means of software plus necessary general-purpose hardware platforms. Of course, they can also be implemented by hardware, but in many cases the former is a better implementation method. Based on this understanding, the technical solution of the present invention, or the part that contributes to the prior art, can be embodied in the form of a software product. This computer software product is stored in a storage medium (such as ROM / RAM, magnetic disk, optical disk) as described above, and includes several instructions to cause a terminal device (which may be a mobile phone, computer, server, or network device, etc.) to execute the methods described in the various embodiments of the present invention.

[0163] The above are merely preferred embodiments of the present invention and do not limit the scope of the patent. Any equivalent structural or procedural transformations made based on the description and drawings of the present invention, or direct or indirect applications in other related technical fields, are similarly included within the scope of patent protection of the present invention.

Claims

1. A method for monitoring the status of occupants in a room using multi-millimeter wave frequency band fusion, characterized in that, The method includes: S1: Deploy millimeter-wave radar sensing units that support different millimeter-wave frequency bands in the target room, use the millimeter-wave radar sensing units to collect multi-band millimeter-wave signals in the room in real time, and compensate the collected multi-band millimeter-wave signals based on an adaptive cross-band heterogeneous source compensation mechanism to obtain compensated multi-band millimeter-wave signals. S2: Extract features from the compensated multi-band millimeter wave signal to obtain significant representation features of different millimeter wave frequency bands; S3: Perform unified fusion representation of the salient representation features of different millimeter wave frequency bands to obtain frequency band fusion features. Utilize the personnel status recognition model to receive the frequency band fusion features and output the personnel status in the room. S4: Sort the room status of personnel output by the personnel status recognition model according to the acquisition time of the associated multi-band millimeter wave signal to obtain the room status time sequence. Monitor the risk status of the room status time sequence. If a high-risk status is detected, an early warning will be automatically triggered.

2. The method for monitoring the status of people in a room using multi-millimeter wave frequency band fusion as described in claim 1, characterized in that, Millimeter-wave radar sensing units supporting different millimeter-wave frequency bands are deployed in the target room. These units are used to collect multi-band millimeter-wave signals in the room in real time, including: The millimeter-wave sensing unit consists of a signal transmitting device, a signal receiving antenna, a mixing module, and a signal processing module; The signal transmitting device in the millimeter-wave radar sensing unit transmits continuous frequency-modulated signals into the room and receives the echo signals reflected after encountering a target. The frequency-modulated signal is in the form of electromagnetic waves. The mixing module performs frequency mixing processing on the frequency-modulated signal and the echo signal, and extracts the beat frequency signal to form an echo sequence. The signal processing module performs a one-dimensional fast Fourier transform on the echo sequence to generate a range spectrum. The range spectrum represents the echo signal intensity at different range intervals in the signal transmission direction of the millimeter-wave radar sensing unit. The range interval is the maximum ranging of the millimeter-wave radar sensing unit divided into multiple range intervals. For the range spectrum associated with multiple consecutive frequency-modulated signals, a second fast Fourier transform is performed on the time series to obtain the range-Doppler matrix. The range-Doppler matrix represents the echo signal intensity in different range intervals and different radial velocities in the signal transmission direction of the millimeter-wave radar sensing unit, and is used to identify moving targets and the speed of the moving targets. An angle estimation algorithm is used to estimate the phase difference of multiple receiving antennas at the same distance in the range spectrum to obtain a range-angle matrix. The range-angle matrix represents the echo signal intensity at different distances and azimuth angles in the signal transmission direction of the millimeter-wave radar sensing unit, and is used to reflect the angular distribution of targets in the room. The echo sequence, range spectrum, range-Doppler matrix, and range-angle matrix are used as millimeter-wave signals collected by the millimeter-wave radar sensing unit, and the millimeter-wave signals collected by the millimeter-wave radar sensing units of different millimeter-wave frequency bands are used to form a multi-band millimeter-wave signal.

3. The method for monitoring the status of people in a room using multi-millimeter wave frequency band fusion as described in claim 2, characterized in that, The acquired multi-band millimeter-wave signals are compensated based on an adaptive cross-band heterogeneous source compensation mechanism, resulting in compensated multi-band millimeter-wave signals, including: Acquire the collected multi-band millimeter wave signals and extract the millimeter wave signals corresponding to different millimeter wave frequency bands; The quality of the millimeter-wave signal is assessed, and the signal quality is determined. If the signal quality of the millimeter-wave signal is lower than a preset quality threshold, the signal is considered lost. The millimeter-wave signal is then labeled as the millimeter-wave signal to be compensated, and neighboring millimeter-wave signals are adaptively extracted. A cross-band heterogeneous source compensation mechanism is used to compensate the millimeter-wave signal. The neighboring millimeter-wave signal is the one among the multi-band millimeter-wave signals that is closest to the frequency band of the millimeter-wave signal to be compensated and whose signal quality is higher than a preset quality threshold. Millimeter-wave signals; The compensation formula for the millimeter-wave signal L to be compensated is: in, This represents the compensation result of the millimeter-wave signal L to be compensated, where L is the signal to be compensated. * φ represents the neighboring millimeter-wave signal of the millimeter-wave signal L to be compensated. norm (L * ;L) indicates that the adjacent millimeter-wave signal L * The scale in the range spectrum is mapped to the scale range of the millimeter-wave signal L to be compensated, where the scale includes the number of range intervals in the range spectrum, the sequence length of the echo sequence, the matrix specifications of the range-Doppler matrix and the range-angle matrix, and the matrix specifications include the number of matrix rows and the number of matrix columns. MLP(·) represents a multilayer perceptron; exp(·) denotes an exponential function with base to the natural constant, f(L), f(L) * The numbers L1 and L2 represent the millimeter-wave signal L1 to be compensated and the adjacent millimeter-wave signal L2, respectively. * The corresponding millimeter wave frequency band, This represents the millimeter-wave signal L to be compensated and the adjacent millimeter-wave signal L. * Frequency band differences between them.

4. The method for monitoring the status of people in a room using multi-millimeter wave frequency band fusion as described in claim 3, characterized in that, Feature extraction was performed on the compensated multi-band millimeter-wave signal to obtain salient representation features of different millimeter-wave frequency bands, including: The millimeter wave signals corresponding to different millimeter wave frequency bands in the compensated multi-band millimeter wave signal are extracted. According to the size of the millimeter wave frequency band, the millimeter wave frequency band is divided into high frequency band, mid frequency band and low frequency band. Adaptive feature extraction is adopted for the high frequency band, mid frequency band and low frequency band millimeter wave signals to obtain the significant representation features of different millimeter wave frequency bands. The method for extracting the adaptability features of the high-frequency millimeter-wave signal is as follows: The energy of any distance interval in the range-Doppler matrix of a millimeter-wave signal within the range of micro-motion velocity is calculated and used as the micro-motion energy spectrum of the high-frequency millimeter-wave signal. The micro-motion energy spectrum is then used as a significant characteristic of the high-frequency millimeter-wave band. The method for extracting the adaptability features of the mid-frequency millimeter-wave signal is as follows: The extracted echo signal intensity in the range spectrum is higher than the preset range echo intensity. The range intervals are defined, and the mean of the velocity spread in the range-Doppler matrix across all extracted range intervals is calculated as the motion monitoring feature of the mid-frequency millimeter-wave signal. The echo signal intensity distribution at different azimuth angles is extracted from the range-angle matrix. The echo signal intensity distribution is the difference between the maximum echo signal intensity at different azimuth angles and the mean echo signal intensity in the range-angle matrix. The standard deviation of the calculated standard deviation is used to represent the motion monitoring characteristics and the echo signal intensity distribution as significant features of the mid-frequency millimeter wave band. The method for extracting the adaptability features of the low-frequency millimeter-wave signal is as follows: The significant characteristics of the low-frequency millimeter-wave band include the normalized signal-to-noise ratio of the millimeter-wave signal and the mean of the echo sequence.

5. The method for monitoring the status of people in a room using multi-millimeter wave band fusion as described in claim 4, characterized in that, The salient representation features of different millimeter-wave frequency bands are uniformly fused to obtain frequency band fusion features, including: Calculate the mean salient representation features of all millimeter-wave frequency bands in the high-frequency band as the high-frequency band component; Calculate the mean salient representation features of all millimeter-wave frequency bands in the mid-frequency band as the mid-frequency component; Calculate the mean salient representation features of all millimeter-wave frequency bands in the low-frequency band as the low-frequency component; The high-frequency, mid-frequency, and low-frequency components are spliced ​​together to form the frequency band fusion feature.

6. The method for monitoring the status of people in a room using multi-millimeter wave band fusion as described in claim 1, characterized in that, The system utilizes a personnel status recognition model to receive frequency band fusion features and outputs the status of people in the room, including: The personnel state recognition model includes a feature alignment layer, a frequency band weighted fusion layer, and a state recognition layer. The feature alignment layer uses a linear transformation to map high-frequency, mid-frequency, and low-frequency components to a unified dimension. The frequency band weighted fusion layer is used to calculate the attention weights of the high-frequency, mid-frequency, and low-frequency components after unifying the dimension, and performs weighted processing on the high-frequency, mid-frequency, and low-frequency components after unifying the dimension to obtain frequency band weighted fusion features. Attention weights for high-frequency, mid-frequency, and low-frequency components are generated based on the normalized signal-to-noise ratio mean. Where exp(·) represents an exponential function with the natural constant as the base, γ represents the attention control parameter, and γ is set to 0.2; ω1, ω2, and ω3 represent the attention weights of the high-frequency component, mid-frequency component, and low-frequency component, respectively. The state recognition layer is in the form of a lightweight multilayer perceptron, used to receive frequency band weighted fusion features and output the predicted probability distribution of personnel state. The personnel state category with the highest predicted probability is selected as the personnel state in the room. The predicted probability distribution of personnel state is a vector composed of the predicted probabilities of different personnel state categories. The personnel state categories include stationary, slow walking, fast moving, drastic changes, and multiple people moving.

7. The method for monitoring the status of people in a room using multi-millimeter wave frequency band fusion as described in claim 6, characterized in that, The room occupant status output by the occupant status recognition model is sorted according to the acquisition time of the associated multi-band millimeter wave signals to obtain a time series sequence of room occupant status, including: The length of the time sequence of the status of people in the room is Len, where the time sequence of the status of people in the room acquired at time t using multi-band millimeter wave signals is: H t =[H(t-Len+1),H(t-Len+2),...,H(t-1),H((t)]; Among them, H t Let H(t-Len+1), H(t-Len+2), H(t-1), and H(t) represent the room status of people associated with the multi-band millimeter-wave signals collected at time t, respectively. Time t represents the collection time of any multi-band millimeter-wave signal.

8. The method for monitoring the status of people in a room using multi-millimeter wave band fusion as described in claim 7, characterized in that, The system monitors the temporal sequence of the status of people in the room to assess risk. If a high-risk status is detected, an alert is automatically triggered, including: Set risk weight coefficients for different personnel status categories, and replace the risk weight coefficients with the interrelated personnel statuses in the time series of personnel status in the room to form a time series of risk weight coefficients; The weighted average risk value of the time series of risk weight coefficients is calculated. If the weighted average risk value is higher than the preset risk threshold, it indicates that a high-risk state has been detected and an early warning is automatically triggered. The time sequence of the status of people in the room H t The corresponding time series sequence of risk weight coefficients is h t The time series sequence h of the risk weight coefficients t The weighted average risk value is: g j =exp[-60β·((t-j)]; Among them, S t The time series sequence h represents the risk weight coefficients. t The weighted average risk value, h((j) represents the risk weight coefficient associated with the state H(j) of the people in the room at time j, j∈[t-Len+1,t]; g j represents the weighting coefficient of the risk weighting coefficient h(j), 60 represents the time interval in seconds between adjacent moments, and β represents the time decay factor, set to 0.05; The preset risk threshold is set to 0.

75. The higher the weighted average risk value, the more likely the people in the room are to be in a state of sudden, violent, uncontrollable or abnormal activity, and there is a higher probability of dangerous events or abnormal behaviors in the environment.

9. A multi-millimeter-wave band fusion room occupant status monitoring system, characterized in that, The room occupant status monitoring system includes a data acquisition device, a feature extraction module, and a status monitoring module: The data acquisition device is used to deploy millimeter-wave radar sensing units supporting different millimeter-wave frequency bands in the target room, and to collect multi-band millimeter-wave signals in the room in real time using the millimeter-wave radar sensing units. The collected multi-band millimeter-wave signals are then compensated based on an adaptive cross-band heterogeneous source compensation mechanism to obtain compensated multi-band millimeter-wave signals. The feature extraction module is used to extract features from the compensated multi-band millimeter wave signal to obtain significant representation features of different millimeter wave frequency bands; The status monitoring module is used to perform unified fusion representation of the significant representation features of different millimeter wave frequency bands to obtain frequency band fusion features. The personnel status recognition model receives the frequency band fusion features and outputs the status of personnel in the room. The status of personnel in the room output by the personnel status recognition model is sorted according to the acquisition time of the associated multi-frequency band millimeter wave signals to obtain the time sequence of personnel status in the room. The risk status is monitored on the time sequence of personnel status in the room. If a high-risk status is detected, an early warning is automatically triggered. To achieve the multi-millimeter wave band fusion method for room personnel status monitoring as described in any one of claims 1-8.

Images