A multi-millimeter wave frequency band fusion room personnel state monitoring method and system

Abstract

The application relates to the technical field of millimeter wave communication, and discloses a multi-millimeter wave frequency band fusion room personnel state monitoring method and system, the method comprising the following steps: compensating collected multi-frequency band millimeter wave signals based on an adaptive cross-frequency band heterogenous compensation mechanism; performing feature extraction and unified fusion representation on the compensated multi-frequency band millimeter wave signals to obtain frequency band fusion features; and recognizing personnel states by using a personnel state recognition model; and performing risk state monitoring and risk early warning on the time sequence of the personnel states in the room. The application collects multi-frequency band millimeter wave signals, extracts significant representation features of different millimeter wave frequency bands, recognizes personnel states based on multi-dimensional significant representation features, realizes high-robustness recognition of various behavior states such as the stillness, movement and violent change of personnel in the room in a complex environment, and realizes timely early warning and automatic response to high-risk states by constructing a state sequence and a risk discrimination mechanism.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of millimeter wave communication, and particularly relates to a multi-millimeter wave frequency band fusion room personnel state monitoring method and system. BACKGROUND

[0002] With the increasing requirements of precision, real-time and non-contact for personnel state perception in smart home, intelligent security, medical care and industrial operation application scenarios, personnel state monitoring technology based on millimeter wave radar has gradually become a research hotspot. Millimeter wave radar has the advantages of strong anti-interference ability, high penetration, good environmental adaptability, etc., and can realize the detection of human motion characteristics in dark, shielding and non-cooperative conditions. Especially in scenes with high privacy and high reliability requirements, millimeter wave radar has obvious advantages compared with traditional image sensors, and is widely used in fall detection, abnormal activity recognition, vital sign monitoring and other fields.

[0003] In existing research, some schemes try to combine multi-modal perception means to comprehensively judge the personnel state. For example, patent CN113573237B proposes a personnel state monitoring method, system and terminal based on face authentication and position perception. The scheme realizes the monitoring of personnel behavior in isolation state through the state monitoring APP in the terminal for registration and face authentication, combines electronic fence and mobile device position perception, and realizes the monitoring of personnel behavior in isolation state. Based on face recognition failure or abnormal position information, the state management in the prevention and control scene is re-timed. Although this method has practicality in identity binding and position verification, it relies on vision and terminal interaction, lacks support for non-cooperative and non-contact state perception, and cannot be applied to scenes where the target is outside the field of view, the light is poor, and the complex shielding.

[0004] In addition, in the traditional personnel monitoring method, it usually relies on cameras, infrared sensors and other devices. However, these methods often have problems such as privacy leakage, great influence of light, low precision, etc., and are not suitable for personnel state detection in public place environment. And the traditional single frequency band millimeter wave radar monitoring system has a large decline in monitoring accuracy when facing target body shielding, reflection attenuation or multipath interference problems due to fixed frequency band.

[0005] In view of the problem, the present application provides a multi-scale millimeter wave frequency band fusion room personnel state monitoring method and system, which combines the advantages of multiple millimeter wave frequencies through a multi-millimeter wave frequency band fusion method, makes up for the shortcomings of a single frequency band, and improves the precision and robustness of personnel state monitoring. SUMMARY

[0006] The application provides a multi-millimeter wave frequency band fusion room personnel state monitoring method and system, through deployment of a multi-frequency band radar sensing unit, complementary characteristics of different frequency bands to space penetration and accuracy are introduced, and overall data integrity and data quality are improved based on a cross-frequency band heterogenous compensation mechanism, high-robustness monitoring of a dynamic occlusion environment is realized, a frequency band adaptive significant feature extraction mode is designed, for example, high-frequency band extraction of micro-motion energy spectrum, medium-frequency band extraction of speed extension and echo intensity distribution, low-frequency band extraction of signal-to-noise ratio and energy statistics, the specificity and identification resolution of frequency band perception are improved; since single-frequency band state recognition is easily affected by noise, occlusion or motion blur, the output is unstable, and the real behavior change cannot be continuously reflected, the application inputs the personnel state recognition model by uniformly fusing the multi-frequency band significant features, the accuracy and anti-interference ability of recognition are significantly improved, and the state dynamic tracking and risk perception are realized by combining the state output time sequence.

[0007] To achieve the above-mentioned purpose, the application provides a multi-millimeter wave frequency band fusion room personnel state monitoring method, comprising the following steps:

[0008] S1: deploying a millimeter wave radar sensing unit supporting different millimeter wave frequency bands in a target room, collecting multi-frequency band millimeter wave signals in the room in real time by using the millimeter wave radar sensing unit, and compensating the collected multi-frequency band millimeter wave signals based on an adaptive cross-frequency band heterogenous compensation mechanism to obtain compensated multi-frequency band millimeter wave signals;

[0009] S2: performing feature extraction on the compensated multi-frequency band millimeter wave signals to obtain significant representation features of different millimeter wave frequency bands;

[0010] S3: uniformly fusing the significant representation features of different millimeter wave frequency bands to obtain frequency band fusion features, receiving the frequency band fusion features by using a personnel state recognition model, and outputting personnel states in the room;

[0011] S4: sorting the personnel states in the room output by the personnel state recognition model according to the collection time of the associated multi-frequency band millimeter wave signals to obtain a time sequence of personnel states in the room, and performing risk state monitoring on the time sequence of personnel states in the room, if a high-risk state is monitored, a warning is automatically triggered.

[0012] As a further improved method of the application:

[0013] Optionally, the millimeter wave radar sensing unit supporting different millimeter wave frequency bands is deployed in the target room, and the multi-frequency band millimeter wave signals in the room are collected in real time by using the millimeter wave radar sensing unit, comprising:

[0014] The millimeter wave sensing unit is composed of a signal transmitting device, a signal receiving antenna, a mixing module and a signal processing module;

[0015] The signal transmitting device in the millimeter wave radar sensing unit transmits continuous frequency-modulated signals to the room, and receives echo signals reflected after encountering targets, wherein the frequency-modulated signals are in the form of electromagnetic waves, a mixing module mixes the frequency-modulated signals and the echo signals, and extracts beat signals to form an echo sequence;

[0016] A signal processing module performs one-dimensional fast Fourier transform on the echo sequence to generate a range profile, wherein the range profile represents echo signal intensities at different distance intervals in the signal transmission direction of the millimeter wave radar sensing unit, and the distance intervals are divided from the maximum range of the millimeter wave radar sensing unit;

[0017] The range profiles associated with the continuous multiple frequency-modulated signals are subjected to secondary fast Fourier transform in time series to obtain a range-Doppler matrix, wherein the range-Doppler matrix represents echo signal intensities at different distance intervals and different radial velocities in the signal transmission direction of the millimeter wave radar sensing unit, and is used to identify moving targets and the speed of the moving targets;

[0018] An angle estimation algorithm is used to estimate the phase difference of multiple receiving antennas at the same distance in the range profile to obtain a range-angle matrix, wherein the range-angle matrix represents echo signal intensities at different distances and different azimuth angles in the signal transmission direction of the millimeter wave radar sensing unit, and is used to reflect the angle distribution of targets in the room;

[0019] The echo sequence, the range profile, the range-Doppler matrix, and the range-angle matrix are taken as millimeter wave signals collected by the millimeter wave radar sensing unit, and millimeter wave signals collected by millimeter wave radar sensing units of different millimeter wave frequency bands are taken as multi-band millimeter wave signals.

[0020] Optionally, the collected multi-band millimeter wave signals are compensated based on an adaptive cross-band heterogeneous compensation mechanism to obtain compensated multi-band millimeter wave signals, including:

[0021] The collected multi-band millimeter wave signals are obtained, and millimeter wave signals corresponding to different millimeter wave frequency bands are extracted;

[0022] The quality of the millimeter wave signals is evaluated, and the signal quality of the millimeter wave signals is obtained, if the signal quality of the millimeter wave signals is lower than a preset quality threshold The millimeter wave signals are marked as to-be-compensated millimeter wave signals, and adjacent millimeter wave signals of the millimeter wave signals are adaptively extracted, and a cross-band heterogeneous compensation mechanism is used to compensate the millimeter wave signals, wherein the adjacent millimeter wave signals are millimeter wave signals in the multi-band millimeter wave signals that are closest to the frequency band of the to-be-compensated millimeter wave signals and have a signal quality higher than the preset quality threshold ;

[0023] The compensation formula of the millimeter wave signal L to be compensated is:

[0024]

[0025] wherein, represents the compensation result of the millimeter wave signal L to be compensated, L * represents the adjacent millimeter wave signal of the millimeter wave signal L to be compensated, φ norm (L * ; L) represents mapping the scale in the adjacent millimeter wave signal L * to the scale range of the millimeter wave signal L to be compensated, wherein the scale includes the number of distance intervals in the distance spectrum, the sequence length of the echo sequence, the matrix specification of the distance-Doppler matrix and the distance-angle matrix, and the matrix specification includes the number of matrix rows and the number of matrix columns;

[0026] MLP(·) represents a multi-layer perception machine;

[0027] exp(·) represents an exponential function with a natural constant as the base, f(L), f(L * ) respectively represents the millimeter wave frequency band corresponding to the millimeter wave signal L to be compensated and the millimeter wave signal L * adjacent to the millimeter wave signal L to be compensated, represents the frequency band difference item between the millimeter wave signal L to be compensated and the millimeter wave signal L * adjacent to the millimeter wave signal L to be compensated.

[0028] Optionally, feature extraction is performed on the compensated multi-band millimeter wave signal to obtain significant representation features of different millimeter wave frequency bands, including:

[0029] The millimeter wave signals corresponding to different millimeter wave frequency bands in the compensated multi-band millimeter wave signal are extracted, the millimeter wave frequency bands are divided into high frequency bands, medium frequency bands and low frequency bands according to the size of the millimeter wave frequency bands, and adaptive feature extraction methods are adopted for the millimeter wave signals of the high frequency bands, the medium frequency bands and the low frequency bands to obtain significant representation features of different millimeter wave frequency bands;

[0030] The adaptive feature extraction method of the millimeter wave signal of the high frequency band is:

[0031] The energy of any distance interval in the distance-Doppler matrix of the millimeter wave signal in the range of the micro-motion velocity is calculated as the micro-motion energy spectrum of the millimeter wave signal of the high frequency band, and the micro-motion energy spectrum is taken as the significant representation feature of the millimeter wave frequency band of the high frequency band;

[0032] The adaptive feature extraction method of the millimeter wave signal of the medium frequency band is:

[0033] The echo signal intensity in the distance spectrum is extracted, which is higher than the preset distance echo intensity a distance interval of the distance-Doppler matrix, and calculates a mean value of a velocity spread of all the extracted distance intervals in the distance-Doppler matrix as a moving body monitoring feature of the millimeter wave signal in the middle frequency band;

[0034] extracts echo signal intensity distribution of different azimuth angles in the distance-angle matrix, where the echo signal intensity distribution is a difference between maximum echo signal intensity of different azimuth angles and a mean value of echo signal intensity in the distance-angle matrix, and calculates a standard deviation, to obtain a significant representation feature of the millimeter wave frequency band in the middle frequency band by taking the moving body monitoring feature and the echo signal intensity distribution as the significant representation feature;

[0035] The adaptive feature extraction manner of the millimeter wave signal in the low frequency band is:

[0036] The significant representation feature of the millimeter wave frequency band in the low frequency band includes a normalized signal-to-noise ratio of the millimeter wave signal and a mean value of an echo sequence.

[0037] Optionally, the significant representation features of different millimeter wave frequency bands are uniformly fused to obtain a frequency band fusion feature, including:

[0038] A mean value of the significant representation features of all the millimeter wave frequency bands in the high frequency band is calculated as a high frequency band component;

[0039] A mean value of the significant representation features of all the millimeter wave frequency bands in the middle frequency band is calculated as a middle frequency band component;

[0040] A mean value of the significant representation features of all the millimeter wave frequency bands in the low frequency band is calculated as a low frequency band component;

[0041] The high frequency band component, the middle frequency band component and the low frequency band component are spliced as the frequency band fusion feature.

[0042] Optionally, a personnel state recognition model is used to receive the frequency band fusion feature and output a personnel state in the room, including:

[0043] The personnel state recognition model includes a feature alignment layer, a frequency band weighted fusion layer and a state recognition layer, the feature alignment layer adopts a linear transformation manner to map the high frequency band component, the middle frequency band component and the low frequency band component to a uniform dimension, the frequency band weighted fusion layer is used to calculate attention weights of the high frequency band component, the middle frequency band component and the low frequency band component after the uniform dimension, and to perform weighted processing on the high frequency band component, the middle frequency band component and the low frequency band component after the uniform dimension to obtain a frequency band weighted fusion feature;

[0044] The attention weights of the high frequency band component, the middle frequency band component and the low frequency band component are generated based on a mean value of the normalized signal-to-noise ratio:

[0045]

[0046] wherein exp(·) represents an exponential function with a natural constant as a base, γ represents an attention control parameter, and γ is set as 0.2; ω1, ω2, and ω3 represent attention weights of high, medium, and low frequency band components, respectively;

[0047] The state recognition layer is in the form of a lightweight multi-layer perception machine, configured to receive the frequency band weighted fusion features and output a predicted probability distribution of the personnel state, and select a personnel state category with the highest predicted probability as the personnel state in the room, wherein the predicted probability distribution of the personnel state is a vector composed of predicted probabilities of different personnel state categories, and the personnel state category includes static, slow walking, fast moving, violent changing, and multiple people activities.

[0048] Optionally, the personnel state in the room output by the personnel state recognition model is sorted according to the collection time of the associated multi-frequency band millimeter wave signal to obtain a time sequence of the personnel state in the room, including:

[0049] The length of the time sequence of the personnel state in the room is Len, and the time sequence of the personnel state in the room of the multi-frequency band millimeter wave signal collected at time t is:

[0050] H t = [H(t-Len+1), H(t-Len+2),..., H(t-1), H(t)];

[0051] wherein H t represents the time sequence of the personnel state in the room of the multi-frequency band millimeter wave signal collected at time t, H(t-Len+1), H(t-Len+2), H(t-1), and H(t) represent the personnel state in the room associated with the multi-frequency band millimeter wave signal collected at time t-Len+1, t-Len+2, t-1, and t, respectively, and time t represents the collection time of any multi-frequency band millimeter wave signal.

[0052] Optionally, the time sequence of the personnel state in the room is monitored for a risk state, and if a high-risk state is monitored, a warning is automatically triggered, including:

[0053] Setting risk weight coefficients for different personnel state categories, replacing the risk weight coefficients with the mutually associated personnel states in the time sequence of the personnel state in the room to form a time sequence of risk weight coefficients;

[0054] Calculating a weighted average risk value of the time sequence of risk weight coefficients, and if the weighted average risk value is higher than a preset risk threshold, a high-risk state is monitored, and a warning is automatically triggered;

[0055] The time sequence of the personnel state in the room H t corresponding to the time sequence of risk weight coefficients is ht wherein the weighted average risk value of the risk weight coefficient time sequence h t is:

[0056]

[0057] g j =exp[-60β·(t-j)];

[0058] wherein S t represents the weighted average risk value of the risk weight coefficient time sequence h t , h(j) represents the risk weight coefficient associated with the room personnel state H(j) at time j, j∈[t-Len+1,t];

[0059] g j represents the weighted coefficient of the risk weight coefficient h(j), 60 represents the time interval in seconds between adjacent time points, and β represents a time decay factor, which is set to 0.05.

[0060] The preset risk threshold is set to 0.75; the higher the weighted average risk value, the more the room personnel state tends to be sudden, violent, uncontrollable or abnormal activity, and the higher the probability of dangerous events or abnormal behaviors in the environment.

[0061] To solve the above problems, the application also provides a room personnel state monitoring system with multi-millimeter wave frequency band fusion, which comprises a data acquisition device, a feature extraction module and a state monitoring module.

[0062] The data acquisition device is used to deploy millimeter wave radar sensing units supporting different millimeter wave frequency bands in the target room, use the millimeter wave radar sensing units to collect multi-frequency band millimeter wave signals in the room in real time, and compensate the collected multi-frequency band millimeter wave signals based on an adaptive cross-frequency band heterogeneous compensation mechanism to obtain compensated multi-frequency band millimeter wave signals.

[0063] The feature extraction module is used to extract features from the compensated multi-frequency band millimeter wave signals to obtain significant representation features of different millimeter wave frequency bands.

[0064] The state monitoring module is used to uniformly fuse and represent the significant representation features of different millimeter wave frequency bands to obtain frequency band fusion features, use a personnel state recognition model to receive the frequency band fusion features, and output the room personnel state, sort the room personnel state output by the personnel state recognition model according to the collection time of the associated multi-frequency band millimeter wave signals to obtain a room personnel state time sequence, and monitor the risk state of the room personnel state time sequence, and if a high-risk state is monitored, automatically trigger an early warning.

[0065] To achieve a multi-millimeter wave frequency band fusion room personnel state monitoring method as described above.

[0066] To solve the above problems, the application further provides an electronic device, comprising:

[0067] A memory stores at least one instruction;

[0068] A communication interface enables electronic device communication; and

[0069] A processor executes the instructions stored in the memory to implement the above-mentioned multi-millimeter wave frequency band fusion room personnel state monitoring method.

[0070] To solve the above problems, the application further provides a computer readable storage medium, the computer readable storage medium stores at least one instruction, the at least one instruction is executed by the processor in the electronic device to implement the above-mentioned multi-millimeter wave frequency band fusion room personnel state monitoring method.

[0071] Compared with the prior art, the application provides a multi-millimeter wave frequency band fusion room personnel state monitoring method and system, which has the following advantages:

[0072] Firstly, the application introduces a frequency band division mechanism, divides the multi-frequency band millimeter wave signal into high, medium and low frequency bands according to the physical characteristic differences of the millimeter wave frequency band, and respectively adopts adaptive feature extraction strategies, thereby significantly improving the capture ability of effective behavior information in each frequency band signal. Further explanation, the high frequency band millimeter wave signal has higher distance and speed resolution, which is suitable for micro-motion detection, so the energy of the micro-motion speed interval in its distance-Doppler matrix is extracted as the micro-motion energy spectrum, which helps to identify fine-grained dynamics such as breathing and gestures; the medium frequency band millimeter wave signal balances between dynamic perception and angle resolution, extracts the speed extension and azimuth angle reflection intensity distribution of the target area with strong echo intensity, effectively improves the accuracy of motion target number, structure and direction perception; and the low frequency band millimeter wave signal has stronger penetration ability and stability, which is suitable for preliminary judgment of whether the target exists, and extracting its signal-to-noise ratio and echo sequence average can enhance the modeling ability of target existence and perception credibility under the shielding area. Through the extraction of the above significant features, the application realizes the differential interpretation and structured induction of multi-frequency band information, avoids the scale mismatch and expression redundancy problems in the information fusion process, improves the robustness, explainability and real-time perception ability of complex personnel state changes of multi-frequency band fusion modeling, and has good engineering deployability and practical application value.

[0073] Meanwhile, the application introduces a time decay weight function in the time aggregation process of the risk weight coefficient, constructs an exponential time weight using the time interval from each time to the current time in the time sequence of the risk weight coefficient, compared with the traditional average method, this mechanism can highlight the dominant role of the risk weight coefficient at the current time on the risk, while avoiding the serious interference of the historical risk weight coefficient on the risk assessment result, thereby improving the timeliness and accuracy of the risk response, especially suitable for real-time response scenarios such as fall prediction and multi-state transition judgment. BRIEF DESCRIPTION OF DRAWINGS

[0074] Figure 1 A flowchart of a multi-millimeter wave frequency band fusion room personnel state monitoring method provided by an embodiment of the application is shown.

[0075] The implementation, functional features and advantages of the application will be further described with reference to the embodiments and the accompanying drawings. DETAILED DESCRIPTION

[0076] It should be understood that the specific embodiments described herein are merely intended to explain the application, and are not intended to limit the application.

[0077] Embodiments of the application provide a multi-millimeter wave frequency band fusion room personnel state monitoring method. The execution subject of the multi-millimeter wave frequency band fusion room personnel state monitoring method includes but is not limited to at least one of electronic devices such as a server, a terminal, etc. that can be configured to execute the method provided by the embodiments of the application. In other words, the multi-millimeter wave frequency band fusion room personnel state monitoring method can be executed by software or hardware installed in a terminal device or a server device, and the software can be a blockchain platform. The server includes but is not limited to a single server, a server cluster, a cloud server or a cloud server cluster, etc.

[0078] Reference Figure 1 Embodiment 1 of the application is:

[0079] S1: deploying a millimeter wave radar sensing unit supporting different millimeter wave frequency bands in a target room, collecting multi-frequency band millimeter wave signals in the room in real time by using the millimeter wave radar sensing unit, and compensating the collected multi-frequency band millimeter wave signals based on an adaptive cross-frequency heterogeneous compensation mechanism to obtain compensated multi-frequency band millimeter wave signals.

[0080] Deploying a millimeter wave radar sensing unit supporting different millimeter wave frequency bands in a target room, collecting multi-frequency band millimeter wave signals in the room in real time by using the millimeter wave radar sensing unit, includes:

[0081] Specifically, the plurality of millimeter wave radar sensor units respectively work at 24GHz, 60GHz and 77GHz frequency bands, are installed at positions having complementary visual angles in the room, and collect time sequences uniformly scheduled by the master module to realize perception and compensation fusion of multi-band and full-space personnel state information, wherein the 24GHz millimeter wave radar sensor unit is deployed at a high position of a corner of the wall for global perspective coverage, the 60GHz millimeter wave radar sensor unit is deployed in a key activity area to realize mesoscale posture recognition, and the 77GHz millimeter wave radar sensor unit is used for near-field micro-motion and physiological state perception.

[0082] It should be noted that the wavelengths corresponding to the 24GHz, 60GHz and 77GHz frequency bands decrease in turn, the penetration ability of the associated signals also decreases in turn, but the resolution increases in turn, and higher precision behavior monitoring can be performed. By deploying multi-band millimeter wave radar sensor units in the room, joint coverage and complementary collection of space-band are realized. Specifically, each millimeter wave radar sensor unit faces different directions, covers the entire room, realizes visual angle complementation, uses 24GHz for full-field coverage, and uses 60GHz / 77GHz to position key areas to realize frequency band complementation.

[0083] In the embodiments of the present application, taking a bedroom as an example, the 24GHz millimeter wave radar sensor unit is deployed at a high position of a corner of the wall of the bedroom and is inclined toward the center of the bedroom; the 60GHz millimeter wave radar sensor unit is deployed on the wall opposite the bed in the bedroom and is horizontally directed to the bed for posture perception and turning-over trajectory perception; and the 77GHz millimeter wave radar sensor unit is deployed in the center of the ceiling and is vertically downward for high-resolution analysis of breathing, heartbeat, hand movement, etc.

[0084] The millimeter wave sensor unit is composed of a signal transmitting device, a signal receiving antenna, a mixing module and a signal processing module.

[0085] The signal transmitting device in the millimeter wave radar sensor unit transmits continuous frequency modulation signals to the room and receives echo signals reflected after encountering a target, wherein the frequency modulation signals are in the form of electromagnetic waves, the mixing module performs mixing processing on the frequency modulation signals and the echo signals to extract beat signals to form an echo sequence; in the present embodiment, the continuous frequency modulation signals are linear frequency modulation continuous waves.

[0086] The signal processing module performs one-dimensional fast Fourier transform on the echo sequence to generate a range spectrum, wherein the range spectrum represents the echo signal intensity at different distance intervals in the signal transmitting direction of the millimeter wave radar sensor unit, and the distance interval is a plurality of distance intervals divided from the maximum ranging of the millimeter wave radar sensor unit.

[0087] The distance spectrum associated with the continuous multiple frequency modulation signals is subjected to a second fast Fourier transform in time series to obtain a range-Doppler matrix, which represents the echo signal intensity at different distance intervals and different radial velocities in the signal transmission direction of the millimeter wave radar sensor unit, and is used to identify the moving target and the speed of the moving target. Specifically, the radial velocity is the radial velocity relative to the signal transmission direction, wherein the positive radial velocity indicates that the target is moving away from the millimeter wave radar sensor unit, and the negative radial velocity indicates that the target is moving towards the millimeter wave radar sensor unit.

[0088] The phase difference of multiple receiving antennas at the same distance in the distance spectrum is subjected to angle estimation to obtain a range-angle matrix, which represents the echo signal intensity at different distances and different azimuth angles in the signal transmission direction of the millimeter wave radar sensor unit, and is used to reflect the angle distribution of the target in the room.

[0089] The echo sequence, the distance spectrum, the range-Doppler matrix and the range-angle matrix are taken as the millimeter wave signals collected by the millimeter wave radar sensor unit, and the millimeter wave signals collected by the millimeter wave radar sensor units at different millimeter wave frequency bands are taken as multi-band millimeter wave signals, and the representation form of the multi-band millimeter wave signals is:

[0090]

[0091] wherein L t represents the multi-band millimeter wave signals collected at time t, time t represents the collection time of any multi-band millimeter wave signal, L t (i) represents the millimeter wave signals collected by the millimeter wave radar sensor unit at the i-th millimeter wave frequency band in the multi-band millimeter wave signals L t , K represents the number of millimeter wave frequency bands, wherein the frequency band size of the 1st to Kth millimeter wave frequency bands increases in turn;

[0092] respectively represent the echo sequence, the distance spectrum, the range-Doppler matrix and the range-angle matrix in the multi-band millimeter wave signals L t (i).

[0093] As a preferred embodiment of the present application, if the distance spectrum shows that a significant echo signal intensity is detected at 3 meters, the range-Doppler matrix shows that a significant echo signal intensity is detected at 3 meters and a speed of -0.6 meters per second, and the range-angle matrix shows that a significant echo signal intensity is detected at 3 meters and a direction of -45°, it indicates that the target is 3 meters in front of the left side of the millimeter wave radar sensor unit and is moving towards the position of the millimeter wave radar sensor unit.

[0094] Compensate the collected multi-band millimeter wave signals based on an adaptive cross-band heterogeneous compensation mechanism to obtain compensated multi-band millimeter wave signals, including:

[0095] Obtain the collected multi-band millimeter wave signals and extract the millimeter wave signals corresponding to different millimeter wave frequency bands;

[0096] Perform quality assessment on the millimeter wave signals to obtain the signal quality of the millimeter wave signals, and if the signal quality of the millimeter wave signals is lower than a preset quality threshold , mark the millimeter wave signals as to-be-compensated millimeter wave signals, and adaptively extract adjacent millimeter wave signals of the millimeter wave signals, and compensate the millimeter wave signals using a cross-band heterogeneous compensation mechanism, wherein the adjacent millimeter wave signals are millimeter wave signals in the multi-band millimeter wave signals that are closest to the to-be-compensated millimeter wave signals in frequency band and have a signal quality higher than the preset quality threshold ;

[0097] The signal quality assessment method of the millimeter wave signals is:

[0098] Q(L t (i))=SNR(L t (i))+ρ(L t (i))+Var(L t (i));

[0099]

[0100] wherein Q(L t (i)) represents the signal quality of the millimeter wave signal L t (i), SNR(L t (i)) represents the normalized signal-to-noise ratio of the millimeter wave signal L t (i), ρ(L t (i)) represents the effective distance interval ratio of the millimeter wave signal L t (i), and Var(L t (i)) represents the normalized echo distribution uniformity of the millimeter wave signal L t (i);

[0101] represents the maximum echo signal strength in the distance-Doppler matrix , represents the average echo signal strength of the echo signal strength in the distance-Doppler matrix that is lower than a preset echo signal strength threshold (for example, 10 decibels), represents the standard deviation of the echo signal strength in the distance-Doppler matrix that is lower than a preset echo signal strength threshold (for example, 10 decibels), represents a preset maximum signal-to-noise ratio (for example, 40 decibels);

[0102] represents a distance spectrum the echo intensity of the middle range is greater than a preset distance echo intensity a number of distance intervals (for example, 15 decibels), represents a distance spectrum the total number of distance intervals in the middle range;

[0103] represents an echo sequence a sequence standard deviation, represents a preset maximum sequence standard deviation (for example, 10), the echo sequence is a discrete sampling point, and the unit is volts;

[0104] It should be noted that the normalized signal-to-noise ratio represents whether the target is prominent in the background, and can reflect the overall clarity of the signal and the existence of a strong target. The greater the normalized signal-to-noise ratio, the clearer the millimeter wave signal. The effective distance interval ratio measures whether the target region is dominant in energy, and can filter out the case where the background noise intensity is high but the target is weak. The greater the ratio, the stronger the target and the clearer the boundary. The normalized echo distribution uniformity represents whether the target is stably present, and suppresses flickering or time-varying interference signals. The greater the value, the more stable and reliable the millimeter wave signal.

[0105] The compensation formula of the to-be-compensated millimeter wave signal L is:

[0106]

[0107] wherein, represents a compensation result of the to-be-compensated millimeter wave signal L, L * represents a neighboring millimeter wave signal of the to-be-compensated millimeter wave signal L, φ norm (L * ; L) represents mapping a scale in the neighboring millimeter wave signal L * to a scale range of the to-be-compensated millimeter wave signal L, wherein the scale includes a number of distance intervals in the distance spectrum, a sequence length of the echo sequence, a matrix specification of the distance-Doppler matrix and the distance-angle matrix, and the matrix specification includes a number of matrix rows and a number of matrix columns;

[0108] MLP(·) represents a multi-layer perception machine;

[0109] exp(·) represents an exponential function with a natural constant as a base, f(L), f(L * ) represent millimeter wave frequency bands corresponding to the to-be-compensated millimeter wave signal L and the neighboring millimeter wave signal L * , respectively, represents a millimeter wave frequency band corresponding to the to-be-compensated millimeter wave signal L and the neighboring millimeter wave signal L *a frequency band difference item between the frequency bands.

[0110] Specifically, to solve the problem that single frequency band millimeter wave signals are prone to signal attenuation or target loss under dynamic occlusion, multipath interference and non-line-of-sight conditions, the application introduces a quality evaluation function, comprehensively normalizes three-dimensional indexes of signal-to-noise ratio, effective distance interval proportion and normalized echo distribution uniformity, establishes a unified discrimination mechanism for millimeter wave signal reliability, realizes dynamic evaluation of millimeter wave signal sampling quality of different frequency bands, significantly enhances the recognition ability of abnormal signals, and effectively avoids the risk of false detection or missed detection; for the information island phenomenon existing in the process of spectrum scheduling or channel switching of multi-frequency band millimeter wave signals, a cross-frequency band compensation mechanism is proposed, when the signal quality of a certain frequency band is insufficient, the adjacent frequency band signal can be automatically called to perform scale alignment and completion, and the frequency band difference item is introduced, so that the signals closer to the frequency band have higher alignment credibility, thereby realizing adaptive fusion regulation based on spectral similarity, and significantly improving the stability and physical consistency of the compensation signal.

[0111] S2: performing feature extraction on the compensated multi-frequency band millimeter wave signal to obtain significant representation features of different millimeter wave frequency bands.

[0112] The significant representation features of different millimeter wave frequency bands are obtained by performing feature extraction on the compensated multi-frequency band millimeter wave signal, including:

[0113] The millimeter wave signals corresponding to different millimeter wave frequency bands in the compensated multi-frequency band millimeter wave signal are extracted, the millimeter wave frequency bands are divided into high frequency bands, medium frequency bands and low frequency bands according to the size of the millimeter wave frequency bands, and adaptive feature extraction methods are adopted for the millimeter wave signals of the high frequency bands, the medium frequency bands and the low frequency bands to obtain significant representation features of different millimeter wave frequency bands, wherein the significant representation features are the dominant features of the millimeter wave frequency bands.

[0114] The adaptive feature extraction method of the millimeter wave signal of the high frequency band is:

[0115] The energy of any distance interval in the range of micro-motion velocity in the distance-Doppler matrix in the millimeter wave signal is calculated as the micro-motion energy spectrum of the millimeter wave signal of the high frequency band, and the micro-motion energy spectrum is taken as the significant representation feature of the millimeter wave frequency band of the high frequency band. Specifically, the millimeter wave signal L t The calculation method of the micro-motion energy spectrum of (i) is:

[0116]

[0117] Wherein, represents the micro-motion energy spectrum of the millimeter wave signal L t (i), represents the distance-Doppler matrix the echo signal strength at the rth distance interval and radial velocity v, R i denotes the number of distance intervals of the ith millimeter wave frequency band, [-δ, δ] denotes the range of micro-motion velocity, and δ is set to 2 meters per second;

[0118] The adaptive feature extraction manner of the millimeter wave signal in the middle frequency band is as follows:

[0119] extracting a distance interval in which the echo signal strength is higher than a preset distance echo strength in the distance spectrum, and calculating a mean value of velocity spreads in the distance-Doppler matrix at all the extracted distance intervals as a motion body monitoring feature of the millimeter wave signal in the middle frequency band;

[0120] The velocity spread is a difference between a maximum velocity and a minimum velocity that are higher than a preset echo signal strength threshold in a distance interval. When there are multiple motion bodies (such as two people walking in front of and behind each other) in a distance interval, they will show different or even opposite velocities, causing the velocity spread to increase, which is used to determine whether multiple motion individuals exist in the room or whether a person has complex motion (such as a part of the body being stationary and a part of the body moving);

[0121] extracting echo signal strength distribution of different azimuth angles in the distance-angle matrix, wherein the echo signal strength distribution is a difference between a maximum echo signal strength of different azimuth angles and a mean value of echo signal strengths in the distance-angle matrix, and the calculated standard deviation takes the motion body monitoring feature and the echo signal strength distribution as a significant representation feature of the millimeter wave frequency band in the middle frequency band;

[0122] The adaptive feature extraction manner of the millimeter wave signal in the low frequency band is as follows:

[0123] The significant representation feature of the millimeter wave frequency band in the low frequency band includes a normalized signal-to-noise ratio of the millimeter wave signal and an echo sequence mean value. It should be noted that the signal-to-noise ratio of the millimeter wave signal L t (i) is wherein denotes a maximum echo signal strength in the distance-Doppler matrix denotes a mean value of echo signal strengths in the distance-Doppler matrix wherein the echo signal strengths are lower than a preset echo signal strength threshold (for example, 10 decibels), denotes a standard deviation of echo signal strengths in the distance-Doppler matrix wherein the echo signal strengths are lower than a preset echo signal strength threshold (for example, 10 decibels).

[0124] ​​S3: uniformly fusing the significant representation features of different millimeter wave frequency bands to obtain a frequency band fusion feature, receiving the frequency band fusion feature by using a personnel state recognition model, and outputting a personnel state in the room.

[0125] Uniformly fusing the significant representation features of different millimeter wave frequency bands to obtain a frequency band fusion feature includes:

[0126] Calculating the mean of the significant representation features of all millimeter wave frequency bands in the high frequency band as a high frequency band component;

[0127] Calculating the mean of the significant representation features of all millimeter wave frequency bands in the middle frequency band as a middle frequency band component;

[0128] Calculating the mean of the significant representation features of all millimeter wave frequency bands in the low frequency band as a low frequency band component;

[0129] Splicing the high frequency band component, the middle frequency band component, and the low frequency band component as the frequency band fusion feature.

[0130] Receiving the frequency band fusion feature by using a personnel state recognition model, and outputting a personnel state in the room includes:

[0131] The personnel state recognition model includes a feature alignment layer, a frequency band weighted fusion layer, and a state recognition layer, the feature alignment layer adopts a linear transformation manner to map the high frequency band component, the middle frequency band component, and the low frequency band component to a unified dimension, the frequency band weighted fusion layer is used to calculate the attention weights of the high frequency band component, the middle frequency band component, and the low frequency band component after the unified dimension, and to perform weighted processing on the high frequency band component, the middle frequency band component, and the low frequency band component after the unified dimension to obtain a frequency band weighted fusion feature;

[0132] It should be noted that the mean of the normalized signal-to-noise ratio of the millimeter wave signal of all millimeter wave frequency bands in the high frequency band is obtained, the mean of the normalized signal-to-noise ratio of the millimeter wave signal of all millimeter wave frequency bands in the middle frequency band is obtained, and the mean of the normalized signal-to-noise ratio of the millimeter wave signal of all millimeter wave frequency bands in the low frequency band is obtained.

[0133] Based on the mean of the normalized signal-to-noise ratio, the attention weights of the high frequency band component, the middle frequency band component, and the low frequency band component are generated:

[0134]

[0135] Wherein, exp(·) represents an exponential function with a natural constant as a base, γ represents an attention control parameter, and γ is set to 0.2; ω1, ω2, ω3 represent the attention weights of the high frequency band component, the middle frequency band component, and the low frequency band component, respectively.

[0136] The state recognition layer is in the form of a lightweight multi-layer perception machine, configured to receive the frequency band weighted fusion features and output a prediction probability distribution of the personnel state, and select a personnel state category with the highest prediction probability as the personnel state in the room, wherein the prediction probability distribution of the personnel state is a vector composed of prediction probabilities of different personnel state categories, and the personnel state category includes static, slow walking, fast moving, violent movement and multiple person activities. It should be noted that the lightweight multi-layer perception machine is composed of two ReLU activation functions and one Softmax activation function.

[0137] Specifically, the features corresponding to static are low high-frequency micro-motion energy spectrum and small change in mid-frequency echo signal intensity distribution, the features corresponding to slow walking are high mid-frequency speed spread and low high-frequency micro-motion energy spectrum, the features corresponding to fast moving are high mid-frequency speed spread and high high-frequency micro-motion energy spectrum, the features corresponding to violent movement are high high-frequency micro-motion energy spectrum and low low-frequency normalized signal-to-noise ratio, and the features corresponding to multiple person activities are high mid-frequency speed spread and high low-frequency echo signal intensity distribution.

[0138] It should be noted that static means that the personnel in the room keeps the posture stable for a long time, the position does not change obviously, and the trunk and limb movements are minimal; slow walking means that the personnel in the room walks in the room with small amplitude and low speed, accompanied by slight body or gait movements; fast moving means that the personnel in the room moves quickly at a high speed, such as running, rapid walking, emergency avoidance, etc.; violent movement means that the personnel in the room has strong posture or attitude changes in a short time, such as falling down, sitting down, suddenly lying down; and multiple person activities means that there are two or more personnel activities in the room at the same time, with multiple spatial motion targets or dynamic angle targets.

[0139] S4: The personnel state in the room output by the personnel state recognition model is sorted according to the collection time of the associated multi-frequency band millimeter wave signal to obtain a time sequence of the personnel state in the room, and the time sequence of the personnel state in the room is monitored for a risk state, and if a high-risk state is monitored, a warning is automatically triggered.

[0140] The personnel state in the room output by the personnel state recognition model is sorted according to the collection time of the associated multi-frequency band millimeter wave signal to obtain a time sequence of the personnel state in the room, including:

[0141] The length of the time sequence of the personnel state in the room is Len, and the time sequence of the personnel state in the room of the multi-frequency band millimeter wave signal collected at time t is:

[0142] H t = [H(t-Len+1), H(t-Len+2),..., H(t-1), H(t)];

[0143] wherein, H t denotes the time sequence of the room personnel state of the multi-band millimeter wave signal collected at time t, H(t-Len+1), H(t-Len+2), H(t-1), H(t) represent the room personnel states associated with the multi-band millimeter wave signal collected at time t-Len+1, t-Len+2, t-1 and t, respectively, in sequence. Specifically, the time interval between adjacent time points is 1 minute. It should be noted that if the steps S1 to S3 are used to monitor the personnel state of the multi-band millimeter wave signal, the obtained room personnel state is associated with the collected multi-band millimeter wave signal.

[0144] The time sequence of the room personnel state is monitored for a risk state, and if a high-risk state is detected, a warning is automatically triggered, including:

[0145] The risk weight coefficients of different personnel state categories are set, the risk weight coefficients are replaced with the room personnel states associated with each other in the time sequence of the room personnel state, to form a time sequence of risk weight coefficients; wherein the risk weight coefficients of the first to fifth personnel state categories are 0.1, 0.2, 0.5, 0.9 and 0.6, respectively, and the first to fifth personnel state categories are stationary, slow walking, fast moving, violent changing and multiple activities, respectively.

[0146] The weighted average risk value of the time sequence of risk weight coefficients is calculated, and if the weighted average risk value is higher than a preset risk threshold, a high-risk state is detected, and a warning is automatically triggered.

[0147] In the embodiments of the present application, the time sequence of the room personnel state H t The corresponding time sequence of risk weight coefficients is h t , wherein the weighted average risk value of the time sequence of risk weight coefficients h t is:

[0148]

[0149] g j = exp[-60β·(t-j)];

[0150] wherein, S t denotes the weighted average risk value of the time sequence of risk weight coefficients h t , h(j) denotes the risk weight coefficient associated with the room personnel state H(j) at time j, j ∈ [t-Len+1, t];

[0151] g jThe weighted coefficient represents a risk weight coefficient h(j), 60 represents a time interval in seconds between adjacent time instants, and β represents a time decay factor, which is set to 0.05;

[0152] The preset risk threshold is set to 0.75; the higher the weighted average risk value, the more the state of the person in the room tends to be sudden, intense, uncontrollable or abnormal activity, and the higher the probability of a dangerous event or abnormal behavior in the environment;

[0153] Specifically, the application introduces a time decay weight function in the time aggregation process of the risk weight coefficient, and constructs an exponential time weight using the time interval from each time instant in the time sequence of the risk weight coefficient to the current time instant. Compared with the traditional average method, this mechanism can highlight the dominant role of the risk weight coefficient of the current time instant in the risk, while avoiding the serious interference of the historical risk weight coefficient on the risk assessment result, thereby improving the timeliness and accuracy of the risk response, especially for real-time response scenarios such as fall prediction and multi-state transition judgment.

[0154] Embodiment 2:

[0155] A multi-millimeter wave frequency band fusion room personnel state monitoring system, comprising a data acquisition device, a feature extraction module, and a state monitoring module:

[0156] The data acquisition device is used to deploy millimeter wave radar sensing units supporting different millimeter wave frequency bands in the target room, use the millimeter wave radar sensing units to collect multi-band millimeter wave signals in the room in real time, and compensate the collected multi-band millimeter wave signals based on an adaptive cross-band heterogeneous compensation mechanism to obtain compensated multi-band millimeter wave signals;

[0157] The feature extraction module is used to extract features from the compensated multi-band millimeter wave signals to obtain significant representation features of different millimeter wave frequency bands;

[0158] The state monitoring module is used to uniformly fuse and represent the significant representation features of different millimeter wave frequency bands to obtain frequency band fusion features, use a personnel state recognition model to receive the frequency band fusion features, and output the personnel state in the room. The personnel state in the room output by the personnel state recognition model is sorted according to the collection time of the associated multi-band millimeter wave signals to obtain a time sequence of the personnel state in the room. The time sequence of the personnel state in the room is monitored for a risk state, and if a high-risk state is detected, a warning is automatically triggered;

[0159] To realize a multi-millimeter wave frequency band fusion room personnel state monitoring method as described in Embodiment 1.

[0160] It should be understood that the embodiments are only for illustration and are not limited in the scope of the patent application by this structure.

[0161] It should be noted that the above-mentioned embodiment numbers of the present application are only for description, not representing the advantages and disadvantages of the embodiments. Also, the terms "comprising", "containing" or any other variants thereof in the present text are intended to cover the non-exclusive inclusion, so that the processes, devices, articles or methods including a series of elements not only include those elements, but also include other elements not explicitly listed or inherent to such processes, devices, articles or methods. Without more limitations, the element defined by the statement "comprising a" does not exclude the presence of other identical elements in the processes, devices, articles or methods including the element.

[0162] Through the above description of the embodiments, those skilled in the art can clearly understand that the above-mentioned embodiment methods can be realized by means of software and the necessary general hardware platform, of course, they can also be realized by hardware, but in many cases the former is a better embodiment. Based on such understanding, the technical solutions of the present application can be embodied in the form of a software product, which is stored in a storage medium (such as a ROM / RAM, a magnetic disk, an optical disk) as described above, and includes a number of instructions for making a terminal device (which can be a mobile phone, a computer, a server, or a network device, etc.) execute the methods described in various embodiments of the present application.

[0163] The above is only the preferred embodiment of the present application, and does not limit the patent scope of the present application, and any equivalent structure or equivalent process transformation using the content of the present application specification and drawings, or direct or indirect application in other related technical fields, are also included in the patent protection scope of the present application.

Claims

1. A multi-millimeter wave band fusion room personnel state monitoring method, characterized in that, The method comprises: S1: deploying a millimeter wave radar sensing unit supporting different millimeter wave frequency bands in a target room, collecting multi-band millimeter wave signals in the room in real time by using the millimeter wave radar sensing unit, and compensating the collected multi-band millimeter wave signals based on an adaptive cross-band heterogeneous compensation mechanism to obtain compensated multi-band millimeter wave signals, comprising: acquiring the collected multi-band millimeter wave signals and extracting millimeter wave signals corresponding to different millimeter wave frequency bands; The quality of the millimeter wave signal is evaluated, and the signal quality of the millimeter wave signal is obtained If the signal quality of the millimeter wave signal is lower than a preset quality threshold , the millimeter wave signal is marked as a millimeter wave signal to be compensated, and the adjacent millimeter wave signal of the millimeter wave signal is adaptively extracted, and a cross-band heterogenous compensation mechanism is used to compensate the millimeter wave signal, wherein the adjacent millimeter wave signal is a millimeter wave signal with a frequency band closest to the frequency band of the millimeter wave signal to be compensated and a signal quality higher than a preset quality threshold The millimeter wave signal to be compensated The compensation formula is: ； wherein, represents a compensation result of the millimeter wave signal to be compensated, represents a neighboring millimeter wave signal of the millimeter wave signal to be compensated, represents mapping a scale in the neighboring millimeter wave signal to a scale range of the millimeter wave signal to be compensated, wherein the scale includes a distance interval number in a distance spectrum, a sequence length of an echo sequence, a matrix specification of a distance-Doppler matrix and a distance-angle matrix, and the matrix specification includes a matrix row number and a matrix column number.​​ MLP stands for multi-layer perceptron; denotes an exponential function with a natural constant as base, denotes a to-be-compensated millimeter wave signal, denotes a neighboring millimeter wave signal, corresponding to a millimeter wave frequency band, denotes a to-be-compensated millimeter wave signal, denotes a neighboring millimeter wave signal, denotes a frequency band difference item between the to-be-compensated millimeter wave signal and the neighboring millimeter wave signal. S2: extracting features from the compensated multi-band millimeter wave signals to obtain significant representation features of different millimeter wave frequency bands; S3: uniformly fusing the significant representation features of different millimeter wave frequency bands to obtain frequency band fusion features, receiving the frequency band fusion features by using a personnel state recognition model, and outputting the personnel state in the room; S4: sorting the personnel state in the room output by the personnel state recognition model according to the collection time of the associated multi-band millimeter wave signals to obtain a time sequence of the personnel state in the room, and monitoring the risk state of the time sequence of the personnel state in the room, and if a high-risk state is monitored, a warning is automatically triggered.

2. A multi-millimeter wave band fusion room personnel state monitoring method according to claim 1, characterized by, Deploying a millimeter wave radar sensing unit supporting different millimeter wave frequency bands in a target room, collecting multi-band millimeter wave signals in the room in real time by using the millimeter wave radar sensing unit, comprising: The millimeter wave radar sensing unit is composed of a signal transmitting device, a signal receiving antenna, a mixing module and a signal processing module; The signal transmitting device in the millimeter wave radar sensing unit transmits continuous frequency modulation signals to the room and receives echo signals reflected after encountering targets, wherein the frequency modulation signals are in the form of electromagnetic waves, the mixing module performs mixing processing on the frequency modulation signals and the echo signals, and extracts beat signals to form an echo sequence; The signal processing module performs one-dimensional fast Fourier transform on the echo sequence to generate a range profile, wherein the range profile represents the echo signal intensity at different distance intervals in the signal transmission direction of the millimeter wave radar sensing unit, and the distance interval is divided into multiple distance intervals by the maximum ranging of the millimeter wave radar sensing unit; Performing secondary fast Fourier transform on the distance profiles associated with continuous multiple frequency modulation signals in the time sequence to obtain a range-Doppler matrix, wherein the range-Doppler matrix represents the echo signal intensity at different distance intervals and different radial velocities in the signal transmission direction of the millimeter wave radar sensing unit, and is used to identify moving targets and the speed of the moving targets; An angle estimation algorithm is used to estimate the phase difference of multiple receiving antennas at the same distance in the range profile to obtain a range-angle matrix, wherein the range-angle matrix represents the echo signal intensity at different distances and different azimuth angles in the signal transmission direction of the millimeter wave radar sensing unit, and is used to reflect the angle distribution of the targets in the room; The echo sequence, the range profile, the range-Doppler matrix and the range-angle matrix are used as the millimeter wave signals collected by the millimeter wave radar sensing unit, and the millimeter wave signals collected by the millimeter wave radar sensing units of different millimeter wave frequency bands are used to form multi-band millimeter wave signals.

3. A multi-millimeter wave band fusion room personnel state monitoring method according to claim 1, characterized by, Extracting features from the compensated multi-band millimeter wave signals to obtain significant representation features of different millimeter wave frequency bands, comprising: extracting the millimeter wave signals corresponding to different millimeter wave frequency bands in the compensated multi-band millimeter wave signals, dividing the millimeter wave frequency bands into high frequency bands, medium frequency bands and low frequency bands according to the size of the millimeter wave frequency bands, and adopting adaptive feature extraction methods for the millimeter wave signals of the high frequency bands, the medium frequency bands and the low frequency bands to obtain significant representation features of different millimeter wave frequency bands; the adaptive feature extraction method for the millimeter wave signals of the high frequency bands is: calculating the energy of any distance interval in the range of micro-motion speed in the range-doppler matrix of the millimeter wave signals as the micro-motion energy spectrum of the millimeter wave signals of the high frequency bands, and taking the micro-motion energy spectrum as the significant representation feature of the millimeter wave frequency band of the high frequency bands; the adaptive feature extraction method for the millimeter wave signals of the medium frequency bands is: extracting distance bins in which echo signal intensity is higher than preset distance echo intensity from the distance profile and calculating the mean of the velocity spreads in the distance-Doppler matrix at all the extracted distance bins as a moving body monitoring feature of the millimeter wave signal in the intermediate frequency band. extracting the echo signal intensity distribution of different azimuth angles in the range-angle matrix, wherein the echo signal intensity distribution is the difference between the maximum echo signal intensity of different azimuth angles and the average echo signal intensity in the range-angle matrix, and the calculated standard deviation takes the motion body monitoring feature and the echo signal intensity distribution as the significant representation feature of the millimeter wave frequency band of the medium frequency bands; the adaptive feature extraction method for the millimeter wave signals of the low frequency bands is: the significant representation features of the millimeter wave frequency bands of the low frequency bands include the normalized signal-to-noise ratio of the millimeter wave signals and the echo sequence average value.

4. A multi-millimeter wave band fusion room personnel state monitoring method according to claim 3, characterized by, unified fusion representation is performed on the significant representation features of different millimeter wave frequency bands to obtain a frequency band fusion feature, including: calculating the average value of the significant representation features of all millimeter wave frequency bands of the high frequency bands as a high frequency band component; calculating the average value of the significant representation features of all millimeter wave frequency bands of the medium frequency bands as a medium frequency band component; calculating the average value of the significant representation features of all millimeter wave frequency bands of the low frequency bands as a low frequency band component; splicing the high frequency band component, the medium frequency band component and the low frequency band component as the frequency band fusion feature.

5. The multi-millimeter wave band fusion-based room occupancy state monitoring method of claim 1, wherein, The personnel state recognition model receives the frequency band fusion feature and outputs the personnel state in the room, including: The personnel state recognition model includes a feature alignment layer, a frequency band weighted fusion layer and a state recognition layer, the feature alignment layer adopts a linear transformation method to map the high frequency band component, the medium frequency band component and the low frequency band component to a unified dimension, the frequency band weighted fusion layer is used to calculate the attention weight of the high frequency band component, the medium frequency band component and the low frequency band component after the unified dimension, and to perform weighted processing on the high frequency band component, the medium frequency band component and the low frequency band component after the unified dimension to obtain a frequency band weighted fusion feature; The attention weight of the high frequency band component, the medium frequency band component and the low frequency band component is generated based on the average value of the normalized signal-to-noise ratio: ； wherein, denotes an exponential function with a natural constant as base, denotes an attention control parameter, set to 0.2; denotes an attention weight of the high frequency band component, the medium frequency band component and the low frequency band component, respectively. The state recognition layer is in the form of a lightweight multilayer perceptron, which is used to receive the frequency band weighted fusion feature and output the prediction probability distribution of the personnel state, and the personnel state category with the highest prediction probability is selected as the personnel state in the room, wherein the prediction probability distribution of the personnel state is a vector composed of the prediction probabilities of different personnel state categories, and the personnel state category includes static, slow walking, fast moving, violent changing and multiple activities.

6. A multi-millimeter wave band fusion room occupancy state monitoring method according to claim 5, characterized in that, The room occupant status output by the occupant status recognition model is sorted according to the acquisition time of the associated multi-band millimeter wave signals to obtain a time series sequence of room occupant status, including: The length of the time sequence of the status of people in the room is Len, where the time sequence of the status of people in the room acquired at time t using multi-band millimeter wave signals is: ； wherein, denotes the time sequence of the state of the person in the room associated with the multi-band millimeter wave signal collected at time t, denotes the time sequence of the state of the person in the room associated with the multi-band millimeter wave signal collected at time t, , , and t, wherein t represents the collection time of any multi-band millimeter wave signal.

7. A multi-millimeter wave band fusion room occupancy state monitoring method according to claim 6, characterized in that, The system monitors the temporal sequence of the status of people in the room to assess risk. If a high-risk status is detected, an alert is automatically triggered, including: Set risk weight coefficients for different personnel status categories, and replace the risk weight coefficients with the interrelated personnel statuses in the time series of personnel status in the room to form a time series of risk weight coefficients; The weighted average risk value of the time series of risk weight coefficients is calculated. If the weighted average risk value is higher than the preset risk threshold, it indicates that a high-risk state has been detected and an early warning is automatically triggered. the room occupants status time series the corresponding risk weight coefficient time series is wherein the risk weight coefficient time series the weighted average risk value is ； ； wherein, a weighted average risk value, a risk weight coefficient associated with, ;​​ a risk weight coefficient a weighting coefficient, denotes the time interval in seconds between adjacent time instants, denotes a time decay factor, set to 0.05; The preset risk threshold is set to 0.

75. The higher the weighted average risk value, the more likely the people in the room are to be in a state of sudden, violent, uncontrollable or abnormal activity, and there is a higher probability of dangerous events or abnormal behaviors in the environment.

8. A multi-millimeter wave band fusion room personnel state monitoring system, characterized by, The room occupant status monitoring system includes a data acquisition device, a feature extraction module, and a status monitoring module: The data acquisition device is used to deploy millimeter-wave radar sensing units supporting different millimeter-wave frequency bands in the target room, and to collect multi-band millimeter-wave signals in the room in real time using the millimeter-wave radar sensing units. The collected multi-band millimeter-wave signals are then compensated based on an adaptive cross-band heterogeneous source compensation mechanism to obtain compensated multi-band millimeter-wave signals. The feature extraction module is used to extract features from the compensated multi-band millimeter wave signal to obtain significant representation features of different millimeter wave frequency bands; The status monitoring module is used to perform unified fusion representation of the significant representation features of different millimeter wave frequency bands to obtain frequency band fusion features. The personnel status recognition model receives the frequency band fusion features and outputs the status of personnel in the room. The status of personnel in the room output by the personnel status recognition model is sorted according to the acquisition time of the associated multi-frequency band millimeter wave signals to obtain the time sequence of personnel status in the room. The risk status is monitored on the time sequence of personnel status in the room. If a high-risk status is detected, an early warning is automatically triggered. To achieve the multi-millimeter wave band fusion method for room personnel status monitoring as described in any one of claims 1-7.

Images