Indoor monitoring and control system and method

Abstract

An indoor monitoring and control system and method, an indoor monitoring device comprising a millimeter-wave radar (1) and a computing unit (2) configured to: in response to radar signal data transmitted by the millimeter-wave radar (1); the calculation unit (2) analyzes micro-Doppler spectrum signals larger than a separation threshold value from the micro-Doppler spectrogram formed through the preprocessing step based on the preset separation threshold value; feature values are extracted based on the micro-Doppler time-frequency diagram after background noise suppression processing; and classifying the radar signal data based on the constructed classification model and the extracted feature value to obtain an indoor condition category corresponding to the radar signal data. According to the system and the method, private data such as displacement, azimuth angle and moving speed of people do not need to be monitored, the data processing amount is small, the speed of obtaining a monitoring result is higher, and the indoor control system acquires and calculates main indoor interference factors, so that the monitoring accuracy of indoor people is improved, and the experience of people is more comfortable.

Description

Indoor monitoring and control system and method Technical Field

[0001] The present invention relates to the field of millimeter wave radar technology, and in particular to an indoor monitoring and control system and method. Background Art

[0002] The number of smart homes in use is growing rapidly and is expected to exceed 500 million in the next few years. Only increasingly digital and sophisticated devices can make homes smart. However, smart devices require energy even when they are off. Even in standby mode, they need to respond instantly to user input, such as voice control or updates from the smart home or the internet. Furthermore, when no one is home, there is no need for smart devices to operate and consume energy in standby mode.

[0003] Accurately monitoring the presence of people indoors is a critical technical issue, as it determines whether smart devices need to be in standby mode. Currently, existing technologies have proposed using millimeter-wave radar to detect the presence of people indoors and integrate it with smart homes. Compared to existing solutions that place devices such as thermostats, smart speakers, and digital assistants in normal standby mode, a more effective way to reduce energy consumption is to put smart devices into "deep sleep mode" when no one is indoors. For some smart devices, this can save a few watts or even a small amount of energy. However, for specialized applications such as TVs, laptops, audio systems, and air conditioners, the energy savings can exceed 100W. By using millimeter-wave radar systems to sense the presence or movement of people indoors, smart devices can automatically switch to sleep mode when no one is indoors. Since millimeter-wave radar systems consume only a few milliwatts, with a maximum power consumption of 0.1W, their power consumption is significantly lower than the energy requirements of electronic devices in "startup" or standard standby mode. Using millimeter-wave radar systems to detect the presence of people indoors and activate smart devices is a future trend in smart home appliance control and is in line with the trend of energy-saving and environmentally friendly intelligent systems.

[0004] For example, Chinese patent CN113267773A discloses a method for accurately detecting and precisely locating indoor occupants based on millimeter-wave radar. This method primarily detects and locates indoor occupants based on high-precision millimeter-wave radar measurements of their distance, azimuth, and velocity, meeting the requirements for millimeter-wave radar as an indoor scene sensing device. The main processing steps include: data frame reconstruction after AD sampling, radar signal processing, micro-motion feature extraction of stationary occupants, tracking and locating groups of mobile occupants, and advanced application driving. This invention, based on a method for accurately detecting and precisely locating indoor occupants using millimeter-wave radar, can be applied to various millimeter-wave radar platforms to detect and locate indoor occupants. It primarily focuses on scenarios where millimeter-wave radar detects the presence, movement, and stillness of indoor occupants, laying the foundation for the widespread application of millimeter-wave radar in indoor occupant detection scenarios in the future.

[0005] However, the existing technology generally sets the millimeter-wave radar just above the doorway to the room to collect indoor data. The above patent requires the extraction of micro-motion features of stationary people and the tracking and positioning of groups of mobile people in order to determine whether people are present indoors. During the monitoring process, the above patent needs to process a lot of data, which increases the workload of the data processor, and also increases the data processing time and the control feedback time of the data processor to the smart device. If the movement features of people are not extracted, then the data obtained cannot exclude the dynamic data of indoor curtains, ventilation equipment and other equipment or items as interference data, resulting in large errors in the results of indoor monitoring using millimeter-wave radar due to the presence of interference data. This is also the fundamental reason why the above invention must extract the movement features of people.

[0006] So, how to effectively calculate and eliminate indoor interference factors, how to improve the accuracy of judging whether a person is indoors without extracting the person's movement characteristics, that is, how to simplify the steps of collecting data and calculating whether a person is indoors, are technical problems that are completely unresolved by current existing technologies.

[0007] For another example, Chinese patent CN110687816A discloses a millimeter-wave radar-based smart home control system and method. The system includes a millimeter-wave radar system, a signal processing system, an artificial intelligence classification system, and a central control system. The millimeter-wave radar system transmits a linear frequency-modulated continuous wave signal to the radar-illuminated scene and receives echo signals reflected from the scene, processing them to obtain intermediate-frequency raw data. The signal processing system processes the raw data to obtain feature data, which is then transmitted to the artificial intelligence classification system. The artificial intelligence classification system performs offline training and online classification on the millimeter-wave radar feature data, and transmits it to the central control system. The central control system provides real-time control, monitoring, and communication for the system.

[0008] For example, Chinese patent CN112762581A discloses a millimeter-wave radar-based intelligent air conditioning control method. This method aims to automatically determine the home scene and select the corresponding air conditioning control mode based on millimeter-wave radar signal data, thereby achieving intelligent air conditioning control. To this end, the method includes: establishing a room's inherent environmental model based on millimeter-wave radar measurement results; detecting a person's current location and dwell time using millimeter-wave radar; determining a corresponding home scene based on the person's current location, dwell time, current time, and the room's inherent environmental model; and determining the air conditioning operating mode based on the home scene.

[0009] Chinese patent CN114488838A discloses a millimeter-wave radar-based smart home control method, electronic device, and system. The system identifies a user's cooking time and behavior in a cooking area during that time. If a user's trajectory after leaving the cooking area shows a trend toward the refrigerator, there's a high probability they're headed for the refrigerator and need to open it to access ingredients. Therefore, when the user leaves the cooking area, the camera is heated using a first power level, enabling good photography when the user opens the refrigerator.

[0010] Chinese patent CN111796527A discloses an intelligent control system based on millimeter-wave radar. The intelligent switch of the intelligent control system is connected to the millimeter-wave radar sensor through a controller, and the wireless communication module is respectively connected to the controller, the intelligent switch, and the control system; the millimeter-wave radar sensor sends the signal generated after receiving the millimeter wave to the controller; the controller processes the signal through the signal processing circuit to obtain the human body detection result; the main control circuit board is respectively connected to the wireless communication module and the controller, and controls the action of the intelligent switch according to the human body detection result or control instruction.

[0011] As shown above, existing technologies focus on improving control methods. However, their data processing fails to account for interference from the smart devices themselves. Consequently, interference from these factors can increase monitoring errors. For example, current data processing cannot eliminate interference from curtains and air conditioners, leading to errors in the millimeter-wave radar system's detection of indoor human presence. Therefore, eliminating interference from devices themselves on millimeter-wave radar monitoring results remains a technical challenge that has yet to be addressed.

[0012] In addition, on the one hand, there are differences in understanding among those skilled in the art; on the other hand, the applicant studied a large number of documents and patents when making the present invention, but due to space limitations, not all details and contents are listed in detail. However, this does not mean that the present invention does not have the characteristics of these prior arts. On the contrary, the present invention already has all the characteristics of the prior art, and the applicant reserves the right to add relevant prior art to the background technology.

[0013] Summary of the Invention

[0014] Existing technology cannot accurately calculate and eliminate interference from objects or devices that interfere with indoor data collection. Therefore, it is necessary to collect and process data on the movement characteristics, distance, and azimuth of people indoors. This not only increases the amount of data processing, but also, for users who only need to determine whether a person is indoors, the current technology undoubtedly prolongs calculation time, slows information feedback, and requires large storage devices. Therefore, the main drawbacks of existing devices using millimeter radar waves for indoor monitoring of people are high cost, large data calculation requirements, large sample data volumes, and time delays in obtaining accurate results.

[0015] Based on the above shortcomings of the existing technology, the present invention hopes to provide a calculation method that can accurately calculate the interference data of indoor objects or equipment and eliminate interference, thereby eliminating the need to collect and calculate the indoor personnel movement feature data, personnel movement trajectory, distance, speed and other data, thereby reducing the overall amount of data calculation and achieving rapid and accurate judgment of whether people are indoors.

[0016] Furthermore, because existing technologies use millimeter-wave radar to collect data such as the movement characteristics and mobile positioning of indoor personnel, there are discussions about whether personal privacy is violated, as well as the possible spread of personal privacy data due to data leaks, and even the direct acquisition of privacy data due to server attacks. This means that although the above patents can be implemented in theory, they are rarely installed and used in reality. Especially for companies or units involved in confidentiality and sensitive work, purchasing patents like the above may involve personal privacy data, so some units have a natural resistance to purchasing such equipment. This means that even if the above patents can be produced, they cannot be widely sold, resulting in poor sales.

[0017] In view of the shortcomings of the existing technology, the present invention hopes to provide an indoor monitoring device that does not involve the collection and processing of privacy data such as personal mobility characteristics, so as to achieve true industrialization and high sales.

[0018] The present invention provides an indoor monitoring device, comprising at least a millimeter-wave radar and a computing unit. The computing unit is configured to: in response to radar signal data transmitted by the millimeter-wave radar, analyze, based on a preset separation threshold, micro-Doppler spectrum signals greater than the separation threshold from a micro-Doppler spectrum formed through a preprocessing step; extract at least one eigenvalue based on a micro-Doppler time-frequency graph after background noise suppression processing; and classify the radar signal data based on a constructed classification model and the extracted eigenvalues ​​to obtain an indoor situation category corresponding to the radar signal data.

[0019] The indoor monitoring device of the present invention changes the analysis method of the radar signal and classifies the effective radar signals to obtain the characteristics of the indoor state, thereby reducing the types of data collected and the amount of data processing. Moreover, the data collected by the millimeter-wave radar of the present invention does not contain data such as the position, displacement, and movement speed of indoor people, thereby simplifying the data processing steps and reducing the purchaser's concerns about the privacy of the monitored person, thereby encouraging users in need to purchase. The present invention is a product that can be accepted by market users and have a willingness to purchase, rather than a theoretical monitoring device. Based on the status of whether there is a person indoors, the present invention can lay a good promotion foundation for the popularization of smart home appliances, and facilitate the control unit of the smart home appliance to quickly perform corresponding home appliance control based on the monitoring results of whether there is a person indoors.

[0020] Preferably, the step of pre-processing the radar signal data by the radar signal calculation unit includes at least: calculating the mean of the first radar signal data: N represents the length of the radar signal sequence; n represents the signal sequence number; the mean is subtracted from the first radar signal data s(n) to suppress the zero Doppler component, and the second radar signal data is obtained Perform a short-time Fourier transform on the second radar signal data to obtain the short-time Fourier transform results of human motion and various interferences. The short-time Fourier transform result is expressed as STFT(t,f). Based on the short-time Fourier transform result STFT(t,f), a micro-Doppler spectrum is obtained: Spectrogram(t,f)=|STFT(t,f)| 2 , wherein the first radar signal data is the initial radar signal data.

[0021] Preferably, the step of the calculation unit analyzing the micro-Doppler signal greater than the separation threshold comprises at least: setting a separation threshold for separating the background noise and the micro-Doppler signal of the target in the micro-Doppler spectrum to suppress the background noise; treating the portion of the micro-Doppler spectrum below the separation threshold as noise and setting it as a noise characteristic value, and treating the portion of the micro-Doppler spectrum greater than the separation threshold as a valid micro-Doppler signal; and expressing the spectrum formed after separation from the noise as:

[0022] Where th represents the separation threshold;

[0023] F(t,f) represents the micro-Doppler spectrum after background noise suppression, and Spectrogram(t,f) represents the micro-Doppler spectrum before background noise suppression.

[0024] Preferably, the characteristic values ​​extracted by the calculation unit from the micro-Doppler time-frequency diagram include at least: the mean and standard deviation of the centroid, the mean and standard deviation of the bandwidth, the mean and standard deviation of the Doppler frequency intervals of the upper and lower contours, the proportion of valid micro-Doppler signals greater than the separation threshold in the micro-Doppler spectrum, the maximum peak value of the Doppler frequency corresponding to the upper contour and / or the maximum interval of the Doppler frequency between the peaks.

[0025] Preferably, the indoor situation categories corresponding to the radar signal data include at least:

[0026] The first type of indoor situation refers to the indoor situation when people enter the room;

[0027] The second type of indoor situation refers to the indoor situation when people leave the room;

[0028] The third type of indoor situation refers to the indoor situation where the first interference factor exists;

[0029] The fourth type of indoor situation refers to the indoor situation where the second interference factor exists;

[0030] The fifth type of indoor situation refers to an indoor situation where people are not indoors and the first interference factor and the second interference factor do not exist.

[0031] Preferably, the calculation unit is further configured to: count the status of people entering and leaving the room based on the indoor situation category output by the classification unit, wherein the initial state of the statistical data is set to 0, when the classification model classifies a group of second radar signal data as the first category of indoor situation, the statistical value is increased by 1; when a group of second radar signal data is classified as the second category of indoor situation, the statistical value is reduced by 1; if the data value of the statistical state is 0 at this time, the statistical value state remains unchanged; when the second radar signal data is classified as the third, fourth or fifth category of indoor situation, the statistical value state remains unchanged.

[0032] Preferably, the calculation unit extracts the upper and lower profile Doppler frequency intervals in the following manner: C = mean(F(i, j)) + α*std(F(i, j)) f span (j) = f(u j )-f(l j )

[0033] Among them, C represents the contour threshold, α represents the scale factor less than 1; fspan (j) represents the Doppler frequency interval of the upper and lower contours, mean(F(i,j)) represents the mean value of the micro-Doppler spectrum matrix, std(F(i,j)) represents the standard deviation of the micro-Doppler spectrum, u j Indicates the u-th time block corresponding to the upper envelope j Doppler blocks, l i Indicates the lth time block corresponding to the lower envelope i Doppler blocks.

[0034] The present invention also provides an indoor monitoring device, comprising at least a millimeter-wave radar and a computing unit, the computing unit comprising at least: a signal analysis unit for responding to radar signal data transmitted by the millimeter-wave radar, the computing unit analyzing micro-Doppler spectrum signals greater than the separation threshold from the micro-Doppler spectrum formed in the preprocessing step based on a preset separation threshold; a feature extraction unit for extracting at least one feature value based on the micro-Doppler time-frequency graph after background noise suppression;

[0035] a classification unit, configured to classify the radar signal data based on the constructed classification model and the extracted feature values ​​to obtain an indoor situation category corresponding to the radar signal data;

[0036] The statistical unit is used to count the status of people entering and leaving the room based on the indoor situation category output by the classification unit.

[0037] The present invention also provides an indoor monitoring method, which comprises: in response to radar signal data transmitted by a millimeter-wave radar, a computing unit analyzes micro-Doppler signals greater than the separation threshold from a micro-Doppler spectrum formed through a preprocessing step based on a preset separation threshold; and the computing unit extracts at least one characteristic value based on the micro-Doppler time-frequency graph after background noise suppression;

[0038] The calculation unit classifies the radar signal data based on the constructed classification model and the extracted feature values ​​to obtain an indoor situation category corresponding to the radar signal data.

[0039] Preferably, the method further comprises: the indoor situation categories corresponding to the radar signal data include at least: a first category of indoor situation, which refers to an indoor situation in which a person enters a room;

[0040] The second type of indoor situation refers to the indoor situation when people leave the room;

[0041] The third type of indoor situation refers to the indoor situation where the first interference factor exists;

[0042] The fourth type of indoor situation refers to the indoor situation where the second interference factor exists;

[0043] The fifth type of indoor situation refers to an indoor situation where people are not indoors and the first interference factor and the second interference factor do not exist.

[0044] Existing technologies for monitoring people indoors require not only millimeter-wave radar but also cameras. This collects and processes large amounts of data, with a wide variety of data types. This leads to large calculation errors in subsequent analysis and inaccurate control systems. Consequently, the accuracy of existing control systems is not high in practical applications.

[0045] In response to the shortcomings of the existing technology, the present invention also provides an indoor control system, which at least includes a millimeter-wave radar, a computing unit and a control unit. The computing unit analyzes the status of indoor occupants from the signal collected by the millimeter-wave radar and sends it to the control unit. The control unit sends corresponding control instructions to at least one smart device based on the status of indoor occupants and a preset control strategy. The computing unit at least includes: a feature extraction unit, which extracts at least one feature value from the micro-Doppler time-frequency diagram after background noise suppression; a classification unit, which classifies radar signal data based on the constructed classification model and feature value to determine the status of indoor occupants.

[0046] The indoor control system of the present invention collects and calculates interference data of objects or equipment, thereby improving the accuracy of monitoring indoor personnel, thereby improving the accuracy of the control unit's control of smart devices, making the human experience more comfortable.

[0047] The feature extraction unit of the present invention reduces the amount of data collection and data processing, and also reduces the time for data processing and the space for data storage. The data transmission delay of the control unit of the present invention is significantly reduced.

[0048] The monitoring end of the present invention only has millimeter wave radar, which reduces the amount of data collection and data processing, as well as the time and space for data processing. The data transmission delay of the control system of the present invention is significantly reduced.

[0049] Preferably, the calculation unit further includes a signal analysis unit, which is configured to: process the portion of the micro-Doppler spectrum below a preset separation threshold as noise and set it as a noise characteristic value, and set the portion of the micro-Doppler spectrum greater than the separation threshold as a valid micro-Doppler signal; and express the micro-Doppler spectrum after background noise suppression as

[0050] Where th represents the separation threshold;

[0051] F(t,f) represents the micro-Doppler spectrum after noise suppression; Spectrogram(t,f) represents the micro-Doppler spectrum before noise suppression. The feature extraction unit receives the micro-Doppler spectrum after background noise suppression from the signal analysis unit.

[0052] Preferably, the calculation unit further includes a preprocessing unit, which is configured to: calculate the mean value based on the data sent by the millimeter wave radar: N represents the length of the radar signal sequence; n represents the signal sequence number; the mean is subtracted from the first radar signal data s(n) to suppress the zero Doppler component and obtain the second signal radar signal data Second signal radar signal data Perform short-time Fourier transform to obtain the short-time Fourier transform results STFT(t,f) of human body motion and various interferences; based on the short-time Fourier transform results STFT(t,f), obtain the micro-Doppler spectrum: Spectrogram(t,f)=|STFT(t,f)| 2 ; The preprocessing unit sends the micro-Doppler spectrum information to the signal analysis unit.

[0053] Preferably, the specified features extracted by the feature extraction unit from the micro-Doppler time-frequency diagram include at least one or more of the following: the mean and standard deviation of the centroid, the mean and standard deviation of the bandwidth, the mean and standard deviation of the Doppler frequency intervals of the upper and lower contours, the proportion of valid micro-Doppler signals greater than the separation threshold in the micro-Doppler spectrum, the maximum peak of the Doppler frequency corresponding to the upper contour, and the maximum interval of the Doppler frequencies between the peaks.

[0054] Preferably, the feature extraction unit extracts the upper and lower profile Doppler frequency intervals in a manner including: C = mean(F(i, j)) + α*std(F(i, j)) f span (j) = f(u j )-f(l j )

[0055] Among them, C represents the contour threshold, α represents the scale factor less than 1; f span (j) represents the Doppler frequency interval of the upper and lower contours, mean(F(i,j)) represents the mean value of the micro-Doppler spectrum matrix, std(F(i,j)) represents the standard deviation of the micro-Doppler spectrum, u j Indicates the u-th time block corresponding to the upper envelope j Doppler blocks, l i Indicates the lth time block corresponding to the lower envelope i Doppler blocks.

[0056] Preferably, the method of constructing the classification model at least includes: setting training set data, wherein the training set data is composed of specified features extracted from the micro-Doppler spectrum after background noise suppression; performing classification training on a random forest classifier based on the training data composed of the specified features extracted from the micro-Doppler spectrum to form a classification model.

[0057] Preferably, the calculation unit also includes a statistical unit, which is configured to: set the initial state of the statistical data to 0, and when the classification model judges a group of input data as the first type of indoor situation, the statistical value is increased by 1; when a group of input data is judged as the second type of indoor situation, the statistical value is reduced by 1; if the data value of the statistical state is 0 at this time, the statistical value state remains unchanged; when the input data is judged as the third, fourth or fifth type of indoor situation, the statistical value state remains unchanged; the control unit determines the corresponding control strategy based on the data value sent by the statistical unit.

[0058] The present invention provides an indoor control method, which includes: analyzing the status of indoor occupants from radar signals collected by a millimeter-wave radar and sending the analysis result to a control unit; the control unit sending corresponding control instructions to at least one smart device based on the status of the indoor occupants and a preset control strategy; and the method also includes: extracting at least one eigenvalue from a micro-Doppler time-frequency diagram after background noise suppression processing; and classifying radar signal data based on a constructed classification model and the eigenvalues ​​to determine the status of indoor occupants.

[0059] Preferably, the method for obtaining the micro-Doppler spectrum after background noise suppression includes:

[0060] The portion below the preset separation threshold is treated as noise and set as the noise characteristic value, and the portion greater than the threshold is set as a valid micro-Doppler signal;

[0061] The micro-Doppler spectrum after background noise suppression is expressed as

[0062] Where th represents the separation threshold;

[0063] F(t,f) represents the micro-Doppler spectrum after noise suppression; Spectrogram(t,f) represents the micro-Doppler spectrum before noise suppression.

[0064] Preferably, the method further comprises: before suppressing background noise in the micro-Doppler spectrum, performing micro-Doppler spectrum extraction; wherein the micro-Doppler spectrum extraction method at least includes:

[0065] Calculate the mean based on the data sent by the millimeter wave radar: N represents the length of the radar signal sequence; n represents the signal sequence number;

[0066] Subtract the mean from the first radar signal data s(n) to suppress the zero Doppler component and obtain the second radar signal data

[0067] Second signal radar signal data Perform short-time Fourier transform to obtain the short-time Fourier transform results STFT(t,f) of human body motion and various interferences;

[0068] The micro-Doppler spectrum is obtained based on the short-time Fourier transform result STFT(t,f): Spectrogram(t,f)=|STFT(t,f)| 2 ;

[0069] The pre-processing unit sends the information of the micro-Doppler spectrum to the signal analysis unit. BRIEF DESCRIPTION OF THE DRAWINGS

[0070] FIG1 is a schematic diagram of simplified module connection relationships of an indoor monitoring device according to a preferred embodiment of the present invention;

[0071] Figure 2 is a micro-Doppler spectrum of a scene where a person walks out of a room;

[0072] Figure 3 is a micro-Doppler spectrum of a scene where a person walks into a room;

[0073] Figure 4 is a micro-Doppler spectrum of a scene where a person walks out of a room under fan interference;

[0074] Figure 5 is a micro-Doppler spectrum of a scene where a person walks into a room under fan interference;

[0075] Figure 6 is a micro-Doppler spectrum under fan operation interference;

[0076] Figure 7 is a micro-Doppler spectrum under curtain interference;

[0077] Figure 8 is a micro-Doppler spectrum of a scene where a person walks out of a room with curtain interference;

[0078] Figure 9 is a micro-Doppler spectrum of a scene where a person walks into a room with curtain interference;

[0079] FIG10 is a schematic diagram showing a comparison of the micro-Doppler spectra of curtains, fans, and people entering and leaving a room without interference before and after noise suppression;

[0080] FIG11 is an example diagram of the peak point and peak extension of the upper envelope in the micro-Doppler spectrum;

[0081] FIG12 is a simplified schematic diagram of module connection relationships of the indoor control system provided by the present invention.

[0082] Reference Signs List

[0083] 1: Millimeter-wave radar; 2: Computing unit; 21: Preprocessing unit; 22: Signal analysis unit; 23: Feature extraction unit; 24: Classification unit; 25: Statistics unit; 3: Control unit. DETAILED DESCRIPTION

[0084] The following is a detailed description with reference to the accompanying drawings.

[0085] Millimeter-wave radar 1: This radar operates in the millimeter-wave band. Millimeter waves range from 30 to 300 GHz (wavelengths from 1 to 10 mm). When setting up a millimeter-wave radar indoors, it's typically placed directly above the entrance to the room to collect indoor data.

[0086] In the present invention, the acquisition time of the millimeter wave radar 1 is preferably set as follows: data with a time length of two seconds is regarded as a group of radar signal data.

[0087] The computing unit 2 in the present invention is used to receive data transmitted by the millimeter-wave radar, process and analyze the data, and thereby determine whether a person is indoors. The computing unit 2 is connected to the millimeter-wave radar 1 via a wired or wireless communication method. Preferably, the computing unit 2 and the millimeter-wave radar 1 are connected wirelessly to reduce the impact of indoor wiring and the damage it may cause to the interior aesthetics.

[0088] The computing unit 2 can be a single processor or a combination of multiple sub-processors. The computing unit 2 can be a dedicated integrated circuit, a processor, or a microprocessor. The computing unit 2 can also be a server, a cloud server, or a server cluster.

[0089] Micro-motion: micro-movement of the target or target component other than the translational motion of the center of mass, such as vibration, rotation and acceleration.

[0090] The micro-Doppler effect is a phenomenon in which the target's micro-motions generate additional frequency modulation on the radar return signal, generating Doppler sidebands around the main body of a maneuvering target. Because micro-motion signatures are often unique manifestations of target motion, extracting these subtle motion features using modern signal processing techniques can provide new approaches for radar non-cooperative target detection and identification.

[0091] A micro-Doppler spectrogram is a visual representation of the spectrum formed by the micro-Doppler signal collected by the radar.

[0092] Example 1

[0093] The indoor monitoring device of the present invention comprises at least one millimeter wave radar 1 and at least one computing unit 2. The indoor monitoring device of the present invention can also be called an indoor monitoring system.

[0094] The calculation unit 2 includes at least a signal analysis unit 22 , a feature extraction unit 23 , a classification unit 24 and a statistics unit 25 .

[0095] Preferably, the calculation unit 2 further includes a preprocessing unit 21. The preprocessing unit 21 is configured to preprocess the received data.

[0096] The signal analysis unit 22 is configured to perform denoising processing on the micro-Doppler spectrum based on a set separation threshold.

[0097] The feature extraction unit 23 is configured to extract a specified micro-Doppler feature based on the received micro-Doppler spectrum.

[0098] The classification unit 24 is configured to: have a preset classification model, perform feature classification based on the classification model and the feature values ​​sent by the feature extraction unit 23 to extract interference data values. The classification model is formed in a training manner based on a random forest algorithm and training set data.

[0099] The counting unit 25 is configured to count the number of times a person enters and exits the room.

[0100] As shown in Figure 1 , the preprocessing unit 21 establishes a data connection with the signal analysis unit 22 to transmit the preprocessed data to the signal analysis unit 22. The signal analysis unit 22 also establishes a data connection with the feature extraction unit 23 to transmit the extracted, denoised micro-Doppler spectrum to the feature extraction unit 23. The feature extraction unit 23 also establishes a data connection with the classification unit 24 to classify the extracted feature information based on the classification model. The classification unit 24 also establishes a data connection with the statistics unit 25 to transmit the classification results to the statistics unit 25.

[0101] The calculation unit 2 of the present invention can also establish a signal connection relationship with the control unit of the smart home appliance to send the classification results and the information counted by the statistical unit 25 to the control unit, so that the control unit can control the operation of the smart home appliance according to the preset strategy based on the result of whether a person is indoors.

[0102] The indoor monitoring device of the present invention performs the indoor monitoring method as follows.

[0103] The present invention uses curtains and ventilation equipment as interference factors for illustration. The curtain interference factor and ventilation equipment interference factor in the example of the present invention can also be replaced by other interference factors for calculation and interference elimination.

[0104] In the present invention, the shaking of indoor curtains is the first interference factor, and the air disturbance of the ventilation equipment is the second interference factor.

[0105] The millimeter wave radar 1 sends data of two seconds in length to the calculation unit 2 as a group of first radar signals s(n).

[0106] The preprocessing unit 21 receives the first radar signal s(n) sent by the millimeter wave radar 1 and performs data preprocessing. The steps of data preprocessing include:

[0107] S1: Calculate the mean of the first radar signal:

[0108] Where n represents the signal number, and N represents the length of the radar signal sequence.

[0109] S2: Subtract the mean from the first radar signal s(n) to suppress the zero Doppler component and obtain the second radar signal data The second radar signal data Perform short-time Fourier transform to obtain the short-time Fourier transform results of human body motion and various interferences. The result of short-time Fourier transform is expressed as STFT(t,f).

[0110] Where s(n) represents the input data, i.e., the first radar signal data; represents the processed second radar signal data; η represents the mean; t represents the time interval; and f represents the frequency within the time interval.

[0111] The basic principle of short-time Fourier transform is to divide the signal into many small time intervals, and then use Fourier transform to analyze each time interval in order to determine the frequency that exists in that time interval.

[0112] S3: Express the micro-Doppler spectrum as: Spectrogram(t,f)=|STFT(t,f)| 2 .

[0113] Micro-Doppler spectrogram is a widely used method for displaying the time-varying spectral density of a time-varying signal. It is a spectrum-time representation that provides information about the actual changes in the signal's spectral content over time. Micro-Doppler spectrogram does not require the preservation of the signal's phase information.

[0114] Figures 2 through 9 show the micro-Doppler spectra obtained by the present invention in various scenarios. These include the micro-Doppler spectra for human entry and exit and interference activities. Figures 2 through 9 show that the micro-Doppler spectra for human entry and exit and various types of interference activities have different distribution characteristics. This difference forms the basis for subsequent feature extraction to distinguish these types of activities.

[0115] Figure 2 shows the micro-Doppler spectrum for a scene where a person walks out of a room. Figure 3 shows the micro-Doppler spectrum for a scene where a person walks into a room. Figure 4 shows the micro-Doppler spectrum for a scene where a person walks out of a room with fan interference. Figure 5 shows the micro-Doppler spectrum for a scene where a person walks into a room with fan interference. Figure 6 shows the micro-Doppler spectrum for a scene where a fan is running. Figure 7 shows the micro-Doppler spectrum for a scene where curtains are present. Figure 8 shows the micro-Doppler spectrum for a scene where a person walks out of a room with curtains present. Figure 9 shows the micro-Doppler spectrum for a scene where a person walks into a room with curtains present.

[0116] The pre-processing unit 21 converts the pre-processed micro-Doppler spectrum Spectrogram(t,f)=|STFT(t,f)| 2 The signal is sent to the signal analysis unit 22. The step of the signal analysis unit 22 analyzing the micro-Doppler spectrum includes at least the following steps.

[0117] S4: Set a separation threshold. The separation threshold is used to suppress noise in the micro-Doppler spectrum to improve the accuracy of micro-Doppler feature extraction of human activities and interference activities.

[0118] S5: Suppress noise. The part below the separation threshold is treated as noise and set to a very small number. The part above the separation threshold is the required effective micro-Doppler signal.

[0119] S6: The micro-Doppler spectrum after background noise suppression is expressed as

[0120] Where th represents the separation threshold; F(t,f) represents the micro-Doppler spectrum after background noise suppression; Spectrogram(t,f) represents the micro-Doppler spectrum before background noise suppression.

[0121] As shown in the formula, the noise characteristic value is -130. The effective micro-Doppler signal above the threshold is 10log 10 (Spectrogram(t,f)), where Spectrogram(t,f)>th. Here, an example of a very small number is -130.

[0122] Figure 10 shows a comparison of the micro-Doppler spectra of curtains, a fan, and a person entering or exiting a room without interference before and after noise suppression. This comparison clearly demonstrates the effective suppression of background noise, significantly reducing the impact of noise on the micro-Doppler signals of a person entering or exiting a room and interfering activities, facilitating subsequent feature extraction.

[0123] The signal analysis unit 22 sends the valid micro-Doppler signal selected according to the separation threshold to the feature extraction unit 23. The feature extraction unit 23 is used to extract the specified features in the micro-Doppler time-frequency diagram.

[0124] The feature extraction unit 23 extracts features by:

[0125] S7: Extract Doppler centroid f c The formula for (j) includes: The Doppler centroid can be considered as an estimate of the center of gravity of the micro-Doppler signature.

[0126] Extract Doppler bandwidth B c The formula for (j) includes:

[0127] F(i,j) represents the value of the micro-Doppler spectrum in the i-th Doppler block and the j-th time block, and f(i) represents the value of the Doppler frequency in the i-th Doppler block.

[0128] S8: Extracting the Doppler centroid f c (j) and Doppler bandwidth B c (j) After that, calculate the Doppler centroid f c (j) and Doppler bandwidth B c (j) Mean and standard deviation.

[0129] The specified features extracted from the micro-Doppler time-frequency maps include: the mean and standard deviation of the centroid, the mean and standard deviation of the bandwidth, and the mean and standard deviation of the upper and lower contour Doppler frequency intervals.

[0130] S9: Extract the mean and standard deviation of the Doppler frequency intervals of the upper and lower contours of the micro-Doppler spectrum.

[0131] Specifically, the formula for calculating the upper and lower profile Doppler frequency interval is as follows: C = mean(F(i, j)) + α*std(F(i, j)) f span (j) = f(u j )-f(l j )

[0132] Where C represents the contour threshold, which is used to accurately extract the upper and lower contours. α represents a scaling factor less than 1, which is used to adjust the value of the contour threshold. span (j) represents the Doppler frequency interval of the upper and lower contours; mean(F(i,j)) represents the mean value of the micro-Doppler spectrum matrix, std(F(i,j)) represents the standard deviation of the micro-Doppler spectrum, u j Indicates the u-th time block corresponding to the upper envelope j Doppler blocks, l i Indicates the lth time block corresponding to the lower envelope i Doppler blocks.

[0133] After obtaining the values ​​of the upper and lower profile Doppler frequency intervals, the average value and standard deviation of the upper and lower profile Doppler frequency intervals are calculated.

[0134] S10: Extract the contour size of the micro-Doppler spectrum.

[0135] The size of the micro-Doppler spectrum outline represents the proportion of the noise-suppressed micro-Doppler spectrum above the threshold. The micro-Doppler spectrum matrix has a total of N × T values, where N represents the number of Doppler bins and T represents the number of time bins.

[0136] A separation threshold th is defined as th = mean(F(i, j)) - 0.05*std(F(i, j)). There are K elements in the micro-Doppler spectrum matrix that are higher than the separation threshold.

[0137] Then, the formula for extracting the contour size of the micro-Doppler spectrum is as follows:

[0138] S11: extracting the average value of the Doppler frequency peak in the micro-Doppler spectrum and performing peak expansion.

[0139] FIG11 is an example diagram of the peak point and peak extension of the upper envelope in the micro-Doppler spectrum.

[0140] Assume that the upper envelope has m peaks, and the Doppler frequency corresponding to each peak is p i ,i∈m. The formula for the average value of the Doppler frequency peak is as follows:

[0141] Where mean(f c ) represents the average value of the Doppler centroid.

[0142] The peak spread in the micro-Doppler spectrum is the maximum interval of Doppler frequencies between peaks, and the formula is as follows: s =p max -p min .

[0143] p max Indicates the Doppler frequency of the largest peak. p min Indicates the Doppler frequency of the minimum peak.

[0144] The feature extraction unit 23 sends the designated features extracted from the denoised micro-Doppler spectrum to the classification unit 24 .

[0145] The classification unit 24 is preferably a classifier, which has a classification model set therein. The classification model is constructed based on a random forest algorithm. Preferably, the algorithm of the classification model is not limited to the random forest algorithm, and can also be other algorithms capable of classification.

[0146] The classification model is formed by classification training based on the random forest algorithm and training set data. The training set data consists of data with specified features extracted from the radar signal data after steps S1 to S11 above. The training set data is used to train the classification model.

[0147] The number of decision trees for the random forest classifier is set to 150. The input of the random forest classifier is the nine features described in the feature extraction section, and the output is the category of the radar signal data.

[0148] The steps to build a classification model include:

[0149] A set of radar signal data is subjected to short-time Fourier transform and noise separation, and the nine eigenvalues ​​extracted are used as the input eigenvalues ​​of the classification model.

[0150] The radar signal data corresponding to the input feature values ​​can be divided into five categories:

[0151] The first type of indoor situation refers to the indoor situation when people enter the room;

[0152] The second type of indoor situation refers to the indoor situation when people leave the room;

[0153] The third type of indoor situation refers to the indoor situation where the first interference factor exists;

[0154] The fourth type of indoor situation refers to the indoor situation where the second interference factor exists;

[0155] The fifth type of indoor situation refers to an indoor situation where people are not indoors and the first interference factor and the second interference factor do not exist.

[0156] The classification unit 24 performs classification processing on the received feature values ​​and sends the data type to the statistics unit 25. The statistics unit 25 receives and counts the classification data and the frequency of occurrence of each category.

[0157] The statistical unit 25 is preferably a counter. The initial state of the counter is set to 0, and the state change of the counter is only related to the detected entry and exit.

[0158] When classification unit 24 classifies a set of second radar signal data as a first indoor situation, indicating a person entering a room, the counter status is incremented by 1. When a set of second radar signal data is classified as a second indoor situation, indicating a person leaving a room, the counter status is decremented by 1. If the counter status is 0 at this time, the counter status remains unchanged. When the data is classified as interference, the counter status remains unchanged.

[0159] After the random forest classifier is trained, the indoor monitoring device of the present invention can accurately detect whether a person is indoors without being affected by the first or second interference factors. Therefore, the present invention has a high degree of accuracy in monitoring indoor occupants, requires less data to calculate, processes data quickly, and provides faster feedback.

[0160] The present invention improves the accuracy of indoor human monitoring by collecting and calculating interference data from the main indoor interference sources: curtains (the first interference factor) and ventilation equipment (the second interference factor). For example, the present invention divides indoor data collected by radar into five categories: human entry and exit data with or without fan or curtain interference, curtain shaking data, fan rotation data, and radar data collected when no one is present and no curtains or fans are moving. Based on these five types of data, the present invention performs data analysis, feature extraction, data classification, and data training, which can obtain accurate results for indoor monitoring, accurately determine whether there are people indoors, and provide more operational solutions for future smart home appliances to perform classified intelligent control based on the presence or absence of people indoors.

[0161] The advantages of the present invention also include: there is no need to monitor the displacement, azimuth and movement speed of the person, which reduces the amount of data calculation and the storage space required for data storage. Therefore, the monitoring device of the present invention has less data processing volume and faster data result feedback.

[0162] Example 2

[0163] This embodiment is a further improvement of embodiment 1, and repeated contents will not be repeated here.

[0164] Based on the defect in the existing technology that the self-interference factors of smart devices cannot be eliminated, the present invention hopes to provide an indoor control system and method that can eliminate the self-interference factors of smart devices, so that the results of monitoring the status of indoor people through millimeter radar waves are more accurate.

[0165] The present invention provides an indoor control system and method, and may also be a data processing method and device based on millimeter radar waves.

[0166] The indoor control system of the present invention includes at least a millimeter-wave radar 1, a computing unit 2, and a control unit 3. The millimeter-wave radar 1 is used to collect data on designated objects or devices in the room, such as data on people, first data on objects, and second data on devices.

[0167] The control unit 3 can send control instructions to devices connected by wire or wirelessly according to pre-stored control strategies to control the operation of the smart devices.

[0168] Specifically, the control unit 3 is used to control the designated equipment in the room according to the data results sent by the calculation unit 2 and the preset control strategy.

[0169] The control unit 3 may be a dedicated integrated chip, a processor, or a microprocessor that receives the status information of the indoor occupants sent by the calculation unit 2 and controls the smart device according to a preset control strategy.

[0170] The calculation unit 2 includes at least a signal analysis unit 22 , a feature extraction unit 23 , a classification unit 24 and a statistics unit 25 .

[0171] Preferably, the calculation unit 2 further includes a preprocessing unit 21. The preprocessing unit 21 is configured to preprocess the received data.

[0172] The signal analysis unit 22 is configured to perform denoising processing on the micro-Doppler spectrum based on a set threshold.

[0173] The feature extraction unit 23 is configured to extract a specified micro-Doppler feature value based on the received micro-Doppler spectrum.

[0174] Classification unit 24 is configured to: Pre-set a classification model. A random forest classifier is trained based on training data consisting of specified features extracted from the micro-Doppler spectra to form a classification model. Feature classification is performed based on the classification model and the feature values ​​transmitted by feature extraction unit 23 to extract interference data values. Statistics unit 25 is configured to count the number of times people enter and exit the room.

[0175] As shown in Figure 12, the preprocessing unit 21 establishes a data connection with the signal analysis unit 22 to transmit the preprocessed data to the signal analysis unit 22. The signal analysis unit 22 establishes a data connection with the feature extraction unit 23 to transmit the extracted, de-noised micro-Doppler spectrum to the feature extraction unit 23. The feature extraction unit 23 establishes a data connection with the classification unit 24 to classify the extracted feature information based on the classification model. The classification unit 24 establishes a data connection with the statistics unit 25 to transmit the classification results to the statistics unit 25.

[0176] The calculation unit 2 of the present invention can also establish a signal connection relationship with the control unit of the smart home appliance to send the classification results and the information counted by the statistical unit 25 to the control unit, so that the control unit can control the operation of the smart home appliance according to the preset strategy based on the result of whether a person is indoors.

[0177] The indoor monitoring device of the present invention performs an indoor monitoring method as described below.

[0178] The present invention uses curtains and ventilation equipment as interference factors for illustration. The curtain interference factor and ventilation equipment interference factor in the example of the present invention can also be replaced by other interference factors for calculation and interference elimination.

[0179] In the present invention, the shaking of indoor curtains is the first interference factor, and the air disturbance of the ventilation equipment is the second interference factor.

[0180] The millimeter wave radar 1 sends two seconds of data as a set of input data to the calculation unit 2 .

[0181] The preprocessing unit 21 receives the input data s(n) sent by the millimeter wave radar 1 and performs data preprocessing. The steps of data preprocessing include:

[0182] S21: Calculate the average value based on the data sent by millimeter wave radar 1: N represents the length of the radar signal sequence; n represents the signal sequence number.

[0183] S22: Subtract the mean from the first radar signal data s(n) to suppress the zero Doppler component and obtain the second radar signal data Second radar signal data

[0184] Second signal radar signal data Perform a short-time Fourier transform to obtain the short-time Fourier transform results of human body motion and various interferences. The result of the short-time Fourier transform is expressed as STFT(t,f).

[0185] The micro-Doppler spectrum is obtained based on the short-time Fourier transform result STFT(t,f): Spectrogram(t,f)=|STFT(t,f)| 2 Spectrogram(t,f) represents the micro-Doppler spectrum.

[0186] Micro-Doppler spectrogram is a widely used method to display the time-varying spectral density of time-varying signals. It is a spectrum-time representation and provides the actual change of the spectral content of the signal over time. Micro-Doppler spectrogram does not need to retain the phase information of the signal.

[0187] The pre-processing unit 21 sends the information of the micro-Doppler spectrum to the signal analysis unit 22 .

[0188] The basic principle of short-time Fourier transform is to divide the signal into many small time intervals, and then use Fourier transform to analyze each time interval in order to determine the frequency that exists in that time interval.

[0189] S23: Get micro-Doppler spectrum Spectrogram(t,f)=|STFT(t,f)| 2 . t represents the time interval; f represents the frequency within the time interval.

[0190] As shown in Figures 2 through 9, the micro-Doppler spectra in various scenarios are different. Figures 2 through 9 show the micro-Doppler spectra for eight scenarios: a person walking out of a room, walking into a room, walking out of a room with fan interference, walking into a room with fan interference, fan interference only, curtain interference only, walking out of a room with curtain interference, and walking into a room with curtain interference.

[0191] The preprocessing unit 21 preprocesses the obtained micro-Doppler spectrum: Spectrogram(t,f)=|STFT(t,f)| 2 The signal is sent to the signal analysis unit 22. The step of performing signal separation on the micro-Doppler spectrum by the signal analysis unit 22 includes at least the following steps.

[0192] Set the separation threshold. The separation threshold is used to suppress noise in the micro-Doppler spectrum to improve the accuracy of micro-Doppler feature extraction of human activities and interference activities.

[0193] S24: Suppress noise. The data portion below the separation threshold is treated as noise data and set as the noise characteristic value. The portion above the separation threshold is selected as the required effective micro-Doppler signal. The noise characteristic value is minimized.

[0194] S25: The micro-Doppler spectrum after noise suppression is shown as follows:

[0195] Where th represents the separation threshold; as shown in the formula, the noise characteristic value is preferably -130. The effective micro-Doppler signal above the threshold is 10log 10 (Spectrogram(t,f)), where Spectrogram(t,f)>th. F(t,f) represents the micro-Doppler spectrum after background noise suppression; Spectrogram(t,f) represents the micro-Doppler spectrum before noise suppression. Here, an example of a very small number is -130.

[0196] The signal analysis unit 22 sends the denoised micro-Doppler spectrum to the feature extraction unit 23. The feature extraction unit 23 is used to extract specified features from the micro-Doppler time-frequency spectrum.

[0197] The feature extraction unit 23 extracts features by:

[0198] S26: The specified features extracted from the denoised micro-Doppler time-frequency map include: the mean and standard deviation of the centroid, the mean and standard deviation of the bandwidth, and the mean and standard deviation of the upper and lower contour Doppler frequency intervals.

[0199] S27: Calculate characteristic information in the denoised micro-Doppler time-frequency graph. The characteristic information includes, but is not limited to, the proportion of valid micro-Doppler signals greater than the separation threshold in the micro-Doppler spectrum, the maximum peak value of the Doppler frequency corresponding to the upper contour, and the maximum interval of Doppler frequencies between peak values.

[0200] Specifically, as shown in Figure 10, the first row, from left to right, shows the micro-Doppler spectra before noise suppression for the conditions of curtain interference, fan interference, a person entering the room with no interference, and a person leaving the room with no interference. The second row, from left to right, shows the micro-Doppler spectra after noise suppression for the conditions of curtain interference, fan interference, a person entering the room with no interference, and a person leaving the room with no interference. The comparison clearly shows that background noise has been effectively suppressed, significantly reducing the impact of noise on the micro-Doppler signals of human entry and exit movements and interfering activities, facilitating subsequent feature extraction.

[0201] The feature extraction unit 23 extracts each designated feature in the following manner.

[0202] The formula for extracting the Doppler centroid is:

[0203] The formula for extracting the Doppler bandwidth is:

[0204] F(i,j) represents the value of the micro-Doppler spectrum in the i-th Doppler block and the j-th time block, and f(i) represents the value of the Doppler frequency in the i-th Doppler block.

[0205] After obtaining the values ​​of each Doppler centroid and Doppler bandwidth, calculate the Doppler centroid f c (j) and Doppler bandwidth B c (j) Mean and standard deviation.

[0206] Extract the interval f of the Doppler frequency interval between the upper and lower contours span The formula for (j) is: C = mean(F(i,j)) + α*std(F(i,j)) f span (j) = f(u j )-f(l j )

[0207] Where C represents the contour threshold, which is used to accurately extract the upper and lower contours. α represents a scaling factor less than 1, which is used to adjust the value of the contour threshold. mean(F(i,j)) represents the mean value of the micro-Doppler spectrum matrix, std(F(i,j)) represents the standard deviation of the micro-Doppler spectrum, and u j Indicates the u-th time block corresponding to the upper envelope j Doppler blocks, l i Indicates the lth time block corresponding to the lower envelope i Doppler bins. F(u,j) represents the value of the micro-Doppler spectrum at the uth Doppler bin and the jth time bin. F(l,j) represents the value of the micro-Doppler spectrum at the lth Doppler bin and the jth time bin.

[0208] After extracting the intervals of the upper and lower contour Doppler frequency intervals, the mean and standard deviation of the upper and lower contour Doppler frequency intervals are calculated.

[0209] The method for extracting the contour size of the micro-Doppler spectrum is as follows.

[0210] f s Indicates the contour size. The contour size represents the proportion of the noise-suppressed micro-Doppler spectrum above the threshold. The micro-Doppler spectrum matrix has N × T values, where N represents the number of Doppler bins and T represents the number of time bins. Define the interval threshold, th = mean(F(i,j)) - 0.05 * std(F(i,j)). The number of elements in the micro-Doppler spectrum matrix above this threshold is K. The formula for extracting the contour size of the spectrum is:

[0211] Extract the average of the Doppler frequency peaks.

[0212] Figure 11 shows an example of the peak points and peak extension of the upper envelope in the micro-Doppler spectrum. As shown in Figure 11, the micro-Doppler spectrum shows several peaks of Doppler frequency. The Doppler frequency peaks in Figure 11 include Peak 1, Peak 2, and Peak 3.

[0213] Assume that there are m peaks in the upper envelope, and the Doppler frequency corresponding to each peak is p i ,i∈m. The formula for the average value of the Doppler frequency peak is as follows:

[0214] Among them, mean(f c ) represents the average value of the Doppler centroid.

[0215] The peak spread in the micro-Doppler spectrum is the maximum interval of Doppler frequencies between peaks, and the formula is as follows: s =p max -pmin .

[0216] The feature extraction unit 23 sends the designated features extracted from the denoised micro-Doppler spectrum to the classification unit 24 .

[0217] The classification unit 24 is preferably a classifier, which has a classification model set therein. The classification model is constructed based on a random forest algorithm. Preferably, the algorithm of the classification model is not limited to the random forest algorithm, and can also be other algorithms capable of classification.

[0218] The classification model is formed by classification training based on the random forest algorithm and training set data. The training set data consists of data with specified features extracted from the radar signal data after the above steps S21 to S27. The training set data is used to train the classification model.

[0219] The number of decision trees for the random forest classifier is set to 150. The input of the random forest classifier is the nine features described in the feature extraction section, and the output is the category of the radar signal data.

[0220] The steps to build a classification model include:

[0221] A set of radar signal data is subjected to short-time Fourier transform and noise separation, and the nine eigenvalues ​​extracted are used as the input eigenvalues ​​of the classification model.

[0222] The radar signal data corresponding to the input feature values ​​can be divided into five categories:

[0223] The first type of indoor situation refers to the indoor situation when a person walks into the room.

[0224] The second type of indoor situation refers to the indoor situation when people walk out of the room.

[0225] The third type of indoor situation refers to the indoor situation where the first interference factor exists.

[0226] The fourth type of indoor situation refers to the indoor situation where the second interference factor exists.

[0227] The fifth type of indoor situation refers to an indoor situation where people are not indoors and the first interference factor and the second interference factor do not exist.

[0228] The third, fourth, and fifth types of indoor situations can all be classified as interference.

[0229] The classification unit 24 performs classification processing on the received feature values ​​and sends the data type to the statistics unit 25. The statistics unit 25 receives and counts the classification data and the frequency of occurrence of various indoor situations.

[0230] The statistical unit 25 is preferably a counter. The initial state of the counter is set to 0, and the state change of the counter is only related to the detected entry and exit.

[0231] When the classification unit 24 classifies a set of input data as the first indoor situation (a person walking into a room), the counter status is incremented by 1. When the input data is classified as the second indoor situation (a person walking out of a room), the counter status is decremented by 1. If the counter status is 0 at this time, the counter status remains unchanged. When the data is classified as a disturbance state, the counter status remains unchanged.

[0232] After data training is completed, the indoor monitoring device of the present invention can accurately detect whether a person is indoors without being interfered with by the first or second interference factors. Therefore, the present invention has a high degree of accuracy in monitoring indoor occupants, requires less data to be calculated, processes data quickly, and provides faster feedback.

[0233] The present invention improves the accuracy of indoor human monitoring by collecting and calculating interference data from the main indoor interference sources: curtains (the first interference factor) and ventilation equipment (the second interference factor). For example, the present invention divides indoor data collected by radar into five categories: human entry and exit data with or without fan or curtain interference, curtain shaking data, fan rotation data, and radar data collected when no one is present and no curtains or fans are moving. Based on these five types of data, the present invention performs data analysis, feature extraction, data classification, and data training, which can obtain accurate results for indoor monitoring, accurately determine whether there are people indoors, and provide more operational solutions for future smart home appliances to perform classified intelligent control based on the presence or absence of people indoors.

[0234] For example, the classification unit can calculate indoor occupancy monitoring results based on real-world data collected by the millimeter-wave radar and send these results to the control unit. The control unit then selects a corresponding control strategy based on the monitoring results and sends control instructions to the corresponding smart devices to achieve precise control.

[0235] For example, a millimeter-wave radar collects data on people entering and exiting a room, curtain data, and air conditioning data. Computation unit 2 calculates this data and determines that no one is present in the room. If a person's absence from the room exceeds a preset first time threshold, the control unit sends a shutdown command to the air conditioner, shutting it down. The control unit also sends a shutdown command to the curtain control unit, closing the curtains.

[0236] When the length of time that the person is not indoors does not exceed the preset first time threshold, the control unit does not send the corresponding control instruction to the designated smart device.

[0237] When the monitoring result of indoor personnel changes from an unmanned state to a occupied state, and the length of time the person stays indoors exceeds a second time threshold, the control unit sends corresponding control instructions to the designated smart device, such as sending control instructions to the air conditioner and curtain controller to open or close the curtains and turn on the air conditioner.

[0238] As described above, the control system of the present invention can perform data analysis and issue accurate control instructions based on the collected data of only one monitoring end of the millimeter wave radar, thereby achieving accurate control of the smart device.

[0239] The advantages of the present invention also include: there is no need to monitor the displacement, azimuth and movement speed of the person, which reduces the amount of data calculation and the storage space required for data storage. Therefore, the monitoring device of the present invention has less data processing volume and faster data result feedback.

[0240] After training, Computing Unit 2 calculates indoor occupant monitoring results based on real-world data collected by the millimeter-wave radar and sends these results to the Control Unit. The Control Unit then selects a control strategy based on these results and sends control instructions to the corresponding smart devices to achieve precise control.

[0241] For example, the millimeter-wave radar collects data on people entering and exiting a room, curtain data, and air conditioning data. The calculation unit 2 calculates this data and determines that no one is inside. If a person is absent from the room for longer than a preset time threshold, the control unit sends a shutdown command to the air conditioner, turning it off. The control unit also sends a shutdown command to the curtain control unit, closing the curtains.

[0242] When the length of time that the person is not indoors does not exceed the preset first time threshold, the control unit does not send the corresponding control instruction to the designated smart device.

[0243] When the monitoring result of indoor personnel changes from an unmanned state to a occupied state, and the length of time the person stays indoors exceeds a second time threshold, the control unit sends corresponding control instructions to the designated smart device, such as sending control instructions to the air conditioner and curtain controller to open or close the curtains and turn on the air conditioner.

[0244] As described above, the control system of the present invention can perform data analysis and issue accurate control instructions based on the collected data of only one monitoring end of the millimeter wave radar, thereby achieving accurate control of the smart device.

[0245] It should be noted that the above-mentioned specific embodiments are exemplary, and those skilled in the art can come up with various solutions inspired by the disclosure of the present invention, and these solutions also fall within the scope of the disclosure of the present invention and fall within the scope of protection of the present invention. Those skilled in the art should understand that the present invention specification and its drawings are illustrative and do not constitute a limitation on the claims. The scope of protection of the present invention is defined by the claims and their equivalents. The present invention specification contains multiple inventive concepts, such as "preferably", "according to a preferred embodiment" or "optionally", which means that the corresponding paragraph discloses an independent concept, and the applicant reserves the right to file a divisional application based on each inventive concept.

Claims

1. An indoor monitoring device, comprising at least a millimeter wave radar (1) and a computing unit (2), characterized in that: The computing unit (2) is configured to: In response to radar signal data transmitted by the millimeter wave radar (1), the calculation unit (2) analyzes micro-Doppler spectrum signals greater than the separation threshold from the micro-Doppler spectrum formed through the pre-processing step based on a preset separation threshold; extracting at least one eigenvalue based on the micro-Doppler time-frequency map after background noise suppression processing; The radar signal data is classified based on the constructed classification model and the extracted feature values ​​to obtain an indoor situation category corresponding to the radar signal data.

2. The indoor monitoring device according to claim 1, characterized in that: The step of pre-processing the radar signal data by the computing unit (2) at least comprises: Calculating a mean of the first radar signal data; subtracting the mean value from the first radar signal data to suppress a zero Doppler component, thereby obtaining second radar signal data; Performing a short-time Fourier transform on the second radar signal data to obtain a short-time Fourier transform result of human motion and various interferences; The micro-Doppler spectrum is obtained based on the results of short-time Fourier transform; The first radar signal data is initial radar signal data.

3. The indoor monitoring device according to claim 1 or 2, characterized in that: The step of the calculation unit (2) analyzing the micro-Doppler signal greater than the separation threshold comprises at least: Setting a separation threshold for separating background noise and a micro-Doppler signal of a target in a micro-Doppler spectrum to suppress the background noise; The portion of the micro-Doppler spectrum below the separation threshold is treated as noise and set as a noise characteristic value, and the portion of the micro-Doppler spectrum greater than the separation threshold is treated as a valid micro-Doppler signal.

4. The indoor monitoring device according to any one of claims 1 to 3, characterized in that: The characteristic values ​​extracted by the calculation unit (2) from the micro-Doppler time-frequency diagram include at least: The mean and standard deviation of the centroid, the mean and standard deviation of the bandwidth, the mean and standard deviation of the Doppler frequency intervals between the upper and lower contours, the proportion of valid micro-Doppler signals greater than the separation threshold in the micro-Doppler spectrum, the maximum peak of the Doppler frequency corresponding to the upper contour and / or the maximum interval of Doppler frequencies between peaks.

5. The system according to any one of claims 1 to 4, characterized in that: The computing unit is further configured to: The indoor situation category output by the classification unit (24) is used to count the status of people entering and leaving the room, wherein: Set the initial state of the statistics to 0, When the classification model classifies a set of second radar signal data as the first type of indoor situation, the statistical value is increased by 1; When a set of second radar signal data is classified as the second type of indoor situation, the statistical value is reduced by 1; if the data value of the statistical state is 0 at this time, the statistical value state remains unchanged; When the second radar signal data is classified into the third, fourth, or fifth indoor situation, the statistical value state remains unchanged.

6. An indoor monitoring method, characterized in that: The method at least comprises: In response to radar signal data transmitted by the millimeter wave radar (1), the calculation unit (2) analyzes micro-Doppler spectrum signals greater than the separation threshold from the micro-Doppler spectrum formed through the pre-processing step based on a preset separation threshold; The calculation unit (2) extracts at least one characteristic value based on the micro-Doppler time-frequency diagram after background noise suppression; The calculation unit (2) classifies the radar signal data based on the constructed classification model and the extracted feature values ​​to obtain an indoor situation category corresponding to the radar signal data.

7. The indoor monitoring method according to claim 6, characterized in that: The step of preprocessing the radar signal data at least includes: Calculating a mean of the first radar signal data; subtracting the mean value from the first radar signal data to suppress a zero Doppler component, thereby obtaining second radar signal data; Performing a short-time Fourier transform on the second radar signal data to obtain a short-time Fourier transform result of human motion and various interferences; The micro-Doppler spectrum is obtained based on the results of short-time Fourier transform; The first radar signal data is initial radar signal data.

8. The indoor monitoring method according to claim 6 or 7, characterized in that: The step of the calculation unit (2) analyzing the micro-Doppler signal greater than the separation threshold comprises at least: Setting a separation threshold for separating background noise and a micro-Doppler signal of a target in a micro-Doppler spectrum to suppress the background noise; The portion of the micro-Doppler spectrum below the separation threshold is treated as noise and set as a noise characteristic value, and the portion of the micro-Doppler spectrum greater than the separation threshold is treated as a valid micro-Doppler signal.

9. An indoor control system comprising at least a millimeter-wave radar (1), a computing unit (2), and a control unit (3), wherein the computing unit (2) analyzes the indoor occupant status from the signal collected by the millimeter-wave radar (1) and sends the signal to the control unit (3), and the control unit (3) sends corresponding control instructions to at least one smart device based on the indoor occupant status and a preset control strategy. It is characterized in that The calculation unit (2) at least comprises: A feature extraction unit (23) extracts at least one feature value from the micro-Doppler time-frequency map after background noise suppression; A classification unit (24) classifies the radar signal data based on the constructed classification model and the characteristic value to determine the status of indoor personnel.

10. The indoor control system according to claim 9, characterized in that: The computing unit (2) further comprises a signal analysis unit (22), wherein the signal analysis unit (22) is configured to: The part of the micro-Doppler spectrum below the preset separation threshold is treated as noise and set as the noise characteristic value, and the part of the micro-Doppler spectrum greater than the separation threshold is set as a valid micro-Doppler signal; The feature extraction unit (23) receives the micro-Doppler spectrum after background noise suppression from the signal analysis unit (22).

11. The indoor control system according to claim 9 or 10, characterized in that: The computing unit (2) further comprises a pre-processing unit (21), wherein the pre-processing unit (21) is configured to: Calculating a mean value based on data sent by the millimeter wave radar (1); subtracting the mean value from the first radar signal data to suppress a zero-Doppler component and obtain second radar signal data; Performing short-time Fourier transform on the second signal radar signal data to obtain short-time Fourier transform results of human body motion and various interferences; The micro-Doppler spectrum is obtained based on the results of short-time Fourier transform; The pre-processing unit (21) sends the information of the micro-Doppler spectrum to the signal analysis unit (22).

12. The indoor control system according to any one of claims 9 to 11, characterized in that: The calculation unit (2) further comprises a statistical unit (25), wherein the statistical unit (25) is configured to: Set the initial state of the statistics to 0, When the classification model judges a set of input data as the first type of indoor situation, the statistical value increases by 1; When a set of input data is judged as the second type of indoor situation, the statistical value is reduced by 1; If the data value of the statistical status is 0 at this time, the statistical value status remains unchanged; When the input data is judged to be the third, fourth or fifth category of indoor conditions, the statistical value status remains unchanged; The control unit (3) determines a corresponding control strategy based on the data value sent by the statistical unit (25).

13. An indoor control method, comprising: The indoor occupant status is analyzed from a radar signal collected by a millimeter-wave radar (1) and sent to a control unit (3); the control unit (3) sends a corresponding control instruction to at least one smart device based on the indoor occupant status and a preset control strategy, wherein the method further comprises: extracting at least one eigenvalue from the micro-Doppler time-frequency map after background noise suppression; The radar signal data is classified based on the constructed classification model and the characteristic value to determine the status of indoor personnel.

14. The indoor control method according to claim 13, characterized in that: Methods for obtaining a micro-Doppler spectrum after background noise suppression include: The part of the micro-Doppler spectrum below the preset separation threshold is treated as noise and set as the noise characteristic value, and the part of the micro-Doppler spectrum greater than the separation threshold is set as a valid micro-Doppler signal; The feature extraction unit (23) receives the micro-Doppler spectrum after background noise suppression from the signal analysis unit (22).

15. The indoor control method according to claim 13 or 14, characterized in that: Before suppressing background noise in the micro-Doppler spectrum, micro-Doppler spectrum extraction is performed; Among them, the micro-Doppler spectrum extraction method includes at least: Calculating a mean value based on data sent by the millimeter wave radar (1); subtracting the mean value from the first radar signal data to suppress a zero-Doppler component and obtain second radar signal data; Performing short-time Fourier transform on the second signal radar signal data to obtain short-time Fourier transform results of human body motion and various interferences; The micro-Doppler spectrum is obtained based on the results of short-time Fourier transform; The pre-processing unit (21) sends the information of the micro-Doppler spectrum to the signal analysis unit (22).

Images