Method of monitoring, monitoring device, computer program product and monitoring system

Abstract

The application provides a monitoring method, a monitoring device, a computer program product and a monitoring system. The method comprises the following steps: acquiring user data of a user, wherein the user data comprises one or more of a position, a movement track, a body state, biochemical data and physiological data; determining whether the user is at risk according to the user data; in the case that it is determined that the user is at risk, generating prompt information and warning according to the prompt information. The scheme solves the problem of poor monitoring efficiency of a bathroom space in the prior art.

Description

Technical Field

[0001] This application relates to the field of Internet of Things (IoT) technology, and more specifically, to a monitoring method, monitoring device, computer program product, and monitoring system. Background Technology

[0002] With the deepening development of Internet of Things (IoT) technology and smart home applications, the level of intelligence in bathroom spaces, which are frequently used in family life and have extremely high requirements for privacy protection, is gradually improving. Currently, for safety and health monitoring in bathroom scenarios, existing technologies mainly rely on infrared sensors or single fall detection devices.

[0003] Existing non-contact sensors, such as infrared pyroelectric sensors, suffer from severely limited performance when faced with the unique high temperature, high humidity, and water mist interference characteristic of bathroom environments. Water vapor strongly absorbs infrared signals, and the steam in the bathroom causes multipath reflection and signal attenuation, resulting in an extremely high false alarm rate. Therefore, current monitoring efficiency in bathroom spaces is poor. Summary of the Invention

[0004] The main objective of this application is to provide a monitoring method, monitoring device, computer program product, and monitoring system to at least solve the problem of poor monitoring efficiency in bathroom spaces in the prior art.

[0005] To achieve the above objectives, according to one aspect of this application, a monitoring method is provided, comprising: acquiring user data collected from a user, wherein the user data includes one or more of location, movement trajectory, body shape, biochemical data, and physiological data; determining whether the user poses a risk based on the user data; and, if the user is determined to pose a risk, generating a prompt message and issuing an early warning based on the prompt message.

[0006] Optionally, the user data is collected based on millimeter-wave radar. Acquiring the collected user data includes: acquiring the signal emitted by the millimeter-wave radar to obtain the emitted signal; acquiring the signal reflected by the user's human body to obtain the echo signal; combining the emitted signal and the echo signal to obtain a millimeter-wave dataset; and inputting the millimeter-wave dataset into a physiological recognition model to obtain the user data corresponding to the millimeter-wave dataset.

[0007] Optionally, before inputting the millimeter-wave dataset into the physiological recognition model to obtain the user data corresponding to the millimeter-wave dataset, the method further includes: obtaining a recognition model, wherein the recognition model is one of a CNN model, an RNN model, and an LSTM model; forming a training set by combining the historical millimeter-wave dataset and the corresponding user labels, and training the recognition model using the training set to obtain the physiological recognition model, wherein the user labels are the historical user data corresponding to the historical millimeter-wave dataset in the training set.

[0008] Optionally, the risk includes the risk of falling, the physiological data includes heart rate, and determining whether the user is at risk based on the user data includes: determining that the user is at risk of falling when the user's posture changes abruptly and the rate of change of heart rate is greater than a preset heart rate change threshold, wherein the posture change is the user's posture changing from standing to sitting or lying down.

[0009] Optionally, the risk includes the risk of being stationary. Determining whether the user is at risk based on the user data includes: determining that the user is at risk of being stationary if the user spends more than a preset time threshold in the same location and the movement trajectory does not change.

[0010] Optionally, the prompt information includes a first prompt information and / or a second prompt information, and the warning based on the prompt information includes at least one of the following: if it is determined that the user is at risk of remaining stationary, control the smart speaker to play music based on the first prompt information, wherein the first prompt information is information to prompt the user using music; control the lighting device to turn on the light mode based on the second prompt information, wherein the second prompt information is information to prompt the user using light.

[0011] Optionally, the user data further includes facial spectral data, and the prompt information includes third prompt information. When it is determined that the user faces the risk, the prompt information is generated, including: calculating a weighted average of the biochemical data, the physiological data, and the facial spectral data to obtain a comprehensive score; if the comprehensive score is greater than a preset score threshold, it is determined that the user faces the risk, and the third prompt information is generated, wherein the third prompt information indicates that the user's biochemical data, physiological data, and facial spectral data are abnormal.

[0012] Optionally, the prompt information includes a fourth prompt information, which is generated when it is determined that the user has the risk. The fourth prompt information is generated when the biochemical data is not within a preset normal biochemical range and the user has the risk. The fourth prompt information is a message indicating that the user has the risk and that the biochemical data is abnormal.

[0013] Optionally, the prompt information includes one or more of a fifth prompt information, a sixth prompt information, and a seventh prompt information. The warning based on the prompt information includes one of the following: controlling the smart mirror to flash an alarm light based on the fifth prompt information, wherein the fifth prompt information is a prompt to the user using the smart mirror; controlling the smart speaker to emit an inquiry voice based on the sixth prompt information, wherein the inquiry voice is a voice asking the user if they need help, and the sixth prompt information is a voice prompt to the user; and if no response voice is received, or if the response voice indicates that the user needs help, sending the seventh prompt information to a target device, wherein the target device is the device of the user's emergency contact, and the seventh prompt information is a prompt to the emergency contact via SMS or telephone.

[0014] According to another aspect of this application, a monitoring device is provided, comprising: a first acquisition unit, configured to acquire user data collected from a user, wherein the user data includes one or more of location, movement trajectory, body posture, biochemical data, and physiological data; a determination unit, configured to determine whether the user poses a risk based on the user data; and an early warning unit, configured to generate a prompt message and issue an early warning based on the prompt message if the user is determined to pose the risk.

[0015] According to another aspect of this application, a computer program product is provided, comprising a computer program that, when executed by a processor, implements the steps of any of the monitoring methods.

[0016] According to another aspect of this application, a monitoring system is provided, comprising: a millimeter-wave radar; a smart speaker; a lighting device; a smart toilet; a smart mirror; and a controller, wherein the controller is communicatively connected to the millimeter-wave radar, the smart speaker, the lighting device, the smart toilet, and the smart mirror, and the controller is used to execute any of the monitoring methods described herein.

[0017] By applying the technical solution of this application, the millimeter-wave radar signal will not be affected by water mist or other interferences. The strong absorption interference of water vapor on the sensing signal is eliminated at the hardware sensing level, ensuring signal integrity in the high-humidity environment of the bathroom. This reduces signal loss or false triggering caused by environmental interference, fundamentally reducing the false alarm rate. The millimeter-wave radar can accurately capture millimeter-level chest micro-movements (for extracting heart rate) and centimeter-level body displacements (for extracting trajectory). By analyzing this high-precision data, it can accurately identify "falls" (drastic changes in posture or position) and "stillness" (no trajectory change for a long time), solving the monitoring failure problem caused by environmental interference in the prior art and improving the safety monitoring efficiency of the bathroom space. Attached Figure Description

[0018] The accompanying drawings, which form part of this application, are used to provide a further understanding of this application. The illustrative embodiments and descriptions of this application are used to explain this application and do not constitute an undue limitation of this application. In the drawings:

[0019] Figure 1 A hardware structure block diagram of a mobile terminal performing a monitoring method according to an embodiment of this application is shown;

[0020] Figure 2 A schematic flowchart of a monitoring method according to an embodiment of this application is shown;

[0021] Figure 3 A flowchart illustrating the monitoring and early warning process is shown;

[0022] Figure 4 A structural block diagram of a monitoring device provided according to an embodiment of this application is shown.

[0023] The above figures include the following reference numerals:

[0024] 102. Processor; 104. Memory; 106. Transmission device; 108. Input / output device. Detailed Implementation

[0025] It should be noted that, unless otherwise specified, the embodiments and features described in this application can be combined with each other. This application will now be described in detail with reference to the accompanying drawings and embodiments.

[0026] To enable those skilled in the art to better understand the present application, the technical solutions in the embodiments of the present application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present application, and not all embodiments. Based on the embodiments in the present application, all other embodiments obtained by those of ordinary skill in the art without creative effort should fall within the scope of protection of the present application.

[0027] It should be noted that the terms "first," "second," etc., in the specification, claims, and accompanying drawings of this application are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that such data can be interchanged where appropriate for the embodiments of this application described herein. Furthermore, the terms "comprising" and "having," and any variations thereof, are intended to cover non-exclusive inclusion; for example, a process, method, system, product, or apparatus that comprises a series of steps or units is not necessarily limited to those steps or units explicitly listed, but may include other steps or units not explicitly listed or inherent to such processes, methods, products, or apparatus.

[0028] With the popularization of smart home IoT technology, the need for safety and health monitoring in bathrooms, as frequently used private spaces, is becoming increasingly prominent. Existing technologies suffer from privacy violations, such as camera surveillance. Traditional infrared or visual sensors have high false alarm rates in high-temperature and water-mist environments, and their detection dimensions are limited, focusing only on fall detection and lacking correlation analysis with physiological indicators such as emotional health. Furthermore, the lack of interconnectivity among smart devices in bathrooms makes it difficult to form coordinated early warning systems.

[0029] As described in the background section, the monitoring efficiency of bathroom spaces in the prior art is poor. To solve the above problems, embodiments of this application provide a monitoring method, a monitoring device, a computer program product, and a monitoring system.

[0030] The technical solutions of the present invention will be clearly and completely described below with reference to the accompanying drawings in the embodiments of the present invention.

[0031] The methods and embodiments provided in this application can be executed on a mobile terminal, computer terminal, or similar computing device. Taking running on a mobile terminal as an example, Figure 1 This is a hardware structure block diagram of a mobile terminal for a monitoring method according to an embodiment of the present invention. Figure 1 As shown, a mobile terminal may include one or more ( Figure 1Only one is shown in the diagram. A processor 102 (which may include, but is not limited to, a microprocessor MCU or a programmable logic device FPGA, etc.) and a memory 104 for storing data are also shown. The mobile terminal may further include a transmission device 106 for communication functions and an input / output device 108. Those skilled in the art will understand that... Figure 1 The structure shown is for illustrative purposes only and does not limit the structure of the mobile terminal described above. For example, the mobile terminal may also include components that are more... Figure 1 The more or fewer components shown, or having the same Figure 1 The different configurations shown.

[0032] The memory 104 can be used to store computer programs, such as application software programs and modules, like the computer program corresponding to the monitoring method in this embodiment of the invention. The processor 102 executes various functional applications and data processing by running the computer program stored in the memory 104, thereby implementing the above-described method. The memory 104 may include high-speed random access memory and may also include non-volatile memory, such as one or more magnetic storage devices, flash memory, or other non-volatile solid-state memory. In some instances, the memory 104 may further include memory remotely located relative to the processor 102, and these remote memories can be connected to the mobile terminal via a network. Examples of the aforementioned networks include, but are not limited to, the Internet, corporate intranets, local area networks, mobile communication networks, and combinations thereof. The transmission device 106 is used to receive or send data via a network. Specific examples of the aforementioned networks may include wireless networks provided by the mobile terminal's communication provider. In one example, the transmission device 106 includes a network interface controller (NIC), which can be connected to other network devices via a base station to communicate with the Internet. In one example, the transmission device 106 may be a radio frequency (RF) module, which is used to communicate with the Internet wirelessly.

[0033] This embodiment provides a monitoring method that runs on a mobile terminal, computer terminal, or similar computing device. It should be noted that the steps shown in the flowchart in the accompanying drawings can be executed in a computer system such as a set of computer-executable instructions. Although a logical order is shown in the flowchart, in some cases, the steps shown or described may be executed in a different order than that shown here.

[0034] Figure 2 This is a flowchart illustrating the monitoring method according to an embodiment of this application. Figure 2 As shown, the method includes the following steps:

[0035] Step S201: Obtain the collected user data, wherein the user data includes one or more of the following: location, movement trajectory, body shape, biochemical data, and physiological data;

[0036] Specifically, the monitoring method can be applied to specific spaces, such as bathrooms. User data is collected using millimeter-wave radar, which is located within the bathroom. The millimeter-wave radar is deployed high on the ceiling or walls of the bathroom to achieve comprehensive coverage of the entire space. The millimeter-wave radar transmits frequency-modulated continuous wave (FMCW) and receives the echo signals reflected from the human body. To overcome interference from high temperature, high humidity, and water mist in the bathroom environment, the echo signals are first preprocessed, including a water vapor self-calibration algorithm to compensate for the attenuation and phase effects of water vapor, ensuring data accuracy. Subsequently, the time-domain signal is converted into a range-Doppler spectrum or point cloud data using a Fast Fourier Transform (FFT). Deep learning models, such as Convolutional Neural Networks (CNNs) or Long Short-Term Memory (LSTMs), are then used to process the point cloud data to extract key user features. Here, "location" refers to the user's three-dimensional coordinates within the bathroom space; "movement trajectory" refers to the user's position sequence over time, used to analyze movement status; and the physiological data is heart rate, which is calculated by extracting micro-motion signals from the chest cavity, followed by signal demodulation and filtering. User data can be one or more combinations of the above, such as containing only location and heart rate, or simultaneously containing location, trajectory, and heart rate.

[0037] Millimeter-wave radar can be installed at the geometric center of the ceiling in the bathroom, or directly above the center of the shower / toilet area. The top-view angle can minimize the obstruction of the radar beam by the limbs, and can completely capture the user's movement trajectory on the ground. In addition, millimeter-wave radar can also be installed in the corner of the bathroom, which can save space and does not affect the spatial layout.

[0038] The biochemical data mentioned in this application refers to biochemical indicators in user excrement (mainly urine) collected by sensors built into the smart toilet. This biochemical data includes, but is not limited to, cortisol concentration, glucose concentration, and ketone body concentration. The physiological data mentioned in this application includes heart rate, respiratory rate, blood oxygen saturation, etc.

[0039] Step S202: Determine whether the user poses a risk based on the aforementioned user data;

[0040] Specifically, the risks include the risk of falling or the risk of remaining stationary. Based on collected user data, real-time analysis is performed using a pre-defined risk assessment logic. For the "risk of remaining stationary," the user's movement trajectory is monitored. If the user's position coordinates within a pre-defined monitoring area remain unchanged within a pre-defined time threshold, and the trajectory shows no significant displacement, the user is initially identified as stationary. For the "risk of falling," movement trajectory and heart rate changes are monitored. If the user's movement trajectory is characterized by a sudden change in posture—that is, a sharp drop in body center of gravity height within a short time window, or a drastic change in body axis tilt angle, such as an instantaneous shift from a standing posture to a near-horizontal posture—and the rate of change in heart rate exceeds a pre-defined heart rate change threshold during or immediately after this change, a fall risk is identified. Furthermore, if only prolonged stillness is detected, without the detection of normal breathing or slight movement, it may also be classified as a high-risk stillness state.

[0041] This application utilizes a millimeter-wave radar deployed in a bathroom to transmit frequency-modulated continuous wave signals and receive echoes reflected from the human body. It estimates the direction of arrival (DOA) using the phase difference of a multi-antenna array, and combines fast Fourier transforms in the range and Doppler dimensions with constant false alarm rate (CFAR) detection to convert polar coordinates into three-dimensional points in a Cartesian coordinate system. Furthermore, it generates the user's movement trajectory and real-time location through time-series correlation and filtering algorithms. Simultaneously, it identifies the region of interest (ROI) where the user's body is located, extracts subtle phase or amplitude variations in the echo signal, removes noise interference through principal component analysis, and separates the heart rate component using bandpass filtering. The user's heart rate is then calculated through peak detection or spectral analysis. This allows for high-precision synchronous acquisition of location, trajectory, and physiological data in a contactless environment resistant to water mist interference.

[0042] The risk of immobility mentioned in this application refers to a user being in an abnormal, prolonged state of stillness within the bathroom space, possibly due to physical discomfort or sudden illness that renders them unable to move, rather than normal toilet or bathing behavior. The risk of falling refers to a user's sudden drop in center of gravity due to loss of balance, slipping, or sudden illness, causing them to instantly change from a standing or sitting position to a lying or sitting position.

[0043] Step S203: If it is determined that the above-mentioned user has the above-mentioned risk, generate a prompt message and issue a warning based on the prompt message.

[0044] Specifically, when a user is determined to be at risk of immobility or falling, a corresponding alert message is generated. This alert message is not a simple text message, but a data packet containing specific control instructions. For example, if the risk of immobility is determined, the alert message may include instructions such as "play soothing music" or "switch to soft lighting"; if the risk of falling is determined, the alert message may include instructions such as "flash warning lights," "issue a voice inquiry," or "send an emergency notification." By parsing this alert message, smart devices in the bathroom can be triggered to respond in a coordinated manner. For example, the smart speaker can be controlled to play preset soothing music to calm the user or attract their attention, the lighting equipment can be controlled to adjust its brightness and color temperature, or the smart mirror can be controlled to flash warning lights and issue a voice inquiry. In the case of an emergency risk, the alert message can also trigger a process of sending an alarm notification to the user's preset emergency contact device or automatically dialing an emergency number.

[0045] In this embodiment, the millimeter-wave radar signal is not affected by water vapor or other interferences. This eliminates the strong absorption interference of water vapor on the sensor signal at the hardware sensing level, ensuring signal integrity in the high-humidity environment of the bathroom. This reduces signal loss or erroneous triggering caused by environmental interference, fundamentally lowering the false alarm rate. The millimeter-wave radar can accurately capture millimeter-level chest micro-movements (for heart rate extraction) and centimeter-level body displacements (for trajectory extraction). By analyzing this high-precision data, it can accurately identify "falls" (drastic changes in posture or position) and "immobility" (no trajectory change for a long time), solving the monitoring failure problem caused by environmental interference in existing technologies and improving the safety monitoring efficiency of bathroom spaces.

[0046] Specifically, when user data includes location, the system determines whether a user poses a risk based on that location. Specifically, if a user remains in the same location for more than a preset time threshold, the system determines that the user poses a risk. (This risk could be a stationary risk).

[0047] Specifically, when user data includes movement trajectories, the system determines whether a user poses a risk based on these trajectories. Specifically, if the duration for which the user's movement trajectory remains unchanged exceeds a preset threshold, the system determines that the user poses a risk. (This risk could be a static risk).

[0048] Specifically, when user data includes body posture, the presence of risk is determined based on body posture. In particular, a sudden change in the user's posture indicates a risk (which could be the risk of falling).

[0049] Specifically, when user data includes biochemical data, the system determines whether a user poses a risk based on the biochemical data. If the biochemical data is outside the preset normal range, the system determines that the user poses a risk (in which case the user is only notified that the biochemical data is abnormal).

[0050] Specifically, when user data includes physiological data, the system determines whether a user is at risk based on this physiological data. This physiological data includes heart rate; if the rate of change in heart rate exceeds a preset threshold, the system determines that the user is at risk of falling (this risk can be classified as fall risk).

[0051] In the specific implementation process, the aforementioned user data is collected based on millimeter-wave radar. Acquiring the collected user data includes: acquiring the signal emitted by the millimeter-wave radar to obtain the transmitted signal; acquiring the signal reflected by the user's human body to obtain the echo signal; combining the transmitted signal and the echo signal to obtain a millimeter-wave dataset; and inputting the millimeter-wave dataset into a physiological recognition model to obtain the aforementioned user data corresponding to the millimeter-wave dataset.

[0052] In this solution, traditional optical or infrared sensors are easily obscured by water mist or interfered with by background thermal radiation in the high humidity environment of the bathroom, resulting in data distortion. The millimeter-wave radar in this solution has the physical properties of penetrating water vapor, and the data obtained is of high quality, ensuring good data accuracy. Moreover, it does not collect user images, thus achieving privacy and security monitoring.

[0053] In the specific implementation process, before inputting the above millimeter-wave dataset into the physiological recognition model to obtain the user data corresponding to the above millimeter-wave dataset, the above method further includes: obtaining a recognition model, wherein the recognition model is one of a CNN model, an RNN model, or an LSTM model; forming a training set by combining the historical millimeter-wave dataset and the corresponding user labels, and training the above recognition model using the above training set to obtain the above physiological recognition model, wherein the above user labels are the historical user data corresponding to the above historical millimeter-wave dataset in the above training set.

[0054] In this scheme, the bathroom environment is highly variable (e.g., different user body sizes, different water vapor concentrations, different background noise). By constructing the training set containing historical millimeter wave datasets and user labels, deep learning training is performed using recognition models such as LSTM or CNN. This enables the model to automatically learn the nonlinear mapping relationship between millimeter wave signals and user data. Through the training process, the physiological recognition model can adapt to the specificity of the bathroom environment, thus enabling it to output user data more accurately when millimeter wave datasets are input.

[0055] In bathroom environments, millimeter-wave radar typically operates in the 24GHz, 60GHz, or 77GHz bands, transmitting frequency-modulated continuous wave (FMCW). The transmitted signal is a linearly modulated frequency signal whose frequency varies linearly with time. When this signal is reflected by the user's body, it generates an echo signal, which carries distance, velocity, and angle information. Due to the high temperature, high humidity, and water mist present in bathroom environments, water molecules absorb and scatter the radar signal, leading to signal attenuation. Therefore, before combining the transmitted signal and the echo signal, the echo signal is preprocessed, including a water vapor self-calibration algorithm, to compensate for environmental interference. The combination method typically involves mixing to generate an intermediate frequency signal, which is then converted into range-Doppler maps or point cloud data using a Fast Fourier Transform (FFT) to form the aforementioned millimeter-wave dataset. The aforementioned identification model is used to extract features from the complex millimeter-wave data and map them to user data. For example, Convolutional Neural Networks (CNNs) can be used to extract spatial features from distance-Doppler maps to identify posture; Long Short-Term Memory Networks (LSTMs) can be used to process time-series data, capturing the temporal dependencies of user actions to identify trajectories or micro-motion signals; Recurrent Neural Networks (RNNs) can also be used for sequence data processing. A large amount of historical millimeter-wave datasets collected in simulated or actual bathroom environments are gathered, and corresponding user tags are labeled by professionals or using high-precision contact sensors. These user tags include real location coordinates, real movement trajectory samples, and real heart rate values. The historical millimeter-wave datasets are paired with the user tags to form a training set, which is then input into the aforementioned recognition model for supervised learning training. The model parameters are optimized until the model converges, resulting in the aforementioned physiological recognition model capable of accurately predicting user data. In practical applications, the real-time collected millimeter-wave dataset is input into the trained physiological recognition model. The model, through feature extraction and data inference, outputs the aforementioned user data, including the aforementioned location, movement trajectory, and heart rate.

[0056] During monitoring, the radar first continuously emits millimeter-wave signals. These electromagnetic waves can penetrate the water vapor commonly found in bathrooms and reach the human body. The radar receiver then captures the echo signals reflected from the body. By analyzing signal delay, frequency changes, and phase changes, the user's spatial location, movement speed, and body contours can be determined. For micro-motion signals such as heartbeats and respiration, the chest cavity's movement causes periodic, minute changes in the echo phase. Using signal processing algorithms (such as Fourier transform and phase demodulation), the frequency components corresponding to heartbeats and respiration can be separated from the echo, thus extracting features such as heart rate and respiratory rate.

[0057] The millimeter-wave radar preferably operates in the 60GHz or 77GHz millimeter-wave band. The electromagnetic waves in this band have short wavelengths, strong directionality, and penetrability, effectively penetrating common bathroom environments such as hot water steam, fog, and glass partitions. This overcomes the failure problems of traditional optical sensors (such as cameras) in humid environments due to lens fogging or insufficient light, and the susceptibility of infrared sensors to background thermal radiation interference in high-temperature and high-humidity environments. The radar transmitter continuously emits frequency-modulated continuous waves or pulse signals. These signals propagate through the bathroom space and reflect off the user's skin and internal organs. The receiver captures these weak echo signals. The received raw radio frequency signal is preprocessed to extract echo data containing three key types of information. First, spatial location and macroscopic motion information: the distance between the user and the radar is calculated by measuring the time of flight (ToF) of the echo signal; the user's movement speed is analyzed by measuring signal frequency changes using the Doppler effect; and the user's coordinate position, movement trajectory, and body contours (e.g., standing, sitting, or falling posture) in three-dimensional space are analyzed using direction-of-arrival estimation or MIMO array technology. Second, microscopic physiological characteristics: the human heartbeat and lung respiration cause periodic micro- to millimeter-level fluctuations in the chest cavity, which lead to periodic micro-changes in the phase of the reflected echo. High-precision phase demodulation technology is used to extract the phase change sequence, and signal processing algorithms such as Fast Fourier Transform (FFT) or Short-Time Fourier Transform (STFT) are used to convert the time-domain phase change sequence into a frequency-domain signal, thereby separating the significant frequency components corresponding to heart rate and respiratory rate. Through peak detection or spectral center frequency calculation, the user's real-time heart rate and respiratory rate, and other physiological characteristics, can be accurately extracted. The above process enables non-contact, all-weather, and high-precision vital sign monitoring, providing key physiological data for subsequent emotion computing models.

[0058] Millimeter-wave radar can sense subtle human movements remotely, its core strength lying in the precise manipulation and analysis of millimeter-wave signals. The entire process can be broken down into the following key steps:

[0059] First, signal transmission and acquisition: The radar first transmits millimeter waves of a specific frequency band through its antenna. These electromagnetic waves can penetrate water vapor and mist in the bathroom, and after shining on the human body, they are reflected, with some of the energy returning to the radar's receiving antenna.

[0060] Second, basic motion information analysis: Radar can accurately determine the target's distance by calculating the time difference between electromagnetic wave transmission and reception. By analyzing the frequency changes of continuous wave signals caused by the Doppler effect, the target's radial velocity can be directly measured. Using an antenna array composed of multiple antennas, the target's azimuth angle can be calculated by analyzing the wave arrival phase difference.

[0061] Third, micro-motion signal detection: This is crucial for monitoring vital signs such as heartbeat and respiration. Physiological activities like heartbeat and respiration cause periodic, minute fluctuations in the chest cavity. While these fluctuations are imperceptible to the naked eye, they alter the phase of the reflected echo. Radar, through extremely sensitive phase demodulation technology, can capture these nanometer- or even micrometer-level displacement changes. Subsequently, using spectral analysis tools such as Fourier transform, the time-domain signal is converted into a frequency-domain signal, thereby separating the low-frequency and high-frequency components corresponding to respiration and heartbeat, ultimately extracting the respiratory rate and heart rate.

[0062] During monitoring, in addition to extracting micro-motion signals to obtain physiological characteristics such as heart rate and respiratory rate, the aforementioned millimeter-wave radar also analyzes signal delay, frequency changes, and phase changes to determine the user's spatial position, movement speed, and body contours, among other basic motion information. Specifically, using the principle of frequency-modulated continuous wave radar, the distance resolution between the user and the radar is calculated by measuring the time difference between the transmitted signal and the received echo signal; the radial movement speed of the user relative to the radar is analyzed by analyzing the Doppler frequency shift of the echo signal; and the energy distribution of the user at different angles is analyzed by combining the phase difference or amplitude distribution of the received signal from the receiving antenna array with beamforming or MUSIC algorithms, thereby determining the user's horizontal azimuth and vertical elevation angles, and constructing the user's three-dimensional spatial position coordinates. Based on the continuously acquired spatial position coordinate sequence, trajectory tracking algorithms such as Kalman filtering or particle filtering are used to smooth noise and reconstruct the user's movement trajectory. Simultaneously, based on radar micro-Doppler features or point cloud clustering results, the user's body contours are analyzed, identifying the user into a specific posture category, such as standing, sitting, squatting, fallen, or lying down. For example, when a user's height suddenly decreases and their center of gravity changes drastically, or when the spatial distribution of key points on the human body matches the characteristics of a fall, it is determined that the user has fallen. The analysis of the above basic motion information provides an intuitive kinematic basis for judging the user's behavioral state.

[0063] During filtering, the water vapor interference self-calibration algorithm first senses the water vapor concentration and distribution characteristics in the current environment through radar signals or auxiliary sensors (such as temperature and humidity sensors). Utilizing a water vapor attenuation and scattering model of millimeter waves, the radar echo signal is compensated or filtered to reduce signal distortion caused by humidity. Adaptive filtering or machine learning methods can be employed here, dynamically subtracting interference components caused by water vapor by comparing environmental echoes without targets with real-time echoes, thereby improving the accuracy of the monitoring data.

[0064] The execution process of the aforementioned water vapor interference self-calibration algorithm includes four stages: environmental perception, model building, signal compensation, and dynamic correction. First, in the environmental perception stage, independent temperature and humidity sensors deployed in the bathroom monitor air temperature and relative humidity in real time, and estimate the current water vapor concentration gradient by combining this with the background noise power spectral density received from the radar. Second, in the model building stage, a physical model of water vapor attenuation and scattering of millimeter waves is constructed based on electromagnetic wave propagation theory. This model treats water vapor as a medium with a specific complex permittivity, calculating the insertion loss and phase shift of the water vapor layer on the transmitted and echo signals. Next, in the signal compensation stage, the original radar echo signal is preprocessed using the aforementioned model parameters. If an adaptive filtering method is used, the environmental echo when there is no target is used as a reference, i.e., radar data when the bathroom is empty or has only a static background of people, as a noise reference. The filter coefficients are adjusted using the least mean square error or recursive least squares algorithm to estimate and subtract the broadband noise component caused by water vapor scattering in real time. If machine learning methods are used, a deep neural network can be pre-trained. The input includes raw IQ data containing water vapor interference and ambient temperature and humidity data. The output is either a denoised, clean physiological signal or compensated amplitude / phase data. Training with a large amount of labeled data under different humidity conditions allows the network to learn the non-linear characteristics of water vapor interference, thus more accurately removing the interference. Finally, in the dynamic correction phase, the ambient echo and real-time echo are continuously compared when there is no target. When the bathroom door is detected to be closed and no one is entering, a clean ambient echo is collected as a reference template. When the user enters, the difference between the real-time echo and the reference template is calculated in real time. The portion of the difference that matches known water vapor interference characteristics is considered noise and subtracted, retaining only the effective signal generated by human body reflection. This dynamic subtraction mechanism can effectively handle situations where water vapor concentration increases instantaneously or accumulates locally during showering, ensuring that the extraction of heart rate, respiration, and body posture data is not affected by drastic fluctuations in ambient humidity.

[0065] The core idea is to identify and separate the interference components in the signal caused by water vapor. Specifically, this includes:

[0066] First, interference modeling and adaptive filtering can be used to collect a pure "water vapor interference signal" as a reference, either in advance or in a static environment without a target. During real-time monitoring, the algorithm dynamically compares the received mixed signal with this reference signal and continuously adjusts the filter parameters using criteria such as minimum mean square error, thereby "subtracting" the components related to water vapor interference from the mixed signal.

[0067] Secondly, advanced denoising based on machine learning can be used for more complex situations. For example, a large amount of radar echo data under different humidity conditions can be collected and labeled with "clean" target signals (such as the corresponding data collected in a dry environment). Then, a neural network model (such as a U-Net structure) can be trained to learn the mapping relationship from "signals with water vapor noise" to "clean signals". Once the model is trained, it can directly denoise the real-time acquired signals, demonstrating strong environmental adaptability.

[0068] Adjusting filter parameters is a process of continuous trial and error, with the goal of achieving optimal filtering results. The specific steps are as follows: First, guess a value and set an initial parameter for the filter (equivalent to roughly setting a value). Calculate the error. Process the signal with the current parameters to obtain a "denoised signal" and calculate the error between it and the desired signal. Fine-tune in the direction of reducing the error. The core principle is that if the error is large, adjust more; if the error is small, adjust less. The algorithm will automatically calculate a fine-tuning amount based on the current error magnitude and the input signal, and update the parameters accordingly.

[0069] Third, the process of "calculating error -> fine-tuning parameters" is continuously repeated every time a new data point is received. Through massive amounts of continuous fine-tuning, the filter parameters gradually stabilize to a set of optimal values, thus filtering out water vapor interference most accurately. Simply put, it's like an automatic thermostatic faucet that continuously fine-tunes the mixing ratio of hot and cold water (parameters) based on the difference between the current water temperature (error) and the set water temperature (target) until water of the perfect temperature flows out.

[0070] In some embodiments, the aforementioned risks include the risk of falling, the aforementioned physiological data includes heart rate, and determining whether the user is at risk based on the aforementioned user data includes: determining that the user is at risk of falling when the user's posture changes abruptly and the rate of change of the aforementioned heart rate is greater than a preset heart rate change threshold, wherein the posture change is the user's posture changing from standing to sitting or lying down.

[0071] This solution detects the risk of a fall by comprehensively judging the user's heart rate variability and body posture. When a sudden change in posture occurs (from standing to sitting or lying down) and the heart rate variability exceeds a preset threshold, it indicates that the change is not an active behavior but a fall event, thus confirming a fall risk. This multimodal fusion mechanism effectively reduces the false alarm rate, improves the reliability of fall detection, and ensures the accuracy of warnings in high-risk scenarios such as bathrooms.

[0072] Specifically, the user data is monitored in real time, and risk identification is performed through a logical judgment module. For the risk of immobility, the user's position coordinates are first tracked. If the user's position coordinates within the preset monitoring area remain unchanged for multiple consecutive time frames, and the movement trajectory shows no displacement change, a timer is started. When the immobility time exceeds the preset duration threshold, combined with the characteristic of no change in the movement trajectory, it is determined that the user poses the risk of immobility. This situation may correspond to the user fainting, coma, or being unable to move due to excessive fatigue. For the risk of falling, the rate of change of spatial coordinates in the movement trajectory is monitored. When a sharp drop in the user's center of gravity height is detected within a short time window (e.g., 3 seconds), and a drastic change in body tilt angle (e.g., from 90 degrees to 180 degrees), a preliminary alarm is triggered. At this time, the heart rate data is further examined. The rate of change of heart rate within the short time window before and after the sudden change in posture is calculated. If the rate of change of heart rate is greater than the preset heart rate change rate threshold, it is determined that the user poses the risk of falling.

[0073] In some embodiments, the aforementioned risks include the risk of being stationary. Determining whether a user poses a risk based on the aforementioned user data includes: determining that the user poses the risk of being stationary if the user remains in the same location for a period of time exceeding a preset time threshold and the user's movement trajectory remains unchanged.

[0074] This solution effectively identifies stationary risks by monitoring the duration of a user's time in the same location and changes in their movement trajectory. This detection mechanism, based on both time and space dimensions, improves the reliability of stationary risk detection and ensures the accuracy of early warnings in high-risk scenarios such as bathrooms.

[0075] The preset duration threshold can be differentiated according to the specific functional areas within the bathroom space to balance the timeliness of safety warnings with the tolerance for errors in normal behavior. Specifically, in the shower area, considering that users may briefly stop moving due to washing their body or applying toiletries, but abnormal situations such as slipping or unconsciousness usually lead to prolonged stillness, the preset duration threshold is preferably set to 60 to 90 seconds, for example, 60 seconds, so as to trigger an early warning in time after a user slips. In the toilet area, given that toileting itself involves a relatively long period of stillness and may be accompanied by normal behaviors such as reading, the preset duration threshold is preferably set to 120 to 180 seconds, for example, 150 seconds, to avoid false alarms for normal toileting behavior. In addition, this threshold can be dynamically adjusted based on the user's historical behavior data, or a tiered warning mechanism can be adopted, that is, a gentle voice inquiry is made when the short-term threshold is reached, and when the preset duration threshold is reached, it is confirmed as a risk of stillness and an emergency warning procedure is executed, thereby achieving accurate monitoring of the user's status in different bathroom scenarios.

[0076] The "unchanged movement trajectory" mentioned in this scheme refers to the user's centroid coordinate fluctuation range in three-dimensional space being less than a preset positional drift threshold within the monitoring period, and the user not undergoing significant displacement from one area to another. For example, if the user remains within the same 0.5-meter radius of the shower area, and the maximum offset of their centroid coordinates in the X, Y, and Z directions is less than 10 centimeters, then the user's movement trajectory at that location is determined to have remained unchanged.

[0077] In one optional scheme, the preset duration threshold is set to 60 seconds, and the preset heart rate change threshold is set to 30 BPM (beats / minute).

[0078] In the specific implementation process, the above-mentioned prompt information includes a first prompt information and / or a second prompt information. The warning based on the above-mentioned prompt information includes at least one of the following: when it is determined that the above-mentioned user is at risk of remaining stationary, the smart speaker is controlled to play music based on the above-mentioned first prompt information, wherein the above-mentioned first prompt information is information to prompt the above-mentioned user through music; the lighting device is controlled to turn on the light mode based on the above-mentioned second prompt information, wherein the above-mentioned second prompt information is information to prompt the above-mentioned user through light.

[0079] In this solution, music has a psychological calming effect, while lighting attracts the user's attention without causing visual stimulation. This multi-sensory, low-intensity reminder method enhances the user experience in the bathroom setting.

[0080] Specifically, soothing music can be played. The rhythm range of soothing music is close to the user's average heart rate, the harmonic structure of soothing music is preferably in a major key or natural minor key, and the spectral energy distribution of soothing music is preferably concentrated in the mid-low frequency range of 200Hz-2000Hz, which can soothe the user's emotions.

[0081] Specifically, a soft lighting mode can be activated. The color temperature of the soft lighting mode is preferably between 2700K and 3000K, which is a warm-toned light. This color temperature range is rich in red light components, which can suppress the inhibitory effect of melatonin secretion. For example, the brightness of the soft lighting mode is preferably less than 50% of the ambient background brightness. The light distribution of the soft lighting mode is preferably uniform omnidirectional lighting or soft localized focused lighting, with a smooth gradient of light intensity and no flicker to ensure visual comfort.

[0082] When the gateway or cloud server determines that the user poses a risk of remaining stationary, it first classifies the risk level as "moderate risk" or "mild risk," meaning the user may be in a state of confusion but still showing weak responsiveness, or in a state of low mood or imbalance. At this time, a non-intrusive, low-intensity warning interaction is executed. On one hand, a command is sent to the smart speaker connected to the smart gateway, causing the smart speaker to play the preset first warning message, i.e., soothing music. The audio volume can be set to a gradually increasing mode, for example, an initial volume of 30 decibels, increasing by 5 decibels every 10 seconds until reaching 45 decibels, to gently wake or soothe the user. On the other hand, commands can be sent simultaneously or sequentially to the lighting devices in the bathroom, controlling them to switch to the soft lighting mode corresponding to the second warning message. For example, adjusting the originally bright cool white light to warm yellow light, or implementing a breathing light effect. The first and second warning messages can be executed independently or in combination. If no response from the user to the music or light is detected, the warning strategy can be upgraded to trigger a higher-intensity alarm or contact emergency contacts.

[0083] In some embodiments, the aforementioned user data also includes facial spectral data, and the aforementioned prompt information includes third prompt information. When it is determined that the user poses the aforementioned risk, the prompt information is generated, including: calculating the weighted average of the aforementioned biochemical data, the aforementioned physiological data, and the aforementioned facial spectral data to obtain a comprehensive score; if the aforementioned comprehensive score is greater than a preset score threshold, it is determined that the user poses the aforementioned risk, and the aforementioned third prompt information is generated, wherein the aforementioned third prompt information indicates that the aforementioned biochemical data, the aforementioned physiological data, and the aforementioned facial spectral data of the user are abnormal.

[0084] This scheme combines biochemical, physiological, and facial spectral data for multimodal assessment. A weighted average composite score quantifies the joint risk contribution of the multi-source data. When the composite score exceeds a preset threshold, it indicates consistency or complementarity among various anomalous signals, improving analytical accuracy and reducing false alarm rates.

[0085] Specifically, user data also includes user spectral data, collected using a smart mirror. The smart mirror has a built-in multispectral camera or near-infrared spectral sensor to collect the user's facial spectral data. This facial spectral data primarily reflects the user's facial micro-expression features, skin blood perfusion, and the degree of paleness or flushing. First, the user data is acquired, including heart rate, respiratory rate, and posture data extracted from millimeter-wave radar; biochemical data, such as cortisol and glucose levels in urine; and the facial spectral data. To eliminate the influence of data with different dimensions, the three types of data are first normalized, converting them into standardized values ​​between 0-100 or 0-1. Then, a weighted average is calculated as a comprehensive score based on the reliability and weight of each data source. For example, if the user is in a stationary risk situation, the weight of biochemical data may be higher, set to 0.4; the weight of facial spectral data is set to 0.3; and the weight of radar data is set to 0.3. When the comprehensive score exceeds a preset threshold, the user is deemed to be at serious risk, and the aforementioned fourth warning message is generated. The aforementioned fourth prompt can be displayed on the smart mirror screen with details of the specific abnormal indicators, or a high-risk warning can be broadcast via voice, or even directly linked to an emergency call.

[0086] Facial spectral data refers to the non-contact acquisition of user facial images and derived physiological and emotional characteristics data through multispectral sensors built into smart mirrors, such as visible light and near-infrared light cameras. Specifically, this data includes not only parameters reflecting changes in facial color, but also blood perfusion indicators such as heart rate, heart rate variability (HRV), and blood oxygen saturation extracted through remote photoplethysmography (rPPg) technology, as well as micro-expression features (such as frowning and drooping corners of the mouth) and eye features (such as pupil diameter and blinking frequency) derived from facial action unit (AU) analysis.

[0087] like Figure 3 As shown, the system begins operation once the user enters the bathroom. Millimeter-wave radar continuously emits signals, utilizing its ability to penetrate water vapor in the bathroom to non-contactly monitor the user's movement trajectory, posture, and micro-motion signals (such as chest rise and fall caused by a heartbeat). Simultaneously, while the user is using the smart toilet, biomarkers such as cortisol in the urine can be detected. A smart mirror then captures the user's facial spectral data. This data can be transmitted in real-time to a cloud server via the network. If the user prioritizes privacy and security, the data can be transmitted only to a local smart gateway for processing.

[0088] Whether using a cloud server or a local gateway, the processing logic is as follows: First, the radar signal is preprocessed, such as by using a water vapor interference self-calibration algorithm to effectively filter out moisture interference and ensure data accuracy. Then, the emotion calculation model is invoked to fuse and analyze heart rate features extracted from the radar, such as HRV, cortisol levels detected by the toilet, and behavioral data (such as prolonged periods of stillness), to calculate the current user's emotional state and risk level.

[0089] The calculation process begins with normalization and feature extraction of data from various sensors. For example, time-domain / frequency-domain features of heart rate variability (HRV) are calculated from radar signals; quantitative values ​​of biochemical indicators such as cortisol are obtained from smart toilets; and resting duration and bradykinesia are extracted from behavioral data. These multimodal features are then input into a pre-trained emotion calculation model (which can be based on machine learning or deep learning models). The model then fuses and calculates the contribution weights of different features to the emotional state. Finally, combined with expert rules or risk thresholds (such as elevated cortisol, decreased HRV, and simultaneous occurrence of behavioral abnormalities), the model outputs an assessment of the user's current emotional state and the corresponding risk level (e.g., low, medium, high).

[0090] The core task of emotion computing models is to integrate data from different sources and of different natures into a unified, quantifiable risk indicator.

[0091] This solution involves feature extraction and standardization. First, the model needs to preprocess the raw data provided by each sensor, transforming it into digital features that can be compared and calculated. For example, extracting time-domain and frequency-domain features of heart rate variability from radar signals; obtaining quantified values ​​of cortisol concentration from smart toilets; and statistically analyzing resting duration and calculating the amplitude of movements from behavioral data.

[0092] This approach involves cross-modal feature fusion, whereby the model needs to combine these features from different dimensions. A simple method is to use a weighted summation based on expert knowledge, assigning importance weights to different features. A more advanced approach is to use a deep learning model (such as the Transformer) for fusion. The model automatically learns the complex relationships between different features through an attention mechanism. For example, it might discover that the simultaneous presence of the features "elevated cortisol levels" and "significantly decreased HRV" is a strong indicator of an "anxiety" state.

[0093] This solution involves risk classification and decision-making. Finally, the model will output the final risk level based on the comprehensive score calculated through fusion and the preset threshold rules.

[0094] The specific integration process is as follows:

[0095] 1. Standardize Scoring. Convert all sensor data into a uniform "anomaly score" (between 0 and 1). Radar detects high stress → stress score 0.8. Toilet flush detects high cortisol → biochemical score 0.9. Behavioral monitoring detects slowed movement → behavioral score 0.7.

[0096] Second, calculate the total score. Assign different weights to different scores based on their importance, and then calculate a comprehensive risk score. For example, the formula is: Comprehensive Risk Score = (0.8 × 30%) + (0.9 × 50%) + (0.7 × 20%) = 0.833.

[0097] Third, act according to the score. Finally, use this total score to "look up the table" and trigger the corresponding measures.

[0098] Low risk (risk score < 0.3): Record only, do not disturb.

[0099] Medium risk (0.3-0.7): Play music to relax.

[0100] High risk (risk score ≥ 0.7, e.g., 0.83): Immediately activate the alarm (first query locally; if there is no response, then call emergency contacts). In summary, the system converts various data into a comprehensive risk score, and then automatically decides whether to play music or call for help based on this score.

[0101] In some embodiments, the above-mentioned prompt information includes a fourth prompt information, which is generated when it is determined that the user has the above-mentioned risk. The fourth prompt information is generated when the above-mentioned biochemical data is not within a preset normal biochemical range and the user has the above-mentioned risk. The fourth prompt information is a message indicating that the user has the above-mentioned risk and that the above-mentioned biochemical data is abnormal.

[0102] In this solution, by monitoring biochemical indicators in urine, it is possible to determine whether there is a risk by combining user data. This allows for multimodal monitoring, which can more accurately determine whether to generate a fourth alert message.

[0103] Specifically, the biochemical data can be collected by the smart toilet. The biochemical data mentioned in this application refers to the biochemical indicators in the user's excrement (mainly urine) collected by the built-in sensors of the smart toilet. The biochemical data includes, but is not limited to, cortisol concentration, glucose concentration and ketone body concentration.

[0104] In some embodiments, the aforementioned prompt information includes one or more of a fifth, sixth, and seventh prompt information. The warning based on the aforementioned prompt information includes one of the following: Based on the fifth prompt information, controlling the smart mirror to flash an alarm light, wherein the fifth prompt information is information used by the smart mirror to prompt the user; based on the sixth prompt information, controlling the smart speaker to emit an inquiry voice, wherein the inquiry voice is a voice asking the user if they need help, and the sixth prompt information is information used to prompt the user via voice; if no reply voice is received, or if the reply voice indicates that the user needs help, sending the seventh prompt information to a target device, wherein the target device is the device of the user's emergency contact, and the seventh prompt information is information used to prompt the emergency contact via SMS or telephone.

[0105] This solution uses visual alerts from a smart mirror to provide prompts to users without infringing on their privacy or disturbing their families. It can also incorporate auditory prompts to further confirm the user's state of consciousness. External communication is only triggered when the user is unresponsive or explicitly requests assistance. This tiered and progressive alert system improves the response efficiency of health monitoring in bathroom settings.

[0106] When a user is determined to pose the aforementioned risk, the warning operation is executed step by step according to the preset escalation strategy. First, the fifth prompt message is generated, and the smart mirror executes this prompt message by having the LED light strip on the edge of the mirror flash red at a high frequency, while the mirror screen displays a conspicuous "Emergency Help" message or warning icon to attract the user's attention in the bathroom, achieving the first level of local visual warning. If the user does not touch or voice respond to the mirror within the preset first response time threshold, or if facial spectral data detects that the user is unconscious, the sixth prompt message is generated, and the smart speaker deployed in the bathroom is controlled to execute this prompt message, playing the aforementioned inquiry voice, for example, "We have detected that you have not been active for a long time. Do you need help?", with the volume set to 80 decibels to ensure that the sound can cover the ambient noise in the bathroom. If no response voice is received from the user within the preset second response time threshold, or if the speech recognition technology identifies keywords in the response voice, such as "fell down," indicating that the user needs help, it is determined that the user is in a state of inability to cope independently, and the seventh prompt message is generated. The aforementioned seventh alert is sent to the target device via the communication module. This seventh alert may take the form of an SMS notification, containing the user's location, risk type, and a recent snapshot of vital signs data, or it may automatically dial a preset emergency contact number and play a pre-recorded distress audio message. This process forms a multi-level, interconnected early warning mechanism, progressing from mild to severe and from internal to external.

[0107] Inquiry voice prompts are voice prompts played via devices such as smart speakers or smart mirrors when a user is detected to be at risk. These prompts aim to confirm the user's state of consciousness and needs, and guide their response. Specifically, inquiry voice prompts use different tones and content depending on the risk level. For medium risk, a calm and concerned tone is used to ask, "We've detected that you've been still for more than 2 minutes. Are you alright?" to rule out the possibility of normal user activity. For high risk, an urgent and firm tone is used to prompt, "Alert! We've detected that you may have fallen. Please answer 'Need help' or 'No'," to quickly confirm the emergency situation. In addition, specific guiding voice prompts can be generated by combining multimodal data, such as "We've detected that you're pale. Do you need help? Please say 'yes' or 'no'," to help users accurately express their symptoms. These voice prompts all include status information, direct questions, and operational guidance, ensuring effective feedback is obtained even when the user is weak or panicked, thus determining whether to continue monitoring or trigger emergency contact mechanisms.

[0108] For example, when the system detects slowed user movements, abnormal HRV data, and elevated cortisol levels, it determines that there is a risk of state imbalance. Next, it triggers corresponding alerts based on preset risk levels. For instance, for moderate risk, the gateway will control the smart speaker to play soothing music; for high risk, it will send an alert notification to family members' phones via the app. If extreme risk is detected, such as a user suddenly falling or remaining motionless in the shower area for an extended period, it will be immediately identified as a high-risk event.

[0109] Of course, local alerts will be prioritized, such as flashing warning lights on the smart mirror and asking "Do you need help?". If no response is received, emergency procedures will be automatically executed, such as directly calling pre-set emergency contacts. If data from multiple sensors are inconsistent, for example, radar detects a slight tremor but the toilet does not detect abnormal cortisol levels, a cross-validation mechanism will be initiated. It will prioritize data sources with higher confidence (such as biochemical indicators) and mark the event as "requiring continued observation" instead of immediately triggering a high-level alert, thus avoiding false alarms and waiting for more definitive data for a comprehensive judgment.

[0110] Specifically, multiple users may be present in the bathroom at the same time (such as family members taking turns or using it simultaneously). To ensure the accuracy of the information, it is necessary to verify whether the user using the toilet and the user using the shower are the same person, which includes voiceprint recognition and body posture recognition.

[0111] The method further includes identifying whether the user who used the toilet and the user who used the shower are the same person based on the user's historical data. This involves extracting the first body posture feature collected by the millimeter-wave radar and the first voiceprint feature collected by the smart speaker from the user who used the toilet; extracting the second body posture feature collected by the millimeter-wave radar and the second voiceprint feature collected by the smart speaker from the user who used the shower; calculating the first similarity between the first and second voiceprint features, and calculating the second similarity between the first and second body posture features; and determining that the user who used the toilet and the user who used the shower are the same person if the first similarity is greater than a first preset similarity threshold and the second similarity is greater than a second preset similarity threshold.

[0112] When the smart toilet is detected to be in use, this time period is designated as the "toilet use period." At this time, the user's first physical characteristics are extracted from millimeter-wave radar data, including standing height, torso tilt angle, sitting duration, and micro-movement patterns. Simultaneously, if the user interacts with the smart speaker via voice during toilet use (e.g., checking the weather or playing music), their first voiceprint characteristics are extracted. When the shower head is detected to be turned on or the radar detects activity in the shower area, this time period is designated as the "shower period." The user's second physical characteristics are extracted from millimeter-wave radar data, including standing posture, body micro-disturbance patterns caused by water flow, and estimated height. Simultaneously, the user's second voiceprint characteristics generated during showering are extracted, such as humming, voice clips against a water background, or breathing sounds.

[0113] Using a pre-trained voiceprint recognition model, the first similarity between the first and second voiceprint features is calculated. Since the shower environment is noisy, voiceprint extraction may be affected by water noise; therefore, speech segments with a signal-to-noise ratio higher than a preset threshold (e.g., 15dB) are preferentially selected for analysis. If there are insufficient speech segments, the breathing rhythm features during showering can be used as an auxiliary voiceprint comparison. Simultaneously, using a body posture recognition model, the second similarity between the first and second body posture features is calculated. Body posture features mainly focus on the geometric attributes of the human body (such as height and shoulder width ratio) and kinematic dynamics. For example, comparing whether the standing height while using the toilet is consistent with the standing height while showering, and comparing whether the gait frequency and center of gravity movement patterns of the two are highly consistent.

[0114] The first similarity is compared with the first preset similarity threshold (e.g., 0.85), and the second similarity is compared with the second preset similarity threshold (e.g., 0.80). Only when both the first similarity and the second similarity are greater than the first preset similarity threshold are the user using the toilet and the user using the shower considered to be the same person. If either similarity is below the corresponding threshold, they are considered different users. For example, if the voiceprint similarity is high but the body shape similarity is low (e.g., a large difference in height), they are considered different people; if the body shape similarity is high but the voiceprint similarity is low (e.g., a hoarse voice due to a cold), a comprehensive judgment can be made based on a preset strategy, combined with other historical data (e.g., multiple interaction records within the past week), or a higher body shape similarity threshold can be required as compensation.

[0115] Of course, data from different time periods can also be collected to make judgments, such as data within a day or data within a week, to extract user data, biochemical data and facial spectral data of the same person.

[0116] This solution seamlessly monitors a user's behavior and physiological indicators using various smart devices deployed in the bathroom, accurately identifying health risks and providing timely warnings. These include millimeter-wave radar deployed on the bathroom ceiling, a smart toilet with urine detection capabilities, a smart mirror with built-in sensors, and a smart gateway or cloud service for local computation. By employing millimeter-wave radar with a water vapor self-calibration algorithm for seamless sensing, combined with an emotion computing model that fuses multi-source physiological data and local / cloud-based decision-making methods, it solves the problem of achieving accurate and safe health warnings in the high-humidity, high-privacy bathroom environment. It helps users understand their health status, detects emotional changes, and provides precise health warnings, enabling collaborative warning management across devices.

[0117] This application also provides a monitoring device. It should be noted that the monitoring device of this application can be used to execute the monitoring method provided in this application. This device is used to implement the above embodiments and preferred embodiments; details already described will not be repeated. As used below, the term "module" can refer to a combination of software and / or hardware that implements a predetermined function. Although the device described in the following embodiments is preferably implemented in software, hardware implementation, or a combination of software and hardware, is also possible and contemplated.

[0118] The monitoring device provided in the embodiments of this application will be described below.

[0119] Figure 4 This is a structural block diagram of a monitoring device according to an embodiment of this application. Figure 4 As shown, the device includes:

[0120] The first acquisition unit 10 is used to acquire user data collected from users, wherein the user data includes one or more of the following: location, movement trajectory, body shape, biochemical data and physiological data.

[0121] The determining unit 20 is used to determine whether the user poses a risk based on the aforementioned user data.

[0122] The early warning unit 30 is used to generate a prompt message and issue an early warning based on the prompt message when it is determined that the user has the aforementioned risk.

[0123] In this embodiment, the millimeter-wave radar signal is not affected by water vapor or other interferences. This eliminates the strong absorption interference of water vapor on the sensor signal at the hardware sensing level, ensuring signal integrity in the high-humidity environment of the bathroom. This reduces signal loss or erroneous triggering caused by environmental interference, fundamentally lowering the false alarm rate. The millimeter-wave radar can accurately capture millimeter-level chest micro-movements (for heart rate extraction) and centimeter-level body displacements (for trajectory extraction). By analyzing this high-precision data, it can accurately identify "falls" (drastic changes in posture or position) and "immobility" (no trajectory change for a long time), solving the monitoring failure problem caused by environmental interference in existing technologies and improving the safety monitoring efficiency of bathroom spaces.

[0124] In the specific implementation process, the aforementioned user data is collected based on millimeter-wave radar. The first acquisition unit includes a first acquisition module, a second acquisition module, a combination module, and a processing module. The first acquisition module is used to acquire the signal emitted by the millimeter-wave radar to obtain the transmitted signal; the second acquisition module is used to acquire the signal reflected by the user's human body to obtain the echo signal; the combination module is used to combine the transmitted signal and the echo signal to obtain a millimeter-wave dataset; and the processing module is used to input the millimeter-wave dataset into the physiological recognition model to obtain the aforementioned user data corresponding to the millimeter-wave dataset.

[0125] In this solution, traditional optical or infrared sensors are easily obscured by water mist or interfered with by background thermal radiation in the high humidity environment of the bathroom, resulting in data distortion. The millimeter-wave radar in this solution has the physical properties of penetrating water vapor, and the data obtained is of high quality, ensuring good data accuracy. Moreover, it does not collect user images, thus achieving privacy and security monitoring.

[0126] In the specific implementation process, the above-mentioned device further includes a second acquisition unit and a training unit. The second acquisition unit is used to acquire a recognition model before inputting the above-mentioned millimeter-wave dataset into the physiological recognition model to obtain the above-mentioned user data corresponding to the above-mentioned millimeter-wave dataset. The recognition model is one of a CNN model, an RNN model, and an LSTM model. The training unit is used to form a training set by combining the historical millimeter-wave dataset and the corresponding user labels, and to train the above-mentioned recognition model using the above-mentioned training set to obtain the above-mentioned physiological recognition model. The above-mentioned user labels are the historical user data corresponding to the above-mentioned historical millimeter-wave dataset in the above-mentioned training set.

[0127] In this scheme, the bathroom environment is highly variable (e.g., different user body sizes, different water vapor concentrations, different background noise). By constructing the training set containing historical millimeter wave datasets and user labels, deep learning training is performed using recognition models such as LSTM or CNN. This enables the model to automatically learn the nonlinear mapping relationship between millimeter wave signals and user data. Through the training process, the physiological recognition model can adapt to the specificity of the bathroom environment, thus enabling it to output user data more accurately when millimeter wave datasets are input.

[0128] In some embodiments, the aforementioned risks include the risk of falling, the aforementioned physiological data includes heart rate, and the determining unit includes a first determining module. The first determining module is used to determine that the user has the aforementioned risk of falling when the user's posture changes abruptly and the rate of change of the aforementioned heart rate is greater than a preset heart rate change threshold. The abrupt change of posture is the user's posture changing from standing to sitting or lying down.

[0129] This solution detects the risk of a fall by comprehensively judging the user's heart rate variability and body posture. When a sudden change in posture occurs (from standing to sitting or lying down) and the heart rate variability exceeds a preset threshold, it indicates that the change is not an active behavior but a fall event, thus confirming a fall risk. This multimodal fusion mechanism effectively reduces the false alarm rate, improves the reliability of fall detection, and ensures the accuracy of warnings in high-risk scenarios such as bathrooms.

[0130] In some embodiments, the aforementioned risks include the risk of being stationary. The determining unit includes a second determining module, which is used to determine that the user has the aforementioned risk of being stationary if the user has been in the same location for a period of time longer than a preset time threshold and the movement trajectory representation has not changed.

[0131] This solution effectively identifies stationary risks by monitoring the duration of a user's time in the same location and changes in their movement trajectory. This detection mechanism, based on both time and space dimensions, improves the reliability of stationary risk detection and ensures the accuracy of early warnings in high-risk scenarios such as bathrooms.

[0132] In the specific implementation process, the above-mentioned prompt information includes a first prompt information and / or a second prompt information. The early warning unit includes a first control module and a second control module. The first control module is used to control the smart speaker to play music according to the first prompt information when it is determined that the user is at risk of remaining stationary. The first prompt information is a prompt to the user using music. The second control module is used to control the lighting equipment to turn on the light mode according to the second prompt information. The second prompt information is a prompt to the user using light.

[0133] In this solution, music has a psychological calming effect, while lighting attracts the user's attention without causing visual stimulation. This multi-sensory, low-intensity reminder method enhances the user experience in the bathroom setting.

[0134] In some embodiments, the user data also includes facial spectral data, the prompt information includes a third prompt information, and the warning unit includes a calculation module and a first warning module. The calculation module is used to calculate the weighted average of the biochemical data, the physiological data, and the facial spectral data to obtain a comprehensive score. The first warning module is used to determine that the user has the aforementioned risk if the comprehensive score is greater than a preset score threshold, and generate the third prompt information, wherein the third prompt information is information indicating that the user's biochemical data, physiological data, and facial spectral data are abnormal.

[0135] This scheme combines biochemical, physiological, and facial spectral data for multimodal assessment. A weighted average composite score quantifies the joint risk contribution of the multi-source data. When the composite score exceeds a preset threshold, it indicates consistency or complementarity among various anomalous signals, improving analytical accuracy and reducing false alarm rates.

[0136] In some embodiments, the above-mentioned prompt information includes a fourth prompt information, and the early warning unit includes a second early warning module. The second early warning module is used to generate the above-mentioned fourth prompt information when the above-mentioned biochemical data is not within the preset normal biochemical range and the above-mentioned user has the above-mentioned risk. The above-mentioned fourth prompt information is information that prompts the above-mentioned user that there is the above-mentioned risk and that the above-mentioned biochemical data is abnormal.

[0137] In this solution, by monitoring biochemical indicators in urine, it is possible to determine whether there is a risk by combining user data. This allows for multimodal monitoring, which can more accurately determine whether to generate a fourth alert message.

[0138] In some embodiments, the aforementioned prompt information includes one or more of a fifth, sixth, and seventh prompt information. The warning unit includes a third control module, a fourth control module, and a third warning module. The third control module is used to control the smart mirror to flash warning lights according to the fifth prompt information, wherein the fifth prompt information is information used by the smart mirror to prompt the user. The fourth control module is used to control the smart speaker to emit an inquiry voice according to the sixth prompt information, wherein the inquiry voice is a voice asking the user if they need help, and the sixth prompt information is information used to prompt the user via voice. The third warning module is used to send the seventh prompt information to a target device if no reply voice is received, or if the reply voice indicates that the user needs help, wherein the target device is the device of the user's emergency contact, and the seventh prompt information is information used to prompt the emergency contact via SMS or telephone.

[0139] This solution uses visual alerts from a smart mirror to provide prompts to users without infringing on their privacy or disturbing their families. It can also incorporate auditory prompts to further confirm the user's state of consciousness. External communication is only triggered when the user is unresponsive or explicitly requests assistance. This tiered and progressive alert system improves the response efficiency of health monitoring in bathroom settings.

[0140] The aforementioned monitoring device includes a processor and a memory. The first acquisition unit, determination unit, and early warning unit are all stored as program units in the memory, and the processor executes these program units stored in the memory to achieve their respective functions. All of the above modules are located in the same processor; alternatively, the modules may be located in different processors in any combination.

[0141] The processor contains a kernel, which retrieves the corresponding program units from memory. One or more kernels can be configured, and adjusting kernel parameters can address the problem of poor monitoring efficiency in bathroom spaces in existing technologies.

[0142] The memory may include non-permanent memory in computer-readable media, such as random access memory (RAM) and / or non-volatile memory, such as read-only memory (ROM) or flash RAM, and the memory includes at least one memory chip.

[0143] This invention provides a computer-readable storage medium including a stored program, wherein the program, when running, controls the device containing the computer-readable storage medium to execute the monitoring method.

[0144] This invention provides a processor for running a program, wherein the program executes the monitoring method described above.

[0145] This invention provides a device including a processor, a memory, and a program stored in the memory and executable on the processor. When the processor executes the program, it implements a monitoring method. The device described herein can be a server, PC, PAD, mobile phone, etc.

[0146] This application also provides a computer program product that, when executed on a data processing device, is suitable for performing a monitoring method.

[0147] This application also provides a monitoring system, including a millimeter-wave radar; a smart speaker; a lighting device; a smart toilet; a smart mirror; and a controller, wherein the controller is communicatively connected to the millimeter-wave radar, the smart speaker, the lighting device, the smart toilet, and the smart mirror, and the controller is used to execute any one of the monitoring methods described above.

[0148] It is obvious to those skilled in the art that the modules or steps of the present invention described above can be implemented using general-purpose computing devices. They can be centralized on a single computing device or distributed across a network of multiple computing devices. They can be implemented using computer-executable program code, and thus can be stored in a storage device for execution by a computing device. In some cases, the steps shown or described can be performed in a different order than those described herein, or they can be fabricated as separate integrated circuit modules, or multiple modules or steps can be fabricated as a single integrated circuit module. Thus, the present invention is not limited to any particular combination of hardware and software.

[0149] Those skilled in the art will understand that embodiments of this application can be provided as methods, systems, or computer program products. Therefore, this application can take the form of a completely hardware embodiment, a completely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, this application can take the form of a computer program product embodied on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0150] This application is described with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of this application. It will be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, special-purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, generate instructions for implementing the flowchart... Figure 1 One or more processes and / or boxes Figure 1 A device that provides the functions specified in one or more boxes.

[0151] These computer program instructions may also be stored in a computer-readable storage medium that can direct a computer or other programmable data processing device to function in a particular manner, such that the instructions stored in the computer-readable storage medium produce an article of manufacture including instruction means, which are implemented in a process Figure 1 One or more processes and / or boxes Figure 1 The function specified in one or more boxes.

[0152] These computer program instructions may also be loaded onto a computer or other programmable data processing equipment to cause a series of operational steps to be performed on the computer or other programmable equipment to produce a computer-implemented process, thereby providing instructions that execute on the computer or other programmable equipment for implementing the process. Figure 1 One or more processes and / or boxes Figure 1 The steps of the function specified in one or more boxes.

[0153] In a typical configuration, a computing device includes one or more processors (CPU), input / output interfaces, network interfaces, and memory.

[0154] Memory may include non-persistent memory in computer-readable media, such as random access memory (RAM) and / or non-volatile memory, such as read-only memory (ROM) or flash RAM. Memory is an example of computer-readable media.

[0155] Computer-readable media includes both permanent and non-permanent, removable and non-removable media that can store information using any method or technology. Information can be computer-readable instructions, data structures, modules of programs, or other data. Examples of computer storage media include, but are not limited to, phase-change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technologies, CD-ROM, digital versatile optical disc (DVD) or other optical storage, magnetic tape, disk storage or other magnetic storage devices, or any other non-transferable medium that can be used to store information accessible by a computing device. As defined herein, computer-readable media does not include transient computer-readable media, such as modulated data signals and carrier waves.

[0156] The technical features of the above embodiments can be combined in any way. For the sake of brevity, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0157] It should also be noted that the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such process, method, article, or apparatus. Unless otherwise specified, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes that element.

[0158] The above description is merely a preferred embodiment of this application and is not intended to limit this application. Various modifications and variations can be made to this application by those skilled in the art. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of this application should be included within the protection scope of this application.

Claims

1. A monitoring method, characterized in that, include: Acquire user data collected from users, wherein the user data includes one or more of the following: location, movement trajectory, body shape, biochemical data, and physiological data; Determine whether the user poses a risk based on the user data; If the user is found to pose the risk, a warning message is generated, and an alert is issued based on the warning message.

2. The method according to claim 1, characterized in that, The user data is collected based on millimeter-wave radar. The collected user data includes: The transmitted signal is obtained by acquiring the signal emitted by the millimeter-wave radar. The echo signal is obtained by acquiring the signal reflected from the user's body. The transmitted signal and the echo signal are combined to obtain a millimeter-wave dataset; The millimeter-wave dataset is input into the physiological recognition model to obtain the user data corresponding to the millimeter-wave dataset.

3. The method according to claim 2, characterized in that, Before inputting the millimeter-wave dataset into the physiological recognition model to obtain the user data corresponding to the millimeter-wave dataset, the method further includes: Obtain the recognition model, which is one of the following: CNN model, RNN model, or LSTM model. The historical millimeter-wave dataset and corresponding user tags are combined to form a training set. The recognition model is trained using the training set to obtain the physiological recognition model. The user tags are the historical user data corresponding to the historical millimeter-wave dataset in the training set.

4. The method according to claim 1, characterized in that, The risks include the risk of falls, and the physiological data includes heart rate. Determining whether a user is at risk based on the user data includes: If the user's posture changes abruptly and the rate of change of heart rate is greater than a preset heart rate change threshold, it is determined that the user is at risk of falling. The abrupt change of posture is the user's posture changing from standing to sitting or lying down.

5. The method according to claim 1, characterized in that, The risks include stationary risks. Determining whether a user poses a risk based on the user data includes: If the user remains in the same location for a period of time exceeding a preset time threshold, and the movement trajectory remains unchanged, it is determined that the user poses a risk of remaining stationary.

6. The method according to claim 5, characterized in that, The prompt information includes a first prompt information and / or a second prompt information, and the warning issued based on the prompt information includes at least one of the following: If it is determined that the user is at risk of remaining stationary, the smart speaker is controlled to play music according to the first prompt information, wherein the first prompt information is information that prompts the user with music; Based on the second prompt information, the lighting device is controlled to turn on the light mode, wherein the second prompt information is information that prompts the user using light.

7. The method according to claim 1, characterized in that, The user data also includes facial spectral data, and the prompt information includes third prompt information. When it is determined that the user faces the aforementioned risk, the prompt information is generated, including: Calculate the weighted average of the biochemical data, the physiological data, and the facial spectral data to obtain a comprehensive score; If the overall score is greater than a preset score threshold, it is determined that the user has the risk, and the third prompt information is generated, wherein the third prompt information is information indicating that the user's biochemical data, physiological data and facial spectral data are abnormal.

8. The method according to claim 1, characterized in that, The notification information includes a fourth notification information, which is generated when it is determined that the user faces the aforementioned risk. This notification information includes: If the biochemical data is not within the preset normal biochemical range and it is determined that the user is at risk, the fourth prompt message is generated, wherein the fourth prompt message is a message indicating that the user is at risk and that the biochemical data is abnormal.

9. The method according to any one of claims 1 to 8, characterized in that, The prompt information includes one or more of the fifth, sixth, and seventh prompt information, and the warning issued based on the prompt information includes one of the following: According to the fifth prompt message, the smart mirror is controlled to flash warning lights, wherein the fifth prompt message is information used by the smart mirror to prompt the user; According to the sixth prompt information, the smart speaker is controlled to issue an inquiry voice, wherein the inquiry voice is a voice asking the user if they need help, and the sixth prompt information is information that is prompted to the user by voice. If no response voice is received, or if the response voice indicates that the user needs help, the seventh prompt message is sent to the target device, wherein the target device is the device of the user's emergency contact, and the seventh prompt message is a message sent to the emergency contact via SMS or telephone.

10. A monitoring device, characterized in that, include: The first acquisition unit is used to acquire user data collected from users, wherein the user data includes one or more of the following: location, movement trajectory, body shape, biochemical data, and physiological data. A determining unit is used to determine whether the user poses a risk based on the user data; The early warning unit is used to generate a prompt message when it is determined that the user has the risk, and to issue an early warning based on the prompt message.

11. A computer program product, comprising a computer program, characterized in that, When the computer program is executed by the processor, it implements the steps of the monitoring method according to any one of claims 1 to 9.

12. A monitoring system, characterized in that, include: Millimeter-wave radar; Smart speaker; Lighting equipment; Smart toilet; Smart mirror; The controller is communicatively connected to the millimeter-wave radar, the smart speaker, the lighting device, the smart toilet, and the smart mirror, and is used to execute the monitoring method according to any one of claims 1 to 9.

Images