Intelligent bathroom product multi-modal interaction control system

Abstract

The application discloses a kind of intelligent bathroom product multimodal interaction control systems, belong to intelligent house control technical field.The system includes: intelligent bathroom main control module, voice interaction module, mobile terminal APP interaction module, multi-sensor sensing interaction module, intelligent house linkage module and cloud service module.Voice interaction module uses end cloud collaborative architecture, supports offline / online dual-mode voice instruction analysis;APP interaction module provides graphical remote control and personalized parameter setting interface;Multi-sensor sensing interaction module integrates infrared human body sensing, millimeter wave radar ranging, pressure sensing and environmental perception sensor;The application fuses voice, touch, somatosensory three kinds of interaction modes, constructs multi-level, multi-dimensional intelligent bathroom man-machine interaction system, effectively solves the technical problems that existing product interaction mode is single, intelligent degree is low, lacks home linkage capability, significantly improves user experience, system practicality and safety.

Description

Technical Field

[0001] This invention relates to the field of smart home control technology, and more specifically, to a multimodal interactive control system for smart bathroom products, and more particularly to a comprehensive smart bathroom control scheme that integrates voice interaction, mobile terminal APP interaction, and multi-sensor sensing interaction. Background Technology

[0002] With the rapid development of IoT and AI technologies, smart home products have gradually become commonplace in households. Smart bathroom products, as an important component of smart homes, have also made significant progress in recent years. Currently, the main smart bathroom products on the market include smart toilets, smart shower systems, smart bathroom mirrors, and smart bathroom cabinets. While these products have improved the user experience to some extent, they still have the following shortcomings: First, the interaction methods are limited. Most existing smart bathroom products rely solely on physical buttons or simple remote controls for operation. While some high-end products are equipped with basic voice functions, their recognition accuracy is low and the supported commands are limited, failing to meet the diverse interaction needs of users. For the elderly and people with mobility issues, the single physical button interaction method is particularly inconvenient.

[0003] Secondly, they lack intelligent automatic sensing capabilities. Most smart bathroom products require active user operation to activate their functions, lacking the ability to automatically sense and respond based on user behavior and environmental conditions. For example, they cannot automatically prepare when a user approaches, nor can they automatically save energy or disinfect after the user leaves, resulting in energy waste and insufficient convenience.

[0004] Third, their ability to integrate with other smart home devices is weak. Most current smart bathroom products are independent "information silos," unable to effectively integrate with other smart home devices such as smart lighting, smart curtains, and smart speakers, making it difficult to create a unified smart living experience.

[0005] Fourth, there is a lack of effective conflict arbitration mechanisms when multiple modal inputs occur concurrently. When voice commands, APP commands, and sensor triggers occur simultaneously, existing systems are prone to control confusion or response delays, failing to ensure that user intentions are accurately executed. Summary of the Invention

[0006] In view of this, the purpose of this invention is to provide a multimodal interactive control system for smart bathroom products that can integrate multiple interaction methods, possess intelligent sensing capabilities, and deeply integrate with smart home systems.

[0007] To achieve the above objectives, the present invention provides a multimodal interactive control system for intelligent bathroom products, which includes the following six core modules: (I) Smart Bathroom Main Control Module The smart bathroom main control module, serving as the core brain of the entire system, employs a high-performance embedded processor (such as the ARM Cortex-A series) and runs a real-time operating system, responsible for coordinating and managing the operation of various functional modules. Its main functions include: multi-source data fusion processing, conflict arbitration and decision fusion of multimodal input signals, issuance and execution scheduling of control commands, and system status management and fault diagnosis. The main control module incorporates a lightweight machine learning inference engine (based on the TensorFlow LiteMicro framework, with a model employing a hybrid architecture of decision trees and lightweight neural networks), enabling user behavior prediction and personalized recommendations based on historical usage data. For example, the system can proactively suggest energy-saving mode switching times based on the user's high-frequency usage periods over several consecutive days, or push health reminders when changes in usage habits are detected.

[0008] The main control module is equipped with multiple communication interfaces, including but not limited to UART, I2C, SPI, GPIO, USB, and Ethernet interfaces, to support high-speed data exchange with various sub-modules. The main control module employs a dual-backup firmware design, supporting automatic rollback in case of OTA upgrade failure, ensuring system reliability.

[0009] (ii) Voice Interaction Module The voice interaction module adopts a dual-architecture design combining edge and cloud. A lightweight voice wake-up and command recognition engine (based on a CNN+GRU neural network model, with a model size not exceeding 3MB) is deployed on the edge, supporting offline recognition of preset wake-up words (such as "Hello Xiaozhi," which users can customize via the app) and commonly used control command sets. The control command sets cover three main categories: device on / off control ("Turn on flush," "Turn off seat heating," "Turn on drying"), parameter adjustment ("Increase water temperature," "Increase flushing intensity," "Decrease seat temperature"), and scene switching ("Activate energy-saving mode," "Switch to cleaning mode"), totaling no fewer than 200 predefined command templates.

[0010] In online mode, the voice interaction module connects to the cloud-based voice service platform via WiFi or Bluetooth, utilizing the natural language understanding capabilities of a large model to handle complex semantics and open-ended dialogues. For example, if a user says, "I feel the seat is a bit cold," the cloud engine can interpret it as the control intention to "raise the seat temperature by 2-3℃" and execute it. A collaborative recognition strategy is employed between the device and the cloud: audio data is first processed on the device using VAD (Voice Activity Detection) and wake word detection, with command recognition only performed after wake-up; when the device's confidence level is higher than a preset threshold (e.g., 0.85), it responds locally; otherwise, it automatically switches to cloud processing to balance response speed and recognition accuracy.

[0011] In terms of hardware, the voice interaction module is equipped with a ring microphone array consisting of four MEMS microphones, supporting 5-meter far-field sound pickup, 360-degree sound source localization, echo cancellation, and beamforming. The microphone array uses differential signal acquisition, and in conjunction with a DSP chip, it executes a multi-channel noise suppression algorithm in real time, maintaining a wake-up rate of over 90% even in ambient noise of 70dB. The system is also equipped with a speaker unit (rated power 3W, frequency response range 200Hz-20kHz) for voice feedback playback and music playback.

[0012] (III) Mobile Terminal APP Interaction Module The mobile terminal APP interaction module supports both iOS and Android platforms, allowing users to fully control smart bathroom products by installing a dedicated APP on their smartphones or tablets.

[0013] The app's functional architecture includes the following sub-modules: (1) Device discovery and network configuration sub-module: Supports three methods of network configuration: QR code scanning, Bluetooth-assisted network configuration, and router hotspot network configuration, guiding users to complete device addition in three steps.

[0014] (2) Real-time status monitoring submodule: Displays the operating status of smart bathroom products in real time in the form of dashboards and cards, including current water temperature (accuracy ±0.5℃), water level, seat temperature (accuracy ±0.5℃), instantaneous power and cumulative energy consumption, UV disinfection status, filter lifespan, and other data. The status update frequency is no less than once per second, and abnormal status is indicated by red high-brightness flashing.

[0015] (3) Mode switching submodule: Provides one-click quick switching between four preset usage modes: Cleaning mode, Comfort mode, Energy saving mode and Custom mode. Each mode has an independent preset combination of parameters: Cleaning mode (water temperature 42℃, rinsing intensity level 5, drying temperature level 3); Comfort mode (water temperature 38℃, rinsing intensity level 3, drying temperature level 2, seat temperature 37℃); Energy saving mode (water temperature 32℃, seat temperature off, standby power consumption <0.5W); Custom mode is completely configurable by the user.

[0016] (4) Personalized parameter setting submodule: Allows users to finely adjust the water temperature range (32-42℃ continuously adjustable, in 1℃ increments), rinsing intensity (5 levels linearly adjustable), drying temperature (3 levels adjustable), LED light brightness (0-100% stepless dimming) and color temperature (2700K-6500K adjustable), and supports multi-user profile management (up to 10 independent user profiles), each profile containing a complete combination of preference parameters and usage history.

[0017] (5) Health Data Analysis Submodule: Records data such as usage frequency, single usage duration, and time period distribution for each family member, and generates a visualized health report on a daily / weekly / monthly basis. This submodule also has anomaly detection capabilities. When a significant deviation in usage frequency or duration is detected (exceeding the individual's historical average ±2σ), a gentle reminder will be pushed through the APP.

[0018] The APP and the main control module use the MQTT over TLS 1.3 protocol for encrypted communication, and the device authentication adopts a one-device-one-key mechanism to ensure data transmission security.

[0019] (iv) Multi-sensor sensing and interaction module The multi-sensor sensing and interaction module integrates four types of sensors to achieve comprehensive environmental and user perception: (1) Infrared human body sensing unit: It adopts a high-sensitivity PIR pyroelectric infrared sensor (model such as RE-200B), with a detection angle of 120°, a detection distance of up to 8 meters, and a response time of less than 0.5 seconds, and is used to determine whether a user has entered the toilet area. This unit, together with a Fresnel lens array, effectively reduces false triggering caused by pets or small animals passing by.

[0020] (2) Millimeter-wave radar ranging unit: It adopts a 77GHz millimeter-wave radar chip (such as TI AWR1642) with an accuracy of centimeter level (±2cm). It can accurately measure the distance between the user and the device (range of 0.1-5 meters), the approach speed and the direction of movement (distinguishing between approaching and moving away), and realize three-level distance perception: the far zone (2-5 meters) performs pre-activation operation, the middle zone (0.5-2 meters) enters full-speed monitoring mode, and the near zone (<0.5 meters) performs preparatory action. This unit is not affected by light, water mist and temperature changes, and can work stably in all weather.

[0021] (3) Pressure sensing unit: A thin-film pressure sensor array (8 sensors evenly distributed) is arranged in the seat ring area, with a resolution of 100g and a sampling frequency of 20Hz. The system determines the user's sitting status (not seated / partially seated / fully seated), sitting posture analysis (sitting upright / leaning to the left / leaning to the right / leaning forward / leaning backward) and seat departure detection (triggered after a 2-second delay after the pressure value returns to zero) through multi-contact pressure distribution. The pressure data can also be used for weight estimation (error within ±0.5kg) and long-term health trend monitoring.

[0022] (4) Environmental Sensing Unit: Integrates an SHT30 temperature and humidity sensor (accuracy: temperature ±0.2℃, humidity ±2%RH) and an SGP30 air quality sensor (detection range: TVOC 0-1000ppb, CO2 equivalent concentration 400-60000ppm) to monitor the temperature, humidity, TVOC and CO2 equivalent concentration in the bathroom in real time. The sensor data acquisition frequency is once every 10 seconds, and it is equipped with data filtering and outlier removal algorithms.

[0023] (v) Smart Home Linkage Module The smart home linkage module supports multiple mainstream smart home communication protocols (including Wi-Fi 802.11b / g / n, Zigbee 3.0, Bluetooth Mesh 5.0, Matter protocol, etc.) to enable interconnection between smart bathroom products and other smart devices in the home.

[0024] This module includes a scene engine submodule, which provides three preset types of interactive scene templates: (1) Toilet entry linkage scenario: The infrared human body sensor detects the user's entry → Sends a closing command to the smart curtain controller via Zigbee → Sends a command to the smart lighting system to dim to night light mode (brightness 15%) via Bluetooth Mesh → Sends a pause playback command to the smart speaker via WiFi → The main control module starts the seat preheating program.

[0025] (2) Bathing linkage scenario: The user starts the bathing mode → sends a defogging and heating start command to the smart bathroom mirror → sends an on command (low speed) to the smart exhaust fan → sends an ambient light adjustment command to the smart lighting (color temperature switches to warm light 3000K, brightness is adjusted to 60%) → sends a command to the smart speaker to play preset soothing music.

[0026] (3) Leave and restore the scene: The millimeter-wave radar confirms that the user has left and the pressure sensor confirms that no one is sitting down → Sends a command to the smart lighting to restore normal brightness → The main control module starts the UV / ozone disinfection program (duration 5 minutes) → Use data to synchronize to the cloud → The system enters ultra-low power standby mode (power consumption <0.3W).

[0027] Users can customize linkage rules through the IF-THEN visual rule editor in the app. The rule editor adopts a graphical programming method, allowing users to select trigger conditions (device status changes, sensor values ​​exceeding thresholds, time schedules, etc.) and execution actions (device on / off, parameter adjustment, scene switching, etc.), creating personalized linkage scenarios without requiring programming knowledge. The system supports setting the priority of linkage rules and combining condition logic (AND / OR).

[0028] (vi) Cloud Service Module The cloud service module adopts a microservice architecture (containerized and deployed on a Kubernetes cluster), providing services such as remote device management, data storage and analysis, OTA firmware upgrades, and AI model iteration. The cloud platform supports concurrent access from tens of millions of devices, and a single machine can handle long-term connections for more than 50,000 devices.

[0029] Cloud-based features include: (1) Device registration and authentication management: When the device is activated for the first time, it applies for a unique device ID and key from the cloud. Subsequent communication uses dynamic token authentication.

[0030] (2) Data storage and analysis: User data (after desensitization) is stored in a time series database with a retention period of 180 days, supporting multi-dimensional data aggregation analysis and trend mining.

[0031] (3) OTA firmware upgrade management: Supports batch gray-scale upgrade strategy, upgrade package is transmitted in differential incremental mode (upgrade package size does not exceed 30% of the original firmware size), and supports breakpoint resume and upgrade result rollback confirmation.

[0032] (4) AI model iteration: The cloud regularly retrains the speech recognition model and user behavior prediction model based on the massive user data collected. The optimized model is pushed to the terminal for update after A / B testing.

[0033] The cloud also provides RESTful open API interfaces, supporting standard integration and ecosystem expansion with third-party IoT platforms (such as Xiaomi IoT Platform, Huawei HiLink, Alibaba Cloud IoT, Apple HomeKit, etc.).

[0034] Compared with existing technologies, the beneficial effects of this invention are mainly reflected in the following aspects: This invention organically integrates three modes: voice interaction, touch interaction (APP), and motion-sensing interaction (multi-sensor sensing). Users can freely choose the most convenient interaction method according to different scenarios: use voice control when hands are inconvenient; use APP operation when the environment is noisy; and the system automatically completes preparation work through sensing when no active operation is required. The three interaction modes complement each other, realizing "there is always the most suitable interaction method at any time." The multi-level and multi-dimensional interaction system greatly reduces the threshold for use, especially suitable for the elderly and special groups; through the collaborative work of multi-dimensional sensors such as millimeter-wave radar and pressure sensor array, the system can accurately perceive the user's position, posture, and behavioral intentions. The three-level distance sensing mechanism realizes a progressive response from "pre-activation" to "standby" to "ready," allowing users to enjoy the convenience of automatic device readiness without being aware of it. Compared with the traditional single infrared sensing solution, the multi-sensor fusion solution reduces the false trigger rate by more than 90% and improves the response accuracy to more than 98%. Machine learning-based user behavior prediction enables the system to proactively adapt to user habits, increasing its intelligence over time. Supporting multiple mainstream communication protocols such as Wi-Fi, Zigbee, Bluetooth Mesh, and Matter, smart bathroom products evolve from standalone devices into crucial nodes in a whole-house smart ecosystem. Interconnected scenarios cover the entire lifecycle of a user's bathroom experience—entry, use, and departure—and can be flexibly customized and expanded by the user. Cross-device collaborative optimization not only enhances convenience and comfort but also achieves energy conservation and emission reduction through scenario-based control (such as automatic light shut-off upon leaving the room and automatic standby when unoccupied). A multimodal fusion decision-making mechanism ensures that security protection commands have the highest priority, effectively preventing control chaos caused by concurrent multimodal inputs. A conflict arbitration strategy employs a triple guarantee of "priority layering + time window arbitration + state backtracking compensation," ensuring accurate execution of user intentions under all circumstances. Network communication uses TLS 1.3 encryption, and voice data supports end-side processing (end-side recognition rate reaches over 85%), with encrypted uploads only for complex semantic scenarios, fully protecting user privacy. Dual-backup firmware and OTA gray-scale upgrades further ensure the long-term reliable operation of the system. Attached Figure Description

[0035] Figure 1 This is an overall architecture diagram of the multimodal interactive control system for the intelligent bathroom product of the present invention; Figure 2 Functional block diagram of the smart bathroom main control module; Figure 3 A flowchart of the voice interaction module; Figure 4 A schematic diagram of the interface layout of the interactive module of a mobile terminal APP; Figure 5This is a schematic diagram of the sensor arrangement for a multi-sensor sensing and interaction module. Figure 6 This is a sequence diagram of a typical linkage scenario for smart home linkage modules; Figure 7 A flowchart illustrating the conflict arbitration logic of a multimodal fusion decision-making unit; Figure 8 This is an overall flowchart of the method of the present invention. Detailed Implementation

[0036] The present invention will be further described in detail below with reference to the accompanying drawings and specific embodiments. It should be understood that the following embodiments are for illustration and explanation only and are not intended to limit the scope of protection of the present invention.

[0037] Example 1: System Overall Architecture like Figure 1 As shown, the multimodal interactive control system for smart bathroom products provided by this invention includes six modules: a smart bathroom main control module 10, a voice interaction module 20, a mobile terminal APP interaction module 30, a multi-sensor sensing interaction module 40, a smart home linkage module 50, and a cloud service module 60. The connection relationships between the modules are as follows: The smart bathroom main control module 10 is connected to the voice interaction module 20, the multi-sensor sensing interaction module 40, and the smart home linkage module 50 via an internal bus (such as I2C, SPI, UART); the mobile terminal APP interaction module 30 establishes a wireless connection with the smart bathroom main control module 10 via Wi-Fi / Bluetooth; and the cloud service module 60 establishes a connection with the smart bathroom main control module 10 and the mobile terminal APP interaction module 30 via the Internet.

[0038] The intelligent bathroom main control module 10, as the core of the system, uses a Rockchip RK3588 processor (octa-core ARM Cortex-A76 / A55, 6TOPS computing power NPU) and runs an embedded real-time operating system based on a customized Linux. The main control module is equipped with 2GB of DDR4 memory and 16GB of eMMC storage to meet the computing needs of local speech recognition inference and multi-sensor data fusion processing. The main control module also features a hardware watchdog timer, which automatically resets the system within 30 seconds in case of an anomaly.

[0039] Example 2: System Initialization and Self-Test Process This embodiment describes the complete initialization process after the system is powered on.

[0040] After the system is powered on, the smart bathroom main control module 10 first loads the system configuration file and user preference data stored in the eMMC. The configuration file includes device parameters (such as rated voltage, power limits, etc.), network configuration (WiFi SSID and encryption key), linkage scene definitions, and user profile data (historical preference parameters for each user). After loading is complete, the main control module performs the following self-test steps in sequence: Step 1: Perform a self-test on the voice interaction module 20, including a connectivity test of the microphone array (checking whether the bias voltage of each MEMS microphone is normal, and marking it as abnormal when the deviation exceeds ±5%), a voice coil impedance test of the speaker unit (normal range 4-8Ω), and a model integrity check of the voice recognition engine (SHA-256 hash comparison).

[0041] Step 2: Perform a self-test on the multi-sensor sensing and interaction module 40. The infrared human body sensing unit 41 performs reference level calibration to eliminate environmental baseline drift; the millimeter-wave radar ranging unit 42 executes its built-in self-test program to confirm that the transmission power and receiving sensitivity are within the nominal range; the pressure sensing unit 43 performs no-load zero-point calibration and records the no-load reference values ​​of each sensor; the environmental perception unit 44 triggers an internal self-test and reads the firmware version to confirm normal operation.

[0042] Step 3: Perform a communication self-test on the smart home linkage module 50, and attempt to establish handshake connections with the bound devices such as smart lighting, smart curtains, and smart speakers in turn, recording the online status and protocol version of each device.

[0043] After all modules pass the self-test, the system enters normal operating mode and announces "System Ready" via speaker and pushes a notification via the APP. If any module fails the self-test, the system records an error log (written to a local circular buffer), sends detailed diagnostic information to the user via the APP, and decides whether to allow continued startup based on the severity of the fault (the system starts in degraded mode when a non-critical module fails).

[0044] Example 3: Detailed Solution for Voice Interaction Module like Figure 3 As shown, the workflow of the voice interaction module 20 is as follows: (1) Sound Acquisition Stage: A circular array of four MEMS microphones synchronously acquires ambient sound at a sampling rate of 48kHz and a quantization precision of 24-bit. A digital signal processor (DSP, such as ADSP-21569) executes a beamforming algorithm in real time to orient the main lobe of the microphone array toward the sound source (angular resolution of approximately 15°), with a sidelobe suppression ratio greater than 20dB, effectively enhancing the sound signal in the target direction and suppressing background noise and reverberation interference. The DSP also performs adaptive echo cancellation (AEC) to eliminate interference from the content played by the speaker on the microphone pickup.

[0045] (2) Wake-up word detection stage: A lightweight neural network model on the edge (based on CNN+GRU architecture, model size approximately 2.8MB, quantization precision INT8) continuously detects wake-up words on the enhanced audio stream. The model input is 40-dimensional MFCC features, the time window is 1.5 seconds, and the inference delay does not exceed 100ms. When a preset wake-up word is detected (default is "Hello Xiaozhi", which can be customized by the user via the APP to any word of 3-5 syllables) and the confidence level exceeds 0.80, the system enters the command listening state. The LED indicator switches from standby breathing mode to constant light mode, and the speaker plays a confirmation prompt tone (1kHz / 200ms short tone) to provide auditory feedback to the user.

[0046] (3) Command Recognition Stage: The audio segment after wake-up (maximum 8 seconds) is first sent to the edge command recognition engine for matching. The edge engine has about 200 predefined command templates, covering three categories: device control (on / off and adjustment of various functions, about 120), query and inquiry (current temperature, remaining battery power, usage statistics, etc., about 50), and scene trigger (starting sleep mode, starting guest mode, etc., about 30). The edge recognition adopts an encoder-decoder architecture based on the attention mechanism, and the word error rate (CER) is less than 5% on the test set. If the edge recognition confidence is higher than 0.85, it is directly converted into a control command; if the confidence is between 0.50 and 0.85, the system confirms with the user again via voice ("Do you want...?"); if the confidence is lower than 0.50 or the user issues an open dialogue request, the audio data is encrypted with AES-256 and uploaded to the cloud voice service platform for large model processing.

[0047] (4) Feedback Output Stage: After the control command is executed, the speech synthesis unit (TTS, based on a lightweight TTS engine on the edge, model size of about 15MB) converts the execution result into natural language speech broadcast. The broadcast delay is no more than 500ms (from the completion of the command to the start of speech output). The TTS engine supports two voice options, male and female, and speech speed adjustment of 0.8-1.2 times. The broadcast content can be set to a concise mode (such as "adjusted to 39 degrees") or a detailed mode (such as "the water temperature has been adjusted to 39 degrees for you, and the current seat temperature is 37 degrees") according to the user's preference.

[0048] Example 4: Detailed Scheme for Multi-Sensor Sensing Interaction like Figure 5 As shown, the sensor arrangement scheme of the multi-sensor sensing and interaction module 40 is as follows (taking a smart toilet as an example): Infrared human body sensing unit 41: Installed at the top front of the device, the Fresnel lens detection angle is tilted downwards by approximately 30° (covering the ground area from 0.5 meters to 4 meters in front of the device), with a horizontal coverage angle of 120°, covering the bathroom entrance area. When a person is detected entering, a "person entered" event is triggered, and the system immediately activates the voice interaction module 20, switching from deep sleep mode to low-power standby mode (power consumption increases from 0.1W to 0.5W), and simultaneously illuminates the guide night light (brightness 20%, warm yellow light).

[0049] Millimeter-wave radar ranging unit 42: Embedded in the center of the front panel of the device (approximately 40cm above the ground), the radar operates in the 77-81GHz frequency band, with a transmit power not exceeding 10dBm, meeting CE and FCC certification requirements. This unit divides the detection space into three distance ranges: Far zone (2-5 meters): When a target is detected, the user's historical preference configuration is loaded (initial identity matching is performed by recognizing the user's walking speed and body shape characteristics); Mid zone (0.5-2 meters): All sensors are activated to enter full-speed monitoring mode (sensor sampling frequency is increased from 10Hz to 50Hz); Near zone (<0.5 meters): When it is determined that the user is about to use the device, preparatory actions are automatically performed—the seat ring is heated to the preset temperature (default 37℃), and the cover automatically opens slowly (opening speed is approximately 3 seconds to fully open). The radar can also distinguish the direction of movement of people approaching and moving away through the micro-Doppler effect, triggering a positive response only in the approaching direction to avoid accidental triggering when the user leaves.

[0050] Pressure sensing unit 43: Consists of 8 thin-film pressure sensors (model such as FSR-402) evenly distributed on the seat surface (3 at the front, 3 in the middle, and 2 at the rear). A 12-bit ADC voltage divider circuit reads the pressure value at each point, with a sampling frequency of 20Hz. The system determines the user's sitting status based on the pressure distribution: partial sitting is defined as the total pressure value consistently exceeding 200g and the maximum pressure at a single point exceeding 100g; full sitting is defined as at least 6 sensor pressure values ​​exceeding 50g; and leaving the seat is defined as all sensor pressure values ​​being less than 10g for 2 seconds. Posture analysis is based on the shift in the center of gravity of the pressure distribution: a shift exceeding 30% of the seat width is defined as off-center sitting. The system automatically triggers the flushing program after a 2-second delay upon detecting a leaving-seat event (automatically selecting based on sitting duration: a gentle flush for less than 60 seconds and a strong flush for more than 60 seconds). Long-term accumulation of pressure data (archived by user identity) can be used for bowel habit monitoring.

[0051] Environmental sensing unit 44: The SHT30 temperature and humidity sensor is installed in the ventilation area inside the device (approximately 30cm from the ground), collecting ambient temperature and humidity data every 10 seconds (temperature measurement range -40 to 125℃, humidity measurement range 0-100%RH). The SGP30 air quality sensor is installed on the top of the device, monitoring TVOC (measurement range 0-1000ppb, resolution 1ppb) and CO2 equivalent concentration (measurement range 400-60000ppm, resolution 1ppm) in real time. When an environmental anomaly is detected (humidity > 85%RH for more than 30 seconds, TVOC > 0.6mg / m³, or CO2 > 1200ppm), the system automatically activates the exhaust fan (by sending a command via the smart home linkage module) or sends a notification to the APP.

[0052] Example 5: Smart Home Interconnection Scenarios like Figure 6 As shown, the following describes in detail the working process of the smart home linkage module 50, taking the complete linkage scenario of "nighttime toilet use" as an example: At time T0 (00:35:12): Infrared human body sensing unit 41 detects that the user is walking from the bedroom to the bathroom, triggering the "person approaching" event. The system records the event timestamp and the direction of the user's approach.

[0053] T0+1 seconds (00:35:13): The system sends an "On (brightness 30%, color temperature 2700K warm light)" command to the smart floor light in the corridor via the Zigbee communication channel of the smart home linkage module 50, providing users with a weak lighting guide and avoiding glare from strong light that might affect sleep. The linkage command uses a confirmation and retransmission mechanism (retransmitted 3 times, 200ms interval) to ensure reliable execution.

[0054] T0+5 seconds (00:35:17): Millimeter-wave radar 42 confirms that the user has entered the bathroom area (distance < 2 meters, direction of movement towards the device). The system executes the following concurrent linkage operations: sends a "fully closed" command to the smart curtain controller (to protect privacy, the window closes at a speed of approximately 15cm / s, taking approximately 8 seconds from opening to fully closing); sends a "close" command to the main bathroom lighting (no main light needed at night, providing nighttime visual adaptation protection for the user); and sends "seat preheating to 37°C" and "turn on night light mode (brightness 15%)" commands to the smart toilet.

[0055] T0+15 seconds (00:35:27): The user arrives in front of the smart toilet (millimeter-wave radar distance <0.5 meters). The toilet seat opens automatically with a soft closure (opening speed approximately 5° / s, taking approximately 3 seconds to go from fully closed to fully open), and the night light softly illuminates. At this time, the user can choose voice commands (such as "Xiao Zhi, turn on the heater" with the default heater temperature of 35°C), APP operation, or direct use.

[0056] During use (00:35:30-00:38:00): The pressure sensing unit 43 detects that the user is fully seated, and the system suspends all linkage operations that may disturb the user (such as incoming call reminders, music playback prompts, etc.) and enters privacy protection mode. The environmental sensing unit 44 continuously monitors the air quality, and automatically turns on the exhaust fan at low speed (noise <30dB) when the CO2 concentration exceeds 1000ppm.

[0057] End of use (00:38:05): Pressure sensor unit 43 detects an event of leaving the seat (total pressure value returns to zero for 2 seconds) and automatically executes the flushing program: based on the sitting time (2 minutes and 35 seconds in this scenario), it automatically selects the "large flush" mode (6L of water). After flushing is completed, the UV sterilization lamp (wavelength 254nm, sterilization time 3 minutes) is automatically activated.

[0058] Leaving Phase (00:38:30): Millimeter-wave radar 42 detects that the user is moving away (distance > 2 meters and continuously moving away from the direction of movement). The system performs the following operations: sends a "restore original state" command to the smart curtains (if they were previously half-open, they will be restored to half-open); sends a "turn off" command to the bathroom lighting (executed after a 3-second delay, so that there is still some lighting when the user leaves); the smart toilet's UV sterilization mode continues to run for the preset time (5 minutes) and then automatically stops; all usage data (usage time, duration, flushing mode, etc.) is encrypted and uploaded to the cloud.

[0059] Throughout the process, the user only needed to say one sentence when they needed warm air; all other operations were completed automatically by the system, achieving a truly "seamless intelligent" experience.

[0060] Example 6: A Case Study of a Multimodal Fusion Decision-Making Mechanism like Figure 7As shown in the figure, this embodiment illustrates the working process of the multimodal fusion decision-making mechanism through a specific conflict scenario.

[0061] Scenario 1: Multimodal concurrency conflicts Simultaneously, when the user executes the "Turn off seat heating" command on the app, the sensor triggers the automatic execution of the "seat preheating" command. The system detects that the P2 level (app command) and P3 level (sensor command) arrive within 200ms. The multimodal fusion decision unit determines that the P2 level is higher than the P3 level according to the priority rule and executes the app's "Turn off seat heating" command. At the same time, the state backtracking and compensation mechanism checks after 30 seconds: if there are no new user commands and the infrared sensor still detects that the user is nearby, the "seat preheating" is retried.

[0062] Scenario 2: Security Command Override The sensor detects a leakage current signal, and the system generates a P0-level safety protection command. At this time, the voice command (P1 level) is executing "increase flushing intensity". The fusion decision unit immediately stops the executing voice command, sends an emergency stop command to the actuator, and simultaneously triggers the leakage current circuit breaker to cut off the power supply (response time <50ms). After the safety command is executed, the system records the event and returns to the safe standby state.

[0063] Scenario 3: Arbitration within the same priority time window The user utters two voice commands consecutively within 500ms: "Turn on flush" and "Turn off flush." ​​The fusion decision unit adopts the "last to arrive" principle, executing the "Turn off flush" command to avoid performing actions inconsistent with the user's latest intention. The arbitration log records the arrival timestamps of both commands and the arbitration result for easy troubleshooting later.

[0064] Example 7: Personalized User Profiling and Adaptive Learning This embodiment describes the specific process by which the system uses a machine learning engine to predict user behavior and personalize the system.

[0065] The system creates a unique user profile for each family member. When a user first uses the system, they need to register their identity via voice or the app (voiceprint registration or manual selection). After that, the system automatically identifies the current user through voiceprint recognition (when using voice commands) or by matching usage time and behavior patterns (when using sensor interaction).

[0066] The profile contains data in the following dimensions: (1) Preference parameters: water temperature preference (e.g., 39.5℃ for user A), flushing intensity preference (e.g., 4 levels for user B), seat temperature preference (e.g., 36℃ for user C), light color temperature and brightness preference (e.g., warm light 2700K / 20% for night mode), music preference (playlist ID).

[0067] (2) Behavioral patterns: typical usage time (e.g., user A's habit of 06:30-06:45 and 22:00-22:15), average usage time (e.g., 3.2 minutes), usage frequency (average 3.2 times / day).

[0068] (3) Learning weight: The system’s historical confidence in each user’s command, used to dynamically adjust the sensitivity of personalized recognition.

[0069] The system incorporates a lightweight time-series prediction model (based on LSTM, with 64 hidden layers). Using usage data from the past 7 days as input (15-minute time resolution), it predicts the probability of usage for each time period within the next 24 hours. When the predicted probability exceeds 0.7, the system enters a "pre-ready" state 15 minutes in advance: the seat ring begins to preheat (but maintains a low temperature of about 30°C to avoid energy waste), and the voice interaction module switches to high-performance mode (slightly increasing power consumption but reducing response latency).

[0070] In terms of personalized recommendations, when the system detects that a user actively adjusts a parameter (such as changing the rinsing intensity from level 3 to level 4) three times in a similar time period (±30 minutes), it will proactively push a suggestion on the fourth use: "We have detected that you have recently preferred a stronger rinsing intensity. Do you want to switch to level 4 as the default setting?" After the user confirms, the user's profile will be updated.

[0071] The system generates a weekly summary of household usage reports, which is pushed to the app. The report includes a comparison of usage frequency for each member, energy consumption trends, and suggested energy-saving strategies.

[0072] Example 8: System Security and Emergency Fault Handling This embodiment describes the system's security protection mechanism and emergency response process during multimodal interaction.

[0073] Security protection instruction level: The system defines the following trigger conditions to automatically generate P0 level security protection instructions: (1) Leakage protection: Built-in leakage current detection module (sensitivity 30mA, response time <100ms), after triggering, immediately cuts off the power supply to the heater and motor, only keeping the control circuit and communication circuit running, and broadcasts "Leakage detected, power supply cut off, please contact after-sales service" and pushes alarms via APP.

[0074] (2) Overheat protection: Triple redundant temperature monitoring (seat ring temperature sensor NTC×2, ambient temperature sensor×1). When the detected value of any temperature sensor exceeds the preset safety threshold (seat ring > 45℃, heating module > 85℃), it will be triggered, immediately stop heating and start forced cooling (fan running at full speed).

[0075] (3) Anti-pinch protection: During the opening and closing process of the cover plate, when the motor drive current rises abnormally (exceeding 150% of the normal value) and the Hall sensor detects an abnormal position, it is determined that a hand or obstacle is pinched, and the motor is immediately stopped and reversed by 5° to release the pressure.

[0076] Fault emergency handling process: Local fault priority handling principle: All safety-related faults are independently judged and handled locally by the main control module, without relying on cloud communication. The handling process when a fault occurs is as follows: Step 1 (0-50ms): The sensor or detection module triggers a fault signal, and the main control module responds via hardware interrupt.

[0077] Step 2 (50-200ms): Execute emergency protection actions (cut off power / stop the motor / start heat dissipation, etc.), and simultaneously record fault scene data (sensor snapshot, timestamp, current execution instruction, etc.).

[0078] Step 3 (200ms-1s): Send a fault prompt to the user through voice broadcast and changes of the LED status light (such as three short beeps + red LED flashing).

[0079] Step 4 (1s-5s): Push a fault notification via the APP, and upload the fault log to the cloud synchronously if the WiFi network is available.

[0080] Step 5 (Recovery phase): After the user confirms the fault handling (via the APP or power-off restart), the system automatically performs a full sensor self-check, and switches to the normal working mode after confirming that everything returns to normal.

[0081] Communication fault handling: When the system detects that the connection with the cloud is interrupted, it automatically switches to the pure local working mode (offline mode). In the offline mode, the functions of voice command recognition, induction trigger, and linkage execution (limited to devices within the local area network) are completely normal, and only the complex semantic understanding relying on the cloud and the cross-Internet linkage functions are restricted. After the cloud connection is restored, the system automatically synchronizes the operation logs generated during the offline period.

[0082] Embodiment 9: Energy-saving mode and green operation strategy This embodiment describes the energy-saving control strategy of the system under the multi-modal interaction framework.

[0083] The system automatically adjusts the power consumption state according to the user's activity state and working mode, and defines the following four levels of power consumption states (as shown in the table below):

[0084] The system uses machine learning to predict user usage patterns. When it anticipates a user's upcoming usage time (e.g., 6:00-7:00 AM on weekdays), it switches the power consumption status from "deep sleep" to "low-power standby" 15 minutes in advance. This eliminates the need for preheating during actual use and avoids energy waste caused by 24 / 7 standby. Long-term statistics show that this strategy can save approximately 45% more energy than 24 / 7 standby mode.

[0085] The user interaction experience is unaffected in energy-saving mode: when the user arrives, the system switches from energy-saving mode to full-speed operation in less than 2 seconds (the seat ring takes about 10-15 seconds to heat up to the preset temperature, the cover opens slowly and the heating is carried out simultaneously, and the user can complete the preheating before sitting down).

[0086] Example 10: Multi-device collaboration and distributed control This embodiment describes how the system works in a scenario where multiple smart bathroom devices work together.

[0087] This invention's system is not only applicable to a single smart toilet, but can also be deployed simultaneously in the same bathroom with multiple smart bathroom devices (such as smart toilets, smart shower systems, and smart bathroom cabinets). Each device contains an independent smart bathroom main control module, forming a distributed collaborative network through a local area network (WiFi or Zigbee Mesh).

[0088] Multi-device scenario example: Before showering, users can perform health monitoring on the smart toilet (by measuring body composition data using pressure sensors and bioimpedance sensors), and the data is shared with the smart bathroom cabinet and smart shower system via a local network.

[0089] When a user approaches the shower area, the smart shower system detects their approach using its integrated millimeter-wave radar. It automatically retrieves the user's identity information from the smart toilet (through time and space correlation matching) and loads the user's shower preference parameters (40℃ water temperature, 3 water pressure levels, and a blue gradient ambient light mode), eliminating the need for user voice prompts or app operations. The smart mirror in the bathroom vanity simultaneously displays a summary of health monitoring data and the day's weather information.

[0090] Status synchronization between multiple devices is achieved through a publish-subscribe messaging mechanism (based on a local MQTT broker). Status change messages (such as "User A has left the seat" or "Water temperature has been adjusted to 40℃") are broadcast in JSON format within the local area network, and devices subscribed to relevant messages respond automatically. The message TTL (Time To Live) is set to 5 seconds to prevent expired status from affecting judgment.

[0091] When a device malfunctions, the distributed collaborative network automatically degrades its operation: for example, when the main control module of a smart toilet is offline, the smart shower system can temporarily take over the environmental sensing function (using its own temperature and humidity sensor) and control the exhaust fan through the linkage module to ensure that basic environmental comfort is not interrupted.

[0092] The above description is merely a preferred embodiment of the present invention and is not intended to limit the invention. Any modifications, equivalent substitutions, and improvements made within the spirit and principles of the present invention should be included within the scope of protection of the present invention.

Claims

1. A multimodal interactive control system for intelligent bathroom products, characterized in that, include: The intelligent bathroom main control module is used to coordinate and manage the operation of various functional modules, and is responsible for data fusion processing, decision logic execution and instruction distribution; The voice interaction module is connected to the smart bathroom main control module and is used to receive the user's voice commands and convert them into control commands after semantic parsing. The mobile terminal APP interaction module is connected to the smart bathroom main control module via a network and is used to provide a remote monitoring, parameter setting and manual control interface; A multi-sensor sensing and interaction module is connected to the smart bathroom main control module. It is used to collect user behavior data and environmental data through various sensors, and automatically trigger corresponding control actions based on the data. The smart home linkage module is connected to the smart bathroom main control module and is used to enable smart bathroom products to work collaboratively with other devices in the smart home system through a standardized communication protocol.

2. The intelligent bathroom product multimodal interactive control system according to claim 1, characterized in that, The voice interaction module includes: The microphone array unit, consisting of at least two MEMS microphones, is used to achieve sound source localization and noise suppression; The edge-side speech recognition engine, deployed in a local embedded processor, is used to recognize preset wake words and control command sets in offline mode; The cloud-based speech recognition engine interface is used to upload audio data to the cloud for natural language understanding when the network is connected. The speech synthesis unit is used to convert system feedback information into speech output.

3. The intelligent bathroom product multimodal interactive control system according to claim 1, characterized in that, The mobile terminal APP interaction module includes: The equipment status monitoring submodule is used to display the real-time operating status of smart bathroom products, including water temperature, water level, seat temperature, and energy consumption data. The mode switching submodule provides quick switching between at least four preset usage modes, including cleaning mode, comfort mode, energy-saving mode and custom mode; The personalized parameter setting submodule allows users to finely adjust the water temperature range, rinsing intensity, drying temperature, and light brightness, and save these settings as a user profile. The health data analysis submodule records the user's usage frequency and duration data and generates health reports.

4. The intelligent bathroom product multimodal interactive control system according to claim 1, characterized in that, The multi-sensor sensing and interaction module includes: The infrared human body sensing unit uses a PIR pyroelectric infrared sensor to detect user entry and exit events; Millimeter-wave radar ranging unit is used to accurately measure the distance and direction of movement between the user and smart bathroom products, enabling hierarchical distance perception; A pressure sensing unit is located in the seat ring or foot pedal area to determine the user's sitting position and weight distribution. The environmental sensing unit integrates temperature and humidity sensors and air quality sensors to acquire the current temperature, humidity, and air quality parameters of the bathroom environment.

5. The intelligent bathroom product multimodal interactive control system according to claim 1, characterized in that, The smart home linkage module supports the following linkage scenarios: Toilet access scenario: When a user enters, the smart curtains will automatically close, the smart lighting will be dimmed to night light mode, and the smart speaker playback will be paused. Bathing-related scenarios: When the bathing mode is activated, the smart bathroom mirror's defogging function is automatically adjusted, the exhaust fan is turned on, and the music ambiance is set. Leave recovery scenario: When the system detects that a user has left and confirms that no one is around, it automatically restores the lighting, activates the disinfection mode, and synchronizes data to the cloud.

6. The intelligent bathroom product multimodal interactive control system according to claim 1, characterized in that, The intelligent bathroom main control module also includes a multimodal fusion decision unit, which is configured as follows: It receives input signals from the voice interaction module, the mobile terminal APP interaction module, and the multi-sensor sensing interaction module; The system arbitrates conflicts among input signals based on preset priority rules, with priorities from highest to lowest as follows: safety protection commands > voice commands > APP remote control commands > automatic sensor triggering; A unified control instruction is generated based on the arbitration result and issued to each enforcement agency.

7. The intelligent bathroom product multimodal interactive control system according to claim 6, characterized in that, The multimodal fusion decision unit is also configured to: when multiple signals of the same priority arrive within a preset time window, arbitrate using the principle of the last arriving signal taking effect; and when a high-priority instruction overrides a low-priority instruction, perform state backtracking and compensation operations when conditions permit.

8. The intelligent bathroom product multimodal interactive control system according to claim 1, characterized in that, The smart bathroom main control module has a built-in lightweight machine learning inference engine, which is used to predict user behavior and recommend personalized parameters based on historical usage data.

9. The multimodal interactive control system for intelligent bathroom products according to any one of claims 1 to 8, characterized in that, The system is applied to any one or more combinations of smart toilets, smart shower systems, or smart bathroom cabinets.

Images