Intelligent home control system and method based on edge computing and multi-modal perception

Abstract

This invention discloses a smart home control system and method based on edge computing and multimodal perception. It includes a perception unit layer deployed within the residence and at key locations to collect real-time data on resident status and environmental parameters; a control layer unit, the core decision-making unit of the system, acting as a central node connecting the perception unit layer, execution layer unit, cloud platform, and user interaction layer unit; an execution layer unit composed of independent execution units distributed across various functional areas of the residence, subject to unified scheduling by the control layer unit; a cloud platform deployed on the Internet, communicating bidirectionally with the control layer unit via MQTT / HTTP protocols; and a user interaction layer unit supporting multi-terminal collaboration, allowing users to interact with the system through terminals. This invention, through a closed-loop "perception-decision-execution" system and an edge-autonomous architecture, achieves adaptive adjustment of the residential environment and precise early warning of resident health and safety, significantly improving residential safety and management efficiency.

Description

Technical Field

[0001] This invention relates to a smart home control system and method, particularly a smart home control system and method based on edge computing and multimodal perception for elderly safety care, belonging to the field of smart home and elderly health technology. Background Technology

[0002] In recent years, with the acceleration of global population aging, the proportion of the elderly population has continued to rise. According to data from the National Bureau of Statistics, by the end of 2025, my country's population aged 60 and above will exceed 300 million, accounting for over 21% of the total population, of which more than 50% will live alone or in empty nests. At the same time, the home safety and health monitoring of the elderly have become major social issues of concern. Studies show that falls are the leading cause of accidental death among people aged 65 and above, with approximately 30% to 40% of the elderly experiencing at least one fall each year. If they do not receive timely assistance after a fall, their mortality rate will increase significantly. Furthermore, chronic diseases such as cardiovascular and cerebrovascular diseases and Alzheimer's disease are highly prevalent among the elderly, often manifesting as behavioral abnormalities (such as nocturnal wandering, prolonged periods of inactivity, and sudden changes in toilet habits). Early identification of these abnormalities is crucial for effective intervention.

[0003] However, the security monitoring functions of most traditional residential or so-called "smart home" systems currently widely used in the market are still limited to basic levels such as magnetic door and window switches, smoke detectors, and gas leak alarms. They lack the refined perception of the daily behaviors of the elderly (such as standing, walking, sitting, and lying down), and are even less able to identify high-risk abnormal events such as falls, wandering, or prolonged inactivity. Even some high-end smart home products mainly rely on wearable devices (wristbands, pendants) or a single camera for monitoring. Wearable devices suffer from low user acceptance, forgetting to wear them, and running out of power, resulting in blind spots in monitoring; single cameras are limited by factors such as changes in light, obstruction, and privacy (the elderly are unwilling to install cameras in bathrooms and bedrooms), resulting in high false alarm and false negative rates, and the video upload to the cloud poses a serious privacy risk.

[0004] In terms of health monitoring, there is a lack of systematic and automated recording and analysis methods for physiological and behavioral indicators of the elderly, such as weight changes, toilet frequency, daily activity levels, and sleep quality. For example, increased nocturia and prolonged toilet time in the elderly are often early signs of urinary tract diseases or heart failure, but traditional homes cannot automatically capture such subtle changes. Family members or community doctors can only rely on subjective review, missing the best intervention opportunity. In addition, the problem of individual data confusion is prominent in multi-member households. Existing smart toilets, smart mattresses, smart cameras, and other devices are produced by different manufacturers and lack data interoperability and linkage mechanisms, forming typical "data silos." Family members need to open multiple apps to obtain fragmented information, making it impossible to form a unified health management view, let alone achieve cross-device collaborative early warning.

[0005] The lack of edge computing capabilities is a major weakness of current technologies. Most smart home devices employ a centralized architecture of "terminal data collection + cloud decision-making," where sensors collect data and upload it to a cloud server. The cloud then runs complex models to analyze the data and make decisions before issuing commands to the terminal for execution. This architecture places extremely high demands on network connectivity stability and bandwidth. If the home network fails, the cloud server is under maintenance, or the operator's network becomes congested, the devices often become "deaf," unable to make localized decisions or operate autonomously. For elderly people living alone, network interruptions may be precisely when they need the most monitoring (for example, if an elderly person falls and cannot reconnect to the network). This strong reliance on the cloud significantly reduces system availability and can even create safety hazards.

[0006] Low multimodal perception fusion and insufficient individual identification capabilities are another major technical obstacle. In multi-member family scenarios (such as elderly people living with their children), existing technologies struggle to accurately distinguish behavioral data and health indicators from different individuals. Some solutions attempt to use single identification methods (such as relying solely on RFID tags or visual recognition), but due to the limitations of each technology (RFID tags may be lost or forgotten, and visual recognition is greatly affected by lighting, angle, and occlusion), single identification schemes are prone to identification failures or data confusion in practical applications. The lack of an effective multimodal fusion identification mechanism makes it impossible to accurately establish individual health records, severely restricting the practicality of health monitoring and safety early warning functions.

[0007] For unusual behaviors unique to the elderly (such as falls, staying out of bed at night, prolonged lingering in the bathroom, and wandering around), existing technologies generally lack dedicated detection models and alarm mechanisms. Most smart cameras can only detect movement and cannot distinguish between normal activity and falls; some wearable devices with fall detection functions require the elderly to actively press the alarm, but in actual falls, the elderly often cannot react proactively. In addition, traditional solutions lack acoustic auxiliary recognition (such as the sound of a fall, cries for help, or abnormal coughing), resulting in a high false negative rate and failing to achieve "unobtrusive, 24 / 7" protection.

[0008] Finally, product design is out of touch with the usage habits of the elderly. Existing smart home devices are complex to operate, with small fonts and low voice recognition rates, making them difficult for seniors to use independently. Furthermore, privacy protection is inadequate; seniors are extremely resistant to cameras in private spaces such as bedrooms and bathrooms, affecting the actual deployment rate of these devices. During home renovations, additional sensors and devices often detract from the aesthetics of the home, reducing user acceptance.

[0009] In summary, existing technical solutions for home safety monitoring of the elderly have significant drawbacks, including high system architecture coupling, weak edge autonomy, insufficient multimodal perception fusion, lack of abnormal behavior detection, and poor user experience. There is an urgent need for a comprehensive solution that can overcome these technical bottlenecks. Summary of the Invention

[0010] The technical problem to be solved by this invention is to provide a smart home control system and method based on edge computing and multimodal perception, which solves the problem that traditional smart homes lack proactive perception and real-time alarm for abnormal behaviors such as falls by the elderly, while reducing cloud dependence, protecting privacy, and improving system reliability and user experience.

[0011] To solve the above-mentioned technical problems, the technical solution adopted by the present invention is as follows: A smart home control system based on edge computing and multimodal perception includes a perception unit layer, a control layer unit, an execution layer unit, a cloud platform, and a user interaction layer unit, forming a closed-loop edge autonomous architecture of "perception-decision-execution".

[0012] The sensing unit layer is deployed in key locations within the residence (bedrooms, bathrooms, dining rooms, living rooms, etc.) and includes an environmental sensing module, a personnel sensing module, a visual sensing module, and a health sensing module. The environmental sensing module collects parameters such as temperature, humidity, TVOC, CO2, and noise. The personnel sensing module uses gravity sensors, RFID readers, infrared grids, and pressure sensor arrays to achieve identity recognition, weight tracking, entry and exit monitoring, and capture of details of toilet / bathing behavior. The visual sensing module uses a high-definition camera (with a physical cover) and millimeter-wave radar to fuse and run a lightweight behavior recognition model in real time, outputting behavior labels including normal walking, sitting / lying down, falling, wandering, and prolonged stillness. A fall detection algorithm is specifically designed for the elderly (based on joint judgment of human body key point posture estimation, center of gravity falling speed, and contour aspect ratio changes). The health sensing module includes a sound acquisition unit (recognizing cries for help, impact sounds, and abnormal coughs) and an infrared temperature measurement unit (non-intrusive acquisition of body surface temperature). All sensors are synchronized through a unified timestamp, and RFID / vision / gravity data are fused and bound to individual identities. When a certain type of sensor fails, a backup identification scheme is automatically switched. After local preprocessing, the data is transmitted to the control layer via RS485 / IO / Wi-Fi.

[0013] The control layer unit is the core decision-making unit of the system, namely the smart home control all-in-one machine. It integrates a high-performance MCU (such as STM32H743) and an AI edge computing module (NPU / TPU, computing power ≥0.5TOPS), and runs an embedded real-time operating system (FreeRTOS). Its functions include: Multi-source data fusion: Aligning visual, gravity, RFID, pressure, and sound data on the timeline, and constructing a holographic state map of residents through identity fusion and behavior fusion; Localized decision-making: Built-in multi-sensor fusion algorithm, behavior recognition model (MobileNetV3 / TinyYOLO, inference latency ≤50ms) and self-learning model to establish an individual behavior profile for each resident and dynamically update the baseline. In particular, fall detection adopts dual-modal confirmation: visual posture estimation + millimeter-wave radar respiratory frequency change. When multiple consecutive frames detect a human body with a horizontal angle <30° and a vertical velocity of the center of mass >1.5m / s, a level one warning is triggered. At the same time, the alarm level is increased by combining sound recognition of impact sound or cry for help. Abnormal behaviors (such as staying in the bathroom for more than 30 minutes, not returning to bed for more than 20 minutes at night, or wandering in the same area for more than 5 minutes) are automatically judged based on the individual baseline. Command issuance and scheduling: Based on the decision results, control commands are issued to the execution layer via Modbus / UART / Wi-Fi and feedback is received, forming a closed loop; Data caching and synchronization: The local MicroSD card stores the data of the last 30 days in a loop (timestamp, sensor type, value, resident ID, event type), supports resume transmission after network interruption (queue + compression), and reports after network recovery; Remote communication management: Communicates bidirectionally with the cloud via MQTT / HTTP (TLS1.2 encryption) to receive configurations, rules, firmware upgrades, and synchronize device status.

[0014] The execution layer unit comprises five major regional modules: Residential area implementation units: micro DC inverter air conditioner, activated carbon fresh air system, intelligent temperature control floor, which work together to regulate temperature, humidity and air quality; Dining area execution units: intelligent gas stove (anti-dry burning, automatic gas shut-off), intelligent range hood (linked airflow adjustment), intelligent dining table (heating and heat preservation, tableware recognition, meal record), nutrition voice reminder module, the system recommends recipes based on health data and reminds you to take medicine and drink water; Bathroom area control units: smart toilet / smart toilet seat (seat heating, warm water washing, warm air drying, automatic opening and closing of the lid, built-in pressure sensor and urine routine detection module), automatic cleaning and disinfection unit (UV ultraviolet light + spray), anti-fall handrail (built-in proximity sensor to detect abnormal force), emergency pull cord alarm (low-position installation), automatic disinfection and flushing after the resident leaves, and voice inquiry and care for long-term use. Entertainment area execution units: smart voice speaker (supports opera, news, audiobooks, and dialogue interaction), large-font smart TV / projector (high contrast interface, volume compensation), companion interaction system (actively plays music or simple questions and answers after emotion recognition, reminds you to get up and move around), speaker (timed reminders for taking medication / activity). Safety zone execution unit: alarm light (RGB three colors, red for emergency / yellow for alert / blue for normal), emergency help trigger unit (wall-mounted button + wearable remote control button), remote notification module (multiple notifications via WeChat, SMS, and telephone, can automatically dial 120 and preset relatives). Upon detecting a fall or high-risk abnormality, a local audible and visual alarm is immediately triggered, and alarms are pushed to children, community doctors, and property management via the cloud, supporting an escalation mechanism for unconfirmed alerts.

[0015] The cloud platform provides equipment management, an AI application engine (continuously optimizing models through federated learning), big data processing (generating weekly / monthly health reports from time-series databases and marking fall risk trends), business services (emergency assistance, online consultations, and contactless social interaction), and user management (centralized management of multiple residences and shared permissions among family members).

[0016] The user interaction layer unit includes a computer terminal (Web management backend, batch device management, data statistical analysis, multi-level permissions), a mobile terminal (WeChat mini program / APP, real-time monitoring of falls with automatic pop-up windows, remote control, alarm reception, health reports, graphical rule configuration), and a printer (optional output of health reports, QR code labels, configuration lists).

[0017] This invention also provides a control method for a smart home control system based on edge computing and multimodal sensing, comprising the following steps: S1. Multi-source data acquisition: The perception layer collects environmental data, resident identity ID, weight, behavioral video streams, and audio in real time; S2, Edge-side data fusion: The control unit matches RFID identity with gravity and visual data to confirm individual profiles; S3. Localized Decision Reasoning: The AI ​​module calls a self-learning model to analyze eating duration, toilet frequency, weight changes, activity trajectory, and posture; Fall Detection Logic: When the vision module detects a sudden downward shift of key human body points and a center of gravity acceleration > 9.8 m / s², or when the millimeter-wave radar detects a sudden change in respiratory rate and a human body horizontal height below half of the normal standing height, a fall event is determined by fusion; if an impact sound or a cry for help is detected at the same time, the alarm level is increased; Abnormal behaviors (prolonged stillness, wandering, failure to return to bed at night) are combined with individual timeout thresholds for determination; S4. Command Issuance and Execution: If the environmental standard is exceeded, adjust the fresh air / temperature control; if a fall or abnormal behavior occurs, trigger a local alarm + remote push, and automatically start the emergency help process (call 120, notify relatives). S5. Closed-loop feedback: After execution, the perception layer collects data again and feeds it back to the control layer to correct the model and record the event log.

[0018] Compared with the prior art, the present invention has the following advantages and effects: Edge autonomy and rapid response: Built-in AI edge computing module enables local data processing and decision-making, and can maintain basic environmental control and security monitoring even when the network is down, making it especially suitable for elderly people living alone. Multimodal fusion for accurate identification: Visual, gravity, RFID, pressure, and sound are fused together to achieve an individual identification accuracy of ≥98% and a fall detection sensitivity of ≥95%, effectively avoiding missed and false alarms; Proactive early warning, closed-loop system: It automatically adjusts the environment, cleans the bathroom, and warns of health risks. It can also proactively identify falls and abnormal behaviors and issue tiered alerts, realizing the transformation from "passive treatment" to "proactive prevention". Modular design, senior-friendly: Modular execution unit, furniture-like appearance, large-font operation interface, voice control, support for physical buttons and pull cord alarm, reducing the threshold of use; Privacy protection and security: The camera is physically covered, the original video is not uploaded for local analysis, and the data is transmitted in encrypted form, eliminating the elderly's privacy concerns in private spaces. Attached Figure Description

[0019] Figure 1 This is a schematic diagram of an intelligent home control system based on edge computing and multimodal perception according to the present invention.

[0020] Figure 2 This is a schematic diagram of the structure of an intelligent home control system based on edge computing and multimodal perception according to the present invention. Detailed Implementation

[0021] To illustrate in detail the technical solutions adopted by the present invention to achieve the intended technical objectives, the technical solutions in the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, not all embodiments. Furthermore, the technical means or technical features in the embodiments of the present invention can be replaced without creative effort. The present invention will be described in detail below with reference to the accompanying drawings and embodiments.

[0022] like Figure 1 and Figure 2 As shown, the present invention discloses a smart home control system based on edge computing and multimodal perception. It adopts a layered and decoupled modular design, with each layer interacting with the others via standardized communication interfaces to form a complete closed-loop control chain of "data acquisition—edge decision-making—command execution—feedback optimization." Specifically, it includes a perception unit layer, a control layer unit, an execution layer unit, a cloud platform, and a user interaction layer unit.

[0023] I. Detailed Implementation of the Perception Unit Layer The sensing unit layer is deployed in key locations within the residence, including bedrooms, bathrooms, dining rooms, living rooms, and corridors. It is functionally divided into environmental sensing modules, personnel sensing modules, visual sensing modules, and health sensing modules.

[0024] 1. Environmental Perception Module Temperature and humidity sensors: Employs an SHT30 digital sensor, measuring range -40℃ to 85℃ with an accuracy of ±0.5℃; humidity 0 to 100%RH with an accuracy of ±3%RH. Installed in the bedroom ceiling (measuring overall temperature) and under the elderly person's bed (measuring the local microenvironment), with a sampling frequency of 1Hz. When the bedroom temperature is detected to be below 18℃ (the lower limit of comfort for the elderly) and the elderly person is resting, the system automatically activates the air conditioner's heating mode or an electric heater; when the summer temperature exceeds 28℃, cooling is activated; and when the humidity exceeds 70%, dehumidification or fresh air exchange is activated.

[0025] Air quality sensor: Employs the SGP30 multi-functional sensor to detect TVOC (0~1000ppb) and eCO2 (400~2000ppm). Installed on the bathroom ceiling and at the bedroom vent. When bathroom TVOC > 500ppb, the negative pressure deodorization device and exhaust fan automatically activate; when bedroom eCO2 > 1200ppm, the fresh air system activates to introduce fresh air, preventing dizziness or increased cardiopulmonary burden due to oxygen deficiency in the elderly.

[0026] Noise sensor: Employs a MAX4466 condenser microphone module, detecting noise levels from 30 to 100 dB at a sampling frequency of 10 Hz. Installed in the living room and bedroom. At night (10:00 PM - 6:00 AM), if continuous noise exceeding 50 dB is detected and identified as an abnormal sound (such as falling objects or elderly people groaning), the event is recorded and the air conditioner / fresh air system is reduced to minimize noise disturbance to sleep. Simultaneously, noise data assists in fall detection: if an instantaneous impact sound greater than 70 dB is detected and there is no subsequent voice response within 10 seconds, the fall verification process is triggered.

[0027] 2. Personnel Sensing Module Gravity sensor: Employs an HX711+C3 grade strain gauge sensor with a measuring range of 150kg and an accuracy of ±20g. It is embedded under restaurant chairs and under the cushions of reclining chairs / sofas frequently used by the elderly. The system uses a dynamic weighing algorithm: when the weight change exceeds 500g, data is continuously collected for 10 seconds, and the stable average is taken as the effective weight. Each weighing result is linked to the elderly person's identity identified by RFID or visual recognition, automatically recording the weight change curve. If the weight decreases by more than 5% within a week or shows a significant daily downward trend, the system triggers a malnutrition warning and suggests increasing nutritional intake or seeking medical attention via a mobile app.

[0028] RFID Reader: Employs an RC522 module (13.56MHz) to read ISO15693 standard passive RFID tags. Tags can be integrated into senior wristbands, keychains, or sewn onto clothing. The reader is installed on bedroom door frames, bathroom door frames, and dining room entrances. When an elderly person passes through, the reader reads their unique ID and combines this with the direction of the infrared grid to determine entry or exit. If an elderly person is detected entering the bathroom and not exiting for more than 30 minutes without any RFID record of leaving, it is determined that they may have fallen or are unwell, triggering a voice inquiry and remote alarm.

[0029] Infrared grid sensor: Utilizing an E3F-DS30C4 through-beam infrared grid, installed at the entrance of each functional area, forming 2-4 pairs of infrared beams. The direction (entry / exit) is determined by the order in which the beams are blocked, and the timestamp of each trigger is recorded. The system tracks the frequency and duration of the elderly person's daily entry and exit from each area. For example, if an elderly person's nighttime urination frequency suddenly increases from 1 to 4 times, and each time they stay in the bathroom for more than 20 minutes, sound recognition is used to determine if there is a possible urinary tract infection or early signs of heart failure.

[0030] Pressure sensor array: Employing thin-film pressure sensors (40cm×40cm, 4×4 dot matrix), these sensors are installed on the toilet seat, bathtub bottom, or under the anti-slip mat in the shower area. The sampling frequency is 20Hz, collecting pressure distribution characteristics. An AI model (running on the control unit's NPU) identifies the elderly person's toilet posture (sitting, leaning forward, leaning back), movement type (sitting down, standing up, while using the toilet), and whether a slip has occurred (instantaneous pressure fluctuations with prolonged inactivity). If the system detects sustained pressure on the toilet seat for more than 5 minutes without any slight movement (potentially indicating difficulty defecating or sudden fainting), it automatically asks via voice, "Are you alright? Do you need help?" Three unanswered questions trigger an emergency alarm.

[0031] 3. Visual Perception Module The visual perception module balances privacy protection and security monitoring, employing a fusion solution of high-definition camera and millimeter-wave radar.

[0032] High-definition camera: Employs an OV5640 camera module (1080P resolution, 120° wide-angle, supports infrared night vision). Installed in the living room, hallway, dining room, and bedroom (facing the bed but avoiding dressing areas). The camera module features a physical cover driven by a stepper motor. When the elderly person is at home and the "active monitoring" mode is not activated, the cover is closed by default, physically blocking the lens to ensure privacy. When the system detects abnormal events (such as not returning home late at night, suspected fall) or when children request remote viewing via the app, the cover can only be opened after confirmation by the elderly person's button or authorization from a preset emergency contact. The video stream undergoes local AI analysis; the original video is not uploaded, only structured data (such as "elderly person walks from the living room to the bathroom" and "fall detection confidence level 0.92") is transmitted to the cloud, preventing privacy leaks at the source.

[0033] Millimeter-wave radar: Employing an IWR6843 60GHz radar sensor, installed high on the ceiling or wall, it can penetrate non-metallic materials such as plastic and wood, and is unaffected by light. The radar continuously outputs the elderly person's location coordinates (X, Y), altitude (Z), movement speed, and respiratory rate. When the camera cover is closed, the radar acts as the primary sensor to monitor the elderly person's activity. The stability of respiratory rate is an important indicator for assessing sleep quality and health status: if the elderly person's respiratory rate suddenly increases to over 25 breaths / minute at night and persists for 10 minutes, it may indicate fever or cardiopulmonary abnormalities; if the respiratory rate is below 8 breaths / minute, combined with immobility, it may indicate respiratory arrest or coma, and the system will immediately alarm.

[0034] The specific implementation of the fall detection algorithm: The control unit runs a lightweight posture estimation model based on PoseNet (approximately 2MB after quantization) and processes the camera image in real time at 25fps on the NPU. Extract the key points of the human body: left / right shoulder, left / right hip, left / right knee, left / right ankle. Calculate the following features: (1) Angle between the main axis of the human body and the horizontal plane: connect the midpoint of the left and right hips and the midpoint of the left and right shoulders, and calculate the angle θ between it and the horizontal line. When θ < 30°, it is judged to be close to horizontal (fall or lie down); (2) Velocity of descent of the center of mass: the rate of change of the vertical coordinate of the midpoint of the hip v_y. If v_y > 1.5m / s (free fall feature) and the subsequent θ is < 30° for more than 1 second, it is judged to be a fall; (3) Aspect ratio of the human body bounding box: the aspect ratio is usually > 1.5 when falling (< 0.8 when standing normally). When three consecutive frames simultaneously satisfy θ < 30° and v_y > 1.5 m / s, or the aspect ratio > 1.5 and the radar detects a height below 0.5 m, a Level 1 fall warning is triggered. Simultaneously, if the sound acquisition unit detects an impact sound (spectral characteristic impact type) or a cry for help (keywords such as "help," "help me," etc.), it immediately increases the fall confidence and directly executes the alarm, without waiting for multi-frame visual confirmation. After fall confirmation, the system performs the following actions: a high-volume alarm (85dB, playing "Elderly person has fallen, please help check") is emitted locally, and an emergency message with timestamps and location is pushed to the children's mobile phones, property management, and community elderly care station via the cloud; if the children's app does not respond within 30 seconds, the system automatically dials 120 (using pre-set voice synthesis to explain the address and situation) and unlocks the door (smart lock) to facilitate emergency personnel entry.

[0035] 4. Health Sensing Module Sound acquisition unit: omnidirectional microphone (sensitivity -38dB, signal-to-noise ratio 65dB), sampling rate 16kHz. Deployed at the head of the bed in the bedroom and on the ceiling of the bathroom. Running a lightweight CNN sound recognition model (MFCC features + classifier), it can identify the following acoustic events related to the elderly: (1) cries for help ("Help me", "Help me", "Someone come here"); (2) fall impact sound (short-term high-energy impact); (3) abnormal cough (more than 5 consecutive dry coughs, frequency characteristics consistent with respiratory diseases); (4) vomiting sound; (5) groaning sound (continuous low frequency, possibly pain or discomfort). After identifying the above events, the system automatically records and triggers the corresponding alarm.

[0036] Infrared temperature measurement unit: Employs an MLX90614 non-contact infrared temperature sensor, installed above the bed in the bedroom (50-100cm from the bed surface, 90° field of view). It automatically measures body surface temperature (forehead or cheek) between 3:00 AM and 5:00 AM daily (during deep sleep) or after detecting that the elderly person has entered the bed and remained still for more than 5 minutes. Continuous recording generates a temperature curve. When the body temperature exceeds 37.5℃ or remains below 36.0℃ for more than 24 hours, an abnormal temperature warning is triggered, alerting family members or community doctors. Combined with cough detection, it can indicate the risk of respiratory infection.

[0037] Perception layer collaboration mechanism: All sensor data is aggregated to the control unit via RS485 bus or GPIO, and uniformly timestamped with NTP synchronization. When RFID identifies an elderly person (e.g., entering a restaurant), the system binds the gravity data (weight), camera posture data, and infrared grid dwell time for the next 30 seconds to that identity. If the RFID fails (forgot to wear it), the visual model infers the identity through face / gait recognition and temporarily stores it in a buffer for manual verification. Multi-sensor data is fused using Kalman filtering to reduce the impact of single noise.

[0038] II. Detailed Implementation of the Control Layer Unit The control unit features an aluminum alloy casing, measuring 150×100×40mm, and is mounted on a DIN rail for installation in a low-voltage box or cabinet. The main controller is an STM32H743 (480MHz), and the AI ​​coprocessor is a GAP9 (supporting INT8 quantization computing power of 1 TOPS). Storage includes 16MB SPI Flash firmware + a 256GB MicroSD card (with 30-day cycle overwrite). Communication includes dual RS485 (Modbus RTU), one Ethernet channel, dual-band Wi-Fi (2.4 / 5GHz), and an optional 4G CAT1 module. A 12V 20Ah lithium battery provides ≥4 hours of sensing and communication after a power outage.

[0039] The system software is based on FreeRTOS and is divided into the following tasks: Data acquisition task: Poll each sensor (period can be configured), and prioritize event-triggered events (such as infrared grid interruption).

[0040] Fusion and Inference Task: Triggered every 50ms, this task fuses all sensor data from the most recent 50ms and runs the behavior recognition model. The fall detection subtask runs at a synchronized camera frame rate of 25fps. When a suspicious fall is detected, it retains 10 seconds of radar and audio data before and after the fall and encapsulates it into a structured event.

[0041] Rule engine task: Maintain the condition-action mapping table (user-defined is supported). For example: Conditions: The toilet pressure sensor array detects toilet seat pressure for more than 15 minutes AND the infrared grid shows no sign of leaving → Action: Ask a voice message "Do you need help?" Wait 10 seconds, and if there is no response, trigger a level 2 alarm (push notification to children's app).

[0042] Conditions: Bedroom millimeter-wave radar detects a respiratory rate >25 breaths / minute AND body temperature >37.3℃ → Action: Push a notification "Suspected fever, it is recommended to measure body temperature".

[0043] Conditions: Nighttime (22:00-06:00) Bed pressure sensor shows >20 minutes away from bed AND bathroom infrared sensor does not trigger → Action: Turn on night light, activate corridor camera (cover temporarily opened) to capture one image (encrypted and stored locally), and send "Nighttime Abnormal Bed Away" alarm to children.

[0044] Communication task: The MQTT client maintains a persistent connection with the cloud, and uploaded data uses LZ4 compression. When the network is down, data is written to the local SQLite database and marked as out of synchronization; data is uploaded in batches upon network recovery.

[0045] Detailed implementation of the self-learning model: The system calculates behavioral statistics for each elderly person every 10 days as a cycle. Meal duration: mean μ_eat, standard deviation σ_eat; toilet frequency: mean μ_toilet, standard deviation σ_toilet; nighttime urination frequency: mean μ_night; daytime activity steps (cumulative from millimeter-wave radar trajectory): mean μ_steps; sleep duration: mean μ_sleep.

[0046] If new data exceeds μ±2σ, it is marked as abnormal and a corresponding warning is triggered. For example, if an elderly person usually goes to the toilet 6 times a day, and then goes to the toilet 10 times for 3 consecutive days, each time accompanied by the sound of difficulty urinating (sound recognition), the system will automatically prompt "Abnormal increase in toilet frequency, it is recommended to check urine routine".

[0047] III. Detailed Implementation of Execution Layer Units 1. Dining Area Implementation Unit Intelligent gas stove: Utilizes a top-intake airflow design, built-in thermocouple flameout protection, and a temperature probe (detecting the pot's bottom temperature). If the pot's bottom temperature exceeds 280℃ and no one operates the knob within 30 seconds, it is determined to be dry-burning. It automatically shuts off the gas supply solenoid valve and issues a voice alarm: "Pot is dry-burning, gas turned off." Simultaneously, an alarm is pushed to the cloud. This function can prevent fires if elderly people forget to turn off the stove.

[0048] Smart range hood: Linked to the gas stove, the range hood automatically starts when the gas stove is turned on, adjusting the airflow automatically based on temperature and smoke sensors. It automatically shuts off 5 minutes after the gas stove is turned off.

[0049] Smart dining table: The tabletop integrates gravity sensors (under each seat) to automatically record the start and end times of meals and estimate intake (based on changes in the weight of bowls and plates). The system recommends low-sugar, low-salt recipes based on the elderly person's health data (such as diabetes or hypertension) and reminds them via the display screen on the table or voice prompts, "Grandpa Wang, please eat your vegetables first, then your staple food." If the system detects that the elderly person's lunch time is later than 1:00 PM and they have not taken their medication, it will remind them, "You have not taken your blood pressure medication yet; please take it after your meal."

[0050] Nutrition reminder module: Based on the weight change curve and the doctor's dietary recommendations, it will proactively remind you that "you have lost a lot of weight recently, and it is recommended to increase your protein intake."

[0051] 2. Bathroom area execution unit Smart toilet: Equipped with a seat pressure sensor array (4×4), a urinalysis module (measuring urine glucose, urine protein, and pH value via test strips or optical reflection), and an automatic flushing and drying module. When an elderly person uses the toilet, the seat automatically heats to 38℃; after use, it automatically performs a warm water rinse and warm air drying. Urinalysis data is uploaded to the cloud to generate weekly reports. A persistently positive urine glucose test indicates poor diabetes control; a positive urine protein test indicates a risk of kidney damage.

[0052] Automatic cleaning and disinfection unit: After the elderly person leaves the bathroom, the system delays for 1 minute before turning on the UV-C ultraviolet lamp (6W, wavelength 254nm) to irradiate the toilet bowl and seat surface for 5 minutes. At the same time, the spray cleaning mechanism (hypochlorous acid disinfectant) is activated to automatically spray and disinfect the floor and walls. The entire cleaning process is completed automatically without the elderly person's intervention.

[0053] Anti-fall grab bars: Installed on both sides of the toilet and on the walls of the shower area, the grab bars are equipped with capacitive proximity sensors (to detect whether a hand is gripping the bar) and strain sensors (to detect the magnitude of force). If the system detects that an elderly person is gripping the grab bar but the force suddenly increases (potentially indicating a slip), the system will vibrate the end of the grab bar and provide a voice prompt, "Please hold onto the grab bar, I'm here to help you," while simultaneously notifying family members. If the force disappears instantly and the radar detects a fall within 5 seconds, an alarm will sound immediately.

[0054] Emergency pull cord alarm: A red pull cord is installed 20cm above the ground next to the toilet. An elderly person can pull it with their hand or foot after falling. Once triggered, the system immediately enters the highest level of alarm: an alarm sounds locally, automatically dials 120 (using a pre-recorded address and description of the patient's condition), and simultaneously sends phone calls and text messages to all emergency contacts.

[0055] 3. Entertainment Area Execution Unit Smart voice speaker: Features a wooden casing, adjustable volume up to 80dB, and supports dialect recognition (Sichuan dialect, Shandong dialect, etc.). Seniors can say, "Xiao Zhi, play Peking Opera," "Today's weather," or "Call my son." The speaker also provides timed reminders such as, "10 AM, please go to the balcony to get some sun," and "3 PM, it's time to eat fruit."

[0056] Large-screen smart TVs: 55 inches and above, featuring a minimalist operating system, 70×70mm tile icons, and a high-contrast black and white color scheme. The remote control has only six buttons: power, volume + / -, channel + / -, voice, and SOS. When an elderly person presses and holds the "SOS" button for 2 seconds, the system directly triggers an emergency alarm, similar to the function of a pull cord in the bathroom.

[0057] Companionship and Interaction System: Based on emotion computing, the system uses a camera (authorization is required to open the cover) to recognize the elderly person's facial expressions (sadness, anxiety, calmness) and tone of voice. If it detects that the elderly person has been sitting still on the sofa for a long time and has a depressed expression, the system will proactively play the elderly person's favorite program (based on historical records) or initiate a conversation: "Grandma Zhang, the weather is very nice today, let me tell you a joke." At the same time, it will send a reminder to the children's mobile phones: "Feeling depressed, video call recommended." If the elderly person has a history of falls, it will also remind them: "You have been sitting for an hour, stand up and move around to prevent blood clots."

[0058] 4. Safe Zone Execution Unit Tiered alarm mechanism: Level 3 Alarm (Notification): Environmental parameters exceed limits, the person has been out of bed for ≤20 minutes overnight, slight weight fluctuations, etc. Only an app notification will be sent; no audio or visual alerts will be triggered.

[0059] Level 2 Alarm (Warning): This includes situations such as toilet time exceeding 30 minutes, coughing lasting 10 minutes, or abnormal body temperature. A local voice prompt will ask, "Do you need help?", and notifications will be sent to the app and via SMS. If there is no response within 30 seconds, the alert level will be upgraded to Level 1.

[0060] Level 1 Alarm (Emergency): Confirmed fall, call for help, pull cord trigger, or absence from bed for more than 20 minutes at night without any mobile signal. Immediately activates a local high-decibel siren (85dB) and flashing red alarm light; automatically dials 120 emergency services (voice announcement: "An elderly person has fallen in apartment XX, building XX, XX, requesting emergency medical assistance"); dials preset children's mobile phones (looping until answered); unlocks the smart door lock (if configured) to facilitate emergency personnel entry; and sends a work order to the community elderly care station.

[0061] Remote notification module: Supports three channels: WeChat template messages, SMS, and telephone. Children can set "Do Not Disturb" periods in the APP (e.g., from 22:00 to 07:00 at night, only emergency alarms can interrupt).

[0062] IV. Detailed Implementation of Cloud Platform and User Interaction Layer The cloud platform employs a microservices architecture, deployed on Alibaba Cloud or Tencent Cloud. The time-series database InfluxDB stores minute-level aggregated sensor data (approximately tens of GB per user per year). The AI ​​application engine uses federated learning every two weeks to aggregate anonymized user data and optimize the behavior recognition model (especially the generalization ability for fall detection for elderly users of different body types). Users can view multiple residences (e.g., parents' home, in-laws' home) in batches through the web management backend. The mobile app supports iOS / Android; key pages include: Homepage: Displays the elderly person's real-time status ("At home, active" or "Spent 12 minutes in the bathroom"); if an abnormality occurs, the card turns red and is pinned to the top.

[0063] The monitoring page displays the current ambient temperature, humidity, and air quality; the elderly person's location (living room / bedroom / bathroom / dining room). Clicking "Request Screen" requires the elderly person to press a confirmation button on their device (via a speaker or wall button) to open the camera's live feed, ensuring privacy.

[0064] Health Reports: Automatically generates weekly / monthly reports, including weight change curves, toilet frequency statistics, sleep quality (based on radar respiratory rate and number of times getting out of bed), step count, body temperature trends, and summaries of abnormal events. Children can share the reports with their doctors.

[0065] Rule configuration: Provides a graphical "if-then" editor, such as "If [falls down] then [call 120] [notify all children] [unlock the door]".

[0066] V. Application Examples of Control Methods Taking "Grandpa Li," an elderly man living alone, as an example, here are some typical events that occur during a day when the system is running: 7:30 AM: Grandpa Li gets up. The mattress pressure sensor detects him getting out of bed, and the system automatically turns on the hallway night light (at low brightness) and announces, "Good morning, Grandpa Li. The temperature today is 18℃. Remember to wear a coat." Millimeter-wave radar detects him walking slowly in the bedroom; everything is normal.

[0067] 08:00: Grandpa Li entered the restroom, and his identity was detected by RFID. During his toilet use, pressure sensors and the toilet seat array recorded data for 8 minutes. The system detected no abnormalities.

[0068] 08:15: Grandpa Li entered the restaurant. The gravity sensor recorded his weight as 68.5kg (a decrease of 0.2kg from yesterday, within the normal fluctuation range). The smart dining table recorded a meal duration of 15 minutes, indicating normal intake. A voice reminder appeared: "Please take your blood pressure medication."

[0069] 10:30: Grandpa Li was watching TV in the living room when the server sent a notification: "It's 10:30 AM, please stand up and move around." He got up and walked to the balcony, where the millimeter-wave radar detected his walking speed as 0.8 m / s (normal).

[0070] 13:00: Afternoon nap. Camera cover closed. Radar detected respiratory rate of 16 breaths / minute, stable.

[0071] 15:00: Grandpa Li re-entered the bathroom but unexpectedly slipped and fell in the shower area. Millimeter-wave radar detected a rapid drop in height from 1.6m to 0.3m, accompanied by an increased breathing rate; the sound acquisition unit captured the impact sound and an "ouch." Although the visual camera cover was closed, the system immediately triggered a Level 1 fall alarm based on radar and sound. The local speaker announced, "Grandpa Li, are you alright? Your son and 120 have been notified." His children's phones received a pop-up message: "Fall incident, address xx, 120 has been called." Simultaneously, the integrated device automatically dialed 120 and played the address information. The ambulance arrived within 15 minutes, and thanks to timely treatment, Grandpa Li was unharmed.

[0072] Afterwards, the system uploaded the video footage of the fall (only the locally stored radar heatmap and sound waveform, with no private footage) to the cloud for model optimization, further improving the accuracy of fall detection.

[0073] As can be seen from the above specific implementation methods, the present invention constructs a complete and implementable smart home control system for the elderly, which has advanced features such as edge autonomy, multimodal fusion, accurate fall detection, abnormal behavior alarm, and proactive health intervention. It effectively solves various problems existing in the prior art and significantly improves the home safety level and quality of life of the elderly.

[0074] A control method for a smart home control system based on edge computing and multimodal sensing includes the following steps: S1, Multi-source data acquisition The sensor groups in the perception layer collect residential environmental data (temperature, humidity, TVOC, CO2, noise), resident identity ID (RFID / vision), weight data (gravity sensor), behavioral video stream (high-definition camera), and acoustic characteristics (microphone array) in real time. At the same time, the millimeter-wave radar continuously outputs the resident's location, activity trajectory, and breathing frequency.

[0075] S2, edge-side data fusion The control unit's integrated machine receives multi-source data and uses a multi-sensor fusion algorithm to match the RFID-identified identity ID with the weight data collected by the gravity sensor. It then combines this with the posture estimation results from the visual perception module and the trajectory characteristics from the millimeter-wave radar to confirm the resident's individual profile. When the RFID tag fails (forgotten or lost), the system automatically invokes a visual recognition model (face / gait) for identity inference and correlates it with gravity data, ensuring an accuracy rate of ≥98% for individual data attribution.

[0076] S3, Localized Decision Reasoning The AI ​​edge computing module (NPU / TPU, computing power ≥ 0.5 TOPS) calls the pre-built self-learning behavior model and rule engine to perform localized analysis: The system establishes an individual behavioral baseline profile for each resident, including: baseline weight (weekly average), baseline toilet frequency (day / night), baseline eating duration, baseline number of times the resident gets out of bed at night, baseline number of steps taken, etc., and updates it dynamically (every 10 days, using μ±2σ for anomaly detection).

[0077] Health abnormality assessment: When the gravity sensor detects that the weight has dropped by more than 5% in 7 days or fluctuates more than a threshold (e.g., ±2kg) in a single week, and the visual posture assessment shows that the mental state is depressed, the AI ​​generates a health risk rating of "malnutrition / sudden weight loss".

[0078] When an infrared grid or pressure sensor array detects an abnormal increase (more than baseline + 2σ for 3 consecutive days) or a decrease (less than baseline - 2σ) in the frequency of urination, combined with sound recognition (groaning or coughing due to difficulty urinating), the system determines it as a risk of "urinary / digestive system abnormality".

[0079] When millimeter-wave radar detects that someone has been out of bed for more than 20 minutes at night and has not returned, and there is no record of them entering or leaving the bathroom, it is determined to be a risk of "abnormal nighttime departure from bed" by combining data from infrared grid sensors.

[0080] Safety risk assessment (fall detection): The vision module runs a lightweight pose estimation model (PoseNet) to calculate human key points in real time. If three consecutive frames detect that the angle between the human body's main axis and the horizontal plane is <30° and the vertical velocity of the center of mass is >1.5m / s (fall characteristics), or if the millimeter-wave radar detects that the human body's height suddenly drops below 0.5m and the breathing rate changes abruptly, and the sound acquisition unit identifies an impact sound or a cry for help, then the multimodal fusion determines it as a fall event, and the risk rating is directly set to the highest level.

[0081] Environmental anomaly assessment: The rule engine makes logical judgments based on environmental sensor data and preset thresholds, such as "Bedroom temperature <18℃ and the elderly are resting → start the air conditioner for heating", and "Bathroom TVOC >500ppb → start negative pressure deodorization".

[0082] When the risk rating exceeds the set level (Level 1 / Level 2 / Level 3), the system will automatically trigger a local alarm (voice inquiry, sound and light prompt) and push an abnormality report to the preset emergency contacts (children, community doctor, property management) via WeChat mini program, SMS or telephone.

[0083] S4. Command Issuance and Execution If the environmental parameters are determined to be out of standard (such as excessively high temperature or poor air quality), the all-in-one unit sends a control command to the execution layer to start the fresh air system, air conditioner, temperature-controlled floor or dehumidifier.

[0084] If the resident's behavior is determined to be abnormal (such as taking more than 30 minutes to use the toilet, or remaining still and unresponsive for a long time), a voice care message will be triggered: "Are you okay? Do you need help?" If there is no response after three attempts, the message will automatically escalate to an alarm.

[0085] If a fall or emergency call for help is detected (triggered by the cord alarm or recognition of a distress signal), the highest level of response will be executed immediately: The local area issued an 85dB siren and the red warning lights flashed; The system sends a "fall / help" alert to all emergency contacts via the communication module (4G / 5G / Wi-Fi) and automatically dials 120 emergency number (including preset address and description of the patient's condition). Open the smart door lock to facilitate access for emergency personnel; Send work orders to community elderly care stations.

[0086] If it is determined that the person left the bed at night and did not return home, the hallway night light will be turned on automatically and a reminder will be sent. At the same time, the bedroom camera will be temporarily turned on (with the physical cover open) to capture a low-resolution image (locally encrypted and stored for verification only).

[0087] S5, Closed-loop feedback After the execution layer unit completes its actions (such as air conditioning adjustment, alarm sending, and cleaning / disinfection completion), the sensing unit layer collects environmental parameters and resident status data again and feeds them back to the control layer. The control unit adjusts the decision model based on the feedback results (for example, if the temperature still does not reach the set value after heating, the air conditioning power is increased; if a fall alarm is falsely triggered, the system automatically stores this feature in the error sample library for subsequent model optimization). All event logs (timestamps, sensor data, decision results, and execution actions) are stored locally for 30 days and synchronized to the cloud for long-term health trend analysis and federated learning model iteration.

[0088] Beneficial effects This invention constructs a complete and implementable integrated smart home control system for elderly home safety, possessing advanced features such as edge autonomy, multimodal fusion, modular expansion, and health early warning, effectively solving various problems existing in current technologies. The specific beneficial effects are as follows: 1. Peripheral autonomy, rapid response By integrating an AI edge computing module into the control unit, local data processing and decision-making are achieved. Even in the event of a network outage, the system can still maintain basic environmental control and fall / abnormal behavior monitoring functions (with a backup battery life of ≥4 hours), avoiding the monitoring failure problem caused by cloud delays or network outages in traditional solutions. It is especially suitable for elderly people living alone.

[0089] 2. Multimodal fusion for accurate identification By combining visual recognition (posture estimation), gravity sensing, RFID identification, millimeter-wave radar, and sound analysis, a multi-dimensional resident profile is constructed, effectively solving the problem of data confusion in multi-member households. Fall detection employs a triple verification system of visual, radar, and sound, with a sensitivity of ≥95% and specificity of ≥98%, significantly reducing false alarm and missed alarm rates and achieving seamless, 24 / 7 safety monitoring.

[0090] 3. Proactive early warning, closed-loop system throughout. It has built a complete closed loop from data perception and AI decision-making to execution unit actions, automatically adjusting temperature and humidity, cleaning bathrooms, and warning of health risks. It can also proactively identify events such as falls, abnormal toilet use, and failure to return to bed at night and issue tiered alarms, realizing the transformation from "passive treatment" to "proactive prevention" and significantly improving the living safety and quality of life of the elderly.

[0091] 4. Modular and elderly-friendly design The execution unit adopts a modular interface design, and each module can be independently disassembled and configured as needed, which facilitates maintenance and functional expansion. The appearance integrates furniture design concepts, and the camera is equipped with a physical cover to protect privacy from the source. The interactive interface supports large font, voice control, physical buttons and pull cord alarm, which reduces the threshold for elderly people to use and improves user acceptance.

[0092] 5. Health alerts, continuous optimization By continuously monitoring weight change curves, toileting behavior patterns, and sleep breathing frequency, the system can issue early warnings of health problems (such as sudden weight loss, increased nocturia, and abnormal body temperature), helping family members to promptly identify and address potential health risks in the elderly. Simultaneously, the cloud-based federated learning framework trains models on anonymized data, and the behavior recognition and fall detection models are regularly optimized via over-the-air (OTA) updates, making the system increasingly accurate with use.

[0093] The above description is merely a preferred embodiment of the present invention and is not intended to limit the present invention in any way. Although the present invention has been disclosed above with reference to preferred embodiments, it is not intended to limit the present invention. Any person skilled in the art can make some modifications or alterations to the above-disclosed technical content to create equivalent embodiments without departing from the scope of the present invention. Any simple modifications, equivalent substitutions, and improvements made to the above embodiments without departing from the scope of the present invention, based on the technical essence of the present invention and within the spirit and principles of the present invention, shall still fall within the protection scope of the present invention.

Claims

1. A smart home control system based on edge computing and multimodal sensing, characterized in that: Include The sensing unit layer is deployed inside the residence and in key locations to collect resident status and environmental parameters in real time. The control layer unit is the core decision-making unit of the system, serving as a central node connecting the perception unit layer, the execution layer unit, the cloud platform, and the user interaction layer unit. The execution layer unit consists of independent execution units distributed in various functional areas of the residence, and is subject to unified scheduling by the control layer unit. The cloud platform is deployed on the Internet and communicates bidirectionally with the control layer unit via the MQTT / HTTP protocol; The user interaction layer unit supports multi-terminal collaboration, allowing users to interact with the system through their terminals.

2. The smart home control system based on edge computing and multimodal perception according to claim 1, characterized in that: The sensing unit layer includes The environmental sensing module is used to monitor the quality of the microenvironment inside the residence and provide data for environmental control. The environmental sensing module includes temperature and humidity sensors, air quality sensors, and noise sensors. Among them, the temperature and humidity sensors are deployed in the bedroom and bathroom to collect temperature and humidity data in real time for linkage with air conditioning, temperature-controlled flooring, and fresh air system. The air quality sensors are deployed in the bathroom and bedroom to detect TVOC and CO2 concentrations and automatically activate deodorization devices and fresh air exchange when odor exceeds the standard. The noise sensor is used to monitor the ambient noise level, identify abnormal sounds, and automatically adjust the equipment operation mode at night. The personnel sensing module is used for resident identification, weight tracking, and activity monitoring. This module includes a gravity sensor, an RFID reader, an infrared grid sensor, and a pressure sensor array. The gravity sensor is embedded in the dining room and rest area, using a dynamic weighing algorithm to achieve seamless weight tracking of residents. Each weighing result is linked to their identity, creating an individual weight change curve. The RFID reader is deployed at the entrances of each functional area to read the RFID tags worn by residents, enabling accurate identification and direction determination. The infrared grid sensor is installed at the entrances of functional areas to monitor residents' movements, recording dwell time and activity frequency. The pressure sensor array is laid at the bottom of toilets or bathtubs to collect pressure distribution characteristics and identify details of toilet or bathing behavior. The visual perception module is used for resident posture recognition and behavior capture, especially including fall detection and abnormal behavior recognition. The visual perception module includes a high-definition camera and millimeter-wave radar. The high-definition camera is equipped with a wide-angle lens and infrared night vision function to collect resident activity images in real time, run a behavior recognition model to judge the behavior status, and the model output includes labels such as normal walking, sitting or lying down, falling, wandering, and prolonged stillness. When a fall or abnormal behavior is detected, the system immediately triggers a local alarm and pushes a remote notification. The millimeter-wave radar has penetrating ability and is not affected by light. When the camera's field of view is limited or the cover is closed, it continuously senses the resident's position, activity trajectory and breathing rate to help determine whether a fall has occurred or abnormal vital signs have occurred. The health sensing module is used to collect the acoustic characteristics and body temperature trends of residents. The health sensing module includes a sound acquisition unit and an infrared temperature measurement unit. The sound acquisition unit is deployed in the bedroom and bathroom to identify acoustic events, including cries for help, fall and impact sounds, and abnormal cough sounds. The infrared temperature measurement unit is installed above the rest area to collect the body surface temperature non-contactly when the resident is stationary, forming a body temperature change curve. The perception layer has a collaborative working mechanism. All modules in the perception layer synchronize data through a unified timestamp. RFID identification results are bound to gravity, pressure, and visual data to ensure that all perception data is attributed to the correct individual resident. When a certain type of sensor fails, the system automatically switches to a backup identification scheme. After local preprocessing, the collected data is transmitted to the control layer in real time via RS485 bus, IO interface, or Wi-Fi.

3. The smart home control system based on edge computing and multimodal perception according to claim 1, characterized in that: The functions of the control layer unit include Multi-source data fusion receives data from pressure sensor arrays, RFID readers, environmental sensors, visual perception modules, and health perception modules, aligns and fuses them on the timeline, and constructs a panoramic view of the residential environment and the status of residents. Localized decision-making, with a built-in AI edge computing module NPU / TPU, runs multi-sensor fusion algorithms, behavior recognition models and self-learning models to perform localized analysis on the collected data, identify resident behavior, determine whether environmental parameters exceed the standard, assess health risk level, and especially make real-time judgments on abnormal events such as falls, prolonged stillness, and failure to return to bed at night. Command issuance and scheduling: Based on the decision results, control commands are issued to each unit in the execution layer through the communication interface, and execution feedback is received to form a closed-loop control. Data caching and synchronization: local flash memory and expandable storage card cache historical data and operation logs, supporting the function of resuming data transmission after network interruption; after the network is restored, the data is reported to the cloud platform via Wi-Fi or 5G / 4G module; Remote communication management establishes an MQTT / HTTP connection with the cloud platform via the Internet, receives configuration parameters, rule updates, and firmware upgrade instructions from the cloud, and synchronizes the device status to the cloud in real time.

4. The smart home control system based on edge computing and multimodal perception according to claim 3, characterized in that: The multi-source data fusion specifically includes Spatiotemporal alignment: Visual data, gravity sensor data, and RFID data are aligned on the timeline. The system adopts a timestamp synchronization mechanism, and all sensor data have a unified time reference, which is synchronized through the NTP network or the local RTC clock. Identity fusion: When RFID identifies a resident's identity, gravity data and visual data over a subsequent period of time are linked to that identity; If the RFID tag is not identified, the system uses a visual recognition model to infer the identity and associates it with gravity data. Behavior fusion: By combining the motion characteristics of pressure sensors, the posture estimation results of vision modules, and the activity trajectory of infrared grids, complex behaviors of residents can be identified, including fall detection: by jointly determining whether a fall has occurred based on features such as human key point posture estimation, center of gravity falling speed, and changes in the aspect ratio of the outline.

5. The smart home control system based on edge computing and multimodal perception according to claim 3, characterized in that: The behavior recognition model and self-learning model specifically include Behavior recognition model: A lightweight convolutional neural network is used to run on the NPU. The model input is camera images and sensor time-series data; the model outputs behavior category labels. The model inference latency is ≤50ms, and the sensitivity and specificity of fall detection are ≥95% and ≥98%, respectively. Self-learning mechanism: The system establishes an independent individual behavior profile for each resident, recording the temporal distribution and characteristic parameters of their behavior patterns; through long-term observation, the system dynamically updates the behavioral baseline; Model Updates: The cloud-based federated learning framework is used periodically to aggregate user data for model training, generating optimized model files, which are then pushed to the control unit for incremental updates via OTA.

6. The smart home control system based on edge computing and multimodal perception according to claim 3, characterized in that: The data caching and synchronization specifically include Local storage: All sensor data is stored on a MicroSD card in a preset format, using a circular storage strategy to retain data from the most recent 30 days; Resuming data upload after network interruption: When a network interruption is detected, the system automatically marks the data to be uploaded as "not synchronized" and stores it in a local queue; after the network is restored, the data is uploaded to the cloud in a first-in-first-out order, and data compression is also supported. Data reporting: During normal operation, the system reports aggregated data every 5 minutes by default; event-triggered data is reported immediately.

7. The smart home control system based on edge computing and multimodal perception according to claim 1, characterized in that: The execution layer unit includes The residential area execution unit includes a micro DC inverter air conditioner, an activated carbon fresh air system, and a smart temperature-controlled floor, which are used to regulate the temperature, humidity, and air quality inside the residence. The dining area execution unit includes intelligent cooking aids and an intelligent dining table system to achieve safe cooking, nutrition management, and meal reminders; specifically, it includes: an intelligent gas stove, an intelligent range hood, an intelligent dining table, and a nutrition advice voice broadcast module; the system recommends recipes based on residents' health data and reminds them to take medication and drink water via a mobile app or voice. The bathroom area control unit includes a smart toilet / smart toilet seat, an automatic cleaning and disinfection unit, anti-fall grab bars, and an emergency pull cord alarm. The smart toilet features heated seat, warm water washing, warm air drying, and automatic lid opening and closing, and includes a built-in pressure sensor and health monitoring module. The cleaning and disinfection unit automatically performs UV disinfection and spray cleaning after the occupant leaves. The anti-fall grab bars are installed on both sides of the toilet and shower area, and have built-in capacitive proximity sensors. When the system detects prolonged leaning or abnormal force, it determines whether a fall has occurred. The emergency pull cord alarm is positioned low for easy access for elderly people who may fall and need to pull the cord for help. The entertainment area's execution unit includes a smart voice speaker, a large-screen smart TV / projector, a companion interaction system, and speakers. The smart voice speaker supports voice-activated playback of opera, news, and audiobooks, and can interact with the system. The large-screen TV interface and remote control simplify operation and support gesture control. The companion interaction system can play soothing music or conduct simple Q&A sessions based on the resident's emotional state to alleviate loneliness for elderly people living alone. Speakers are used to play medication reminders, activity reminders, and emergency announcements at set times. The safety zone execution unit includes alarm lights, an emergency assistance trigger unit, and a remote notification module. It triggers local alarms and remote notifications when it detects falls, unauthorized intrusion, prolonged inactivity by residents, or health risks. The alarm lights, installed at the entrances to bedrooms, living rooms, and bathrooms, use high-brightness LEDs and support red, yellow, and blue flashing. Red indicates an emergency, yellow indicates a warning, and blue indicates the system is functioning normally. The emergency assistance trigger unit includes multiple physical buttons and a wearable remote control button. The remote notification module supports sending alarm information containing location, time, and event type to preset emergency contacts.

8. The smart home control system based on edge computing and multimodal perception according to claim 1, characterized in that: The cloud platform includes Device management, maintaining the registration information, online status, firmware version, and configuration parameters of all smart home devices, and supporting OTA remote upgrades; The AI ​​application engine receives structured data uploaded from the control layer and performs in-depth analysis and model training. Based on the federated learning framework, we continuously optimize the behavior recognition model and health assessment model, especially the generalization ability of the fall detection model, using multi-user de-identified data. Big data processing uses a time-series database to store massive amounts of sensor data, generates weekly / monthly health reports for residents, and provides data visualization dashboards that highlight fall risk trends and changes in mobility. Business services, including extended functions: emergency assistance, online medical consultation, and contactless social interaction; User management manages user accounts, device binding relationships, and permission configurations, supporting centralized management of multiple residences and sharing among multiple family members.

9. The smart home control system based on edge computing and multimodal perception according to claim 1, characterized in that: The user interaction layer unit includes On the computer, users can access the management backend of the cloud platform through a web browser. The computer-based functions include batch device management, data statistical analysis, and user permission management. Among them, batch device management allows users to manage multiple residential devices simultaneously, view the online status, operating parameters, and alarm information of each device; data statistical analysis generates cross-device data reports; and user permission management allows users to set multi-level user permissions. On mobile devices, the system is presented as a WeChat mini-program or app. Mobile features include real-time monitoring, remote control, alarm reception, health reports, and rule configuration. Real-time monitoring allows users to view real-time images of the interior of the residence, environmental parameters, and the real-time status of residents. Remote control allows users to manually start execution units and adjust device settings. Alarm reception includes receiving notifications such as fall warnings, health abnormalities, and equipment malfunctions via WeChat template messages, app push notifications, and SMS. Health reports provide individual residents' weekly / monthly health reports, including activity level assessments, sleep quality, and toilet frequency. Rule configuration allows users to customize linkage rules through a graphical interface, such as "If a fall is detected, immediately notify children and call property management." The printer, as an optional output device of the system, connects to a computer or mobile phone via USB or Wi-Fi, allowing users to actively trigger print jobs. Printer functions include health report printing, QR code label printing, and device configuration list printing. Specifically, for health report printing, users can send weekly / monthly health reports from their computers or mobile phones to the local printer to generate paper reports. For QR code label printing, the system generates unique QR code labels for each resident, which can be printed and affixed to the home wall or health record book; scanning the QR code allows for quick access to the resident's health record. During the equipment installation and commissioning phase, the system prints the device configuration list, sensor calibration records, and network parameter information.

10. A control method for a smart home control system based on edge computing and multimodal sensing as described in any one of claims 1-9, characterized in that... Includes the following steps: S1. Multi-source data acquisition: Real-time acquisition of environmental data, resident ID, weight data, behavioral video streams and audio data through various sensor groups in the perception layer; S2. Edge-side data fusion: The all-in-one machine of the control layer unit receives multi-source data, uses a multi-sensor fusion algorithm to match the identity ID identified by RFID with the weight data collected by the gravity sensor, and combines the posture estimation results of the visual perception module to confirm the individual profile of the resident. S3. Localized decision-making and reasoning: The AI ​​edge computing module calls a pre-built self-learning model to analyze residents' meal duration, toilet frequency, weight change curve, activity trajectory, and posture to determine whether there are any health abnormalities or safety risks. At the same time, the rule engine makes logical judgments based on environmental data and preset thresholds. Among them, the fall detection logic is as follows: when the vision module detects that the key points of the human body suddenly shift downward and the center of gravity acceleration exceeds the threshold, or the millimeter-wave radar detects a sudden change in breathing frequency and the horizontal height of the human body is less than half of the normal standing height, it is fused and judged as a fall event. If the sound acquisition unit recognizes an impact sound or a cry for help at the same time, the alarm level is increased. S4. Command Issuance and Execution: If the environmental parameters are determined to be out of standard, the all-in-one unit sends a control command to the execution layer to start the fresh air system or temperature control equipment; if a fall or abnormal behavior is determined to occur, a local alarm is immediately triggered and pushed to the user terminal through the communication module, and the emergency help process can be automatically started at the same time. S5. Closed-loop feedback: After the execution layer unit completes its action, the perception unit layer collects data again and feeds it back to the control layer to correct the decision model and record event logs for subsequent model optimization.

Images