Household intelligent guard system integrating emergency linkage and emotion recognition and method thereof

Abstract

The invention discloses an emergency linkage and emotion recognition integrated household intelligent guard system and method, and relates to the technical field of Internet of Things, artificial intelligence and emergency response, the system comprises the following components: a front-end sensing device, an edge AI processing unit, a voice broadcast interaction module, a remote linkage module and a background data platform; by integrating the front-end sensing device and the edge AI processing unit, multi-dimensional real-time monitoring of human body behaviors, emotional states and environment information is achieved, the abnormal behaviors and emotional states of a user can be accurately recognized by the system through the behavior recognition module, the emotional analysis module and the multi-modal fusion module, and the user experience is improved. And active care and family interaction are carried out through the voice broadcast interaction module, and the remote linkage module is triggered to inform family members or an emergency platform if necessary, so that a closed-loop intervention mechanism is formed, and the intelligent level and the response speed of family safety monitoring are effectively improved.

Description

Technical Field

[0001] The present invention relates to the fields of Internet of Things, artificial intelligence and emergency response technology, and specifically to a home intelligent security system and method integrating emergency linkage and emotion recognition. Background Art

[0002] With the aggravation of the problem of population aging and the increasing attention paid to family safety and health, the home intelligent protection system has gradually become a research hotspot in the field of smart home. The home intelligent protection system aims to realize real-time monitoring and early warning of family members' behavior, emotional state and environmental information by integrating multiple sensors and intelligent analysis technologies, thereby improving the level of family safety and protecting the health and safety of family members. Especially in the families of special groups such as elderly people living alone and patients with chronic diseases, the role of the intelligent protection system is more prominent.

[0003] However, most traditional smart home security systems have the following shortcomings: First, they often rely on a single or a few sensors for data collection, resulting in limited monitoring dimensions and making it difficult to fully and accurately reflect the behavior and emotional state of family members. Second, traditional systems lack intelligence in data processing and analysis, and can often only perform simple threshold judgments, and are unable to deeply understand and accurately identify complex behavioral patterns and emotional changes. In addition, traditional systems also have shortcomings in privacy protection and data security, which can easily cause users to worry about privacy leaks. Finally, the response mechanism of traditional systems is relatively simple and lacks closed-loop intervention capabilities, making it impossible to intervene and notify in a timely and effective manner when abnormal situations are detected.

[0004] In view of the above shortcomings of traditional home intelligent guarding systems, the present invention proposes a home intelligent guarding system and method that integrates emergency linkage and emotion recognition, which is particularly important. Summary of the Invention

[0005] The purpose of the present invention is to make up for the shortcomings of the existing technology and provide a family intelligent guarding system and method that integrates emergency linkage and emotion recognition. It can realize multi-dimensional real-time monitoring of human behavior, emotional state and environmental information by integrating multiple front-end sensing devices, and use edge AI processing units to perform data processing and intelligent analysis to improve the intelligence level of the system. Through the voice broadcast interaction module and remote linkage module, it can realize active care and family interaction, as well as emergency linkage when necessary. At the same time, the system fully considers user privacy protection and data security in its design to ensure the security and controllability of user data.

[0006] In order to solve the above technical problems, the present invention provides the following technical solutions: On the one hand, a home intelligent security system integrating emergency linkage and emotion recognition, the system includes the following components: a front-end sensing device, an edge AI processing unit, a voice broadcast interaction module and a remote linkage module, and a back-end data platform; The front-end sensing device includes a video acquisition module, a millimeter-wave radar module, an infrared sensor, a human sensor, and a sound sensor, and is used to collect multimodal data. The edge AI processing unit integrates a behavior recognition module, a sentiment analysis module, and a multimodal fusion module to perform data processing and intelligent analysis; The voice broadcast interaction module is used to output active caring voice and family interaction content; The remote linkage module connects the family APP and the emergency platform via 4G / 5G / WIFI or offline SMS channel to realize event report push and remote interaction; The backend data platform stores status indicators based on a time series database and generates daily user status reports and trend analysis models.

[0007] Furthermore, the behavior recognition module adopts a dual-stream fusion algorithm based on the temporal features of skeleton points, which specifically includes a spatial feature branch and a temporal feature branch: The improved lightweight convolutional network is used to extract the spatial features of the human skeleton point coordinate sequence in the video image. The network structure consists of 5 layers of depth-separable convolutional layers. The convolution kernel size of each layer is 3×3, the number of channels is 32, 64, 128, 256, and 512 respectively, and the activation function uses the Swish function. The input dimension of the skeleton point coordinate is , is the number of time frames, is the bone point number, is the XY coordinate, and the output dimension after spatial feature extraction is , is the spatial feature dimension; The gated recurrent unit is used to capture the temporal dependency of the skeleton point sequence. The hidden layer dimension is 256 and the input is the output of the spatial feature branch. , after time feature extraction, the output dimension is , is the time feature dimension; The spatial and temporal features are integrated through the cross-modal attention mechanism, and the calculation formula is: in 、 is the learnable weight matrix, is the attention dimension, is the fused temporal feature; The fused features are input into the fully connected layer, and the focus loss function is used to optimize the imbalance problem of falling and abnormal stillness categories. The loss function is: in, is the positive sample weight, To focus on the parameters, they are determined by cross-validation on the validation set.

[0008] Furthermore, the millimeter wave radar module adopts a joint modeling algorithm of micro-Doppler characteristics and respiratory frequency, and the specific steps are as follows: Perform short-time Fourier transform on the radar echo signal to obtain the time-frequency matrix , reduced to a 20-dimensional feature vector through principal component analysis , the calculation formula is: The window length is set to 512 points and the overlap rate is 75%, which is determined to be the optimal parameter combination through experiments; The respiratory component in the radar signal is extracted using an adaptive zero-phase filter, and the instantaneous frequency is calculated using the Hilbert transform. The formula is: ,in The preprocessed radar signal is filtered using a sliding window mean filter to obtain the respiratory rate estimate; Micro-Doppler characteristics and respiratory rate Input gated graph neural network, nodes are radar reflection coefficients of different body parts, edges are spatial relationships between parts, and node states are updated through gating mechanism. The formula is: ,in For nodes exist The hidden state of the moment, is the set of neighbor nodes, It is a learnable parameter obtained through training with radar measured data.

[0009] Furthermore, the speech recognition branch of the sentiment analysis module adopts an improved hybrid attention network with the following specific structure: The speech signal is divided into frames, and the Mel-frequency cepstral coefficients (MFCC) and first-order difference coefficients are extracted to form a 40-dimensional feature vector. , and the dimension is reduced to 128 after linear transformation: in, 、 are learnable parameters, initialized by pre-trained models; Use bidirectional LSTM to capture contextual dependencies, with a hidden layer dimension of 256 and an output Calculate word-level weights via self-attention: in, 、 is the attention parameter, which is obtained by training the emotion-labeled speech data; The word-level weighted features are average-pooled and input into the fully connected layer to generate sentence-level features s, and the weights are calculated by attention: in, is the number of statements, 、 It is a learnable parameter, which is optimized through cross-sentence sentiment correlation analysis; The hierarchical attention features are input into the Softmax layer to output the probabilities of five categories of emotions, including normal, tired, lonely, anxious, and excited. The loss function uses label smoothing regularization: in is a smoothing parameter determined by tuning the sentiment dataset.

[0010] Furthermore, the visual branch of the emotion analysis module adopts an expression recognition network based on a feature pyramid, specifically including: ResNet50 is used as the backbone network. Feature maps of different scales are output in Conv2, Conv3, and Conv4 layers. Feature pyramids are constructed through upsampling and lateral connections. The formula is: in For the Layer feature pyramid output, is the corresponding layer feature of the backbone network, for Convolutional weights, learned through end-to-end training; Connect the key point detection branch to the top layer of the feature pyramid and output the coordinates of 68 facial key points , the loss function is smooth L1 loss: in is the predicted coordinate, is the real coordinate, initialized by 3 million dataset pre-training; The multi-scale feature map is input into the global average pooling layer, combined with the key point geometric features, and outputted through the fully connected layer the probability of 7 categories of expressions. The classification loss is optimized jointly by the focal loss and the triplet loss: in To balance the coefficients, triplet loss improves inter-class discrimination by mining difficult samples.

[0011] Furthermore, the multimodal fusion module adopts a dynamic weight adaptive fusion algorithm, and the specific steps are as follows: Score speech emotion , visual expression score , behavioral abnormality score , calculate the Pearson correlation coefficient of each mode in the historical data , as the modal reliability weight: The correlation coefficient is calculated using a sliding window of data from the past 7 days, reflecting the correlation between the modality and the true sentiment; Introducing real-time environmental factors , generate environmental impact factors through a fully connected network , adjust the modal weight: , in, 、 It is a learnable parameter obtained through training on a multi-scenario test set; Final sentiment rating The weighted sum of each mode is: .

[0012] Furthermore, the active care triggering mechanism adopts a hierarchical decision model based on reinforcement learning, and the specific structure is as follows: User sentiment level , behavioral status , timestamp is the input, the output intervention type, and the policy function is: ,in For high-level status, For the intervention type, the reward function is designed as: are weight coefficients, corresponding to mood improvement, interaction frequency increase, and abnormal response speed respectively; According to the high-level intervention type, specific actions are output. The action space contains 20 preset strategies, and the execution function is: in, For the low-level state, the action selection is optimized through the Q-learning algorithm, with the goal of minimizing intervention delay and user resistance.

[0013] Furthermore, the backend data platform uses a privacy-enhancing algorithm based on a generative adversarial network to achieve anonymized transmission of structured data. The specific process is as follows: When processing structured data such as user behavior and sentiment levels, the backend data platform generates fake data by building a generative adversarial network. This data is mixed with the original data and then transmitted to the cloud or terminal, thereby ensuring data availability while preventing privacy leaks. This algorithm consists of two core modules: The first module is the data generator, which uses the user's original structured data as a condition and combines it with random Gaussian noise to generate fake data. The generator learns the distribution characteristics of real data through a multi-layer fully connected network, making the fake data highly similar to the real data in terms of statistical laws, but without containing the individual identification information of the real user; The second module is the discriminator, whose function is to distinguish whether the input data is real data or forged data. The discriminator uses a deep learning network to make a probabilistic judgment on the authenticity of the data. The generator and the discriminator are continuously optimized through an adversarial training mechanism. After multiple rounds of iterative training, the forged data output by the generator will be difficult for the discriminator to identify, thus achieving data anonymization; During the data transmission process, the system adopts a mixed sending strategy of real data and forged data. Each time it transmits, it randomly chooses to send the original structured data or the forged data output by the generator with a probability of 50%. At the same time, all data is encrypted before transmission, and users can set the data storage time limit in the APP to further strengthen privacy protection.

[0014] On the other hand, a method for intelligent home protection that integrates emergency linkage and emotion recognition is characterized in that the method comprises the following specific steps: S1. Initial configuration and strategy generation: The family app receives basic user data, including age, chronic disease information, sleep and rest schedules, family contact information, and privacy preferences. The system generates an initial recognition strategy based on this data, including behavioral abnormality thresholds and emotional intervention priority rules, and configures the front-end device operating mode based on the room type. S2. Real-time collection and preprocessing of multi-source heterogeneous data: The front-end sensing device layer continuously collects multimodal data, including: The video acquisition module obtains human posture image data for skeleton point sequence extraction and fall detection; The millimeter-wave radar module collects non-visual parameters such as breathing rate and body movement trajectory, replacing video monitoring in private areas; Infrared / human sensors transmit room occupancy data and activity trajectories in real time; The sound sensor collects voice signals for call detection and emotion feature extraction; The raw data is pre-processed by the edge AI processing unit for noise reduction and normalization. Sensitive image data is deleted by default after the skeleton points are extracted locally. S3. Behavior-emotion dual-dimensional intelligent analysis: An action recognition algorithm based on a temporal convolutional network is used on the skeleton point sequence, combined with temporal cluster analysis of sensor data to identify normal activities, abnormal behaviors, and deviations from work and rest schedules. Speech signals are extracted using MFCC features combined with the ResNet+Bi-LSTM model for emotion classification. Facial images are analyzed using the CNN model to analyze features of eyebrows, eyes, and mouth for expression recognition. Multimodal results are then weighted using a confidence-weighted strategy to generate an emotion rating score of 0-10. S4. Graded Response and Closed-Loop Intervention: When the emotional level score is continuously detected to be ≤4 points or abnormal behavior lasts for ≥30 minutes, local interaction is triggered, with voice broadcasting of preset care content, playing pre-recorded voice messages of relatives, lighting up soothing lights, or pushing activity suggestions. If the user responds through voice commands or actions, the system records the intervention as successful; otherwise, it enters the second-level response; Generate a structured event report and push it to the family app and community service platform via 4G / 5G / WIFI. If the network is disconnected, an early warning summary will be sent via SMS. Family members can remotely initiate a voice / video call or request a home inspection by a community grid worker, forming a family-community collaborative response chain. S5. Data Accumulation and Personalized Learning: The backend data platform stores daily status reports based on a time series database. It uses Prophet or RNN models to analyze temporal fluctuations in user behavior and emotions, creating individualized emotional rhythm profiles. The system automatically adjusts recognition thresholds and optimizes intervention strategies based on historical data, enabling dynamic evolution of monitoring-analysis-intervention strategies. S6. Privacy Compliance Enforcement Mechanism: Sensitive areas are monitored by a combination of millimeter-wave radar and human-sensing sensors by default. Image data is not collected. All original images and voice data are deleted only after feature extraction is completed locally in the edge AI processing unit. Only structured labels are transmitted to the cloud. Data transmission is encrypted using AES-256, and storage supports time-limited self-sale and anonymous upload options. Users can authorize relatives to have access rights in a hierarchical manner through the APP.

[0015] Compared with the prior art, the present invention has the following beneficial effects: 1. By integrating front-end sensing devices and edge AI processing units, the system realizes multi-dimensional real-time monitoring of human behavior, emotional state and environmental information. Using the behavior recognition module, emotion analysis module and multimodal fusion module, the system can accurately identify the user's abnormal behavior and emotional state, and provide active care and family interaction through the voice broadcast interaction module. When necessary, it triggers the remote linkage module to notify family members or the emergency platform, forming a closed-loop intervention mechanism, which effectively improves the intelligence level and response speed of home security monitoring.

[0016] II. The system fully considers user privacy protection and data security in design, and the above design fully complies with the minimum necessity and user informed consent principle of the Personal Information Protection Law, to enhance compliance trust. Sensitive areas by default adopt millimeter wave radar + human sensor combination monitoring, avoid direct collection of image data, ensure user privacy, all original images, voice data are deleted after feature extraction in the edge AI processing unit, only structured labels are transmitted to the cloud, reducing the risk of privacy leakage in the data transmission process, at the same time, the background data platform uses a privacy enhancement algorithm based on generative adversarial network to realize the anonymous transmission of structured data, further guarantee the safety of user data, in addition, users can authorize relatives to access through APP, enhance the controllability and transparency of data use. Finally, compared with traditional video monitoring products, the system realizes non-image + image fusion perception, compared with wearable devices, there is no need for user active operation or wearing, compared with security monitoring, emotion recognition and active intervention ability are increased.

[0017] Other advantages, objects, and features of the application will be set forth in part in the following specification taken in conjunction with the accompanying drawings, and in part will become apparent to those skilled in the art from a consideration of the following specification and drawings, or can be learned from the practice of the application. BRIEF DESCRIPTION OF DRAWINGS

[0018] In order to more clearly illustrate the technical solutions in the embodiments of the present application or the prior art, the drawings needed to be used in the embodiments or prior art description will be briefly introduced below. Obviously, the drawings in the following description are only some embodiments of the present application, and those skilled in the art can also obtain other drawings according to these drawings without creative labor.

[0019] Figure 1 The architecture diagram of the family intelligent guardian system; Figure 2 The operation diagram of the flow of the family intelligent guardian method. DETAILED DESCRIPTION

[0020] In order to further illustrate the technical means and effects adopted by the present application to achieve the predetermined application purpose, the specific embodiments, structures, features and effects according to the present application will be described in detail below in combination with the drawings and preferred embodiments.

[0021] Example One : An 80-year-old elderly person living alone, whose children work in other places, suffers from mild hypertension. His daily routine is to get up at 8 a.m. and go to bed at 10 p.m. The system has preset the abnormal behavior detection threshold (for example, if no activity is detected for 2 hours after the regular waking time, an alert will be triggered) and privacy preferences (only millimeter-wave radar + human sensing sensor are enabled in the bedroom area, and video images are not collected) through the family APP.

[0022] Perform short-time Fourier transform (window length 512 points, overlap rate 75%) on the bedroom area echo signal to generate a time-frequency matrix , which is reduced to a 20-dimensional micro-Doppler feature vector by principal component analysis .

[0023] Synchronously extract the respiratory component through an adaptive zero-phase filter and calculate the instantaneous frequency through Hilbert transform , after sliding window mean filtering, the respiratory rate was 14 times / min (normal range).

[0024] Human sensor: Outputs a binary occupancy signal (0 = no one, 1 = someone). At 12:00, it outputs "1" for four consecutive hours without any displacement change, triggering skeleton point timing anomaly detection.

[0025] Behavior recognition algorithm trigger logic Dual-stream fusion network: Spatial feature branch: Input skeleton point coordinate sequence (T = 120 frames, V = 17 points, C = 2 dimensions), through 5 layers of depth-wise separable convolution (activation function Swish), output T × V × Fs (Fs = 64), capture the static features of the relative position of joints (such as the difference in skeleton point distribution between supine and sitting postures).

[0026] Temporal feature branch: The gated recurrent unit (GRU) inputs spatial features and outputs T×V×Ft (Ft=32), capturing the temporal dependency of the skeleton point displacement approaching 0 within 120 minutes.

[0027] Cross-modal attention fusion: through formula Softmax Calculate fusion features ( ), it was found that the "angle between the head-torso-lower limb bone points remained at 180°" in the spatial feature (lying flat posture), and "no change for 3600 consecutive frames" in the temporal feature, and was judged to be "abnormally still".

[0028] Network disconnection redundancy mechanism: If the 4G / WIFI signal is disconnected, the system automatically switches to the SMS channel to send an alert, including: Structured data: abnormality duration (4 hours), respiratory rate (14 times / minute), anonymized skeleton point simplified data based on the adversarial generative network (only the relative positions of joints are retained, without individual identification).

[0029] Emergency contact chain: priority to send the child's mobile phone number, if not answered, 15 minutes later automatically trigger the community emergency platform SMS interface (prestore 3 emergency contacts).

[0030] Medical data linkage: the background data platform retrieves the old person's blood pressure monitoring data in the past 7 days (morning systolic blood pressure mean 135 mmHg) through the time series database, and pushes it to the emergency center with the warning information, to assist in judging the coma inducement (exclude hypertension crisis, tend to hypoglycemia).

[0031] Example Two : Old couple living alone, husband living alone because of his wife's hospitalization, the system collects their rest data (such as: going to bed before 23:00 at night), voice interaction history (average daily communication with the old couple 3 times) and emotion rhythm portrait (based on Prophet model analysis, peak of loneliness at night 21:00-23:00) through family member APP.

[0032] Frame the voice signal (frame length 25ms, frame shift 10ms), extract 13-dimensional MFCC+13-dimensional first-order difference, and form a 40-dimensional feature vector , linear transformation Dimension reduction to 128 dimensions.

[0033] Bidirectional LSTM output ht∈ℝ² 56 , calculate word-level weights through self-attention , find that the proportion of "um" and "good" single word weights increases from 30% to 65% in daily life, and judge as "interaction willingness decreases".

[0034] ResNet50 backbone network outputs 56x56 / 28x28 / 14x14 feature maps at Conv2 / Conv3 / Conv4 layers, and fuses multi-scale information through feature pyramid ( 1x1 convolution).

[0035] The key point detection branch outputs 68-point coordinates, and the smooth L1 loss calculation finds that "the mouth corner droops 15°" and "the eyebrow-eye distance is reduced by 20%" (sad expression features), and the classification branch outputs the "low" probability of 78% through the focal loss+triplet loss (λ=0.5), and the difficult sample mining mechanism improves the inter-class distance by 40%.

[0036] The past 7-day data shows that the voice emotion score Sv is correlated with the true emotion , the vision Sa is , and the behavior Sb is , the initial weights wv=0.38, wa=0.36, wb=0.26 are calculated.

[0037] Real-time environmental factor E = [night (22:00), indoor light intensity 10 lux, no background sound], through the fully connected network Calculate δv=0.1 (speech recognition accuracy improves at night), δa=-0.05 (dark light causes expression detection confidence to decrease), δb=0.08 (sedentary behavior is more abnormal at night), and the adjusted weights w'v=0.42, w'a=0.34, and w'b=0.28.

[0038] Final Rating: =0.42×7.8 (Sv)+0.34×8.2 (Sa)+0.28×6.5 (Sb)=7.62, triggering the "loneliness risk" alarm (threshold ≥7 points).

[0039] High-Level Decision Making: Policy Function ,enter [7.62, long sitting, 22:00], output = "Family memory activation" (weight 0.65), reward function , prioritize improving sentiment scores.

[0040] Low-level action: executing a function , based on the user profile (preferring old photos and rejecting strangers' voices), select the action: Play the "bedtime story" audio recorded by your spouse before he was hospitalized (10 segments are pre-stored and randomly selected).

[0041] Push a "seaside vacation" memory video automatically generated from the family album (including photos of the user and his / her spouse taken in the past three years, with other characters blurred using skeleton point de-identification technology).

[0042] Environmental linkage: Adjust the lighting to warm yellow (color temperature 2700K) and play ocean wave white noise simultaneously (verified by the emotion-environment association model, this combination increases the probability of pleasure by 62%).

[0043] The above description is merely a preferred embodiment of the present invention and does not constitute any form of limitation to the present invention. Although the present invention has been disclosed as above in terms of a preferred embodiment, it is not intended to limit the present invention. Any person skilled in the art can, without departing from the scope of the technical solution of the present invention, make some changes or modifications to equivalent embodiments using the technical contents disclosed above. However, any brief modifications, equivalent changes and modifications made to the above embodiments based on the technical essence of the present invention without departing from the content of the technical solution of the present invention are still within the scope of the technical solution of the present invention.

Claims

1. A smart home security system integrating emergency linkage and emotion recognition, characterized in that: The system consists of the following components: front-end sensing equipment, edge AI processing unit, voice broadcast interaction module, remote linkage module, and back-end data platform: The front-end sensing device includes a video acquisition module, a millimeter-wave radar module, an infrared sensor, a human sensor, and a sound sensor, and is used to collect multimodal data. The edge AI processing unit integrates a behavior recognition module, a sentiment analysis module, and a multimodal fusion module to perform data processing and intelligent analysis; The voice broadcast interaction module is used to output active caring voice and family interaction content; The remote linkage module connects the family APP and the emergency platform via 4G / 5G / WIFI or offline SMS channel to realize event report push and remote interaction; The backend data platform stores status indicators based on a time series database and generates daily user status reports and trend analysis models.

2. The intelligent home security system integrating emergency linkage and emotion recognition according to claim 1 is characterized in that: The behavior recognition module adopts a dual-stream fusion algorithm based on the temporal features of skeleton points, which specifically includes a spatial feature branch and a temporal feature branch: The improved lightweight convolutional network is used to extract the spatial features of the human skeleton point coordinate sequence in the video image. The network structure contains 5 layers of depth-separable convolutional layers, the activation function uses the Swish function, and the skeleton point coordinate input dimension is , is the number of time frames, is the bone point number, is the XY coordinate, and the output dimension after spatial feature extraction is , is the spatial feature dimension; The gated recurrent unit is used to capture the temporal dependency of the skeleton point sequence, and the input is the output of the spatial feature branch. , after time feature extraction, the output dimension is , is the time feature dimension; The spatial and temporal features are integrated through the cross-modal attention mechanism, and the calculation formula is: in 、 is the learnable weight matrix, is the attention dimension, is the fused temporal feature; The fused features are input into the fully connected layer, and the focus loss function is used to optimize the imbalance problem of falling and abnormal stillness categories. The loss function is: in, is the positive sample weight, is the focus parameter.

3. The intelligent home security system integrating emergency linkage and emotion recognition according to claim 1 is characterized in that: The millimeter wave radar module uses a joint modeling algorithm of micro-Doppler characteristics and respiratory frequency. The specific steps are as follows: Perform short-time Fourier transform on the radar echo signal to obtain the time-frequency matrix , reduced to a 20-dimensional feature vector through principal component analysis , the calculation formula is: The window length is set to 512 points and the overlap rate is 75%, which is determined to be the optimal parameter combination through experiments; The respiratory component in the radar signal is extracted using an adaptive zero-phase filter, and the instantaneous frequency is calculated using the Hilbert transform. The formula is: ,in The preprocessed radar signal is filtered using a sliding window mean filter to obtain the respiratory rate estimate; Micro-Doppler characteristics and respiratory rate Input gated graph neural network, nodes are radar reflection coefficients of different body parts, edges are spatial relationships between parts, and node states are updated through gating mechanism. The formula is: ,in For nodes exist The hidden state of the moment, is the set of neighbor nodes, are learnable parameters.

4. The intelligent home security system integrating emergency linkage and emotion recognition according to claim 1 is characterized in that: The speech recognition branch of the sentiment analysis module adopts an improved hybrid attention network with the following structure: The speech signal is divided into frames, and the Mel cepstral coefficients and first-order difference coefficients are extracted to form a 40-dimensional feature vector , and the dimension is reduced to 128 after linear transformation: in, 、 are learnable parameters, initialized by pre-trained models; Use bidirectional LSTM to capture contextual dependencies, with a hidden layer dimension of 256 and an output Calculate word-level weights via self-attention: in, 、 is the attention parameter, which is obtained by training the emotion-labeled speech data; The word-level weighted features are average-pooled and input into the fully connected layer to generate sentence-level features s, and the weights are calculated by attention: in, is the number of statements, 、 It is a learnable parameter, which is optimized through cross-sentence sentiment correlation analysis; The hierarchical attention features are input into the Softmax layer to output the probability of 5 categories of emotions. The loss function uses label smoothing regularization: in is the smoothing parameter.

5. The intelligent home security system integrating emergency linkage and emotion recognition according to claim 1 is characterized in that: The visual branch of the emotion analysis module uses an expression recognition network based on a feature pyramid, specifically including: ResNet50 is used as the backbone network. Feature maps of different scales are output in Conv2, Conv3, and Conv4 layers. Feature pyramids are constructed through upsampling and lateral connections. The formula is: in For the Layer feature pyramid output, is the corresponding layer feature of the backbone network, for Convolutional weights, learned through end-to-end training; Connect the key point detection branch to the top layer of the feature pyramid and output the coordinates of 68 facial key points , the loss function is smooth L1 loss: in is the predicted coordinate, is the real coordinate; The multi-scale feature map is input into the global average pooling layer, combined with the key point geometric features, and outputted through the fully connected layer the probability of 7 categories of expressions. The classification loss is optimized jointly by the focal loss and the triplet loss: in To balance the coefficients, triplet loss improves inter-class discrimination by mining difficult samples.

6. The intelligent home security system integrating emergency linkage and emotion recognition according to claim 1 is characterized in that: The multimodal fusion module adopts a dynamic weight adaptive fusion algorithm, and the specific steps are as follows: Score speech emotion , visual expression score , behavioral abnormality score , calculate the Pearson correlation coefficient of each mode in the historical data , as the modal reliability weight: The correlation coefficient is calculated using a sliding window of data from the past 7 days, reflecting the correlation between the modality and the true sentiment; Introducing real-time environmental factors , generate environmental impact factors through a fully connected network , adjust the modal weight: , in, 、 is a learnable parameter; Final sentiment rating The weighted sum of each mode is: .

7. The intelligent home security system integrating emergency linkage and emotion recognition according to claim 1 is characterized in that: The active care triggering mechanism adopts a hierarchical decision-making model based on reinforcement learning, and its specific structure is as follows: User sentiment level , behavioral status , timestamp is the input, the output intervention type, and the policy function is: ,in For high-level status, For the intervention type, the reward function is designed as: are weight coefficients, corresponding to mood improvement, interaction frequency increase, and abnormal response speed respectively; According to the high-level intervention type, specific actions are output. The action space contains 20 preset strategies, and the execution function is: in, For the low-level state, the action selection is optimized through the Q-learning algorithm, with the goal of minimizing intervention delay and user resistance.

8. The intelligent home security system integrating emergency linkage and emotion recognition according to claim 1 is characterized in that: The backend data platform uses a privacy-enhancing algorithm based on a generative adversarial network to achieve anonymized transmission of structured data. The specific process is as follows: When processing structured data such as user behavior and sentiment levels, the backend data platform generates fake data by building a generative adversarial network. This data is mixed with the original data and then transmitted to the cloud or terminal, thereby ensuring data availability while preventing privacy leaks. This algorithm consists of two core modules: The first module is the data generator, which uses the user's original structured data as a condition and combines it with random Gaussian noise to generate fake data. The generator learns the distribution characteristics of real data through a multi-layer fully connected network, making the fake data highly similar to the real data in terms of statistical laws, but without containing the individual identification information of the real user; The second module is the discriminator, whose function is to distinguish whether the input data is real data or forged data. The discriminator uses a deep learning network to make a probabilistic judgment on the authenticity of the data. The generator and the discriminator are continuously optimized through an adversarial training mechanism. After multiple rounds of iterative training, the forged data output by the generator will be difficult for the discriminator to identify, thus achieving data anonymization; During the data transmission process, the system adopts a mixed sending strategy of real data and forged data. Each time it transmits, it randomly chooses to send the original structured data or the forged data output by the generator with a probability of 50%. At the same time, all data is encrypted before transmission, and users can set the data storage time limit in the APP to further strengthen privacy protection.

9. A method for intelligent home security that integrates emergency linkage and emotion recognition, applicable to the intelligent home security system that integrates emergency linkage and emotion recognition as described in any one of claims 1 to 8, characterized in that: The specific steps of this method are: S1. Initial configuration and strategy generation: The family app receives basic user data, including age, chronic disease information, sleep and rest schedules, family contact information, and privacy preferences. The system generates an initial recognition strategy based on this data, including behavioral abnormality thresholds and emotional intervention priority rules, and configures the front-end device operating mode based on the room type. S2. Real-time collection and preprocessing of multi-source heterogeneous data: The front-end sensing device layer continuously collects multimodal data, including: The video acquisition module obtains human posture image data for skeleton point sequence extraction and fall detection; The millimeter-wave radar module collects non-visual parameters such as breathing rate and body movement trajectory, replacing video monitoring in private areas; Infrared / human sensors transmit room occupancy data and activity trajectories in real time; The sound sensor collects voice signals for call detection and emotion feature extraction; The raw data is pre-processed by the edge AI processing unit for noise reduction and normalization. Sensitive image data is deleted by default after the skeleton points are extracted locally. S3. Behavior-emotion dual-dimensional intelligent analysis: An action recognition algorithm based on a temporal convolutional network is used on the skeleton point sequence, combined with temporal cluster analysis of sensor data to identify normal activities, abnormal behaviors, and deviations from work and rest schedules. Speech signals are extracted using MFCC features combined with the ResNet+Bi-LSTM model for emotion classification. Facial images are analyzed using the CNN model to analyze features of eyebrows, eyes, and mouth for expression recognition. Multimodal results are then weighted using a confidence-weighted strategy to generate an emotion rating score of 0-10. S4. Graded Response and Closed-Loop Intervention: When the emotional level score is continuously detected to be ≤4 points or abnormal behavior lasts for ≥30 minutes, local interaction is triggered, with voice broadcasting of preset care content, playing pre-recorded voice messages of relatives, lighting up soothing lights, or pushing activity suggestions. If the user responds through voice commands or actions, the system records the intervention as successful; otherwise, it enters the second-level response; Generate a structured event report and push it to the family app and community service platform via 4G / 5G / WIFI. If the network is disconnected, an early warning summary will be sent via SMS. Family members can remotely initiate a voice / video call or request a home inspection by a community grid worker, forming a family-community collaborative response chain. S5. Data Accumulation and Personalized Learning: The backend data platform stores daily status reports based on a time series database. It uses Prophet or RNN models to analyze temporal fluctuations in user behavior and emotions, creating individualized emotional rhythm profiles. The system automatically adjusts recognition thresholds and optimizes intervention strategies based on historical data, enabling dynamic evolution of monitoring-analysis-intervention strategies. S6. Privacy Compliance Enforcement Mechanism: Sensitive areas are monitored by a combination of millimeter-wave radar and human-sensing sensors by default. Image data is not collected. All original images and voice data are deleted only after feature extraction is completed locally in the edge AI processing unit. Only structured labels are transmitted to the cloud. Data transmission is encrypted using AES-256, and storage supports time-limited self-sale and anonymous upload options. Users can authorize relatives to have access rights in a hierarchical manner through the APP.

Images