Smart home nursing early warning control method and device and electronic equipment

Abstract

The embodiment of the invention discloses a smart home nursing early warning control method and device and electronic equipment. A specific embodiment of the method comprises the following steps: acquiring information of a to-be-nursed object in a target family area; performing behavior recognition processing on the information of the to-be-nursed object to obtain behavior recognition information; performing early warning level identification on the behavior identification information to obtain early warning level identification information; and performing control operation on controlled equipment corresponding to the target family area according to the early warning level identification information. According to the embodiment, the energy consumption of the controlled equipment can be reduced, the endurance time is prolonged, and the safety nursing reliability and the early warning accuracy are improved.

Description

Technical Field

[0001] The embodiments disclosed herein relate to the field of computer technology, and more specifically to smart home monitoring and early warning control methods, devices, and electronic devices. Background Technology

[0002] Smart home monitoring and early warning control is a technology that uses smart home devices to monitor the behavior of the person being monitored and to activate controlled devices (such as alarms, smart locks, and communication devices) for security monitoring. Currently, the typical approach for smart home monitoring and early warning control is as follows: First, monitoring information about the target area is acquired based on sensors such as door locks or cameras. Then, in response to determining that the monitoring information meets preset rules or thresholds, control operations on the controlled devices are triggered.

[0003] However, in practice, it has been found that when the above-mentioned method is used for care and early warning control, the following technical problems often exist: Since the monitoring information comes from basic status information obtained by a single or a small number of sensors (e.g., door lock open / close status), and is easily affected by environmental interference, the false alarm rate is high during the care and early warning control process, resulting in a decrease in the accuracy of the early warning and a decline in the reliability of safety care. The information disclosed in this background section is only intended to enhance the understanding of the background of the present disclosure concept, and therefore may contain information that does not constitute prior art known to those skilled in the art. Summary of the Invention

[0004] The summary portion of this disclosure is intended to provide a brief overview of the concepts, which will be described in detail in the detailed description portion that follows. This summary portion is not intended to identify key or essential features of the claimed technical solutions, nor is it intended to limit the scope of the claimed technical solutions.

[0005] Some embodiments of this disclosure propose a smart home monitoring and early warning control method, device, and electronic device to solve one or more of the technical problems mentioned in the background section above.

[0006] In a first aspect, some embodiments of this disclosure provide a smart home care early warning control method, including: acquiring information on objects to be cared for in a target home area; performing behavior recognition processing on the information on objects to be cared for to obtain behavior recognition information; performing early warning level recognition on the behavior recognition information to obtain early warning level recognition information; and performing control operations on controlled devices corresponding to the target home area based on the early warning level recognition information.

[0007] Secondly, some embodiments of this disclosure provide a smart home care early warning control device, including: an acquisition unit configured to acquire information about a person to be cared for in a target home area; a behavior recognition unit configured to perform behavior recognition processing on the information about the person to be cared for to obtain behavior recognition information; an early warning level recognition unit configured to perform early warning level recognition on the behavior recognition information to obtain early warning level recognition information; and a control unit configured to perform control operations on controlled devices corresponding to the target home area based on the early warning level recognition information.

[0008] Thirdly, some embodiments of this disclosure provide a smart lock, including: one or more processors; and a storage device having one or more programs stored thereon, which, when executed by one or more processors, cause the one or more processors to implement the method described in any of the implementations in the first aspect.

[0009] Fourthly, some embodiments of this disclosure provide a door, including the smart door lock described in the third aspect above.

[0010] Fifthly, some embodiments of this disclosure provide a computer-readable medium having a computer program stored thereon, wherein the computer program, when executed by a processor, implements the method as described in any of the implementations of the first aspect.

[0011] The various embodiments of this disclosure have the following beneficial effects: the smart home monitoring and early warning control method of some embodiments of this disclosure can reduce the energy consumption of controlled devices and improve battery life. Specifically, the reason for the increased energy consumption and decreased battery life of controlled devices is that the monitoring information comes from basic status information obtained by a single or a small number of sensors, and is easily affected by environmental interference, resulting in a high false alarm rate during the monitoring and early warning control process. This causes the linked controlled devices to perform a large number of invalid control operations (e.g., frequent alarms, frequent locking, frequent push notifications), increasing the working time of the controlled devices, leading to increased energy consumption and decreased battery life. Based on this, the smart home monitoring and early warning control method of some embodiments of this disclosure can first obtain the information of the object to be monitored in the target home area. Here, the information of the object to be monitored is multi-dimensional and multi-modal, providing data support for subsequent behavior recognition processing. Then, the behavior recognition processing is performed on the above-mentioned information of the object to be monitored to obtain behavior recognition information. Here, behavior recognition processing is performed on the multimodal information of the person under care, transforming the original monitoring information into behavior recognition information with semantic meaning of the behavior of the person under care. This behavior recognition information accurately reflects the current behavioral state of the person under care, improving the accuracy of subsequent warning level identification and thus reducing the false alarm rate of subsequent control operations. Next, the behavior recognition information is used to identify warning levels, resulting in warning level identification information. Here, the behavior recognition information is mapped to warning level identification information reflecting the degree of abnormality in the behavior of the person under care, reducing the possibility of misjudging low-risk abnormal behavior as a high-level warning and lowering the false alarm rate. Finally, based on the warning level identification information, control operations are performed on the controlled devices corresponding to the target home area. Here, a tiered control strategy is executed according to the warning level identification information, ensuring that the controlled devices only enter working mode under necessary warning levels, reducing the possibility of increased operating time due to ineffective control operations. Therefore, this smart home care warning control method can reduce the energy consumption of controlled devices and improve their battery life. Attached Figure Description

[0012] The above and other features, advantages, and aspects of the embodiments of this disclosure will become more apparent from the accompanying drawings and the following detailed description. Throughout the drawings, the same or similar reference numerals denote the same or similar elements. It should be understood that the drawings are schematic, and elements are not necessarily drawn to scale.

[0013] Figure 1 This is a flowchart of some embodiments of the smart home monitoring and early warning control method according to the present disclosure; Figure 2 This is a structural schematic diagram of some embodiments of the smart home monitoring and early warning control device according to the present disclosure; Figure 3This is a schematic diagram of the structure of an electronic device suitable for implementing some embodiments of the present disclosure. Detailed Implementation

[0014] Embodiments of this disclosure will now be described in more detail with reference to the accompanying drawings. While some embodiments of this disclosure are shown in the drawings, it should be understood that this disclosure can be implemented in various forms and should not be construed as limited to the embodiments set forth herein. Rather, these embodiments are provided to provide a more thorough and complete understanding of this disclosure. It should be understood that the accompanying drawings and embodiments of this disclosure are for illustrative purposes only and are not intended to limit the scope of protection of this disclosure.

[0015] It should also be noted that, for ease of description, only the parts relevant to the invention are shown in the accompanying drawings. Unless otherwise specified, the embodiments and features described in this disclosure can be combined with each other.

[0016] It should be noted that the concepts of "first" and "second" mentioned in this disclosure are used only to distinguish different devices, modules or units, and are not used to limit the order of functions performed by these devices, modules or units or their interdependencies.

[0017] It should be noted that the terms "a" and "a plurality of" used in this disclosure are illustrative rather than restrictive, and those skilled in the art should understand that, unless otherwise expressly indicated in the context, they should be understood as "one or more".

[0018] The names of messages or information exchanged between multiple devices in the embodiments of this disclosure are for illustrative purposes only and are not intended to limit the scope of such messages or information.

[0019] This disclosure will now be described in detail with reference to the accompanying drawings and embodiments.

[0020] Figure 1 A flowchart 100 of some embodiments of the smart home care early warning control method according to the present disclosure is shown. The smart home care early warning control method includes the following steps: Step 101: Obtain information on the individuals requiring care in the target family area.

[0021] In some embodiments, the executing entity (e.g., an electronic device) of the above-described smart home monitoring and early warning control method can acquire information about the object to be monitored in the target home area via a wired or wireless connection. The target home area can be the indoor area where the object to be monitored is located. The object to be monitored can be a family member of the smart home who needs monitoring. The information about the object to be monitored can be sensor data reflecting the behavioral state of the object to be monitored, acquired from various sensors. This information may include, but is not limited to, at least one of the following: door lock status information, video information, and activity monitoring information set. The door lock status information can be information on the door lock's open / closed status with a timestamp, acquired from a smart door lock. The video information can be a sequence of video frames containing the object to be monitored, acquired by a camera. The activity monitoring information in the activity monitoring information set can be sensor data reflecting the activities of the object to be monitored within the target home area, acquired by an indoor activity monitoring sensor. The indoor activity monitoring sensor may include, but is not limited to, at least one of the following: an infrared sensor, a millimeter-wave radar sensor, and a sound sensor.

[0022] Step 102: Perform behavior recognition processing on the information of the person to be cared for to obtain behavior recognition information.

[0023] In some embodiments, the aforementioned executing entity may perform behavior recognition processing on the information of the person to be cared for to obtain behavior recognition information. This behavior recognition information may be structured information reflecting the behavioral state of the person to be cared for.

[0024] In some optional implementations of certain embodiments, the above-mentioned behavior recognition processing of the information of the person to be cared for to obtain behavior recognition information may include the following steps: The first step is to identify the individuals requiring care by performing identity verification. This identity verification information can be uniquely identifying each individual.

[0025] In practice, the aforementioned execution entity can first, in response to determining that the aforementioned door lock status information meets the triggering condition, input the aforementioned video information into a trained face vector extraction model to obtain a set of face feature vectors. The face feature vectors in this set can represent the face information of the person being cared for in the aforementioned video information. The aforementioned triggering condition can be the condition that the aforementioned door lock status information is in an unlocked state or that an opening event has occurred. The aforementioned face vector extraction model can be a model that performs face detection and feature extraction processing on the input video information and outputs a set of face feature vectors. The aforementioned face vector extraction model can be a model composed of a face location detection model and a face feature extraction model connected in series. The aforementioned face location detection model can be a RetinaFace model, and the aforementioned face feature extraction model can be a face feature extraction network trained using the ArcFace loss function. For example, the aforementioned face feature extraction network can be ResNet50 (50-layer Residual Network). Then, a cosine similarity algorithm is used to perform similarity matching between the aforementioned face feature vector set and a preset face feature database to obtain identity information. The aforementioned preset facial feature database can be a pre-set database that stores the facial feature vectors of the individuals to be cared for.

[0026] The second step involves performing multi-dimensional behavior recognition on the aforementioned information about the person under care to obtain behavior identification information. This behavior identification information can be a label representing the behavior category of the person under care at the current moment. For example, the label for the behavior category can include, but is not limited to, at least one of the following: leaving the house, entering the house, approaching but not leaving, or lingering at the door. In practice, the executing entity can first input the aforementioned video information into a trained object detection model to obtain a sequence of object location information. The object location information in this sequence can be the bounding box coordinates of the person under care in a single video frame. The object detection model can be a model that performs object detection on the input video information and outputs a sequence of object location information. For example, the object detection model can be a YOLOv8s (You Only Look Once version 8 Small) model. Then, using the SORT (Simple Online and Realtime Tracking) algorithm, motion trajectory recognition is performed on the sequence of object location information to obtain object trajectory information. This object trajectory information can reflect the movement trajectory of the person under care in the aforementioned video information. Next, the trajectory information of the cared-for individual is input into a multi-dimensional behavior recognition model to obtain behavior identification information. This multi-dimensional behavior recognition model can be a model that extracts features and classifies the input trajectory information of the cared-for individual, outputting behavior identification information. This multi-dimensional behavior recognition model can be an XGBoost (eXtremeGradient Boosting) model.

[0027] The third step is to generate behavioral identification information based on the aforementioned identity information and behavioral identification information. In practice, the executing entity can use the aforementioned identity information and behavioral identification information as behavioral identification information.

[0028] Optionally, the above-mentioned object behavior recognition processing of the multimodal family care information to obtain care identification information may include the following steps: The first step is to obtain historical information on individuals requiring care. This historical information can be information on at least one individual requiring care obtained before the current time.

[0029] The second step involves generating time-series information about the individuals requiring care, based on the aforementioned information about historical individuals requiring care. This time-series information can be a vector sequence reflecting the status of individuals requiring care within the target family area over a preset period. The preset period can be a pre-defined time window ending at the current time. For example, the preset period could be 72 hours. In practice, the executing entity can use the PAA (Piecewise Aggregate Approximation) algorithm to encode the time-series features of the information about individuals requiring care and the historical information about historical individuals requiring care, obtaining a vector sequence of individuals requiring care as the time-series information.

[0030] The third step is to generate the status information of the person to be cared for based on the aforementioned time-series information. This status information can be information about different status categories of the person to be cared for within a preset period, along with their duration. These different status categories can include, but are not limited to, at least one of the following: active at home, inactive at home, and not at home. In practice, the executing entity can input the aforementioned time-series information into a trained status recognition model for the person to be cared for to obtain the status information. The aforementioned status recognition model for the person to be cared for can be a model that extracts temporal features and classifies the status from the input time-series information, outputting the status information. For example, the aforementioned status recognition model for the person to be cared for can be a model composed of two layers of LSTM (Long Short-Term Memory) network, a fully connected layer, and a Softmax classification layer connected in series.

[0031] The fourth step is to generate behavior recognition information based on the aforementioned status information. In practice, the executing entity can perform rule matching between the status information and a preset abnormal status database to obtain behavior recognition information. The preset abnormal status database can be a pre-defined database that stores a set of abnormal status condition information. The abnormal status condition information in the set can be rule information for determining whether the aforementioned status information is abnormal, and information with abnormal status labels. The abnormal status label can be the type of abnormality in the aforementioned status information. For example, the abnormal status label could be "The elderly person has not left home for 72 hours."

[0032] In addressing the technical problems mentioned above by adopting technical solutions, and considering the application scenario—a scenario requiring precise and safe care in complex daily family activities (e.g., multi-member mixed families with elderly members and children)—the following technical problem often arises: Due to significant differences in behavioral patterns among family members and the limited range of motion of those being cared for, methods based on motion trajectory for behavior recognition in this scenario have low accuracy (e.g., interaction and occlusion in the motion trajectories of multiple members lead to trajectory breaks). This results in a high false alarm rate during caregiving and early warning control, causing numerous ineffective control operations on the linked controlled devices, increasing their operating time and energy consumption, and ultimately reducing the safety and reliability of the care. Considering the following requirements for this application scenario: complex daily family activities, the ability to judge complex behavioral patterns, and sensitivity to subtle movements, we have decided to adopt the following solution: Optionally, the above-mentioned behavior recognition processing of the information of the person to be cared for to obtain behavior recognition information may include the following steps: The first step is to generate identification information based on the information of the person to be cared for. This step can be implemented by referring to the first step in some optional implementations of step 102 in certain embodiments, and will not be repeated here.

[0033] The second step involves extracting pose features from the video information, including the information about the object to be cared for, to obtain a pose feature vector. This pose feature vector reflects the object's positional changes and motion state within the video information. In practice, the executing entity can input the video information into a trained pose feature extraction model to obtain the pose feature vector. This pose feature extraction model can be a model that extracts and fuses motion features from the input video information, outputting a pose feature vector. It can be a model composed of a motion state extraction model and a feature fusion encoding model connected in series. The feature fusion encoding model can be a one-dimensional convolutional layer. Alternatively, it can be a model composed of a keypoint state extraction model and an optical flow motion feature extraction model connected in parallel. The keypoint state extraction model can be an HRNet-W32 (High-Resolution Network-Width 32) model. The optical flow motion feature extraction model can be a PWC-Net (Pyramid, Warping, and Cost Volume Network) model.

[0034] The third step involves performing multimodal feature enhancement processing on the activity monitoring information set included in the information of the person under care, resulting in an activity monitoring feature vector. This activity monitoring feature vector characterizes the environmental state of the target home area and the activities of the person under care within the home. In practice, the implementing entity can first determine the signal trigger frequency and average signal intensity of the activity monitoring information corresponding to the infrared sensor as the first monitoring feature vector. This first monitoring feature vector characterizes the frequency and amplitude of movement of the person under care. Then, using MFCC (Mel-Frequency Cepstral Coefficients), features are extracted from the activity monitoring information corresponding to the sound sensor to obtain a second monitoring feature vector. This second monitoring feature vector characterizes the sound information in the target home area environment. Finally, the first and second monitoring feature vectors are concatenated to obtain the activity monitoring feature vector.

[0035] The fourth step involves performing cross-modal feature aggregation on the aforementioned posture feature vector and activity monitoring feature vector based on the identification information, resulting in an aggregated multimodal feature vector. This aggregated multimodal feature vector represents the correlation between the posture feature vector and the activity monitoring feature vector. In practice, the executing entity can first use One-Hot Encoding to encode the identification information, obtaining an identification vector. This identification vector can be a numerical representation of the identification information. Then, the identification vector, posture feature vector, and activity monitoring feature vector are concatenated to obtain a concatenated multimodal feature vector. Finally, this concatenated multimodal feature vector is input into the trained cross-modal feature aggregation model to obtain the aggregated multimodal feature vector. The cross-modal feature aggregation model can be a model that extracts features and aggregates information from the input concatenated multimodal feature vector, outputting the aggregated multimodal feature vector. This model can be a model composed of a Transformer encoder and a pooling layer connected in series. For example, the pooling layer mentioned above could be a global average pooling layer.

[0036] The fifth step involves performing hierarchical behavior classification processing on the aggregated multimodal feature vectors to obtain initial behavior identification information. This initial behavior identification information can be semantic information reflecting the behavior, actions, and intentions of the person under care in the video information, along with their confidence levels. In practice, the executing entity can first input the aggregated multimodal feature vectors into the trained hierarchical behavior recognition model to obtain basic action labels, composite behavior labels, and behavior intention labels. The basic action labels can be labels reflecting the basic actions of the person under care and their corresponding confidence levels. For example, the basic action labels can include, but are not limited to, at least one of the following: walking, standing, sitting. The composite behavior labels can be information reflecting the current behavior of the person under care and its corresponding confidence levels. For example, the composite behavior label could be "loitering at the door." The behavior intention labels can be information reflecting the behavioral trends of the person under care and their corresponding confidence levels. For example, the behavior intention label could be "temporarily leaving." The hierarchical behavior recognition model can be a model composed of a basic action recognition model, a composite behavior recognition model, and a behavior intention recognition model connected in series. The basic action recognition model described above can be a model that performs feature classification on the input aggregated multimodal feature vector and outputs a basic action label. For example, the basic action recognition model can be an FCNN (Fully Connected Neural Network). The composite action recognition model described above can be a model that performs feature fusion and classification on the input basic action label and the aggregated multimodal feature vector and outputs a composite action label. The composite action recognition model can be a three-layer TCN (Temporal Convolutional Network). The behavior intent recognition model described above can be a model that performs feature extraction and linear projection on the input basic action label, composite action label, and aggregated multimodal feature vector and outputs a behavior intent label. The behavior intent recognition model can be a Transformer encoder. Then, the basic action label, the composite action label, and the behavior intent label described above are determined as the initial behavior identification information.

[0037] Step 6: Based on the aforementioned identity information, perform behavior detection processing on the aforementioned initial behavior identification information to obtain behavior recognition information. In practice, the executing entity can first use the aforementioned identity information as an index to perform rule matching between the aforementioned initial behavior identification information and a preset behavior detection rule base to obtain target behavior status identification information. The aforementioned target behavior status identification information can be information reflecting whether the behavior status of the person under care is abnormal, and the type of abnormal behavior status when abnormal. For example, the aforementioned abnormal behavior status type can be the type of falling in place and not moving for 60 seconds. The aforementioned preset behavior detection rule base can be a pre-set database that stores mapping rules between the initial behavior identification information and target behavior status identification information corresponding to different people under care. Step 7: Based on the aforementioned behavior recognition information, perform control operations on the controlled devices corresponding to the aforementioned target home area. The implementation method of this step can refer to the implementation method of steps 103-104, and will not be repeated here.

[0038] The above-mentioned technical solution and related content, as an inventive point of this disclosure, solves the second technical problem: "Increased energy consumption of controlled equipment, reduced safety of the person under care, and reduced reliability of safety care." Factors leading to increased energy consumption of controlled equipment, reduced safety of the person under care, and reduced reliability of safety care are often as follows: Due to the large differences in behavioral patterns among members in the above scenarios and the small range of activity of the person under care, the method of behavior recognition based on motion trajectory has low accuracy in recognizing the behavior of the person under care in the above scenarios. This results in a high false alarm rate during the care and early warning control process, causing the linked controlled equipment to perform a large number of invalid control operations, increasing the working time of the controlled equipment, leading to increased energy consumption of the controlled equipment, reduced safety of the person under care, and reduced reliability of safety care. Solving the above factors can achieve the effect of reducing energy consumption of controlled equipment, improving safety of the person under care, and improving the reliability of safety care. To achieve this effect, this disclosure first generates identity identification information based on the above-mentioned information of the person under care. Next, posture feature extraction is performed on the video information included in the information of the person under care, resulting in a posture feature vector. Compared to methods based on motion trajectories, posture feature extraction extracts more refined motion features of the person under care, and the resulting posture feature vector can be used to accurately distinguish different behaviors with similar trajectories. Subsequently, multimodal feature enhancement processing is performed on the activity monitoring information set included in the information of the person under care, resulting in an activity monitoring feature vector. Here, by extracting and fusing features from infrared, sound, and other sensor data, an activity monitoring feature vector is generated that comprehensively characterizes the environmental state of the target home area and the activities of the person under care indoors, effectively compensating for the insufficient posture feature vector information derived from single visual information. Then, based on the aforementioned identity information, cross-modal feature aggregation processing is performed on the posture feature vector and the activity monitoring feature vector to obtain an aggregated multimodal feature vector. Next, hierarchical behavior classification processing is performed on the aggregated multimodal feature vector to obtain initial behavior identification information. Finally, based on the aforementioned identity information, behavior detection processing is performed on the initial behavior identification information to obtain behavior recognition information. Here, hierarchical behavior classification processing can progressively deduce initial behavioral identifiers reflecting the actions, behaviors, and intentions of the person under care. Behavior detection processing then fuses these initial behavioral identifiers to obtain high-confidence behavior recognition information, thereby reducing the false alarm rate of subsequent control operations. Finally, based on the aforementioned behavior recognition information, control operations are performed on the controlled devices corresponding to the target home area. The improved accuracy of behavior recognition information reduces the false alarm rate in the caregiving early warning control process, thus reducing a large number of invalid control operations and lowering the energy consumption of the controlled devices.

[0039] In addressing the technical challenges of precise and safe care in complex family daily activities, the following technical problem often arises in the application scenarios: care scenarios where family members are in recovery or have chronic illnesses (e.g., elderly people recovering from surgery, patients with cardiovascular diseases, etc.). In these scenarios, the following third technical problem persists: the activities of the caregiver are limited, and changes in external activity are not obvious. Methods based on video information and activity detection for early warning and control can only identify whether their external behavioral intentions and activity states are abnormal; they cannot promptly detect potential physiological abnormalities in the caregiver in these scenarios. Anomalies exist in existing methods for caregiving and early warning control based on physiological monitoring data (e.g., heart rate data from smart bracelets). These methods typically perform simple anomaly checks by directly comparing the physiological monitoring data with fixed thresholds. However, the quality of this physiological monitoring data is unstable (e.g., data loss due to loose wear, abnormal fluctuations due to signal interference, and noise), resulting in a high false alarm rate and low accuracy. This leads to numerous ineffective control operations by the linked controlled devices, increasing their operating time, energy consumption, and battery life, ultimately reducing the safety of the person being cared for and lowering the reliability of safety monitoring. To address the following requirements for this application scenario: robustness to physiological monitoring data quality, the ability to identify deviations in physiological monitoring data, and the ability to jointly determine behavioral states and physiological anomalies, we have decided to adopt the following solution: Optionally, the above-mentioned control operation on the controlled device corresponding to the target home area based on the above-mentioned behavior recognition information may include the following steps: The first step involves performing data validity testing on the monitoring data sequence acquired from the target monitoring device to obtain a validity test value. The target monitoring device can be a device capable of monitoring the physiological signals of the person under care. This device may include, but is not limited to, at least one of the following: a smart home device with a millimeter-wave radar sensor (e.g., a smart bed), or a wearable health monitoring device worn by the person under care (e.g., a smartwatch). The monitoring data in the data sequence can be information reflecting the physiological signals (e.g., heart rate, respiratory rate, etc.) of the person under care within a preset period. The validity test value characterizes the quality of the monitoring data sequence. In practice, the executing entity can first use the `resample` function from the Pandas package to perform missing data percentage statistics on the monitoring data sequence, obtaining a missing rate index. This missing rate index can be a scalar reflecting the completeness of the monitoring data sequence. Then, a Hampel filter is used to perform outlier percentage statistics on the monitoring data sequence, obtaining an outlier rate index. This outlier rate index can be a scalar reflecting the proportion of abnormal data in the monitoring data sequence. Next, the outlier rate and missing rate indicators are weighted and fused to obtain the validity test value. This validity test value can be a scalar between 0 and 1. A higher validity test value indicates lower quality of the monitored data sequence.

[0040] The second step involves performing time-series data augmentation on the monitoring data sequence in response to the determination that the validity detection value is less than or equal to a preset validity detection threshold, resulting in an augmented monitoring data sequence. The validity detection threshold can be a pre-set critical value used to determine whether the monitoring data sequence requires further time-series data augmentation. For example, the validity detection threshold could be 0.8. The augmented monitoring data sequence can be a sequence that has undergone missing value imputation and smoothing / denoising. In practice, the executing entity can first use linear interpolation to imput missing values ​​in the monitoring data sequence, obtaining an imputed monitoring data sequence. Then, a moving average filter is used to smooth and denoise the imputed monitoring data sequence, resulting in a denoised monitoring data sequence, which serves as the augmented monitoring data sequence.

[0041] The third step involves extracting multi-dimensional temporal features from the enhanced monitoring data sequence to obtain a monitoring data feature vector set. The monitoring data feature vectors in this set characterize the amplitude distribution and temporal variations of the enhanced monitoring data sequence within continuous sampling segments. In practice, the executing entity can first use a sliding window algorithm to divide the enhanced monitoring data sequence into a set of monitoring data subsequences. This sliding window algorithm can be an algorithm that divides the enhanced monitoring data sequence according to a preset time window and a preset step size. For example, the preset time window could be 60 seconds, and the preset step size could be 10 seconds. The monitoring data subsequences in the set can be all enhanced monitoring data within a preset time window. Then, for each monitoring data subsequence in the set, the following determination steps are performed: First, the statistical characteristic values ​​of the monitoring data subsequence are determined as statistical feature vectors. These statistical characteristic values ​​can include: the mean, maximum, minimum, standard deviation, and variance of the monitoring data subsequence. The aforementioned statistical feature vector can characterize the level and fluctuation range of the parameters of the aforementioned monitoring data subsequence within the corresponding preset time window. The second step uses the FDM (Finite Difference Method) to determine the mean of the first derivatives of the aforementioned monitoring data subsequence as the trend feature vector. This trend feature vector can characterize the changing trend of the parameters of the aforementioned monitoring data subsequence. The third step involves concatenating the aforementioned statistical feature vector and the aforementioned trend feature vector to obtain the monitoring data feature vector.

[0042] The fourth step involves inputting the aforementioned monitoring data feature vector set into the trained deviation detection model to obtain the deviation value set of the enhanced monitoring data sequence. The deviation value in this set characterizes the degree to which the monitoring data sub-sequence corresponding to the aforementioned monitoring data feature vector deviates from the regular monitoring data. The regular monitoring data can be the monitoring data of the person under care in a stable daily state. The deviation detection model can be a model that performs sequence encoding and decoding reconstruction on the input monitoring data feature vector set and outputs deviation information. For example, the deviation detection model can be an autoencoder of an LSTM (Long Short-Term Memory) network.

[0043] The fifth step involves generating deviation event information for the enhanced monitoring data sequence based on the aforementioned deviation value set. This deviation event information can be structured information reflecting whether the physical condition of the person under monitoring is abnormal. For example, the deviation event information could be: a heart rate of 120 beats per minute for 10 minutes. In practice, the executing entity can first determine the monitoring data deviation information set by matching each deviation value in the aforementioned deviation value set with its corresponding monitoring data subsequence. Then, the monitoring data deviation information set is matched with a preset deviation event rule base to obtain the deviation event information. The preset deviation event rule base can be a pre-defined database that stores the monitoring data deviation information set and the rules for judging various types of deviation event information.

[0044] The sixth step is to identify the above deviation event information and the above behavior identification information as target behavior identification information.

[0045] Step 7: Based on the target behavior recognition information mentioned above, control operations are performed on the controlled devices corresponding to the target home area. The implementation method of this step can refer to the implementation method of steps 103-104, and will not be repeated here.

[0046] The above-described technical solution and its related content, as an inventive point of this disclosure, solve the third technical problem: "Increased energy consumption of controlled devices, reduced battery life, decreased safety of the person under care, and reduced reliability of safety monitoring." Factors leading to increased energy consumption of controlled devices, reduced battery life, decreased safety of the person under care, and reduced reliability of safety monitoring are often as follows: In the above scenarios, the activities of the person under care are restricted, and changes in external activities are not obvious. Methods for monitoring and controlling the person under care based on video information and activity monitoring information can only identify whether their external behavioral intentions and activity states are abnormal, but cannot promptly detect potential physiological abnormalities in the person under care in the above scenarios. Existing methods for monitoring and controlling the person under care based on physiological monitoring data typically directly perform simple anomaly judgments by comparing the physiological monitoring data with fixed thresholds. Due to the unstable quality of the aforementioned physiological monitoring data, the false alarm rate and accuracy of monitoring and controlling the person under care are high, leading to a large number of ineffective control operations by the linked controlled devices. This increases the working time of the controlled devices, resulting in increased energy consumption, reduced battery life, decreased safety of the person under care, and reduced reliability of safety monitoring. Solving the above factors can reduce the energy consumption of controlled equipment, increase battery life, improve the safety of the monitored objects, and enhance the reliability of security monitoring. To achieve this effect, this disclosure first performs data validity detection processing on the monitoring data sequence obtained from the target monitoring device to obtain a validity detection value. Then, in response to determining that the validity detection value is less than or equal to a preset validity detection threshold, the monitoring data sequence undergoes time-series data augmentation processing to obtain an enhanced monitoring data sequence. Here, through validity detection, the degree of data missingness and data anomaly in the monitoring data sequence is quantified into a validity detection value that reflects the quality of the monitoring data sequence, enabling the identification and gating of low-quality data. Monitoring data sequences with validity detection values ​​less than or equal to the preset validity detection threshold are then augmented and denoised to reduce invalid calculations and false triggers on abnormal data, thereby reducing processing load and energy consumption caused by invalid linkages. Next, multi-dimensional time-series features are extracted from the enhanced monitoring data sequence to obtain a monitoring data feature vector set. Subsequently, the monitoring data feature vector set is input into a trained deviation detection model to obtain a deviation value set for the enhanced monitoring data sequence. Next, based on the aforementioned deviation value set, deviation event information for the enhanced monitoring data sequence is generated. Here, the enhanced monitoring data sequence is transformed from a continuous numerical sequence into deviation event information with time range, amplitude characteristics, and type semantics. This deviation event information accurately reflects abnormalities in the physiological dimensions of the person under care, improving the coverage and accuracy of subsequent control operations on the controlled devices. Then, the aforementioned deviation event information and the aforementioned behavior recognition information are used to determine the target behavior recognition information. Finally, based on the aforementioned target behavior recognition information, control operations are performed on the controlled devices corresponding to the target home area.In this context, for care scenarios where family members are in recovery or have chronic illnesses requiring care, the target behavior recognition information, which includes deviation event information, can effectively supplement the shortcomings of single behavior recognition, improve the accuracy of care warning and control processes, thereby reducing a large number of invalid control operations, lowering the energy consumption of controlled equipment, increasing battery life, and enhancing the reliability and comprehensiveness of safe care.

[0047] Step 103: Perform early warning level identification on the behavior identification information to obtain early warning level identification information.

[0048] In some embodiments, the aforementioned executing entity can perform warning level identification on the aforementioned behavior identification information to obtain warning level identification information. The warning level identification information may reflect the degree of abnormality of the aforementioned behavior identification information. The warning level identification information may include: a warning level identifier and warning reminder information. The warning level identifier may be an identifier reflecting the severity of the warning or requiring periodic push notifications. The warning level identifier may include: a first-level warning identifier, a second-level warning identifier, and a third-level warning identifier. The warning reminder information may be text information that issues warning information of different levels through the warning level identifier. In practice, the aforementioned executing entity can perform warning level identification on the behavior identification information according to a target user rule engine to obtain warning level identification information. The target user rule engine may be a pre-set rule processing system used to match the aforementioned behavior identification information with user-defined rule conditions to obtain the warning level identification information corresponding to the aforementioned behavior identification information.

[0049] Step 104: Based on the warning level identification information, control the controlled devices corresponding to the target household area.

[0050] In some embodiments, the aforementioned executing entity can control the controlled devices corresponding to the target family area based on the aforementioned warning level identification information. The control operations may include, but are not limited to, at least one of the following: pushing warning reminder information corresponding to the aforementioned warning level identification information to the user terminal, activating alarm prompts, and temporarily locking the door. The types of the aforementioned warning reminder information may include, but are not limited to, at least one of the following: warning reminder information characterizing the current behavior of the person being cared for, and warning reminder information characterizing the duration of the current state of the person being cared for. The duration of the current state of the person being cared for also includes the health status of the person being cared for. For example, the warning reminder information characterizing the current behavior of the person being cared for could be "Note: The elderly person went out alone at 22:10 at night"; the warning reminder information characterizing the duration of the current state of the person being cared for could be "Note: The elderly person has not gone out within 72 hours"; the warning reminder information corresponding to the health status could be "Note: The person being cared for has been monitored to be out of bed for more than 1 hour at night for 2 consecutive days." In practice, the aforementioned executing entity can first match the aforementioned warning level identification information with a preset controlled device control command database to obtain device control commands. The aforementioned equipment control commands can be executable computer instructions that control the controlled equipment. The aforementioned preset controlled equipment control command database can be a pre-set database that stores warning level identification information and mapping relationships between equipment control commands for different controlled devices. Then, the controlled equipment is controlled to execute the control operations corresponding to the aforementioned equipment control commands.

[0051] In some optional implementations of certain embodiments, the control operation on the controlled device corresponding to the target home area based on the aforementioned warning level identification information may include the following steps: The first step involves controlling the controlled device to perform a first control operation when the aforementioned warning level identification information indicates a Level 1 warning. The Level 1 warning information may include a warning level identifier as a Level 1 warning identifier. This Level 1 warning information may reflect changes in the daily behavior or status of the person under care. For example, the warning reminder message corresponding to the Level 1 warning information could be "Reminder: Your child returned home at 17:30." The first control operation may involve controlling the communication device to push the warning reminder message corresponding to the warning level identification information to the user terminal. This first control operation may include, but is not limited to, at least one of the following: pushing the warning reminder message to the primary bound user terminal (e.g., a family administrator's mobile phone), or generating a daily timed (e.g., 20:00) behavior report of the person under care (e.g., number of times they went out, return time, indoor activity time) and pushing it to the primary bound terminal.

[0052] The second step involves controlling the controlled device to perform a second control operation when the aforementioned warning level identification information indicates a second-level warning. The second-level warning information may include a warning level identifier as a second-level warning identifier. This information may reflect a potentially abnormal risk behavior state of the person under care. For example, such potentially abnormal risk behavior states may include: attempting to leave during prohibited hours, entering prohibited areas, leaving home alone at night, or a continuous deviation in physiological monitoring parameters (e.g., monitoring the person under care for more than one hour of nighttime activity for two consecutive days). For example, the warning reminder information corresponding to the second-level warning information could be "Note: The child is attempting to leave during prohibited hours." The second control operation may include, but is not limited to, at least one of the following: simultaneously pushing the warning reminder information to the primary bound user terminal and at least one secondary bound user terminal (e.g., other family members); pushing multiple daily behavior reports (e.g., once each at 12:00 and 20:00) to the user terminal; controlling the smart door lock to temporarily lock; activating camera tracking and recording; and activating the smart speaker to provide voice prompts. The aforementioned second control operation can simultaneously control one or more controlled devices to operate. For example, when the warning level identification information indicates that a child intends to go out during prohibited hours, the second control operation can first control the smart door lock to temporarily lock and prevent the child from accidentally leaving the house. Then, it can control the communication device to simultaneously push the aforementioned warning reminder information to the primary bound user terminal and at least one secondary bound user terminal to remind the parents. Finally, it can activate the smart speaker to play the warning information, etc.

[0053] Third, when the aforementioned warning level identification information is information representing a third-level warning, the controlled device is controlled to perform a third control operation. The aforementioned third-level warning information may include a warning level identifier as a third-level warning identifier. The aforementioned third-level warning information may reflect an emergency abnormal behavior state of the person under care. For example, the aforementioned emergency abnormal behavior state may include: falling and remaining motionless, significant and continuous deviation of physiological monitoring parameters (e.g., heart rate parameter exceeding a preset emergency upper limit threshold for 5 consecutive minutes), etc. For example, the warning reminder information corresponding to the aforementioned third-level warning information may be "Alarm: An elderly person has been detected to have fallen in the living room and has remained motionless for 60 seconds." The aforementioned third control operation may include, but is not limited to, at least one of the following: pushing an emergency alarm to all bound user terminals, dialing a preset voice call (e.g., emergency contact number, 120 emergency number), controlling the alarm to activate the sound and light alarm, activating the lighting system to activate high-brightness mode, and activating the camera for tracking and recording. The aforementioned emergency alarm may utilize an enhanced reminder mechanism to simultaneously push the aforementioned warning reminder information to all bound user terminals. For example, the aforementioned enhanced reminder mechanism may include, but is not limited to, at least one of the following: adding sound prompts to the pushed content, repeated reminders (e.g., repeated pushes every 30 seconds) until user confirmation (e.g., user confirmation of whether to immediately call 120 emergency services), and pushing video or audio clips captured by cameras in the area where the person under care is located to all bound user terminals. The aforementioned third control operation may simultaneously control one or more controlled devices to operate. For example, when the warning reminder information corresponding to the aforementioned third-level warning information is "Alarm: An elderly person has been detected to have fallen in the living room and has not moved for 60 seconds," the aforementioned third control operation may first control the communication device to simultaneously push the aforementioned warning reminder information to all bound user terminals using the enhanced reminder mechanism. Then, activate the lighting system to turn on high brightness mode and control the smart speaker to activate sound and light alarms. Subsequently, activate the camera to track and record the person under care, and push the video clips captured by the camera to all bound user terminals. Afterward, in response to meeting the preset dialing conditions (e.g., receiving confirmation information from the user to dial 120 emergency services, or the user not confirming for 60 seconds), control the communication device to dial 120 emergency services. Finally, the smart lock determines whether to unlock or keep locked based on whether someone is present and their identity.

[0054] The above embodiments of this disclosure have the following beneficial effects: The smart home care monitoring and early warning control method of some embodiments of this disclosure can reduce the energy consumption of controlled devices and improve battery life. Specifically, the reason for the increased energy consumption and decreased battery life of controlled devices is that the monitoring information comes from basic state information obtained by a single or a small number of sensors and is easily affected by environmental interference, resulting in a high false alarm rate during the care monitoring and early warning control process. This causes the linked controlled devices to perform a large number of invalid control operations, increasing the working time of the controlled devices, leading to increased energy consumption and decreased battery life. Based on this, the smart home care monitoring and early warning control method of some embodiments of this disclosure can first obtain the information of the object to be cared for in the target home area. Here, the information of the object to be cared for is multi-dimensional and multi-modal, providing data support for subsequent behavior recognition processing. Then, the behavior recognition processing is performed on the above information of the object to be cared for to obtain behavior recognition information. Here, behavior recognition processing is performed on the multimodal information of the person under care, transforming the original monitoring information into behavior recognition information with semantic meaning of the behavior of the person under care. This behavior recognition information accurately reflects the current behavioral state of the person under care, improving the accuracy of subsequent warning level identification and thus reducing the false alarm rate of subsequent control operations. Next, the behavior recognition information is used to identify warning levels, resulting in warning level identification information. Here, the behavior recognition information is mapped to warning level identification information reflecting the degree of abnormality in the behavior of the person under care, reducing the possibility of misjudging low-risk abnormal behavior as a high-level warning and lowering the false alarm rate. Finally, based on the warning level identification information, control operations are performed on the controlled devices corresponding to the target home area. Here, a tiered control strategy is executed according to the warning level identification information, ensuring that the controlled devices only enter the working state under necessary warning levels, reducing the possibility of increased operating time due to ineffective control operations. Therefore, this smart home care warning control method can reduce the energy consumption of controlled devices and improve their battery life.

[0055] Further reference Figure 2 As an implementation of the methods shown in the above figures, this disclosure provides some embodiments of a smart home monitoring and early warning control device. These device embodiments are similar to... Figure 1 Corresponding to the method embodiments shown, this smart home monitoring and early warning control device can be specifically applied to various electronic devices.

[0056] like Figure 2As shown, a smart home care monitoring and early warning control device 200 includes: an acquisition unit 201, a behavior recognition unit 202, an early warning level recognition unit 203, and a control unit 204. The acquisition unit is configured to acquire information about the object to be monitored in a target home area. The behavior recognition unit is configured to perform behavior recognition processing on the information about the object to be monitored to obtain behavior recognition information. The early warning level recognition unit is configured to perform early warning level recognition on the behavior recognition information to obtain early warning level recognition information. The control unit is configured to perform control operations on the controlled devices corresponding to the target home area based on the early warning level recognition information.

[0057] It is understandable that the various units described in the smart home monitoring and early warning control device 200 are related to the reference. Figure 1 The steps in the described method correspond accordingly. Therefore, the operations, features, and beneficial effects described above for the method also apply to the smart home monitoring and early warning control device 200 and the units contained therein, and will not be repeated here.

[0058] The following is for reference. Figure 3 It shows a schematic diagram of the structure of an electronic device (e.g., an electronic device) 300 suitable for implementing some embodiments of the present disclosure. Figure 3 The electronic device shown is merely an example and should not be construed as limiting the functionality and scope of the embodiments of this disclosure.

[0059] like Figure 3 As shown, the electronic device 300 may include a processing unit (e.g., a central processing unit, a graphics processing unit, etc.) 301, which can perform various appropriate actions and processes according to a program stored in a read-only memory (ROM) 302 or a program loaded from a storage device 308 into a random access memory (RAM) 303. The RAM 303 also stores various programs and data required for the operation of the electronic device 300. The processing unit 301, ROM 302, and RAM 303 are interconnected via a bus 304. An input / output (I / O) interface 305 is also connected to the bus 304.

[0060] Typically, the following devices can be connected to I / O interface 305: input devices 306 including, for example, touchscreens, touchpads, keyboards, mice, cameras, microphones, accelerometers, gyroscopes, etc.; output devices 307 including, for example, liquid crystal displays (LCDs), speakers, vibrators, etc.; storage devices 308 including, for example, magnetic tapes, hard disks, etc.; and communication devices 309. Communication device 309 allows electronic device 300 to communicate wirelessly or wiredly with other devices to exchange data. Although Figure 3An electronic device 300 with various devices is shown; however, it should be understood that it is not required to implement or possess all of the devices shown. More or fewer devices may be implemented or possessed alternatively. Figure 3 Each box shown can represent a device or multiple devices as needed.

[0061] In particular, according to some embodiments of this disclosure, the processes described above with reference to the flowcharts can be implemented as computer software programs. For example, some embodiments of this disclosure include a computer program product comprising a computer program carried on a computer-readable medium, the computer program containing program code for performing the methods shown in the flowcharts. In such embodiments, the computer program can be downloaded and installed from a network via communication device 309, or installed from storage device 308, or installed from ROM 302. When the computer program is executed by processing device 301, it performs the functions defined in the methods of some embodiments of this disclosure.

[0062] It should be noted that, in some embodiments of this disclosure, the computer-readable medium described above may be a computer-readable signal medium or a computer-readable storage medium, or any combination thereof. A computer-readable storage medium may be, for example,—but not limited to—an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any combination thereof. More specific examples of a computer-readable storage medium may include, but are not limited to: an electrical connection having one or more wires, a portable computer disk, a hard disk, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fiber, portable compact disk read-only memory (CD-ROM), optical storage device, magnetic storage device, or any suitable combination thereof. In some embodiments of this disclosure, a computer-readable storage medium may be any tangible medium containing or storing a program that can be used by or in conjunction with an instruction execution system, apparatus, or device. In some embodiments of this disclosure, a computer-readable signal medium may include a data signal propagated in baseband or as part of a carrier wave, carrying computer-readable program code. Such propagated data signals may take various forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination thereof. A computer-readable signal medium can be any computer-readable medium other than a computer-readable storage medium, which can send, propagate, or transmit a program for use by or in connection with an instruction execution system, apparatus, or device. The program code contained on the computer-readable medium can be transmitted using any suitable medium, including but not limited to: wires, optical fibers, RF (radio frequency), etc., or any suitable combination thereof.

[0063] In some implementations, clients and servers can communicate using any currently known or future-developed network protocol such as HTTP (Hypertext Transfer Protocol) and can interconnect with digital data communication (e.g., communication networks) of any form or medium. Examples of communication networks include local area networks (“LANs”), wide area networks (“WANs”), the Internet (e.g., the Internet of Things), and peer-to-peer networks (e.g., ad hoc peer-to-peer networks), as well as any currently known or future-developed networks.

[0064] The aforementioned computer-readable medium may be included in the aforementioned electronic device; or it may exist independently and not assembled into the electronic device. The aforementioned computer-readable medium carries one or more programs that, when executed by the electronic device, cause the electronic device to: acquire information about a person requiring care in a target home area; perform behavior recognition processing on the information about the person requiring care to obtain behavior recognition information; perform warning level identification on the behavior recognition information to obtain warning level identification information; and, based on the warning level identification information, perform control operations on the controlled devices corresponding to the target home area.

[0065] Computer program code for performing operations of some embodiments of this disclosure can be written in one or more programming languages ​​or a combination thereof, including object-oriented programming languages ​​such as Java, Smalltalk, and C++, and conventional procedural programming languages ​​such as the "C" language or similar programming languages. The program code can be executed entirely on the user's computer, partially on the user's computer, as a standalone software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In cases involving remote computers, the remote computer can be connected to the user's computer via any type of network—including a local area network (LAN) or a wide area network (WAN)—or can be connected to an external computer (e.g., via the Internet using an Internet service provider).

[0066] The flowcharts and block diagrams in the accompanying drawings illustrate the architecture, functionality, and operation of possible implementations of systems, methods, and computer program products according to various embodiments of this disclosure. In this regard, each block in a flowchart or block diagram may represent a module, segment, or portion of code containing one or more executable instructions for implementing a specified logical function. It should also be noted that in some alternative implementations, the functions indicated in the blocks may occur in a different order than those indicated in the drawings. For example, two consecutively indicated blocks may actually be executed substantially in parallel, and they may sometimes be executed in reverse order, depending on the functions involved. It should also be noted that each block in the block diagrams and / or flowcharts, and combinations of blocks in the block diagrams and / or flowcharts, can be implemented using a dedicated hardware-based system that performs the specified function or operation, or using a combination of dedicated hardware and computer instructions.

[0067] The units described in some embodiments of this disclosure can be implemented in software or hardware. The described units can also be housed in a processor; for example, a processor may be described as including an acquisition unit, a behavior recognition unit, a warning level recognition unit, and a control unit. The names of these units do not necessarily limit the specific unit; for example, the acquisition unit may also be described as "a unit that acquires information about the individuals to be cared for in a target home area."

[0068] The functions described above in this document can be performed at least in part by one or more hardware logic components. For example, exemplary types of hardware logic components that can be used, without limitation, include: field-programmable gate arrays (FPGAs), application-specific integrated circuits (ASICs), application-specific standard products (ASSPs), system-on-a-chip (SoCs), complex programmable logic devices (CPLDs), and so on.

[0069] The above description is merely a selection of preferred embodiments of this disclosure and an explanation of the technical principles employed. Those skilled in the art should understand that the scope of the invention involved in the embodiments of this disclosure is not limited to technical solutions formed by specific combinations of the above-described technical features, but should also cover other technical solutions formed by arbitrary combinations of the above-described technical features or their equivalents without departing from the above-described inventive concept. For example, technical solutions formed by substituting the above-described features with (but not limited to) technical features with similar functions disclosed in the embodiments of this disclosure.

Claims

1. A smart home monitoring and early warning control method, comprising: Obtain information on caregivers in the target family area; The information of the person to be cared for is processed by behavior recognition to obtain behavior recognition information; The behavior recognition information is used to identify the warning level, thereby obtaining warning level recognition information; Based on the warning level identification information, control operations are performed on the controlled devices corresponding to the target household area.

2. The method according to claim 1, wherein, The step of performing behavior recognition processing on the information of the person to be cared for to obtain behavior recognition information includes: The information of the person to be cared for is used to identify their identity, thereby obtaining identity information; The information of the person to be cared for is subjected to multi-dimensional behavior recognition to obtain behavior identification information; Based on the identity information and the behavior information, behavior identification information is generated.

3. The method according to claim 1, wherein, The step of performing behavior recognition processing on the information of the person to be cared for to obtain behavior recognition information includes: Obtain information on individuals requiring historical care. Based on the information of the person to be cared for and the historical information of the person to be cared for, the time sequence information of the person to be cared for is generated; Based on the time sequence information, generate the status information of the object to be cared for; Based on the state information, behavior recognition information is generated.

4. The method according to any one of claims 1-3, wherein, The step of controlling the controlled devices corresponding to the target household area based on the warning level identification information includes: When the warning level identification information is information representing a first-level warning, the controlled device is controlled to perform a first control operation; When the warning level identification information is information representing a second-level warning, the controlled device is controlled to perform a second control operation. When the warning level identification information is information representing a third-level warning, the controlled device is controlled to perform a third control operation.

5. The method according to claim 1, wherein, The step of performing behavior recognition processing on the information of the person to be cared for to obtain behavior recognition information, and controlling the controlled devices corresponding to the target family area based on the behavior recognition information, includes: Based on the information of the person to be cared for, generate identification information; The video information included in the information of the object to be cared for is processed by posture feature extraction to obtain the posture feature vector of the object to be cared for. Multimodal feature enhancement processing is performed on the activity monitoring information set included in the information of the object to be cared for to obtain an activity monitoring feature vector; Based on the identity information, cross-modal feature aggregation processing is performed on the posture feature vector and the activity monitoring feature vector to obtain an aggregated multimodal feature vector; The aggregated multimodal feature vectors are subjected to hierarchical behavior classification processing to obtain initial behavior identification information; Based on the identity information, the initial behavior identification information is processed by behavior detection to obtain behavior recognition information; Based on the behavior recognition information, control operations are performed on the controlled devices corresponding to the target home area.

6. The method according to claim 5, wherein, The step of controlling the controlled devices corresponding to the target home area based on the behavior recognition information includes: The monitoring data sequence obtained from the target monitoring device is processed to perform data validity detection, and the validity detection value is obtained. In response to determining that the validity detection value is less than or equal to a preset validity detection threshold, the monitoring data sequence is subjected to time-series data augmentation processing to obtain an enhanced monitoring data sequence; Multidimensional time-series feature extraction is performed on the enhanced monitoring data sequence to obtain a monitoring data feature vector set; The feature vector set of the monitoring data is input into the trained deviation detection model to obtain the deviation value set of the enhanced monitoring data sequence; Based on the set of deviation values, deviation event information of the enhanced monitoring data sequence is generated; The deviation event information and the behavior recognition information are identified as target behavior recognition information; Based on the target behavior recognition information, control operations are performed on the controlled devices corresponding to the target home area.

7. A smart home monitoring and early warning control device, comprising: The acquisition unit is configured to acquire information about caregivers in a target household area. The behavior recognition unit is configured to perform behavior recognition processing on the information of the object to be cared for, and obtain behavior recognition information; The warning level identification unit is configured to identify the warning level of the behavior identification information to obtain warning level identification information. The control unit is configured to control the controlled devices corresponding to the target home area based on the warning level identification information.

8. A smart door lock, comprising: One or more processors; Storage device, on which one or more programs are stored, When the one or more programs are executed by the one or more processors, the one or more processors implement the method as described in any one of claims 1-6.

9. A type of door, wherein, The door includes the smart door lock as described in claim 8.

10. A computer-readable medium having a computer program stored thereon, wherein, When the computer program is executed by a processor, it implements the method as described in any one of claims 1-6.

Images