Platform for monitoring home safety of old people based on decision engine multi-device linkage and monitoring method

Abstract

The invention discloses a platform and a monitoring method for monitoring the home safety of old people based on decision engine multi-device linkage, and the method comprises the steps: carrying out the data preprocessing according to the human body home data collected by an edge computing device; according to the human body home data, multi-modal data fusion is carried out to generate behavior identification data, and the multi-modal data fusion comprises one or more of time sequence alignment, modal completion and feature extraction; inputting the behavior recognition data into a local recognition model for real-time behavior analysis, and inputting the behavior recognition data into a cloud decision engine for multi-source data comprehensive deep analysis; and according to an output result of the local identification model and / or the cloud decision engine, performing operation of matching the smart home equipment and / or home behavior alarm processing. According to the invention, multi-device data integration and cooperative linkage measurement can be adopted to monitor the home safety of the elderly.

Description

Technical Field

[0001] The present invention relates to the technical field of security monitoring, and in particular to a platform and a monitoring method for monitoring the safety of elderly people at home based on a decision engine and multi-device linkage. Background Art

[0002] With the advent of an aging society around the world, the number of elderly people living alone has increased significantly. They may face a variety of safety hazards in their daily lives, such as falls, sudden illnesses or accidents. These problems are often aggravated by the fact that no one discovers them in time, which puts great pressure on the elderly's families and social medical resources. In recent years, with the rapid development of technologies such as the Internet of Things (IoT), artificial intelligence (AI) and edge computing, home monitoring systems have gradually upgraded from traditional video monitoring modes to multi-device monitoring modes. These devices include but are not limited to: wearable devices: such as smart bracelets or heart rate monitoring devices, which are used to collect health data of the elderly in real time; environmental sensors: such as smoke alarms, gas sensors and door and window status monitors, which are used to monitor safety hazards in the home environment; smart home devices: such as cameras, smart speakers and emergency buttons, which are used to provide video monitoring and alarm functions.

[0003] However, although the existing home device monitoring system has improved the safety of elderly people at home to a certain extent, the various monitoring devices contained in the existing home device monitoring system are usually independent of each other and cannot achieve efficient data integration and linkage. For example, the data collected by smart bracelets and cameras are usually processed by different cloud platforms, resulting in information island problems and reducing the accuracy and response speed of abnormal situation judgment.

[0004] Therefore, there is an urgent need for a platform and a monitoring method for monitoring the safety of the elderly at home that can integrate data from multiple devices and adopt collaborative linkage measurements. Summary of the invention

[0005] Purpose of the invention: In order to overcome the above shortcomings, the purpose of the present invention is to provide a platform and monitoring method for monitoring the safety of the elderly at home based on a decision engine and multiple devices linked together.

[0006] In order to solve the above technical problems, the present invention provides a monitoring method for monitoring the safety of the elderly at home based on a decision engine and multiple devices linked together, comprising the following steps: Step S1: Perform data preprocessing based on human home data collected by edge computing devices; Step S2: performing multimodal data fusion according to the human body home data to generate behavior recognition data, wherein the multimodal data fusion includes one or more of time series alignment, modality completion, and feature extraction; Step S3: inputting the behavior recognition data into a local recognition model for real-time behavior analysis, and at the same time inputting the behavior recognition data into a cloud decision engine for comprehensive in-depth analysis of multi-source data; Step S4: According to the output results of the local recognition model and / or the cloud decision engine, matching smart home device operations and / or home behavior alarm processing are performed.

[0007] As a preferred embodiment of the present application, in step S1, the method comprises the following steps: Step S11: Collecting the elderly’s behavior images, expression changes, home environment data, elderly’s body data, and smart home device operation data through edge computing devices; Step S12: De-noising and filtering the collected data; Step S13: normalize the collected data to generate human home data.

[0008] 3. According to claim 2, a monitoring method for monitoring the safety of the elderly at home based on the linkage of multiple devices of a decision engine, characterized in that in step S2, the method comprises the following steps: Step S21: according to the human body home data, align the data with uneven time intervals using a linear interpolation method, so as to synchronize the data in the human body home data to a unified time axis through a timestamp; Step S22: Based on the human body home data, the missing data is supplemented using KNN interpolation or statistical methods; Step S23: extracting behavioral features and health features from the aligned and completed data, wherein the behavioral features include one or more of motion trajectories, home environment data, and smart home device operation data, and the health features include one or more of the elderly's facial expression changes and human body data.

[0009] As a preferred embodiment of the present application, in step S3, the method comprises the following steps: Step S31: Generate a behavior data set based on the collected historical behavior recognition data; Step S32: Use the pre-trained MobileNet model and fine-tune it on the behavior dataset using the fine-tuning loss function: in, is the cross entropy loss function, For the real result, is the prediction result, N is the number of data; Step S33: Perform quantization-aware training on the fine-tuned MobileNet model: in, w is the model weight or activation value, ∆ is the quantization step size, and w max and minimum value w min calculate: , b is the quantization bit width; Step S34: convert the trained model into the TensorFlow Lite format and use a compiler to load the trained model into a local edge computing device to form a local recognition model.

[0010] As a preferred embodiment of the present application, in step S3, the method further comprises the following steps: Step S35: Generate a behavior data set based on the collected historical behavior recognition data; Step S36: Extracting behavioral features from the behavioral dataset using an unsupervised learning method: in, is the clustering loss function, is the i-th data point in the behavior dataset, is the jth cluster center; and then the daily behavior pattern model of the elderly at home is constructed; Step S37: Use reinforcement learning method to optimize the strategy of daily behavior pattern model of elderly people at home: in, α is the learning rate, r For instant rewards, γ is the discount factor, is the current state and action, For the next state and action; Step S38: deploy the trained and optimized daily behavior pattern model of the elderly at home to the cloud server to generate a cloud decision engine.

[0011] As a preferred embodiment of the present application, in step S3, the method further comprises the following steps: Step S301: inputting behavior recognition data into a local recognition model and accepting the instant behavior detection result output by the local model; Step S302: judging whether the elderly person at home has abnormal behavior according to the instant behavior detection result; Step S303: If yes, a matching abnormal behavior result is generated and the abnormal behavior result and the behavior identification data are sequentially input into the cloud-based decision engine to perform an abnormal behavior potential risk analysis and a comprehensive in-depth analysis of multi-source data to generate a first behavior decision; if no, the behavior identification data is input into the cloud-based decision engine to perform a comprehensive in-depth analysis of multi-source data to generate a second behavior decision.

[0012] As a preferred embodiment of the present application, after collecting the human body home data, the method further comprises the following steps: Step S101: extracting behavioral images, expression changes, and voice signals of elderly people at home based on human body data collected by edge computing devices, and fusing the extracted data to generate human emotion features; Step S102: detecting the emotional state of the elderly at home according to the human emotional characteristics and generating a care strategy according to the emotional state and the activity level of the elderly at home; Step S103: According to the care strategy, control the matching smart home device to perform the first care operation and / or the second care operation.

[0013] As a preferred embodiment of the present application, after collecting the human body home data, the method further comprises the following steps: Step S1001: judging whether there are smart home devices and human-worn positioning devices containing indoor positioning data according to the human-home data; Step S1002: If yes, extract all positioning data and generate beacon coordinates according to the installation location of the smart home device containing indoor positioning data; if no, cancel the next step; Step S1003: constructing a virtual home geo-fence according to the beacon coordinates and monitoring range requirements; Step S1004: Calculate the real-time position of the elderly at home using trilateration based on the indoor mobile positioning data of the positioning device worn by the human body: ,in, is the coordinate of the i-th beacon, is the distance of the i-th beacon; Step S1005: judging whether the elderly person at home has entered a set danger zone based on the calculated real-time location of the elderly person at home and in combination with the virtual home geographic fence; Step S1006: If yes, preventive operations and / or home behavior alarm processing are performed on the smart home devices in the set dangerous area. If no, return to step S1004 and recalculate the real-time position of the elderly at home.

[0014] As a preferred embodiment of the present application, in step S1001, the method further includes the following steps: Step S10011: assigning a risk area to each constructed virtual home geo-fence, wherein the risk area includes a low-risk area, a medium-risk area, and a dangerous area; Step S10012: Dynamically adjust the boundary range and / or fence position and / or risk level of each virtual home geo-fence based on time information and behavior patterns of elderly people at home.

[0015] The present invention also provides a platform for monitoring the safety of the elderly at home based on a decision engine and multi-device linkage using the monitoring method, comprising: Cloud servers; Smart home devices; Edge computing devices are used to collect behavioral images, facial expression changes, home environment data, body data of the elderly, and operation data of smart home devices; The control unit is connected to the cloud server, smart home devices, edge computing devices, human-worn positioning devices, and terminal devices, respectively, and includes: A data processing module is used to perform data preprocessing based on the human body home data collected by the edge computing device; A data fusion module, used to perform multimodal data fusion according to the human body home data to generate behavior recognition data, wherein the multimodal data fusion includes one or more of time series alignment, feature level fusion, and decision level fusion; A behavior analysis module is used to input the behavior recognition data into a local recognition model for real-time behavior analysis, and at the same time input the behavior recognition data into a cloud decision engine for comprehensive in-depth analysis of multi-source data; The behavior processing module is used to match the operation of smart home devices and / or home behavior alarm processing according to the output results of the local recognition model and / or the cloud decision engine.

[0016] The present invention also provides a computer medium having a computer program stored thereon, and the computer program is executed by a processor to implement the method for monitoring the safety of the elderly at home based on the linkage of multiple devices using a decision engine.

[0017] The present invention also provides a computer, comprising the computer medium.

[0018] The above technical solution of the present application has the following advantages compared with the prior art: 1. Utilize edge computing devices and multimodal data fusion technology to collect and pre-process multi-source data such as behavioral images, facial expression changes, and home environment data of the elderly at home to avoid data isolation problems between smart home devices; integrate scattered data into unified behavior recognition data through methods such as time series alignment, feature-level fusion, and decision-level fusion, thereby improving the accuracy and response speed of abnormal situation judgment.

[0019] 2. The design of local and cloud dual models linkage can effectively reduce missed reports and false alarms, and ensure timely response to abnormal behaviors of elderly people at home.

[0020] 3. It can initiate a variety of linkage measurements based on home scenarios. For example, when an elderly person falls, it can turn off the gas, turn on the lights, and make an emergency call. Through multi-dimensional linkage, the overall actual efficiency of the platform is improved. And the introduction of virtual geographic fences enables the platform to monitor the location of the elderly at home in real time, promptly identify and prevent entry into dangerous areas, and further enhance home safety protection capabilities. BRIEF DESCRIPTION OF THE DRAWINGS

[0021] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on the provided drawings without paying creative work.

[0022] Figure 1 It is a flow chart of a monitoring method for monitoring elderly people's home safety based on the linkage of multiple devices with a decision engine provided by an embodiment of the present invention.

[0023] Figure 2 It is a flow chart of a data preprocessing method provided by an embodiment of the present invention.

[0024] Figure 3 It is a flow chart of a multimodal data fusion method provided by an embodiment of the present invention.

[0025] Figure 4 It is a flow chart of a local recognition model loading method provided by an embodiment of the present invention.

[0026] Figure 5 It is a flow chart of a cloud decision engine deployment method provided by an embodiment of the present invention.

[0027] Figure 6 It is a flow chart of a behavior decision generating method provided by an embodiment of the present invention.

[0028] Figure 7 It is a flow chart of the emotion recognition and care processing method provided by an embodiment of the present invention.

[0029] Figure 8 It is a flow chart of a method for setting a virtual home geo-fence provided in an embodiment of the present invention.

[0030] Fig. 9 It is a flow chart of a method for dynamically adjusting a virtual home geo-fence provided in an embodiment of the present invention.

[0031] Fig.10 It is a connection diagram of a platform for monitoring elderly people's home safety based on a decision engine and multi-device linkage provided by an embodiment of the present invention.

[0032] Fig.11 It is a schematic diagram of module connection of a control unit provided in an embodiment of the present invention.

[0033] Description of the Figures in the Specification: 10. Cloud server, 11. Smart home device, 12. Edge computing device, 13. Control unit, 101. Data processing module, 102. Data fusion module, 103. Behavior analysis module, 104. Behavior processing module. DETAILED DESCRIPTION

[0034] Embodiments of the present invention are described in detail below, examples of which are shown in the accompanying drawings, wherein the same or similar reference numerals throughout represent the same or similar elements or elements having the same or similar functions. The embodiments described below with reference to the accompanying drawings are exemplary and are intended to be used to explain the present invention, and should not be construed as limiting the present invention.

[0035] refer to Figure 1 As shown, in some embodiments, the monitoring method for monitoring the safety of the elderly at home based on the linkage of multiple devices of the decision engine includes the following steps: Step S1: Perform data preprocessing based on human home data collected by edge computing devices.

[0036] Specifically, refer to Figure 2 As shown, in step S1, the method includes the following steps: Step S11: Collect the elderly’s behavior images, expression changes, home environment data, elderly’s body data, and smart home device operation data through edge computing devices.

[0037] Among them, the edge computing devices include but are not limited to smart home gateways, microcomputers, etc., which are specifically set by the debugging personnel according to actual needs; the smart home devices include but are not limited to smoke alarms, temperature and humidity sensors, door and window sensors, ground pressure sensors, heart rate, blood oxygen, body temperature sensors, accelerometers, gyroscopes, air conditioners, refrigerators, lamps, kitchen appliances, etc., which are specifically set by the debugging personnel according to actual needs; further, MQTT or HTTP protocols are used to ensure low-latency data transmission between devices.

[0038] Step S12: De-noising and filtering the collected data; Among them, each smart home device sends the collected data to the edge computing device, which performs sliding mean filtering or Kalman filtering denoising on the data to eliminate sensor noise and outliers to ensure data accuracy; and then performs low-pass filtering on the denoised data to remove irrelevant or abnormal data points. Among them, the sliding mean filter or Coleman filter is selected by the debugging personnel according to actual needs. For example, in this application, reference can be made to the sliding mean filter processing: in, is the denoised data point, is the original data point.

[0039] Step S13: normalize the collected data to generate human home data.

[0040] Among them, after denoising and filtering, Z-score or normalization processing is used to unify the data units and scales from different sources to avoid the impact of dimensional differences on the analysis results.

[0041] Step S2: performing multimodal data fusion according to the human body home data to generate behavior recognition data, wherein the multimodal data fusion includes one or more of time alignment, modality completion, and feature extraction.

[0042] Among them, the timing alignment is to synchronize the data streams of each device to a unified time axis through timestamps, the modality completion is to complete the missing data points using machine learning models (such as KNN interpolation) or statistical methods, and the feature extraction is to extract key features from the aligned and completed data, such as motion trajectory, heart rate variability, etc.

[0043] Therefore, reference Figure 3 As shown, specifically, in step S2, the method includes the following steps: Step S21: according to the human body home data, use linear interpolation to align data with uneven time intervals, so as to synchronize the data in the human body home data to a unified time axis through timestamps; the specific process is: in, is the interpolated time data, For known data, is the timestamp of known data, is the time point that needs to be interpolated; Step S22: Based on the human body home data, the missing data is supplemented using KNN interpolation or statistical methods: in, For The most similar k neighboring data, is the similarity weight, usually defined as the inverse distance weight , for and The Euclidean distance of Step S23: extracting behavioral features and health features from the aligned and completed data, wherein the behavioral features include one or more of motion trajectories, home environment data, and smart home device operation data, and the health features include one or more of the elderly's facial expression changes and human body data.

[0044] Step S3: Input the behavior recognition data into the local recognition model for real-time behavior analysis, and at the same time input the behavior recognition data into the cloud decision engine for comprehensive in-depth analysis of multi-source data.

[0045] Specifically, refer to Figure 4 As shown, in step S3, the method includes the following steps: Step S31: Generate a behavior data set based on the collected historical behavior recognition data.

[0046] Step S32: Use the pre-trained MobileNet model and fine-tune it on the behavior dataset using the fine-tuning loss function: in, is the cross entropy loss function, For the real result, is the prediction result, and N is the number of data.

[0047] Among them, MobileNet is a lightweight convolutional neural network (CNN) that reduces the number of model parameters and computational complexity through deep separable convolution. MobileNet V1, V2 or V3 is selected according to the resources of the edge device, and widthmultiplier( α ) controls the model width. Furthermore, according to actual needs, debuggers can also choose the TinML model. TinML is a machine learning framework optimized for edge devices and supports microcontroller-level deployment. If TinML is selected, TensorFlow Lite Micro or Edge Impulse is used to quantize the model to INT8 precision.

[0048] Step S33: Perform quantization-aware training on the fine-tuned MobileNet model: ,in, w is the model weight or activation value, ∆ is the quantization step size, and w max and minimum value w min calculate: , b is the quantization bit width.

[0049] Step S34: Convert the trained model to TensorFlow Lite format: tflite_model=convert_model(model,optimizations=[INT8]), and then use a compiler (such as TensorFlow Lite Micro Interpreter) to load the trained model to the local edge computing device to form a local recognition model.

[0050] In this way, by running a lightweight model on edge computing devices, real-time detection can be achieved using an optimized reasoning process to ensure a balance between model accuracy and computing efficiency.

[0051] refer to Figure 5 As shown, in some examples, specifically in step S3, the method includes the following steps: Step S35: Generate a behavior data set based on the collected historical behavior recognition data.

[0052] Step S36: extracting behavior features from the behavior data set using an unsupervised learning method, for example, using a K-Means clustering objective function: in, is the clustering loss function, is the i-th data point in the behavior dataset, is the jth cluster center; and then a daily behavior pattern model of the elderly at home is constructed.

[0053] Among them, the normal behavior range: the upper and lower limits of the behavior are calculated based on the mean and standard deviation of the clustering results; the abnormal behavior deviation value: is defined based on the distance beyond the normal range. Step S37: Use reinforcement learning method to optimize the strategy of daily behavior pattern model of elderly people at home: in, α is the learning rate, r For instant rewards, γ is the discount factor, is the current state and action, For the next state and action.

[0054] Among them, after constructing the daily behavior pattern model of the elderly at home, the monitoring sensitivity is adjusted through the above-mentioned reinforcement learning method to optimize the strategy, for example: setting the strategy objective function: in, is the cumulative reward starting from time t, is the discount factor (trade-off between short-term and long-term rewards), is the immediate reward at time t+k; then the strategy gradient is updated: in, is the probability ratio, is the advantage function (estimating the quality of an action), To cut the scope, limit the update range of the strategy; design a reward mechanism based on the accuracy of the alarm and the safety of the elderly at home: Accurate alarm: reward R +1; False alarm: penalty R −1; no anomaly detected: penalty R −2.

[0055] For example, the daily sleep time distribution of elderly people at home is [20:00-05:00]. After unsupervised learning, the normal mode is identified as "staying still in the bedroom at night", and after strategy optimization, non-bedroom movement at night triggers the alarm strategy.

[0056] Step S38: deploy the trained and optimized daily behavior pattern model of the elderly at home to the cloud server to generate a cloud decision engine.

[0057] In some embodiments, reference Figure 6As shown, in step S3, the method further includes the following steps: Step S301: inputting behavior recognition data into a local recognition model and accepting the instant behavior detection result output by the local model.

[0058] Step S302: Analyze whether the elderly person at home has abnormal behavior based on the real-time behavior detection result.

[0059] The abnormal behaviors include but are not limited to elderly people falling, long-term inactivity, abnormal heart rate fluctuations, and environmental abnormalities (such as high temperature environment, excessive smoke concentration, etc.). For example, the abnormal behavior detection process is as follows: The fall detection conditions are: the acceleration threshold is greater than the set acceleration threshold; The inactivity detection condition is: the activity amount is less than the set activity amount threshold and the duration is greater than the set inactivity time threshold.

[0060] Step S303: If yes, execute step S3031: generate matching abnormal behavior results and input the abnormal behavior results and behavior identification data into the cloud-based decision engine in sequence, perform abnormal behavior potential risk analysis and multi-source data comprehensive in-depth analysis, and generate a first behavior decision; if no, execute step S3032: input the behavior identification data into the cloud-based decision engine, perform multi-source data comprehensive in-depth analysis, and generate a second behavior decision.

[0061] For example, if the local recognition model identifies that the elderly person is in a state of long-term inactivity, the cloud decision engine will call more real-time data from smart home devices to further observe the elderly person to determine whether the elderly person has fallen and is still, or is still watching TV or resting. If the elderly person is determined to have fallen and is still, the family's terminal device will be notified and emergency assistance will be called. If the local recognition model does not identify abnormal behavior of the elderly person at home, the cloud decision engine will call the real-time data of smart home devices to analyze the current behavior of the elderly person at home, so as to optimize the local recognition model and cloud decision engine next time.

[0062] Furthermore, the risk score can be set to trigger a corresponding decision, for example, a preset score threshold is set. If the risk score is greater than or equal to the preset score threshold, the first behavior decision is executed; if the risk score is less than the preset score threshold, the second behavior decision is executed.

[0063] Step S4: According to the output results of the local recognition model and / or the cloud decision engine, matching smart home device operations and / or home behavior alarm processing are performed.

[0064] In some embodiments, reference Figure 7 As shown, after collecting the human body home data, the method further includes the following steps: Step S101: Based on the human body data at home collected by the edge computing device, the behavior images, expression changes and voice signals of the elderly at home are extracted and the extracted data are fused to generate human emotion characteristics.

[0065] Among them, the facial and expression features in the behavioral images of the elderly at home are captured through the cameras of smart home devices, and the intonation, sound intensity and speaking speed features in the voice audio are collected, and then integrated to generate the human emotional characteristics of the elderly at home.

[0066] Step S102: Detect the emotional state of the elderly at home according to the human emotional characteristics and generate a care strategy according to the emotional state and the activity level of the elderly at home.

[0067] For example, if it is recognized that an elderly person at home has a smile on his face and speaks a calm voice, then the elderly person at home is judged to be in positive emotions, and a care strategy for playing cheerful music and / or a care strategy for generating pleasant voice playback data can be generated; if it is recognized that the elderly person at home has a non-smiling expression on his face and his voice is silent or fluctuates beyond a threshold, then the elderly person at home is judged to be in negative emotions, and a care strategy for playing soothing music and / or a care strategy for generating soothing voice playback data entered by family members can be generated.

[0068] Step S103: According to the care strategy, control the matching smart home device to perform the first care operation and / or the second care operation.

[0069] Thus, exemplarily, according to the care strategy for generating playing cheerful music and / or the care strategy for generating pleasant voice playback data, the matching smart home device is controlled to perform the corresponding operation of playing cheerful music and / or playing pleasant voice. Furthermore, if the smart home device has a companion robot, the corresponding companion robot can be controlled to interact with the elderly at home for entertainment. According to the care strategy for playing soothing music and / or the care strategy for generating soothing voice playback data entered by family members, the matching smart home device is controlled to perform the corresponding operation of playing soothing music and / or playing soothing voice. Furthermore, if the smart home device has a companion robot, the corresponding companion robot can be controlled to soothe the elderly at home. Furthermore, the family member user terminal can also be notified to remind them to accompany the elderly at home.

[0070] In some embodiments, reference Figure 8 As shown, after collecting the human body home data, the method further includes the following steps: Step S1001: Based on the human home data, determine whether there are smart home devices and human-worn positioning devices containing indoor positioning data, and the human-worn positioning devices include smart bracelets, watches, etc. worn by elderly people at home.

[0071] Step S1002: If yes, extract all positioning data and generate beacon coordinates according to the installation location of the smart home device containing indoor positioning data; if no, cancel the next step; Step S1003: Construct a virtual home geo-fence according to the beacon coordinates and monitoring range requirements.

[0072] Among them, the virtual home geo-fence builds a virtual geographic boundary through indoor positioning technology (such as UWB, or ultra-wideband, or Bluetooth beacons); ultra-wideband (UWB) calculates the distance between the device and the beacon through time-of-flight (ToF) ranging technology; Bluetooth beacons are based on signal strength (RSSI) ranging and triangulation positioning.

[0073] Specifically, if a virtual home geo-fence is used, UWB or Bluetooth beacons are arranged at various important locations in the house, and each beacon records its fixed position (x, y, z) coordinates.

[0074] Among them, the virtual home geo-fence of this application is a regional monitoring logic that can flexibly define the location range (such as kitchen, bathroom, balcony, etc.). When the elderly at home approach or enter these areas, alarms or related reminders are triggered.

[0075] For example, the boundary of the danger zone is set to (x min ,y min ,z min ) and (x max ,y max ,z max ).

[0076] Step S1004: Calculate the real-time position of the elderly at home using trilateration based on the indoor mobile positioning data of the positioning device worn by the human body: in, is the coordinate of the i-th beacon, is the distance of the i-th beacon; Step S1005: Based on the calculated real-time location of the elderly at home and combined with the virtual home geo-fence, determine whether the elderly at home has entered a set danger zone: Step S1006: If yes, preventive operations and / or home behavior alarm processing are performed on the smart home devices in the set dangerous area. If no, return to step S1004 and recalculate the real-time position of the elderly at home.

[0077] Furthermore, a warning range can be set. When the location of the elderly at home approaches the warning range, subsequent preventive operations and / or home behavior alarm processing are automatically triggered; that is: , then trigger the warning.

[0078] For example, if family members set the kitchen as a dangerous area, when the elderly at home approach the kitchen, the smart pots, stoves, natural gas switches, etc. in the kitchen will be automatically turned off as a precaution, and the family members can also be notified through the family terminal devices.

[0079] Specifically, refer to Fig. 9 As shown, in step S1001, the method further includes the following steps: Step S10011: assigning a risk area to each constructed virtual home geo-fence, wherein the risk area includes a low-risk area, a medium-risk area, and a dangerous area; Step S10012: Dynamically adjust the boundary range and / or fence position and / or risk level of each virtual home geo-fence based on time information and behavior patterns of elderly people at home.

[0080] Specifically, first, divide the activity range of the home, such as bedroom, living room, kitchen, bathroom, balcony, etc.; then analyze the behavior of the elderly according to time periods: daytime: more room switching; nighttime: mainly in bedroom and bathroom; then extract high-risk activity areas and times; thus, the boundary range and / or fence position and / or risk level of the virtual home geo-fence can be dynamically adjusted based on the time period.

[0081] In some examples, historical data can be used to analyze the activity patterns of the elderly (such as high-risk time periods and areas), and then dynamically build personalized geo-fences that adapt to the elderly.

[0082] refer to Fig.10 and Fig.11 As shown, in some embodiments, the present application also provides a platform for monitoring elderly people's home safety based on a decision engine and multi-device linkage using the monitoring method, including: Cloud server 10; Smart home devices 11; The edge computing device 12 is used to collect the behavior images, expression changes, home environment data, body data of the elderly, and operation data of smart home devices; The control unit 13 is connected to the cloud server 10, the smart home device 11, the edge computing device 12, the human body wearable positioning device and the terminal device respectively, and includes: The data processing module 101 is used to perform data preprocessing based on the human body home data collected by the edge computing device 12; A data fusion module 102 is used to perform multimodal data fusion according to the human body home data to generate behavior recognition data, wherein the multimodal data fusion includes one or more of time series alignment, feature level fusion, and decision level fusion; The behavior analysis module 103 is used to input the behavior recognition data into the local recognition model for real-time behavior analysis, and at the same time input the behavior recognition data into the cloud decision engine for comprehensive in-depth analysis of multi-source data; The behavior processing module 104 is used to match the operation of smart home devices and / or home behavior alarm processing according to the output results of the local recognition model and / or the cloud decision engine.

[0083] In some embodiments, the present application also provides a computer medium having a computer program stored thereon, and the computer program is executed by a processor to implement a monitoring method for monitoring the safety of an elderly person at home based on a decision engine and multiple devices linked together using the monitoring method.

[0084] In some embodiments, the present application also provides a computer, comprising the computer medium described above.

[0085] In the description of this specification, the description with reference to the terms "one embodiment", "some embodiments", "example", "specific example", or "some examples" etc. means that the specific features, structures, materials or characteristics described in conjunction with the embodiment or example are included in at least one embodiment or example of the present invention. In this specification, the schematic representations of the above terms do not necessarily refer to the same embodiment or example. Moreover, the specific features, structures, materials or characteristics described may be combined in any one or more embodiments or examples in a suitable manner. In addition, those skilled in the art may combine and combine the different embodiments or examples described in this specification and the features of the different embodiments or examples, without contradiction.

[0086] Although the embodiments of the present invention have been shown and described above, it is to be understood that the above embodiments are exemplary and are not to be construed as limitations of the present invention. A person skilled in the art may change, modify, replace and vary the above embodiments within the scope of the present invention.

Claims

1. A monitoring method for elderly people's home safety based on decision engine multi-device linkage, characterized in that: The following steps are involved: Step S1: Perform data preprocessing based on human home data collected by edge computing devices; Step S2: performing multimodal data fusion according to the human body home data to generate behavior recognition data, wherein the multimodal data fusion includes one or more of time series alignment, modality completion, and feature extraction; Step S3: inputting the behavior recognition data into a local recognition model for real-time behavior analysis, and simultaneously inputting the behavior recognition data into a cloud decision engine for comprehensive in-depth analysis of multi-source data; Step S4: According to the output results of the local recognition model and / or the cloud decision engine, matching smart home device operations and / or home behavior alarm processing are performed.

2. According to claim 1, a monitoring method for monitoring elderly people's home safety based on decision engine multi-device linkage is characterized in that: In step S1, the method comprises the following steps: Step S11: Collecting the elderly’s behavior images, expression changes, home environment data, elderly’s body data, and smart home device operation data through edge computing devices; Step S12: De-noising and filtering the collected data; Step S13: normalize the collected data to generate human home data.

3. According to claim 2, a method for monitoring the safety of the elderly at home based on the decision engine and multi-device linkage, characterized in that: In step S2, the method comprises the following steps: Step S21: according to the human body home data, align the data with uneven time intervals using a linear interpolation method, so as to synchronize the data in the human body home data to a unified time axis through a timestamp; Step S22: Based on the human body home data, the missing data is supplemented using KNN interpolation or statistical methods; Step S23: extracting behavioral features and health features from the aligned and completed data, wherein the behavioral features include one or more of motion trajectories, home environment data, and smart home device operation data, and the health features include one or more of the elderly's facial expression changes and human body data.

4. A method for monitoring the safety of the elderly at home based on a decision engine and multiple devices linked together according to claim 1 or 3, characterized in that: In step S3, the method comprises the following steps: Step S31: Generate a behavior data set based on the collected historical behavior recognition data; Step S32: Use the pre-trained MobileNet model and fine-tune it on the behavior dataset using the fine-tuning loss function: in, is the cross entropy loss function, For the real result, is the prediction result, N is the number of data; Step S33: Perform quantization-aware training on the fine-tuned MobileNet model: in, w is the model weight or activation value, ∆ is the quantization step size, and w max and minimum value w min calculate: , b is the quantization bit width; Step S34: convert the trained model into the TensorFlow Lite format and use a compiler to load the trained model into a local edge computing device to form a local recognition model.

5. According to claim 4, a method for monitoring the safety of the elderly at home based on the decision engine and multi-device linkage, characterized in that: In step S3, the method further comprises the following steps: Step S35: Generate a behavior data set based on the collected historical behavior recognition data; Step S36: Extracting behavioral features from the behavioral dataset using an unsupervised learning method: in, is the clustering loss function, is the i-th data point in the behavior dataset, is the jth cluster center; and then the daily behavior pattern model of the elderly at home is constructed; Step S37: Use reinforcement learning method to optimize the strategy of daily behavior pattern model of elderly people at home: in, α is the learning rate, r For instant rewards, γ is the discount factor, is the current state and action, For the next state and action; Step S38: deploy the trained and optimized daily behavior pattern model of the elderly at home to the cloud server to generate a cloud decision engine.

6. According to claim 5, a method for monitoring the safety of the elderly at home based on the decision engine and multi-device linkage, characterized in that: In step S3, the method further comprises the following steps: Step S301: inputting behavior recognition data into a local recognition model and accepting the instant behavior detection result output by the local model; Step S302: judging whether the elderly person at home has abnormal behavior according to the instant behavior detection result; Step S303: If yes, a matching abnormal behavior result is generated and the abnormal behavior result and the behavior identification data are sequentially input into the cloud-based decision engine to perform an abnormal behavior potential risk analysis and a comprehensive in-depth analysis of multi-source data to generate a first behavior decision; if no, the behavior identification data is input into the cloud-based decision engine to perform a comprehensive in-depth analysis of multi-source data to generate a second behavior decision.

7. According to claim 1, a method for monitoring the safety of the elderly at home based on the decision engine and multi-device linkage, characterized in that: After collecting the human body home data, the method further includes the following steps: Step S101: extracting behavioral images, expression changes, and voice signals of elderly people at home based on human body data collected by edge computing devices, and fusing the extracted data to generate human emotion features; Step S102: detecting the emotional state of the elderly at home according to the human emotional characteristics and generating a care strategy according to the emotional state and the activity level of the elderly at home; Step S103: According to the care strategy, control the matching smart home device to perform the first care operation and / or the second care operation.

8. According to claim 1, a method for monitoring the safety of the elderly at home based on the linkage of multiple devices using a decision engine, characterized in that: After collecting the human body home data, the method further includes the following steps: Step S1001: judging whether there are smart home devices and human-worn positioning devices containing indoor positioning data according to the human-home data; Step S1002: If yes, extract all positioning data and generate beacon coordinates according to the installation location of the smart home device containing indoor positioning data; if no, cancel the next step; Step S1003: constructing a virtual home geo-fence according to the beacon coordinates and monitoring range requirements; Step S1004: Calculate the real-time position of the elderly at home using trilateration based on the indoor mobile positioning data of the positioning device worn by the human body: in, is the coordinate of the i-th beacon, is the distance of the i-th beacon; Step S1005: judging whether the elderly person at home has entered a set danger zone based on the calculated real-time location of the elderly person at home and in combination with the virtual home geographic fence; Step S1006: If yes, preventive operations and / or home behavior alarm processing are performed on the smart home devices in the set dangerous area. If no, return to step S1004 and recalculate the real-time position of the elderly at home.

9. The method for monitoring the safety of the elderly at home based on the decision engine multi-device linkage according to claim 8 is characterized in that: In step S1001, the method further includes the following steps: Step S10011: assigning a risk area to each constructed virtual home geo-fence, wherein the risk area includes a low-risk area, a medium-risk area, and a dangerous area; Step S10012: Dynamically adjust the boundary range and / or fence position and / or risk level of each virtual home geo-fence based on time information and behavior patterns of elderly people at home.

10. A platform for monitoring elderly people's home safety based on a decision engine and multi-device linkage using the monitoring method according to any one of claims 1 to 9, characterized in that: include: Cloud servers; Smart home devices; Edge computing devices are used to collect behavioral images, facial expression changes, home environment data, body data of the elderly, and operation data of smart home devices; The control unit is connected to the cloud server, smart home devices, edge computing devices, human-worn positioning devices, and terminal devices, respectively, and includes: A data processing module is used to perform data preprocessing based on the human body home data collected by the edge computing device; A data fusion module, used to perform multimodal data fusion according to the human body home data to generate behavior recognition data, wherein the multimodal data fusion includes one or more of time series alignment, feature level fusion, and decision level fusion; A behavior analysis module is used to input the behavior recognition data into a local recognition model for real-time behavior analysis, and at the same time input the behavior recognition data into a cloud decision engine for comprehensive in-depth analysis of multi-source data; The behavior processing module is used to match the operation of smart home devices and / or home behavior alarm processing according to the output results of the local recognition model and / or the cloud decision engine.

Images