Smart home system

The smart home system addresses the lack of self-learning and predictive capabilities by creating a digital twin model for personalized home automation, allowing precise prediction and adaptive control of home operations.

CN120315312AInactive Publication Date: 2025-07-15UNIV OF ELECTRONICS SCI & TECH OF CHINA ZHONGSHAN INST
View PDF 0 Cites 2 Cited by

Patent Information

Application Number
CN202510589111.9
Authority / Receiving Office
CN · China
Patent Type
Applications(China)
Current Assignee / Owner
Filing Date
2025-05-08
Publication Date
2025-07-15
Estimated Expiration
Not applicable · inactive patent

AI Technical Summary

Technical Problem

Existing smart home systems lack self-learning and predictive capabilities, failing to accurately match user-specific needs and provide true intelligence.

Method used

A smart home system utilizing a user behavior data collection module, shadow intelligence modeling module, home control module, and augmented reality interaction module to create a personalized digital twin model, enabling predictive home automation and user interaction.

Benefits of technology

Enables precise prediction and automatic control of home operations based on user behavior, enhancing user experience through personalized and adaptive home automation.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN120315312A_ABST
    Figure CN120315312A_ABST
Patent Text Reader

Abstract

The invention provides a smart home system, and the system comprises a user behavior data collection module which is used for collecting and processing the daily behavior data of a user, including a position, an activity track, and the frequency and time of using home equipment; the shadow intelligent modeling module is used for creating a digital twin model of the user based on the user behavior data, analyzing user habits through a machine learning algorithm and generating a prediction rule; the home control module is used for executing home operation in advance based on the shadow intelligence prediction result, and the home operation comprises light adjustment, temperature control, home appliance starting and stopping and curtain opening and closing; and the augmented reality interaction module is used for generating a virtual image of a user, displaying a home decision of the shadow intelligence through AR equipment or a terminal, and allowing the user to modify or optimize the home decision. The method has the following beneficial effects: intelligent prediction: the shadow intelligence can predict user behaviors in advance and execute corresponding operation at a proper time point; the air conditioner is turned on in advance, water is boiled and tea is made, and light is adjusted.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present invention relates to a digital twin home system based on "shadow intelligence", which realizes personalized smart home control by creating a digital twin model of the user, and particularly relates to a smart home system. Background Art

[0002] Most existing smart home systems rely on immediate control by users, such as voice commands or manual operations via mobile phone applications, lacking the ability of autonomous learning and prediction, and it is difficult to achieve true intelligence. Some smart homes have automation functions, such as triggering home devices based on time or sensors, but still cannot accurately match the personalized needs of users.

[0003] Digital twin technology is a method for prediction and optimization through digital mapping of physical entities. Applying it to the smart home field can achieve accurate prediction and automatic control by simulating user behavior, improving the user experience. Summary of the Invention

[0004] Aiming at the deficiencies of the existing technology, the purpose of the present invention is to provide a smart home system to solve the problems proposed in the above background art.

[0005] The present invention is realized through the following technical solutions: A smart home system, comprising:

[0006] User behavior data acquisition module: used to collect the daily behavior data of the user, including location, activity trajectory, frequency and time of using home devices, and process it;

[0007] Shadow intelligence modeling module: used to create a digital twin model of the user based on user behavior data, analyze user habits through machine learning algorithms, and generate prediction rules;

[0008] Home control module: used to execute home operations in advance based on the prediction results of shadow intelligence, including light adjustment, temperature control, starting and stopping of household appliances, and opening and closing of curtains;

[0009] Augmented reality interaction module: used to generate a virtual image of the user, and display the home decisions of shadow intelligence through AR devices or terminals, allowing the user to modify or optimize.

[0010] As a preferred embodiment, the user behavior data acquisition module: used to collect the daily behavior data of the user, including location, activity trajectory, frequency and time of using home devices, and process it, which is realized through the following methods:

[0011] Step A1, collect the behavior data of the user, and construct a personalized behavior data set of the user through smart home devices, sensors and user interaction data. The data sources include:

[0012] Home equipment data: switch status and usage time of smart lights, smart curtains, air conditioners, TVs, and water heaters;

[0013] Location and activity data: GPA positioning, Bluetooth / Wi-Fi connection, smart bracelet data;

[0014] Environmental data: temperature and humidity, light intensity, and noise level sensor data;

[0015] User operation data: voice commands, mobile phone APP remote operation, manual keystroke records.

[0016] Step A2: Process the collected data, align the timestamps of different data sources, convert them into a unified format, and then remove unreasonable data, such as sensor errors and missing value filling. Use the min-max normalization method to standardize the data value range and improve the LATM learning efficiency.

[0017] Step A3: Window the processed data, that is, divide the user behavior data into data segments of fixed window length (such as 24 hours as 1 window) in chronological order, and divide the processed data into training set, validation set, and test set: 70% for training, 15% for validation, and 15% for testing to ensure the generalization ability of the model.

[0018] As a preferred implementation, the shadow intelligent modeling module is used to create a digital twin model of a user based on user behavior data, analyze user habits through a machine learning algorithm, and generate prediction rules, which is achieved through the following steps:

[0019] Step B1: Use the long short-term memory network LSTM to model time series. The model structure includes:

[0020] Input layer: time series of input behavior data (device status, user location, environmental parameters);

[0021] LSTM layer: Multi-layer LSTM units extract time series features and identify long-term behavior patterns;

[0022] Fully connected layer (Dense layer): maps the features extracted by LSTM to different prediction categories (such as turning on / off the light);

[0023] Output layer: predicts the user's future behavior, such as predicting the next possible device operation or when the user will arrive home;

[0024] Step B2: Train the modeled digital twin model. Specifically, use the cross entropy loss to calculate the error between the model prediction and the actual label, then use the Adam adaptive optimization algorithm to accelerate the training and perform hyperparameter tuning.

[0025] Hyperparameter tuning refers to: the length of the time window (e.g., the behavior in the past 24 hours is used to predict the next 1 hour).

[0026] The number of LSTM units (e.g., 128, 256 for experimental adjustment).

[0027] The learning rate (e.g., 0.001, using a learning rate decay strategy).

[0028] Step B3: Based on the training results of the LSTM model, construct the user's personalized digital twin model, i.e., the shadow intelligence, for predicting the user's future behavior. The prediction of the user's future behavior is divided into short-term prediction at the hourly level and long-term prediction over days or more. Among them, the short-term prediction (hourly level) includes:

[0029] Predicting the possible activities of the user 1 hour later (e.g., whether to go home, whether to turn on the TV) or predicting the next operation (e.g., turning on the light within 5 minutes after the user enters the bedroom).

[0030] The long-term prediction (daily / weekly level) includes:

[0031] Predicting the user's daily schedule (e.g., habitually turning on the coffee machine at 7:30 in the morning). Or predicting the user's weekend mode vs. weekday mode

[0032] Step B4: Automatically generate home control rules based on the LSTM prediction results:

[0033] Such as "If the user is expected to go home within 30 minutes, turn on the air conditioner in advance."

[0034] "If the LSTM predicts that all lights will be turned off at 23:00 at night, then automatically execute the light-off."

[0035] The user can modify and fine-tune the rules to adjust the behavior of the shadow intelligence through voice or the APP interface.

[0036] As a preferred implementation, the augmented reality interaction module: is used to generate the user's virtual image and display the home decisions of the shadow intelligence through AR devices or terminals, allowing the user to modify or optimize them, which is achieved through the following steps:

[0037] Step C1, the user sees the predicted behavior of the shadow intelligence through AR glasses or the mobile APP and manually adjusts the rules;

[0038] The shadow intelligence provides suggestions through the voice or text interface of the virtual assistant for the user to choose, such as: "Do you need to automatically turn on the water heater.

[0039] Step C2, the user sees the device usage trends predicted by the LSTM shown by the shadow intelligence through AR glasses or the mobile APP. On this interface, the user can manually enable / disable the prediction rules to enhance controllability.

[0040] As a preferred embodiment, it further includes:

[0041] System optimization module: used to adjust the control rules of the shadow intelligence based on user feedback, improve the prediction accuracy. The digital twin model adjusts the rules based on user feedback. If the user manually adjusts the air conditioner temperature multiple times, the system adjusts the output of the LSTM model to make the prediction more in line with the user's habits.

[0042] If the user does not return home at the time predicted by the LSTM, the system reduces the confidence level of the air conditioner starting in advance. Or perform adaptive adjustment through abnormal behavior recognition. Abnormal behavior recognition means that if the user's behavior pattern suddenly changes (such as being away from home for a long time), the shadow intelligence pauses some automatic controls to avoid misoperations.

[0043] After adopting the above technical solutions, the beneficial effects of the present invention are:

[0044] 1. Intelligent prediction: The shadow intelligence can predict the user's behavior in advance and perform corresponding operations at the appropriate time point, such as turning on the air conditioner in advance, boiling water for making tea, adjusting the lights, etc.

[0045] 2. Augmented reality interaction: The user can view the behavior decisions of their shadow intelligence through AR and intervene or adjust them to improve the usage experience.

[0046] 3. Dynamic optimization: The system can continuously optimize the prediction ability of the shadow intelligence based on user feedback and habit changes to ensure that home operations always meet the user's needs. Description of the Drawings

[0047] In order to more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the following will briefly introduce the drawings required for use in the description of the embodiments or the prior art. Obviously, the following drawings are only some embodiments of the present invention. For those of ordinary skill in the art, without creative efforts, other drawings can also be obtained based on these drawings.

[0048] Figure 1 It is the system architecture diagram of a smart home system of the present invention.

[0049] Figure 2 It is the working schematic diagram of the user behavior data collection module.

[0050] Figure 3 It is the working schematic diagram of the shadow intelligence modeling module. Specific Embodiments

[0051] Next, the technical solutions in the embodiments of the present invention will be clearly and completely described in conjunction with the accompanying drawings in the embodiments of the present invention. Obviously, the described embodiments are only a part of the embodiments of the present invention, rather than all the embodiments. All other embodiments obtained by those of ordinary skill in the art based on the embodiments of the present invention without making creative efforts shall fall within the protection scope of the present invention.

[0052] Please refer to Figure 1 , the present invention provides a technical solution: a smart home system, including:

[0053] User behavior data acquisition module: used to collect the daily behavior data of users, including location, activity trajectory, frequency and time of using home appliances, and perform processing;

[0054] Shadow intelligent modeling module: used to create a digital twin model of the user based on the user behavior data, analyze the user's habits through machine learning algorithms, and generate prediction rules;

[0055] Home control module: used to perform home operations in advance based on the prediction results of shadow intelligence, including lighting adjustment, temperature control, starting and stopping of home appliances, and opening and closing of curtains;

[0056] Augmented reality interaction module: used to generate a virtual image of the user, and display the home decisions of shadow intelligence through AR devices or terminals, allowing users to modify or optimize.

[0057] Please refer to Figure 2 , the user behavior data acquisition module: used to collect the daily behavior data of users, including location, activity trajectory, frequency and time of using home appliances, and perform processing, which is realized by the following methods:

[0058] Step A1, collect the behavior data of users, and construct a personalized behavior data set of users through smart home devices, sensors and user interaction data. The data sources include:

[0059] Home appliance data: the on / off state and usage duration of smart lights, smart curtains, air conditioners, TVs, and water heaters;

[0060] Location and activity data: GPA positioning, Bluetooth / Wi-Fi connection, smart bracelet data;

[0061] Environmental data: temperature and humidity, light intensity, noise level sensor data;

[0062] User operation data: voice commands, mobile APP remote operations, manual button records.

[0063] Step A2: Process the collected data, align the timestamps of different data sources, convert them into a unified format, and then eliminate unreasonable data, such as sensor errors and missing value filling. Use the min-max normalization method to standardize the data value range and improve the learning efficiency of LATM.

[0064] Step A3: Window the processed data, that is, divide the user behavior data into data segments of a fixed window length in chronological order (such as 24 hours as one window), and divide the processed data into training set, validation set, and test set: 70% for training, 15% for validation, and 15% for testing to ensure the generalization ability of the model.

[0065] Embodiment 1: This embodiment details the implementation steps and working methods of the shadow intelligent modeling module.

[0066] Please refer to Figure 3 , the shadow intelligent modeling module: used to create a digital twin model of the user based on user behavior data, analyze user habits through machine learning algorithms, generate prediction rules, and is implemented through the following steps:

[0067] Step B1: Use the long short-term memory network LSTM for time series modeling. The model structure includes:

[0068] B1.1, input layer:

[0069] Input the time series of behavior data (device status, user location, environmental parameters); the input layer receives the user behavior data and formats it into time series data for LSTM to learn. Assume that the smart home behavior data we collect includes the following categories:

[0070] 1. Device status (on / off, adjustment value): such as the temperature of the smart air conditioner, the brightness of the light, the opening angle of the curtain, etc., formatted as numerical data.

[0071] 2. User location and activity information: such as GPS coordinates, whether at home, walking / driving status, etc., converted into continuous numerical values or one-hot encoding.

[0072] 3. Environmental parameters: such as indoor temperature and humidity, light intensity, noise level, etc., normalized and then input into the LSTM network.

[0073] To perform time series modeling, we define:

[0074] X t represents the input feature vector at time t.

[0075] X = {X1, X2, …, XT} represents the sequence of user behavior data in the past T time steps.

[0076] B1.2, LSTM layer:

[0077] Multiple LSTM cells extract time series features to identify long-term behavior patterns; LSTM processes time series data through memory cells to identify users' long-term behavior patterns. LSTM includes an input gate, a forget gate, and an output gate, and their calculation methods are as follows:

[0078] Forget gate

[0079] Determines which past information needs to be forgotten at the current moment, and the calculation method is:

[0080] f t = σ(W f · [h t-1 , X t · + b f )

[0081] Where: ft is the output of the forget gate, ranging between [0, 1], 1 represents complete retention, and 0 represents complete forgetting. W f and b f are the weight matrix and the bias term respectively. h t-1 is the hidden state of the previous time step, and X t is the input of the current time step. σ() is the Sigmoid activation function.

[0082] The input gate determines how the new information at the current time step updates the cell state:

[0083] i t = σ(W i · [h t-1 , X t + b i )

[0084]

[0085] Where: i t controls the influence degree of the new input information. is the value of the new candidate memory cell. C t is the updated memory cell state.

[0086] The output gate determines the final output at the current time step:

[0087] o t = σ(W o · [h t-1 , X t + b o )

[0088] h t = o t*tanh(C t )

[0089] where: o t controls the output information at the current time step. h t As the final hidden state of the LSTM, it is passed to the subsequent fully connected layer.

[0090] B1.3, Fully Connected Layer:

[0091] Maps the features extracted by the LSTM to different prediction categories (such as turning on / off the light);

[0092] The time series features extracted by the LSTM need to be mapped through the fully connected layer to predict the specific behavior of the user. For example:

[0093] y t = Softmax(W h ·h t + b h )

[0094] where: y t represents the user behavior predicted at time t (such as turning on the light, closing the window, adjusting the air conditioner temperature, etc.). Softmax calculates the probabilities of each category and selects the category with the highest probability as the final prediction result.

[0095] B1.4, Output Layer:

[0096] Predicts the future behavior of the user, such as predicting the possible next device operation or when the user will arrive home. The output layer finally obtains the prediction result of the user's future behavior. For example: Classification task: Predict whether the user will arrive home within 30 minutes (yes / no). Regression task: Predict to what temperature the air conditioner will be adjusted in the next 10 minutes.

[0097] Step B2, Train the modeled digital twin model. Specifically, use the cross-entropy loss to calculate the error between the model prediction and the actual label, then use the Adam adaptive optimization algorithm to accelerate the training, and perform hyperparameter tuning.

[0098] For the classification task (such as predicting whether the user will arrive home): Use the cross-entropy loss function:

[0099]

[0100] where: y i is the true label, is the predicted probability.

[0101] For the regression task (such as predicting the change in air conditioner temperature): Use the mean squared error (MSE):

[0102]

[0103] Hyperparameter tuning refers to: the length of the time window (e.g., the behavior in the past 24 hours is used to predict the next 1 hour).

[0104] The number of LSTM units (e.g., 128 or 256 for experimental adjustment).

[0105] Learning rate (e.g., 0.001, using a learning rate decay strategy).

[0106] Step B3: Based on the training results of the LSTM model, construct a personalized digital twin model of the user, i.e., the shadow intelligence, for predicting the user's future behavior. The prediction of the user's future behavior is divided into short-term prediction at the hourly level and long-term prediction over a period of more than one day. Among them, the short-term prediction (hourly level) includes:

[0107] Predict the possible activities of the user 1 hour later (e.g., whether to go home, whether to turn on the TV) or predict the next operation (e.g., turn on the light within 5 minutes after the user enters the bedroom).

[0108] The long-term prediction (daily / weekly level) includes:

[0109] Predict the user's daily routine (e.g., habitually turn on the coffee machine at 7:30 in the morning). Or predict the user's weekend mode vs. weekday mode

[0110] Step B4: Automatically generate home control rules based on the LSTM prediction results:

[0111] Such as "If the user is expected to go home within 30 minutes, turn on the air conditioner in advance."

[0112] "If the LSTM predicts that all lights will be turned off at 23:00 at night, then automatically turn off the lights."

[0113] The user can modify and fine-tune the rules to adjust the behavior of the shadow intelligence through voice or the APP interface.

[0114] Example 1.1. To better understand the working principle of the LSTM model in the shadow intelligence modeling module, we will use a specific example with specific values to elaborate in detail on how to use the LSTM model for user behavior prediction. Taking intelligent lighting control as an example, assume that we want to predict whether the user will turn off the bedroom light at 22:00 at night.

[0115] Assume that the shadow intelligence system has collected the user's behavior data in the past 24 hours, including the following features:

[0116] Light status (0 represents the light is off, 1 represents the light is on)

[0117] User location (0 represents the company, 1 represents home)

[0118] Ambient brightness (in lx)

[0119] We will use the LSTM model to predict whether the user will turn off the bedroom light at 22:00 at night.

[0120] The time window we use is the data of the past 24 hours (one hour as a time step), and the target we hope to predict is whether the user will turn off the light at 22:00 at night. The input data of the model is as

[0121] Table 1:

[0122] Time step Lighting state User location Ambient brightness (lx) 0 1 0 200 1 1 0 250 2 1 1 100 ... ... ... ... 22 1 1 50

[0123] Table 1, Input data table of the model

[0124] Working principle of the LSTM model

[0125] The working principle of LSTM can be divided into the following steps:

[0126] Forget gate: Decide which information needs to be forgotten.

[0127] Input gate: Decide which information needs to be updated.

[0128] Output gate: Decide the output at the current moment.

[0129] 1. Forget gate

[0130] The purpose of the forget gate is to decide which historical information is not helpful for the current prediction, that is, which information needs to be forgotten. The formula is:

[0131] f t = σ(W f · [h t-1 , X t + b f )

[0132] Where: h t-1 is the hidden state at the previous moment. X t is the input feature at the current moment.

[0133] Assume that the hidden state h t-1 at the previous moment = 0.5, and the input X t at the current moment = [light status, user location, ambient brightness] = [1, 1, 50].[[]END]

[0134] Weight matrix W f = [0.2, 0.3, 0.5], bias term b f = 0.1.

[0135] Substitute into the formula to calculate the output of the forget gate:

[0136] f t = σ(0.2·0.5 + 0.3·1 + 0.5·50 + 0.1)

[0137] f t = σ(0.1 + 0.3 + 25 + 0.1) = σ(25.5)

[0138] f t ≈ 1

[0139] This means that the forget gate hardly forgets previous memories.

[0140] 2. Input gate

[0141] The input gate determines which new information needs to be updated into the memory cell. The formula is:

[0142] i t = σ(W i · [h t-1 , X t + b i )

[0143]

[0144] Assume the input X t = [1, 1, 50] at the current time step, the weight W i = [0.4, 0.6, 0.2], the bias term b i = 0.2, and the previous state C of the memory cell t-1 = 0.3.

[0145] First, calculate the output of the input gate:

[0146] i t = σ(0.4·0.5 + 0.6·1 + 0.2·50 + 0.2)

[0147] i t = σ(0.2 + 0.6 + 10 + 0.2) = σ(11)

[0148] i t ≈ 1

[0149] Then, calculate the candidate memory cell:

[0150]

[0151] Finally, update the memory cell:

[0152] C t = 1·0.3 + 1·1 = 1.3

[0153] 3. Output gate

[0154] The outputs determine the final output h at the current moment t , which is the user behavior we predict. The formula is:

[0155] o t = σ(W o · [h t-1 , X t + b o )

[0156] h t = o t * tanh(C t )

[0157] Assume that the weight W at the current moment o = [0.5, 0.6, 0.4], the bias term b o = 0.1, the hidden state h at the previous moment t-1 = 0.5, and the memory cell C t = 1.3.

[0158] First, calculate the output of the output gate:

[0159] o t = σ(0.5·0.5 + 0.6·1 + 0.4·50 + 0.1)

[0160] o t = σ(0.25 + 0.6 + 20 + 0.1) = σ(20.95)

[0161] o t ≈ 1

[0162] Finally, calculate the hidden state at the current moment (i.e., the prediction result):

[0163] h t = 1·tanh(1.3) ≈ 1·0.858

[0164] h t ≈ 0.858

[0165] According to the value of h t , use the Softmax activation function to convert the output into prediction probabilities for class prediction. For example, assume that the model outputs a classification result of turning on the light (1) or turning off the light (0). h t = 0.858 can be converted to a probability of 85.8% for turning on the light and a probability of 14.2% for turning off the light.

[0166] 4. Prediction result

[0167] If h tIf the predicted value is greater than a certain threshold (e.g., 0.5), the model predicts that the user will turn off the light; otherwise, the light remains on.

[0168] In this example, h t ≈ 0.858, predicting that the user is likely to keep the light on. Therefore, the system decides whether to turn off the light at 22:00 based on the prediction.

[0169] 5. Model Optimization

[0170] During the training process, we calculate the loss function and optimize it. The specific steps include:

[0171] Calculate the loss: Use the cross-entropy loss to calculate the error between the model prediction and the actual label.

[0172] Optimize the parameters: Update the weights W f , W i , W C , W O , to minimize the loss and improve the model's prediction ability.

[0173] Example 2, Augmented Reality Interaction Module: Used to generate a virtual image of the user and display the home decisions of Shadow Intelligence through an AR device or terminal, allowing the user to modify or optimize them, implemented through the following steps:

[0174] Step C1, the user sees the predicted behavior of Shadow Intelligence through AR glasses or a mobile phone APP and manually adjusts the rules;

[0175] Shadow Intelligence provides suggestions through the voice or text interface of the virtual assistant for the user to choose, such as: "Do you need to automatically turn on the water heater?

[0176] Step C2, the user sees the device usage trend predicted by LSTM shown by Shadow Intelligence through AR glasses or a mobile phone APP. On this interface, the user can manually enable / disable the prediction rules to enhance controllability.

[0177] Example 3, further includes:

[0178] System Optimization Module: Used to adjust the control rules of Shadow Intelligence based on user feedback to improve prediction accuracy. The digital twin model adjusts the rules based on user feedback. If the user manually adjusts the air conditioner temperature multiple times, the system adjusts the output of the LSTM model to make the prediction more in line with the user's habits.

[0179] If the user does not return home at the time predicted by LSTM, the system reduces the confidence in the air conditioner starting in advance. Or make adaptive adjustments through abnormal behavior recognition. Abnormal behavior recognition refers to: If the user's behavior pattern suddenly changes (such as being away from home for a long time), Shadow Intelligence pauses some automatic controls to avoid misoperations.

[0180] Example 4. In this example, how each module works together is elaborated in detail. Based on specific data, the working principles of the user behavior data collection module, shadow intelligence modeling module, home control module, augmented reality interaction module, and system optimization module are described.

[0181] Mr. Zhang is an office worker with a relatively regular daily life. The goal of the system is to continuously collect Mr. Zhang's behavior data, build his digital twin model (shadow intelligence), predict his needs, and adjust home appliances in advance. For example, Mr. Zhang usually goes home around 18:30 on weekdays; after getting home, he is used to turning on the air conditioner in advance, dimming the lights, and opening the curtains to create a comfortable living environment.

[0182] 1. User behavior data collection module: It is used to collect the user's daily behavior data, including location, activity trajectory, frequency and time of using home appliances, and process the data.

[0183] In Mr. Zhang's home, a variety of sensors and intelligent devices are first deployed. These devices form the user behavior data collection module and collect Mr. Zhang's daily behavior data in real time.

[0184] For example, the smart bracelet, mobile phone GPS, and home Wi-Fi positioning jointly record his location information and activity trajectory: Mr. Zhang gets up at home (coordinates approximately 39.90°N, 116.40°E) at 7:00 every morning and leaves for work at 7:30; at 18:20 in the afternoon, the system detects that the signal of his mobile phone starts to weaken, indicating that he is on his way back, and at 18:35, it is confirmed that he has returned home.

[0185] At the same time, the data of home appliance usage is also recorded: the air conditioner at home starts the pre-set mode in advance at about 18:25 on weekdays, adjusting the room temperature to 26°C, the living room lights are adjusted to 60% brightness at 18:40, the curtains are automatically opened at 18:30, and the system will perform the light-off operation before 22:30 every night.

[0186] In addition, the information such as temperature, humidity, and light collected by the environmental sensors is also uploaded synchronously. For example, at 18:30, the outdoor temperature is about 28°C, the indoor temperature is about 24°C, and the humidity is 45%. After being integrated with time stamps, these data form a continuous time series, providing sufficient basis for subsequent modeling.

[0187] 2. Shadow intelligence modeling module: It is used to create a digital twin model of the user based on the user behavior data, analyze the user's habits through machine learning algorithms, and generate prediction rules.

[0188] Based on the collected data, the Shadow Intelligence Modeling Module begins to construct Mr. Zhang's digital twin model. Using the data from the past 30 days, through normalizing the home device status, location information, and environmental parameters within 24 hours and time window slicing, each time step contains features such as [light status (0 or 1), location code (0 represents the company, 1 represents home), indoor temperature, humidity, light intensity].

[0189] The system adopts a deep learning model based on Long Short-Term Memory Network (LSTM) to capture the temporal dependencies of Mr. Zhang's behavior.

[0190] The input layer of the model accepts the processed time series data, and the data of each time step is represented by a vector; next, multiple layers of LSTM units (such as 128 units in the first layer and 256 units in the second layer) gradually extract long-term behavior patterns through the forget gate, input gate, and output gate mechanisms.

[0191] At a certain moment (such as 22:00), when the input vector is [1, 1, 50] (where "1" represents the light is on, the location code is home, and the environmental brightness is 50 lx),

[0192] The model calculates the output of the forget gate: using the formula f t = σ(0.2×0.5 + 0.3×1 + 0.5×50 + 0.1) = σ(25.5) ≈ 1, indicating that the current input is highly correlated with past information;

[0193] Calculate the update ratio of new information and the candidate memory value through the input gate, and then update the memory cell to get C t = 1×0.3 + 1×1 = 1.3;

[0194] Finally, the output gate combines the current memory state to calculate the hidden state h t = 1×tanh(1.3) ≈ 0.858.

[0195] After passing through the fully connected layer and Softmax mapping, this hidden state is interpreted as a prediction for the binary classification task of "whether to turn off the light", and the model outputs a probability of turning off the light of about 85.8%.

[0196] In model training, the error between the true label (for example, the user actually turns off the light before 22:30, and the label is 1) and the predicted probability is calculated through the cross-entropy loss function, such as L = -log(0.858) ≈ 0.154;

[0197] Subsequently, in the backpropagation process, the Adam optimizer is used to update the weights of each layer. Through iterative adjustment, the loss is continuously reduced until the model can accurately predict Mr. Zhang's behavior pattern. After training, the system generates a series of prediction rules. For example: "If a user's homecoming trend is detected between 18:15 and 18:30, the air conditioner is started 5 minutes in advance and the target temperature is set to 26°C"; "About 3 minutes after the user arrives home, the living room lights are automatically adjusted to 60% brightness, and the curtains are drawn at the same time"; "If the user does not actively adjust the lights before 22:30 at night, the lights are automatically turned off."

[0198] 3. Home control module: It is used to execute home operations in advance based on the prediction results of Shadow Intelligence, including light adjustment, temperature control, appliance start / stop, and curtain opening / closing.

[0199] In the home control module, these prediction rules are directly applied. For example, at 18:25 on a working day, according to the model's prediction that the probability of Mr. Zhang arriving home reaches 95%, the system automatically starts the air conditioner and preset the indoor temperature to 26°C. Shortly after Mr. Zhang arrives home, the system automatically adjusts the living room lights to 60% and draws the curtains to ensure a comfortable home environment. When night approaches, the system automatically turns off all lights at 22:30 according to the historical data pattern and switches to the sleep mode, completing a full cycle of prediction and execution.

[0200] 4. Augmented reality interaction module: It is used to generate a virtual image of the user and display the home decisions of Shadow Intelligence through AR devices or terminals, allowing the user to modify or optimize them.

[0201] To make the entire system more interactive and flexible, the augmented reality interaction module is also introduced. Through smart glasses or a mobile phone APP, Mr. Zhang can see the virtual image generated by the system before returning home - the prediction results and operation plans are displayed on the AR interface, such as "It is predicted that you will arrive home at 18:30. The system has started the air conditioner and adjusted the lights to 60%." At the same time, an operation schedule and status icons of each device are provided, allowing Mr. Zhang to make real-time interventions and rule adjustments through voice or gestures (for example, if he feels that the air conditioner temperature is slightly high, he can immediately lower it to 25°C). All these interaction information not only gives Mr. Zhang an intuitive understanding of home automation control, but also transmits his feedback data back to the system.

[0202] The system optimization module continuously receives Mr. Zhang's feedback data and makes fine-tuning. If Mr. Zhang manually adjusts the air conditioner temperature 3 times in the past week, reducing the target temperature from 26°C to 25°C; or occasionally delays turning off the lights, the system will automatically adjust the prediction rules and update the digital twin model to ensure that future operations better meet the actual needs. Eventually, the entire system forms a closed loop: from real-time data collection to accurate prediction based on LSTM, then to automatic execution and user interaction feedback, and finally through the optimization module to continuously improve the prediction rules, enabling the home environment to always be adjusted to the most suitable state before Mr. Zhang arrives home, achieving a truly intelligent and personalized home experience.

[0203] The above are only the preferred embodiments of the present invention and are not intended to limit the present invention. Any modifications, equivalent replacements, improvements, etc. made within the spirit and principle of the present invention shall be included in the protection scope of the present invention.

Claims

1. A smart home system, characterized in that, It includes: User Behavior Data Collection Module: It is used to collect the daily behavior data of users, including location, activity trajectory, frequency and time of using home appliances, and perform processing; Shadow Intelligence Modeling Module: It is used to create a digital twin model of the user based on the user behavior data, analyze user habits through machine learning algorithms, and generate prediction rules; Home Control Module: It is used to perform home operations in advance based on the prediction results of shadow intelligence, including light adjustment, temperature control, start and stop of household appliances, and opening and closing of curtains; Augmented Reality Interaction Module: It is used to generate a virtual image of the user, and display the home decisions of shadow intelligence through AR devices or terminals, allowing users to modify or optimize; 2. The smart home system according to claim 1, wherein: The User Behavior Data Collection Module: It is used to collect the daily behavior data of users, including location, activity trajectory, frequency and time of using home appliances, and perform processing, which is realized by the following methods: Step A1, collect the behavior data of users, and construct a personalized behavior data set of users through smart home devices, sensors and user interaction data; Step A2, process the collected data, align the timestamps of different data sources, convert them into a unified format, and then eliminate unreasonable data and use the min-max normalization method to standardize the data value range; Step A3, window the processed data, that is, cut the user behavior data into data segments with a fixed window length in chronological order, and divide the processed data into training set, validation set and test set; 3. The smart home system according to claim 2, characterized in that: The Shadow Intelligence Modeling Module: It is used to create a digital twin model of the user based on the user behavior data, analyze user habits through machine learning algorithms, and generate prediction rules, which is realized by the following steps: Step B1, use the Long Short-Term Memory Network (LSTM) for time series modeling, and the model structure includes: Input layer: Input the time series of behavior data; LSTM layer: Multiple LSTM units extract time series features and identify long-term behavior patterns; Fully connected layer: Map the features extracted by LSTM to different prediction categories; Output layer: Predict the future behavior of users; Step B2, train the digital twin model after modeling. Specifically, use the cross-entropy loss to calculate the error between the model prediction and the actual label, then use the Adam adaptive optimization algorithm to accelerate the training, and perform hyperparameter tuning; Step B3, construct a personalized digital twin model of the user, that is, shadow intelligence, based on the training results of the LSTM model, which is used to predict the future behavior of users. The prediction of users' future behavior is divided into short-term prediction at the hourly level and long-term prediction over days; Step B4, automatically generate home control rules according to the LSTM prediction results: Users can modify and fine-tune the rules, and adjust the behavior of shadow intelligence through voice or APP interface.

4. A smart home system according to claim 3, wherein: Augmented Reality Interaction Module: It is used to generate a virtual image of the user, and display the home decisions of shadow intelligence through AR devices or terminals, allowing users to modify or optimize, which is realized by the following steps: Step C1, users can see the predicted behavior of shadow intelligence through AR glasses or mobile APPs and manually adjust the rules; Shadow intelligence provides suggestions through the voice or text interface of the virtual assistant for users to choose; Step C2: The user can view the device usage trends predicted by Shadow Intelligence through AR glasses or a mobile phone APP. On this interface, the user can manually enable / disable the prediction rules to enhance controllability.

5. A smart home system according to claim 1 or 4, characterized in that: It also includes: System optimization module: used to adjust the control rules of Shadow Intelligence based on user feedback to improve prediction accuracy. The digital twin model adjusts the rules based on user feedback; Or perform adaptive adjustment through abnormal behavior recognition. Abnormal behavior recognition means that if the user's behavior pattern suddenly changes (such as being away from home for a long time), Shadow Intelligence pauses some automatic controls to avoid misoperations.

Citation Information

Cited By

  • Intelligent temperature control energy-saving system and method based on personnel position detection and prediction

    CN120909374A

  • Control method and device of home smart device, computer readable storage medium and computer program product

    CN122506885A