Smart home equipment control method and device, electronic equipment and medium

Abstract

The embodiment of the invention provides a control method and device of smart home equipment, electronic equipment and a medium. The method comprises the following steps: acquiring home multi-modal data; determining a current home scene according to the home multi-modal data; based on the current home scene, setting an operation state of home equipment; and detecting whether the user is about to arrive at home or not, determining the target home equipment of which the running state is a dormant state when detecting that the user is about to arrive at home, and waking up the target home equipment before the user arrives at home. According to the embodiment of the invention, through a dual judgment mechanism based on scene recognition and user behavior prediction, the current equipment state and the user behavior are predicted at the same time, and compared with a mode of adopting a single control logic, equipment energy consumption can be considered while the control accuracy of the smart home equipment can be effectively improved, and the user experience is improved. And the double effects of improving user experience, saving energy and reducing consumption are achieved.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of smart home devices, and in particular to a smart home device control method and device, electronic device, and readable storage medium. BACKGROUND

[0002] With the popularization of smart home technology, users' demand for the intelligent level and response accuracy of home systems is increasing. However, traditional smart home control methods mainly rely on single sensor triggering or fixed scene modes, and have problems such as rigid control and response lag. In particular, in the case of complex family activities or user behavior prediction, the traditional method adopts a single control logic for device state, which makes it difficult to balance energy consumption and user experience, and cannot meet the demand for accurate control of user personalized services in modern smart home systems. SUMMARY

[0003] In view of the above problems, the present application embodiments are proposed to provide a smart home device control method, device, electronic device, and readable storage medium that overcome the above problems or at least partially solve the above problems.

[0004] In a first aspect, the present application embodiments provide a smart home device control method, which comprises: obtaining home multi-modal data; determining a current home scene according to the home multi-modal data; setting the running state of a home device based on the current home scene; detecting whether a user is about to come home, and determining a target home device whose running state is a hibernation state when it is detected that the user is about to come home, and waking up the target home device before the user comes home.

[0005] Optionally, the setting of the running state of the home device based on the current home scene comprises: obtaining associated home devices corresponding to the current home scene and a recommended running state of the associated home devices; determining a device importance score of the associated home devices according to the home multi-modal data; setting the running state of the home device according to the importance score of the associated home devices and the recommended running state.

[0006] Optionally, the determination of the current home scene according to the home multi-modal data comprises: inputting the home multi-modal data into a scene recognition model to obtain scene probabilities of a plurality of home scenes; Based on the scene probabilities of the multiple home scenarios, candidate home scenarios are determined; if the scene probability of the candidate home scenario is greater than or equal to a preset probability threshold, then the candidate home scenario is determined as the current home scenario. If the probability of the candidate home scene is less than the preset probability threshold, then the previous home scene is determined as the current home scene.

[0007] Optionally, the home multimodal data includes home device data and user usage data; The step of determining the device importance score of the associated home appliances based on the home multimodal data includes: Based on the home appliance data and the user usage data, the importance score of the associated home appliances is determined.

[0008] Optionally, setting the operating status of the home appliances based on the importance score of the associated home appliances and the suggested operating status includes: Based on the device importance score, the power consumption level of the corresponding home device is determined; Set the operating status of the home device according to its power consumption level and recommended operating status.

[0009] Optionally, detecting whether a user is about to arrive home includes: Obtain current time, user location information, and historical behavior data; The current time, the user's location information, and the historical behavior data are input into the user behavior prediction model to obtain the prediction confidence score; the prediction confidence score represents the probability that the user is about to arrive home. Based on the predicted confidence level, it is determined whether the user is about to arrive home.

[0010] Optionally, waking up the target home device before the user arrives home includes: Determine the wake-up priority of the target home device; According to the wake-up priority of the target home devices, the target home devices are woken up sequentially before the user arrives home.

[0011] Optionally, determining the wake-up priority of the target home device includes: Obtain the device wake-up cost of the target home appliance; The wake-up priority of the target home device is determined based on the prediction confidence level, the importance score of the target home device, and the device wake-up cost of the target home device.

[0012] Secondly, embodiments of the present invention provide a device for controlling smart home devices, the device comprising: The data acquisition module is used to acquire multimodal data about the home environment. The scene determination module is used to determine the current home scene based on the home multimodal data; The status setting module is used to set the operating status of home devices based on the current home scene; The device wake-up module is used to detect whether a user is about to arrive home. If the user is about to arrive home, the module determines the target home device that is in a dormant state and wakes up the target home device before the user arrives home.

[0013] Optionally, the status setting module includes: The status acquisition submodule is used to acquire the associated home devices corresponding to the current home scene and the suggested operating status of the associated home devices; The scoring determination submodule is used to determine the device importance score of the associated home devices based on the home multimodal data; The status setting submodule is used to set the operating status of the home devices based on the importance score of the associated home devices and the suggested operating status.

[0014] Optionally, the scene determination module includes: The scene acquisition submodule is used to input the home multimodal data into the scene recognition model to obtain the scene probabilities of multiple home scenes; The first scenario determination submodule is used to determine candidate home scenarios based on the scenario probabilities of the multiple home scenarios; if the scenario probability of the candidate home scenario is greater than or equal to a preset probability threshold, then the candidate home scenario is determined as the current home scenario. The second scenario determination submodule is used to determine the previous home scenario as the current home scenario if the scenario probability of the candidate home scenario is less than the preset probability threshold.

[0015] Optionally, the home multimodal data includes home device data and user usage data; The scoring determination submodule includes: The scoring unit is used to determine the importance score of the associated home appliances based on the home appliance data and the user usage data.

[0016] Optionally, the status setting submodule includes: A power consumption determination unit is used to determine the power consumption level of the corresponding home device based on the device importance score. The status setting unit is used to set the operating status of the home device according to the power consumption level and the recommended operating status of the home device.

[0017] Optionally, the device wake-up module includes: The data acquisition submodule is used to acquire current time, user location information, and historical behavior data; The model output submodule is used to input the current time, the user location information, and the historical behavior data into the user behavior prediction model to obtain the prediction confidence score; the prediction confidence score represents the probability that the user is about to arrive home. The home arrival confirmation submodule is used to determine whether the user is about to arrive home based on the predicted confidence level.

[0018] Optionally, the device wake-up module includes: The priority determination submodule is used to determine the wake-up priority of the target home device; The device wake-up submodule is used to wake up the target home devices sequentially before the user arrives home, according to the wake-up priority of the target home devices.

[0019] Optionally, the priority determination submodule includes: A wake-up cost acquisition unit is used to acquire the device wake-up cost of the target home device; The priority determination unit is used to determine the wake-up priority of the target home device based on the prediction confidence level, the importance score of the target home device, and the device wake-up cost of the target home device.

[0020] Thirdly, embodiments of the present invention provide an electronic device including a processor, a memory, and a program or instructions stored in the memory and executable on the processor, wherein the program or instructions, when executed by the processor, implement the steps of the control method for a smart home device as described in the first aspect.

[0021] Fourthly, embodiments of the present invention provide a readable storage medium storing a program or instructions that, when executed by a processor, implement the steps of the control method for a smart home device as described in the first aspect.

[0022] The embodiments of the present invention have the following advantages: This invention provides a control method, device, electronic device, and medium for smart home devices. The method acquires multimodal home data; determines the current home scene based on the multimodal data; sets the operating state of the home devices based on the current home scene; detects whether a user is about to arrive home; and if so, identifies a target home device in a dormant state and wakes it up before the user arrives. This invention employs a dual-judgment mechanism based on scene recognition and user behavior prediction, simultaneously predicting both the current device state and user behavior. Compared to methods using a single control logic, this effectively improves the accuracy of smart home device control. Attached Figure Description

[0023] To more clearly illustrate the technical solutions of the embodiments of the present invention, the accompanying drawings used in the description of the embodiments of the present invention will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0024] Figure 1 This is a flowchart illustrating the steps of a control method for a smart home device provided in an embodiment of the present invention; Figure 2 This is a flowchart of another method for controlling a smart home device provided in an embodiment of the present invention; Figure 3 This is a structural block diagram of a control device for a smart home device provided in an embodiment of the present invention. Detailed Implementation

[0025] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some, not all, of the embodiments of the present invention. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0026] The terms "first," "second," etc., used in the specification and claims of this invention are used to distinguish similar objects and are not used to describe a specific order or sequence. It should be understood that such data can be interchanged where appropriate so that embodiments of the invention can be implemented in orders other than those illustrated or described herein. Furthermore, in the specification and claims, "and / or" indicates at least one of the connected objects, and the character " / " generally indicates that the preceding and following objects are in an "or" relationship.

[0027] Smart homes, as an important part of modern life, are not only a key vehicle for improving living comfort but also a core element for achieving green, energy-saving, and convenient living. However, with the rapid increase in the number of smart devices, the contradiction between system energy consumption control and user experience assurance is becoming increasingly prominent. Traditional control strategies are clearly insufficient in response accuracy and adaptability when faced with complex family activities, multi-member behavior patterns, and device heterogeneity, making it difficult to achieve the expected goals of intelligent management.

[0028] In recent years, the integration of edge computing and artificial intelligence technologies has provided new solutions for optimizing smart home systems. In particular, multimodal data fusion technology can simultaneously capture multi-dimensional information such as environmental conditions, device operation, and user behavior, creating conditions for achieving refined home control. However, in the face of dynamic and ever-changing home environments and personalized user needs, how to achieve precise responses in device control remains a key technical challenge in the current smart home field.

[0029] In the field of smart home control, rule engines and machine learning-based methods offer feasible solutions for improving the intelligence level of systems. While existing technologies have achieved scenario-based control to some extent, they still generally suffer from problems such as rigid strategies, slow response times, and insufficient personalization. In particular, they lack the ability to accurately perceive and predict user behavior intentions, making it difficult to balance energy consumption and user experience.

[0030] In the existing technology in this field, smart home control mainly adopts the following two methods: Method A: A scene-based device linkage control method. This method uses preset scene configuration templates to adjust the status of associated devices in batches after manual triggering by the user. While this method can achieve basic automation, it heavily relies on active user intervention and lacks the ability to autonomously perceive environmental states and user intentions, resulting in limited intelligence in practical use.

[0031] Method B: Device control method based on a single sensor trigger. This method monitors specific environmental parameters (such as human movement, light intensity, etc.) and automatically controls the device status according to preset thresholds. Although it achieves unattended control, this method lacks the ability to recognize complex behavioral intentions and is prone to misjudgments (such as misjudging a quiet reading session as an unattended state), seriously affecting the user experience.

[0032] Method A, similar to this invention, aims to improve control efficiency through scenario-based strategies, but it relies on manual presets and triggers, making it unable to adapt to dynamically changing family activity needs. Method B, while achieving automated control, suffers from a superficial understanding of user behavior and a simplistic control strategy, making it prone to misoperation when dealing with complex life scenarios. Both methods have significant shortcomings in terms of intelligence, scenario adaptability, and user experience, necessitating the development of a novel smart home control technology based on multimodal perception, possessing predictive capabilities, and capable of autonomously optimizing control strategies.

[0033] One of the core concepts of this invention is to first acquire multimodal home data; determine the current home scene based on the multimodal data; set the operating status of home devices based on the current home scene; detect whether the user is about to arrive home; if the user is about to arrive home, identify the target home device in a dormant state and wake it up before the user arrives home. This invention employs a dual judgment mechanism based on scene recognition and user behavior prediction, simultaneously predicting and processing the current device status and user behavior. Compared to using a single control logic, this effectively improves the accuracy of smart home device control while also considering device energy consumption, achieving the dual effects of improved user experience and energy saving.

[0034] Figure 1 This is a flowchart of the steps of a home appliance control method provided in an embodiment of the present invention.

[0035] like Figure 1 As shown, the method may specifically include the following steps: Step 101: Obtain home multimodal data; This invention can be applied to a home host for controlling smart home devices. The home host can connect to multiple smart home devices. When the home host is first started, it can discover and record the IDs and types of all devices and sensors in the network. During operation, the host acts as a central node, continuously collecting data from various sources.

[0036] In this embodiment of the invention, it is necessary to continuously collect multimodal data from home sensors, such as temperature sensors, light sensors, human infrared sensors, device operating status, and user operation records. This data forms the basis for comprehensive home environment perception and includes multi-dimensional information such as temperature and humidity sensor data, light sensor data, door magnetic switch status, human infrared sensor data, home appliance operating status, program selection, remaining time, user voice commands, device online status, and communication protocol type.

[0037] By acquiring rich multimodal data, comprehensive information support is provided for subsequent scene recognition and device control decisions, which helps improve the system's understanding accuracy of complex home environments.

[0038] Step 102: Determine the current home scene based on the home multimodal data; In this embodiment of the invention, multimodal home data can be input into a pre-trained model. Through feature extraction and deep learning inference, eight predefined home scenarios, including leaving home, returning home, dining, watching movies, sleeping, working, entertaining, and cleaning, can be accurately identified. This effectively integrates features from different modalities to capture typical patterns of various scenarios.

[0039] Through precise scene recognition, the system can accurately understand the user's current activity status and needs, providing a reliable context-aware basis for subsequent device control.

[0040] Step 103: Based on the current home scene, set the operating status of the home devices; In this embodiment of the invention, the operating status of home devices can be set according to the identified home scene. The operating status of home devices may include: active state, sleep state, and standby state, etc.

[0041] By providing personalized device status management tailored to specific scenarios, the system achieves refined energy consumption control while ensuring user comfort, effectively solving the problem of balancing energy consumption and user experience in traditional smart home systems.

[0042] Step 104: Detect whether the user is about to arrive home. If the user is about to arrive home, determine the target home device that is in a dormant state and wake up the target home device before the user arrives home.

[0043] In this embodiment of the invention, time characteristics, geographical location information, and user historical behavior patterns can be comprehensively analyzed to predict the user's intention to return home and their arrival time. When it is predicted that the user is about to arrive home, a target home appliance in a dormant state can be identified and woken up before the user arrives home.

[0044] Through predictive wake-up mechanisms, seamless intelligent services are achieved, allowing users to enjoy a comfortable home environment as soon as they arrive home, significantly enhancing the user experience.

[0045] This invention first acquires multimodal home data; based on this data, it determines the current home scene; based on this scene, it sets the operating status of home devices; it detects whether a user is about to arrive home, and if so, identifies a target home device in a dormant state and wakes it up before the user arrives. This invention employs a dual-judgment mechanism based on scene recognition and user behavior prediction, simultaneously predicting both the current device status and user behavior. Compared to a single control logic approach, this effectively improves the accuracy of smart home device control while also considering energy consumption, achieving both enhanced user experience and energy savings.

[0046] It should be noted that the control method for smart home devices provided in this embodiment of the invention can be executed by a control device for the smart home device, or a control module within that control device for executing the control method for loading the smart home device. This embodiment of the invention uses the execution of the control method for loading the smart home device by the control device of the smart home device as an example to illustrate the control method for home devices provided in this embodiment of the invention.

[0047] Figure 2 This is a flowchart of another method for controlling a smart home device provided in an embodiment of the present invention.

[0048] like Figure 2 As shown, the method may specifically include the following steps: Step 201: Obtain home multimodal data; In this embodiment of the invention, it is necessary to acquire multimodal data of the home environment. This data typically comes from various sensors and smart devices deployed in the home environment, and includes multi-dimensional information such as environmental status, device operation, and user behavior.

[0049] In some examples, data acquisition requires preprocessing such as filtering and standardization to eliminate noise and dimensional differences, ensure data quality, and provide reliable input for subsequent scene recognition and equipment control.

[0050] For example, home multimodal data may include: temperature sensor data, light sensor data, human infrared sensor signals, device status data, such as air conditioner on / off status, power consumption, operating mode, and user operation records, such as App control, voice commands, etc.

[0051] By comprehensively collecting multimodal data, a rich foundation for environmental perception is provided for the system, ensuring the accuracy and reliability of subsequent decisions and overcoming the shortcomings of traditional single sensor data sources.

[0052] Step 202: Determine the current home scene based on the home multimodal data; In this embodiment of the invention, multimodal home data is input into a multimodal fusion network based on an attention mechanism for processing. This network can simultaneously process multi-source data such as environmental sensing, device status, user operations, and time information, and output a probability distribution of a predefined home scene.

[0053] In some embodiments, step 202 may include the following sub-steps: Sub-step S11: Input the home multimodal data into the scene recognition model to obtain the scene probabilities of multiple home scenes; In this embodiment of the invention, the input multimodal data can be feature extracted and fused using a scene recognition model to output probability values ​​for eight predefined scenarios: leaving home, returning home, dining, watching a movie, sleeping, working, entertainment, and cleaning.

[0054] Sub-step S12: Determine candidate home scenes based on the scene probabilities of the multiple home scenes; if the scene probability of the candidate home scene is greater than or equal to a preset probability threshold, then determine the candidate home scene as the current home scene; In this embodiment of the invention, the scenario with the highest scenario probability among multiple home scenarios can be selected as a candidate home scenario. When the scenario probability of the candidate home scenario is greater than or equal to a preset probability threshold, for example, when the probability value exceeds the preset threshold of 0.7, it is determined that the current situation is in that scenario.

[0055] Sub-step S13: If the probability of the candidate home scene is less than the preset probability threshold, then the previous home scene is determined as the current home scene.

[0056] In this embodiment of the invention, the scenario with the highest scenario probability among multiple home scenarios can be selected as the candidate home scenario. When the scenario probability of the candidate home scenario is less than a preset probability threshold, for example, when the probability value is less than the preset threshold of 0.7, the previous valid scenario determination is maintained to avoid scenario misjudgment and frequent switching caused by instantaneous data fluctuations.

[0057] In some examples, the training process for a scene recognition model may include the following steps: First, the collected raw data is preprocessed: During the initial stage of operation (days 1-14), sensor data is continuously collected, such as temperature and humidity, light intensity, human infrared detection, door magnetic sensors, and sound; device status data, such as air conditioner on / off status, light brightness, and television status; user operation records, such as manually turning devices on and off and adjusting parameters; and time information. Users can label certain scenes using the app, such as "I am cooking now" or "I am watching a movie." After labeling 50-100 samples, this labeled data can be used to train the scene recognition model.

[0058] The preprocessed data is then input into the scene recognition model. The model learns data feature patterns under different scenarios. For example, the model integrates multi-source data, including environmental sensing, device status, user operation, and time information. Through the learned feature patterns, it outputs the probability distribution of eight scenarios. For example, the features of the "movie-watching scenario" are: TV on + living room lights dimmed + sound sensor detects audio-visual signals + time is evening; the features of the "sleep scenario" are: bedroom lights off + no human infrared signals + time is 10:00 PM - 6:00 AM. After training, the model can automatically recognize eight common household scenarios: leaving home, returning home, dining, watching a movie, sleeping, working, entertaining, and cleaning. For example, the output at a certain moment is: {Leaving home: 0.05, Returning home: 0.02, Dining: 0.03, Watching a movie: 0.85, Sleeping: 0.01, Working: 0.02, Entertaining: 0.01, Cleaning: 0.01}.

[0059] Finally, scene determination is performed: for example, the highest value in the probability distribution is extracted from the model inference results. When the highest probability is greater than 0.7, the current scene is determined to be the corresponding category. In the example above, the probability of "watching a movie" is 0.85 > 0.7, so the current scene is determined to be a "movie watching scene". If the highest probability is less than or equal to 0.7, it indicates that the current state is ambiguous, and the previous scene determination is maintained to avoid frequent erroneous switching. Through the multimodal fusion scene recognition mechanism, the user's current activity intention can be accurately understood, providing a decision-making basis for refined device control and effectively solving the problem of low accuracy in traditional scene recognition methods.

[0060] Step 203: Obtain the associated home devices corresponding to the current home scene and the suggested operating status of the associated home devices; In this embodiment of the invention, based on the identified home scenario, the corresponding associated devices and their suggested operating states can be queried from a preset scenario-device policy library. The policy library can define initial power consumption level suggestions for each device under different scenarios, serving as a baseline policy for device control.

[0061] In some examples, four power consumption levels can be preset, each containing specific functional limitations and wake-up times. For example: L0 - Active State: 100% power consumption, the device is fully operational, all functions are available, and the response time is <100ms, suitable for devices that the user is currently using. L1 - Standby State: 30-50% power consumption, the device is idle but can respond momentarily, maintaining network connectivity and basic listening, with a response time of <1 second, suitable for devices that the user may use at any time. L2 - Shallow Sleep: 10-20% power consumption, the device disables some functions, maintains minimal network listening, and has a response time of <5 seconds, suitable for devices that the user uses less frequently. L3 - Deep Sleep: 5-10% power consumption, the device retains only minimal network listening, disables most functions, and has a response time of <30 seconds, suitable for devices that the user rarely uses.

[0062] For example, when identified as a "sleep scenario", the policy library suggests: the bedroom air conditioner (L1) maintains the temperature but reduces the fan speed, the bedroom lights (L3) go into deep sleep, the living room TV (L3) go into deep sleep, and the security device (L0) remains on alert.

[0063] The pre-defined scenario-device strategy library provides a basic decision-making framework for device control, ensuring the standardization and consistency of system control, while also providing a benchmark reference for subsequent personalized adjustments.

[0064] Step 204: Determine the device importance score of the associated home appliances based on the home multimodal data; In this embodiment of the invention, a personalized importance score for each associated device is calculated based on home multimodal data. This score comprehensively reflects the device's usability and user preferences in a specific scenario.

[0065] In some embodiments, the home multimodal data includes home device data and user usage data, and step 204 includes the following sub-steps: Sub-step S21: Determine the importance score of the associated home appliances based on the home appliance data and the user usage data.

[0066] In this embodiment of the invention, the importance score can be obtained by weighted calculation of four dimensions: usage frequency, user preference, safety criticality, and energy efficiency level. The usage frequency can be calculated every 30 days. In some examples, the importance score calculation formula can be: Importance score = 0.4 × frequency of use + 0.3 × user preference + 0.2 × safety criticality + 0.1 × energy efficiency rating.

[0067] By using a device importance scoring mechanism, personalized adaptation of control strategies is achieved, taking into full account user habits and preferences, and effectively improving the system's intelligence level and user experience.

[0068] Step 205: Set the operating status of the home devices according to the importance score of the associated home devices and the suggested operating status.

[0069] In this embodiment of the invention, the recommended operating state is dynamically adjusted based on the equipment importance score, and the final equipment control command is generated and executed.

[0070] Step 205 may include the following sub-steps: Sub-step S31: Determine the power consumption level of the corresponding home device based on the device importance score; In this embodiment of the invention, the power consumption level of the corresponding home device can be fine-tuned based on the device importance score.

[0071] In some examples, specific rules may include: if the policy suggests L3 but the score is >0.6, upgrade to L2 (if the score is >0.8, upgrade to L1); if the policy suggests L2 but the score is <0.3, downgrade to L3; if the policy suggests L1 but the score is <0.2, downgrade to L2; security devices are forced to maintain their original level or higher. Example: In a movie-watching scenario, a floor lamp has a policy suggestion of L3 and a score of 0.4, so it is not adjusted for now; if the user manually turns it on repeatedly, raising the score to 0.65, upgrade it to L2 to improve device response speed.

[0072] Sub-step S32: Set the operating status of the home device according to the power consumption level and the recommended operating status of the home device.

[0073] In this embodiment of the invention, a power level switching command is sent to the device to control the device to enter the corresponding operating state (L0-L3), and the control results are recorded locally for subsequent optimization.

[0074] Step 206: Detect whether the user is about to arrive home. If the user is about to arrive home, determine the target home device that is in a dormant state and wake up the target home device before the user arrives home.

[0075] In this embodiment of the invention, the user's intention to go home is determined based on a user behavior prediction model, and predictive wake-up is performed on devices in a dormant state to ensure that critical devices are ready when the user arrives home.

[0076] In some examples, step 206 may include the following sub-steps: Sub-step S41: Obtain the current time, user location information, and historical behavior data; In this embodiment of the invention, input data for behavior prediction can be collected, including the current timestamp, user GPS location information, such as within a 500-meter geofence, and historical behavior data from the past 7 days.

[0077] In some examples, the system clock can be used to obtain a precise current timestamp, including hour, minute, and day of the week information; a user-authorized mobile app can be used to obtain real-time GPS location coordinates and calculate the user's relative position to the home geofence; and the user's behavior history for the past 30 days can be extracted from the local database, including the distribution of daily departure and return times, as well as typical device usage sequences after arriving home.

[0078] By comprehensively collecting multi-dimensional data, we provide comprehensive and accurate input information for behavior prediction, ensuring the reliability and adaptability of the prediction model.

[0079] Sub-step S42 involves inputting the current time, the user's location information, and the historical behavior data into the user behavior prediction model to obtain the prediction confidence score; the prediction confidence score represents the probability that the user is about to arrive home. In this embodiment of the invention, a user behavior prediction model constructed by an LSTM-GRU hybrid network can comprehensively analyze spatiotemporal features and behavioral patterns to output the prediction confidence of a user's imminent return home.

[0080] In some examples, the host runs a behavior prediction model periodically. The model takes into account inputs such as the current time (whether it falls within a historical homecoming time window), the user's GPS location (whether it's within a geofence), and historical behavioral patterns (behavioral patterns during the same period over the past 30 days), and outputs a prediction confidence level that "the user is about to go home." Users can also authorize the provision of a schedule as a basis for judgment, improving the accuracy of predicting user behavioral intentions.

[0081] In some examples, the model outputs a prediction confidence score as a probability value between 0 and 1, representing the likelihood that the user will arrive home within a specific future time window (such as 15 minutes). For instance, if a user enters the geofenced area at 18:10 on a weekday, the model might output a confidence score of 0.87, indicating an 87% certainty that the user will arrive home soon.

[0082] By fusing and analyzing multi-source time-series data through deep learning models, we can accurately capture user behavior patterns and provide a reliable basis for predictive wake-up decisions.

[0083] Sub-step S43: Based on the predicted confidence level, determine whether the user is about to arrive home; In this embodiment of the invention, it can be determined whether a user is about to arrive home based on the predicted confidence level.

[0084] In some examples, when the prediction confidence is greater than or equal to a preset threshold and the user's location is within the geofence, it is determined that "the user is about to arrive home," and the predictive wake-up process is initiated.

[0085] In some examples, the preset threshold can be adaptively adjusted based on historical prediction accuracy. For instance, when the system makes multiple consecutive successful predictions, the threshold will be appropriately lowered to 0.75 to improve sensitivity; conversely, when over-wake-up occurs, the threshold will be raised to 0.85 to reduce the false trigger rate.

[0086] Sub-step S44: Determine the wake-up priority of the target home device; In this embodiment of the invention, all devices currently in L2 or L3 sleep state are scanned, and a wake-up priority is calculated for each device.

[0087] In some embodiments, sub-step S44 may include the following steps: Obtain the device wake-up cost of the target home appliance; In this embodiment of the invention, the wake-up cost of the target home device can be obtained. The wake-up cost is the time required for the device to start earlier to ensure that the user is in a comfortable state when they arrive home.

[0088] For example, water heaters take 30 minutes to heat up, air conditioners take 15 minutes to cool down, and lights only take 1 second to turn on.

[0089] The wake-up priority of the target home device is determined based on the prediction confidence level, the importance score of the target home device, and the device wake-up cost of the target home device.

[0090] In this embodiment of the invention, the wake-up priority of the target home device can be determined based on the weighted sum of the prediction confidence level, the importance score of the target home device, and the device wake-up cost of the target home device.

[0091] For example, the formula for calculating wake-up priority can be: Wake-up priority = 0.5 × prediction confidence + 0.2 × importance score + 0.3 × device wake-up cost Sub-step S45: According to the wake-up priority of the target home devices, wake up the target home devices sequentially before the user arrives home.

[0092] In this embodiment of the invention, the target home devices can be woken up sequentially before the user arrives home, according to their wake-up priority.

[0093] In some examples, the wake-up time is calculated backwards from the user's expected arrival time and the wake-up cost of each device. A gradient wake-up strategy is used to ensure that the device is ready exactly when the user arrives home: Assuming the user is 500 meters away from home and is expected to arrive in 10 minutes, the following wake-up sequence is executed: T-30 minutes: The user has not yet entered the geofence, for example, the entrance to the community is set as the geofence, but the model has predicted the trend of returning home and wakes up the water heater in advance to start heating; T-15 minutes: The user enters the geofence, wakes up the air conditioner to start cooling; T-1 minute: Wakes up the living room lights to standby mode. When the user enters the door, T=0, the door sensor triggers the door opening event, the water heater has reached the set temperature, the air conditioner has lowered the room temperature to the comfortable range, and the lights instantly turn on to the user's preferred brightness.

[0094] This phased, tiered wake-up strategy can improve users' precise control over smart homes and enhance the user experience of proactive smart home regulation.

[0095] This invention first acquires multimodal home data; based on this data, it determines the current home scene; it acquires the associated home devices corresponding to the current scene and their suggested operating states; based on the multimodal home data, it determines the device importance score of the associated home devices; based on the device importance score and suggested operating state, it sets the operating state of the home devices; it detects whether the user is about to arrive home, and if so, it identifies the target home device in a dormant state and wakes it up before the user arrives. This invention constructs a dual judgment mechanism of scene recognition and user behavior prediction, and innovatively introduces a dynamic device importance score based on multimodal data to perform personalized and precise control of device operating states. Compared with methods using a single control logic, this method not only achieves basic energy efficiency management through a scenario-based strategy, but also further combines user behavior prediction and device importance assessment. While ensuring a seamless user experience, it achieves refined and intelligent control of device energy consumption, achieving synergistic optimization of the dual goals of improved user experience and energy saving.

[0096] It should be noted that, for the sake of simplicity, the method embodiments are all described as a series of actions. However, those skilled in the art should understand that the embodiments of the present invention are not limited to the described order of actions, because according to the embodiments of the present invention, some steps can be performed in other orders or simultaneously. Furthermore, those skilled in the art should also understand that the embodiments described in the specification are preferred embodiments, and the actions involved are not necessarily essential to the embodiments of the present invention.

[0097] Figure 3 This is a control device for a smart home device provided in an embodiment of the present invention.

[0098] like Figure 3 As shown, it can specifically include the following modules: Data acquisition module 301 is used to acquire multimodal data of the home. Scene determination module 302 is used to determine the current home scene based on the home multimodal data; The status setting module 303 is used to set the operating status of home devices based on the current home scene; The device wake-up module 304 is used to detect whether the user is about to arrive home. If the user is about to arrive home, the module determines the target home device that is in a dormant state and wakes up the target home device before the user arrives home.

[0099] In this embodiment of the invention, the state setting module includes: The status acquisition submodule is used to acquire the associated home devices corresponding to the current home scene and the suggested operating status of the associated home devices; The scoring determination submodule is used to determine the device importance score of the associated home devices based on the home multimodal data; The status setting submodule is used to set the operating status of the home devices based on the importance score of the associated home devices and the suggested operating status.

[0100] In this embodiment of the invention, the scene determination module includes: The scene acquisition submodule is used to input the home multimodal data into the scene recognition model to obtain the scene probabilities of multiple home scenes; The first scenario determination submodule is used to determine candidate home scenarios based on the scenario probabilities of the multiple home scenarios; if the scenario probability of the candidate home scenario is greater than or equal to a preset probability threshold, then the candidate home scenario is determined as the current home scenario. The second scenario determination submodule is used to determine the previous home scenario as the current home scenario if the scenario probability of the candidate home scenario is less than the preset probability threshold.

[0101] In this embodiment of the invention, the home multimodal data includes home equipment data and user usage data; The scoring determination submodule includes: The scoring unit is used to determine the importance score of the associated home appliances based on the home appliance data and the user usage data.

[0102] In this embodiment of the invention, the state setting submodule includes: A power consumption determination unit is used to determine the power consumption level of the corresponding home device based on the device importance score. The status setting unit is used to set the operating status of the home device according to the power consumption level and the recommended operating status of the home device.

[0103] In this embodiment of the invention, the device wake-up module includes: The data acquisition submodule is used to acquire current time, user location information, and historical behavior data; The model output submodule is used to input the current time, the user location information, and the historical behavior data into the user behavior prediction model to obtain the prediction confidence score; the prediction confidence score represents the probability that the user is about to arrive home. The home arrival confirmation submodule is used to determine whether the user is about to arrive home based on the predicted confidence level.

[0104] In this embodiment of the invention, the device wake-up module includes: The priority determination submodule is used to determine the wake-up priority of the target home device; The device wake-up submodule is used to wake up the target home devices sequentially before the user arrives home, according to the wake-up priority of the target home devices.

[0105] In this embodiment of the invention, the priority determination submodule includes: A wake-up cost acquisition unit is used to acquire the device wake-up cost of the target home device; The priority determination unit is used to determine the wake-up priority of the target home device based on the prediction confidence level, the importance score of the target home device, and the device wake-up cost of the target home device.

[0106] This invention first acquires multimodal home data; based on this data, it determines the current home scene; based on this scene, it sets the operating status of home devices; it detects whether a user is about to arrive home, and if so, identifies a target home device in a dormant state and wakes it up before the user arrives. This invention employs a dual-judgment mechanism based on scene recognition and user behavior prediction, simultaneously predicting both the current device status and user behavior. Compared to a single control logic approach, this effectively improves the accuracy of smart home device control while also considering energy consumption, achieving both enhanced user experience and energy savings.

[0107] As the apparatus embodiment is basically similar to the method embodiment, it is described in a relatively simple manner. For relevant details, please refer to the description of the method embodiment.

[0108] This invention also provides an electronic device, including a processor, a memory, and a program or instructions stored in the memory and executable on the processor. When the program or instructions are executed by the processor, they implement the various processes of the above-described smart home device control method embodiments and achieve the same technical effects. To avoid repetition, they will not be described again here.

[0109] It should be noted that the electronic devices in the embodiments of the present invention include the mobile electronic devices and non-mobile electronic devices described above.

[0110] This invention also provides a readable storage medium storing a program or instructions. When the program or instructions are executed by a processor, they implement the various processes of the above-described control method embodiments for smart home devices and achieve the same technical effects. To avoid repetition, they will not be described again here.

[0111] The processor is the processor in the electronic device described in the above embodiments. The readable storage medium includes computer-readable storage media, such as computer read-only memory (ROM), random access memory (RAM), magnetic disk, or optical disk.

[0112] The various embodiments in this specification are described in a progressive manner, with each embodiment focusing on the differences from other embodiments. The same or similar parts between the various embodiments can be referred to each other.

[0113] Those skilled in the art will understand that embodiments of the present invention can be provided as methods, apparatus, or computer program products. Therefore, embodiments of the present invention can take the form of entirely hardware embodiments, entirely software embodiments, or embodiments combining software and hardware aspects. Furthermore, embodiments of the present invention can take the form of computer program products implemented on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0114] Embodiments of the present invention are described with reference to flowchart illustrations and / or block diagrams of methods, terminal devices (systems), and computer program products according to embodiments of the invention. It will be understood that each block of the flowchart illustrations and / or block diagrams, as well as combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, special-purpose computer, embedded processor, or other programmable data processing terminal device to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing terminal device, generate instructions for implementing the flowchart illustrations and / or block diagrams. Figure 1 One or more processes and / or boxes Figure 1 A device that provides the functions specified in one or more boxes.

[0115] These computer program instructions may also be stored in a computer-readable storage medium that can direct a computer or other programmable data processing terminal device to operate in a particular manner, such that the instructions stored in the computer-readable storage medium produce an article of manufacture including instruction means, which are implemented in a process Figure 1 One or more processes and / or boxes Figure 1 The function specified in one or more boxes.

[0116] These computer program instructions can also be loaded onto a computer or other programmable data processing terminal equipment, causing a series of operational steps to be performed on the computer or other programmable terminal equipment to produce a computer-implemented process, thereby providing instructions that execute on the computer or other programmable terminal equipment for implementing the process. Figure 1 One or more processes and / or boxes Figure 1 The steps of the function specified in one or more boxes.

[0117] Although preferred embodiments of the present invention have been described, those skilled in the art, upon learning the basic inventive concept, can make other changes and modifications to these embodiments. Therefore, the appended claims are intended to be interpreted as including the preferred embodiments as well as all changes and modifications falling within the scope of the present invention.

[0118] Finally, it should be noted that in this document, relational terms such as "first" and "second" are used only to distinguish one entity or operation from another, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Furthermore, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or terminal device that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or terminal device. Without further limitations, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or terminal device that includes said element.

[0119] The control method, apparatus, electronic device, and computer-readable storage medium for a smart home device provided by the present invention have been described in detail above. Specific examples have been used to illustrate the principles and implementation methods of the present invention. The description of the above embodiments is only for the purpose of helping to understand the method and core ideas of the present invention. At the same time, for those skilled in the art, there will be changes in the specific implementation methods and application scope based on the ideas of the present invention. Therefore, the content of this specification should not be construed as a limitation of the present invention.

Claims

1. A control method of a smart home device, the method comprising: The method comprises: obtaining home multi-modal data; determining a current home scene according to the home multi-modal data; setting an operation state of a home device based on the current home scene; detecting whether a user is about to come home, and determining a target home device whose operation state is a hibernation state and waking up the target home device before the user comes home if it is detected that the user is about to come home. 2.The control method of a smart home device according to claim 1, characterized in that, The setting of the operation state of the home device based on the current home scene comprises: obtaining associated home devices corresponding to the current home scene and a recommended operation state of the associated home devices; determining a device importance score of the associated home devices according to the home multi-modal data; setting the operation state of the home device according to the importance score of the associated home devices and the recommended operation state. 3.The control method of a smart home device according to claim 1, characterized in that, The determination of the current home scene according to the home multi-modal data comprises: inputting the home multi-modal data into a scene recognition model to obtain scene probabilities of a plurality of home scenes; determining a candidate home scene according to the scene probabilities of the plurality of home scenes; if the scene probability of the candidate home scene is greater than or equal to a preset probability threshold, the candidate home scene is determined as the current home scene; if the scene probability of the candidate home scene is less than the preset probability threshold, a previous home scene is determined as the current home scene. 4.The control method of a smart home device according to claim 2, characterized in that, The home multi-modal data comprises home device data and user usage data. The determination of the device importance score of the associated home devices according to the home multi-modal data comprises: determining the importance score of the associated home devices according to the home device data and the user usage data. 5.The control method of a smart home device according to claim 2, characterized in that, The setting of the operation state of the home device according to the importance score of the associated home devices and the recommended operation state comprises: determining a power consumption level of a corresponding home device according to the device importance score; setting the operation state of the home device according to the power consumption level of the home device and the recommended operation state. 6.The control method of a smart home device according to claim 1, wherein, The detection of whether the user is about to come home comprises: obtaining current time, user location information and historical behavior data; inputting the current time, the user location information and the historical behavior data into a user behavior prediction model to obtain a prediction confidence; the prediction confidence represents a probability of whether the user is about to come home; determining whether the user is about to come home according to the prediction confidence. 7.The control method of a smart home device according to claim 6, characterized in that, The waking up of the target home device before the user comes home comprises: determining a wake-up priority of the target home device; waking up the target home device in turn before the user comes home according to the wake-up priority of the target home device. 8.The control method of a smart home device according to claim 7, characterized in that, The determination of the wake-up priority of the target home device comprises: obtaining a device wake-up cost of the target home device; determining the wake-up priority of the target home device according to the prediction confidence, the importance score of the target home device and the device wake-up cost of the target home device. 9.A control device of a smart home device, characterized by, The apparatus comprises: a data acquisition module configured to obtain home multi-modal data; a scene determination module configured to determine a current home scene according to the home multi-modal data; a state setting module, configured to set a running state of the home device based on the current home scene; a device wake-up module, configured to detect whether a user is about to arrive home, and in a case where it is detected that the user is about to arrive home, determine a target home device whose running state is a hibernation state, and wake up the target home device before the user arrives home.

10. An electronic device, comprising: The device comprises a processor, a memory, and a program or instruction stored in the memory and executable on the processor, and the program or instruction is executed by the processor to implement the steps of the control method of the smart home device as claimed in claims 1-8.

11. A readable storage medium, characterized by, The readable storage medium stores a program or instruction, and the program or instruction is executed by the processor to implement the steps of the control method of the smart home device as claimed in claims 1-8.

Images