A smart home user intention prediction method and system based on multi-modal behavior symmetry features

Abstract

This invention relates to the field of smart home technology, specifically to a method and system for user behavior analysis and intent prediction in a smart home system. The method acquires multimodal user behavior data, including operating status data of smart home devices, environmental sensor detection data, and server-side control command data. The data undergoes timestamp conversion, behavior encoding, and normalization, dividing it into multiple continuous sub-sequences as model training samples. The preprocessed training samples are input into a time-series prediction model for training. The time-series prediction model sequentially includes a sequence encoding layer, a near-symmetric attention layer, a far-symmetric attention layer, and a prediction output layer. The user intent prediction result is obtained, and corresponding smart home device control commands are generated. The technical solution of this invention can improve the accuracy and stability of user intent prediction in complex application scenarios, providing more reliable technical support for the automatic control and intelligent decision-making of intelligent systems.

Description

Technical Field

[0001] This invention relates to the field of smart home technology, and more specifically to a method and system for user behavior analysis and intent prediction in a smart home system. Background Technology

[0002] User behavior prediction is one of the key technologies in intelligent systems. Its core lies in inferring users' future operational tendencies or behavioral intentions by analyzing their historical behavior data. It is widely used in application scenarios such as recommendation systems, intelligent interaction, smart homes, and automatic control.

[0003] Existing user intent prediction methods typically use user behavior sequences as modeling objects, employing deep learning structures such as recurrent neural networks and attention mechanisms to learn the temporal correlations within these sequences, thereby predicting the user's next action or preference. While these methods can characterize the temporal dependencies of user behavior to some extent, their modeling approaches mostly focus on uniformly modeling the overall temporal dependencies of the behavior sequence, usually employing a single attention structure or a uniform temporal feature extraction method.

[0004] However, in real-world applications, user behavior often exhibits distinct structural characteristics, especially in continuous control scenarios such as smart homes. User actions commonly involve paired state changes and recurring control habits across time periods. For example, device on / off operations often correspond in time, and the control behaviors of different devices may show symmetrical patterns over short or long time scales. Current technologies lack specific modeling methods for these symmetrical behavioral characteristics, making it difficult to effectively distinguish between short-term control symmetries and long-term control patterns formed over time.

[0005] Furthermore, existing user behavior prediction methods typically rely primarily on single-type behavioral data for modeling, failing to fully integrate multimodal data. In smart home scenarios, user intent is not only reflected in device operation sequences but is also closely related to environmental conditions such as temperature and lighting. Most prediction models only use operation sequences as primary input, failing to deeply integrate environmental conditions as an equally important modality, thus ignoring contextual information crucial for intent understanding and leading to prediction bias.

[0006] Therefore, existing technologies still suffer from insufficient understanding of user behavior structure features and limited prediction accuracy in complex application scenarios, and there is an urgent need for a new method for predicting user behavior intent to overcome these shortcomings. Summary of the Invention

[0007] The purpose of this invention is to address the technical problems of existing smart home user intent prediction methods in complex application scenarios, which are difficult to distinguish between modeling "near-symmetric" and "far-symmetric" behavioral patterns due to their simple model structure, and the prediction bias caused by the failure to deeply integrate multimodal data. This invention provides a smart home user intent prediction method and system based on multimodal behavioral symmetry features. By performing unified preprocessing and structured representation of multimodal user behavior data, and introducing a hierarchical modeling mechanism for behavioral symmetry in the time series modeling process, the prediction model can focus on the symmetric control behavior features formed by users at different time scales, thereby more comprehensively and accurately understanding the user's true control intent.

[0008] The technical solution of this invention can improve the accuracy and stability of user intent prediction in complex application scenarios, and provide more reliable technical support for the automatic control and intelligent decision-making of intelligent systems.

[0009] To achieve the above objectives, the technical solution adopted by the present invention is as follows:

[0010] A method for predicting user intent in smart homes based on multimodal behavioral symmetric features includes:

[0011] S1: Acquire multimodal user behavior data, which includes at least: operating status data of home appliances; sensor detection data related to the environment; and control command data from the server; wherein each data record includes the identification of the control device, the control status, and the control timestamp;

[0012] S2: Preprocess the multimodal user behavior data, the preprocessing including:

[0013] The control timestamp is converted into a feature representing the time interval between adjacent control actions; the control device identifier and control status are encoded.

[0014] The time interval, device code, and status code are normalized to eliminate the dimensional differences between different modal data.

[0015] The preprocessed sequence data is divided into multiple consecutive subsequences using a sliding time window method, which serve as training samples for the model.

[0016] S3: Input the preprocessed training samples into the time series prediction model for training. The time series prediction model includes, in sequence:

[0017] Sequence coding layers are used to extract temporal features from input sequence data;

[0018] A near-symmetric attention layer is used to apply first-level attention calculation to the temporal features in order to focus on and extract the manipulation behavior features with corresponding state change relationships formed within a preset time window;

[0019] The far-symmetric attention layer is used to apply a second-level attention calculation to the features processed by the near-symmetric attention layer in order to associate and extract the control behavior features with corresponding state change relationships formed across time periods or multiple control steps.

[0020] A prediction output layer is used to predict the user's subsequent control intentions based on the features output by the far-symmetric attention layer.

[0021] S4: After preprocessing the real-time acquired user behavior sequence in step S2, input it into the trained time-series prediction model to obtain the user intent prediction result;

[0022] S5: Generate corresponding smart home device control commands based on the user intent prediction results.

[0023] In a further optimized manner, the sequence coding layer adopts a gated recurrent unit network structure, where the input of each GRU unit includes the input features at the current time and the hidden state passed by the previous GRU unit.

[0024] Furthermore, the environmental sensor detection data includes indoor environmental data and outdoor environmental data. The indoor environmental data is collected by temperature sensors, humidity sensors, and light intensity sensors installed in the home, while the outdoor environmental data is obtained through communication with a meteorological service platform via a network interface.

[0025] Further optimization involves converting the time interval feature by calculating the difference between the timestamps of two adjacent manipulation records in the sequence to obtain a time interval value in minutes.

[0026] Further optimized, the normalization process employs a minimum-maximum normalization method to linearly transform the original data to the [0,1] interval. The transformation formula is as follows:

[0027] Where x is the original data, VAL_min represents the minimum value of the same type of data, VAL_max represents the maximum value of the same type of data, and y is the normalized result.

[0028] In a further optimized configuration, the near-symmetric attention layer is configured to capture paired manipulation behaviors with a time interval within a preset threshold, and the far-symmetric attention layer is configured to capture paired manipulation behaviors with a time interval exceeding the preset threshold or interspersed with other manipulation behaviors.

[0029] In a further optimized manner, the prediction output layer adopts a fully connected neural network structure to map the features output by the far-symmetric attention layer into a probability distribution of user control over at least one smart home device.

[0030] In a further optimized manner, the generation of the control command includes: selecting control intentions with probability values ​​exceeding a preset threshold based on the predicted control probability distribution, generating corresponding device control commands, and sending them to the corresponding smart home devices via a message publish-subscribe communication protocol.

[0031] Further optimized, and sent to the corresponding smart home devices via the MQTT protocol.

[0032] A smart home user intent prediction system based on multimodal behavioral symmetric features, used to implement the method according to any one of claims 1-8, the system comprising:

[0033] The data acquisition module is used to collect operating status data, environmental sensor data, and user control data of smart home devices;

[0034] A data preprocessing module, connected to the data acquisition module, is used to preprocess the acquired data, including time interval calculation, encoding, and normalization.

[0035] The predictive model module, deployed on a cloud platform, includes:

[0036] The sequence coding unit adopts a GRU network structure; the hierarchical attention unit includes a near-symmetric attention computation unit and a far-symmetric attention computation unit; the prediction output unit adopts a fully connected network structure.

[0037] The control execution module, connected to the prediction model module, is used to generate and issue device control commands based on the prediction results.

[0038] In a further optimized version, the data acquisition module is implemented through a multi-protocol smart gateway, which simultaneously supports Wi-Fi, ZigBee, and 433MHz communication protocols for unified access to different types of smart home devices and sensors.

[0039] In summary, due to the adoption of the above technical solution, the beneficial effects of the present invention are:

[0040] 1. This invention achieves clear and attributable technical effects through the synergy of a "unified data preprocessing workflow" and a "layered behavioral symmetry modeling mechanism." By encoding and normalizing device identifiers, operating states, and timestamps, and converting absolute timestamps into relative time intervals, the differences in dimensions and numerical ranges of multimodal data are reduced. This constructs unified and standardized behavioral sequence features, improving the fusionability of multimodal behavioral data within the same time-series model and laying a data foundation for the model to integrate environmental and control information. Secondly, by constructing a time-series prediction model containing near-symmetric and far-symmetric attention layers, it achieves the distinguishable modeling of paired control behaviors formed within a short period and periodic control behaviors formed across time periods. This allows the model to focus on control behavior features with corresponding state change relationships at different time scales. This enables the model to more comprehensively and precisely characterize user control habits, thereby improving the accuracy and adaptability of user intent prediction in complex application scenarios and alleviating the prediction bias problems caused by the single model structure and insufficient utilization of multimodal data in existing methods.

[0041] 2. By employing a GRU network as the sequence encoding layer and explicitly incorporating indoor and outdoor environmental data, the model's ability to extract features of temporal dependencies in user behavior sequences is enhanced. At the same time, environmental state information is used as a constraint in the user intent prediction process, thereby ensuring that environmental context information is systematically transformed into quantitative features that the model can process. This improves the completeness of multimodal data fusion modeling and the stability of prediction results under changing environmental conditions.

[0042] 3. By specifying the time interval between manipulation behaviors in "minutes" and using min-max normalization to process multi-dimensional behavioral features, the time and state features of user behavior sequences have a unified data scale. At the same time, different time attention ranges are set for near-symmetric attention layers and far-symmetric attention layers, which helps the model distinguish behavioral symmetry patterns formed at different time scales during the modeling process. This reduces the uncertainty in the construction of behavioral symmetry features and improves the stability of model training and prediction.

[0043] 4. The prediction output layer generates a probability distribution of control operations for smart home devices and, combined with preset thresholds, generates control commands. These commands are then sent to the corresponding devices via a message communication protocol. This enables the smart home system to make probability-driven automatic control decisions based on the prediction results, thereby reducing the risk of false triggering due to prediction uncertainty and improving the reliability of the automatic control process. Furthermore, by constructing a system architecture that includes a multi-protocol smart gateway, the aforementioned user intent prediction-based method can achieve unified access and collaborative control across different types of devices, thus transforming the method-level prediction effect into a system-level technical effect that can operate in actual smart home scenarios. Attached Figure Description

[0044] Figure 1 A schematic diagram of the overall architecture of a smart home user intent prediction system according to an embodiment of the present invention;

[0045] Figure 2 A core flowchart of an embodiment of the intent prediction method of the present invention;

[0046] Figure 3 A schematic diagram of the data processing flow of a multi-protocol smart gateway;

[0047] Figure 4 This is a network topology diagram of a smart home device and sensor according to an embodiment of the present invention;

[0048] Figure 5 Schematic diagram of GRU unit structure. Detailed Implementation

[0049] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0050] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this invention pertains. The terminology used herein in the description of the invention is for the purpose of describing particular embodiments only and is not intended to be limiting of the invention. The term "and / or" as used herein includes any and all combinations of one or more of the associated listed items.

[0051] Example 1

[0052] This embodiment uses a smart home system deployed in a three-bedroom, one-living-room house of approximately 120 square meters as an example to fully illustrate the specific implementation of the method and system of the present invention.

[0053] I. System Hardware and Data Acquisition Deployment

[0054] like Figure 1 As shown, the system architecture of this embodiment includes, from bottom to top, a terminal perception layer, a network layer, and a cloud platform layer.

[0055] Terminal sensing layer: Deployed includes 1 ZigBee coordinator node, 8 ZigBee terminal nodes (including 2 sensors each for temperature and humidity, smoke, light, and harmful gases), and approximately 10 smart home devices such as curtain motors, air conditioners, range hoods, and smart lights. The network topology adopts a star topology, as shown in the specific structure below. Figure 3 As shown.

[0056] Network Layer: The smart gateway at the network layer uses a high-performance core board based on the ARM Cortex-A9 architecture as its processing unit. This board integrates a Wi-Fi network card, a ZigBee coordinator module, and a 433MHz low-frequency wireless module, and connects to the home router via an RJ45 interface, achieving multi-protocol converged access and internet connectivity. For details on its internal thread management process responsible for wireless communication, please refer to [link / reference needed]. Figure 2 .

[0057] Cloud platform layer: An MQTT message broker server (such as EMQX) and a MySQL database are deployed. The cloud platform is responsible for receiving and storing all terminal data and running business logic and predictive models.

[0058] During system operation, ZigBee terminal nodes collect indoor environmental data (temperature, humidity, light intensity, smoke concentration, etc.) at a fixed interval (every 5 minutes in this embodiment), and transmit it to the coordinator via the ZigBee network, and then to the smart gateway via serial port. Changes in the operating status (on / off, mode, speed, etc.) of smart home appliances are reported to the gateway in real time via Wi-Fi or 433MHz protocol. Simultaneously, the system uses public APIs from meteorological service platforms such as China Weather Network to acquire outdoor temperature, humidity, and weather conditions at the same frequency. After all data is aggregated by the gateway, it is uploaded to the cloud platform database in real time via Wi-Fi and the home router using the MQTT protocol, forming the original historical dataset. Each data record includes a timestamp, device / sensor identifier, and its status value.

[0059] II. Implementation of a Smart Home User Intent Prediction Method Based on Symmetrical Features of Multimodal Behavior

[0060] The core of the software method in this embodiment, namely the AI ​​prediction module, is implemented based on the TensorFlow deep learning framework. For example... Figure 2 , Figure 3 As shown.

[0061] Step S1: Construct a multimodal training dataset.

[0062] Historical data for a specific user over 30 consecutive days was extracted from the cloud platform database and organized by day. Each day's data sequence includes all device operation records for that user and their corresponding environmental snapshot data. Each training sample data point is composed of three fused parts:

[0063] Device operation logs: Control commands actively issued by the user, such as "turn on the living room lights".

[0064] Synchronized environmental status: When this operation occurs, data such as temperature, humidity, and light intensity are collected by indoor and outdoor sensors.

[0065] Timestamp: Records the precise time when the manipulation occurred.

[0066] Therefore, a set of n-day historical control records for user u, Su = {Su1, Su2, ..., Sun}, is constructed. The control sequence for the i-th day is represented as Sui = {X1, X2, ..., Xm}, and the j-th record in the sequence is defined as the triple Xj = {Dj, Cj, Tj}. Here, Dj is the identifier of the controlled device (e.g., "master bedroom light"), Cj is the control status (e.g., "on"), and Tj is the control timestamp.

[0067] Step S2: Data preprocessing and feature engineering.

[0068] This step aims to transform raw, multimodal, heterogeneous data into standardized, structured input that the model can process.

[0069] Time feature quantification: The absolute timestamp Tj is converted into a feature reflecting the interval between actions. The time interval between adjacent operations is calculated as ΔTj = Tj - T_{j-1} (in minutes). For the first record of each day, ΔT1 is set to 0. This conversion is crucial for identifying behavioral symmetry, especially near-symmetric behavior.

[0070] Categorical feature encoding: Discrete features such as device identifier Dj and control status Cj are converted into unique integer IDs using label encoding. For example, "living room light" is encoded as 1, "air conditioner" is encoded as 2; "on" is encoded as 1, and "off" is encoded as 0.

[0071] Multimodal data normalization: Due to the significant differences in the dimensions and numerical ranges of device IDs, status IDs, time intervals, and various environmental data (such as temperature values), normalization is necessary. This embodiment employs a minimum-maximum normalization method to linearly scale various features to the [0,1] interval. For feature x, the formula for calculating its normalized value y is:

[0072] Among them, VAL min and VAL max These are the minimum and maximum values ​​of the feature across all values ​​in the training set over the entire time period (30 days in this example).

[0073] Constructing the training sample sequence: The long sequence is divided into fixed-length samples for model input using a sliding time window method. Set the time window length, such as L=15 (i.e., each sample contains 15 consecutive records), and the sliding step size S=1. The L normalized feature vectors within each window constitute the input S_input, and the next real manipulation action immediately following this window is used as the predicted label S_label.

[0074] Step S3: Train a time-series prediction model based on a hierarchical attention mechanism.

[0075] The specific architecture of the model, and the key details of its construction and training are as follows:

[0076] Model input and GRU encoding layer:

[0077] The input layer receives tensors of shape (batch size, L, feature dimension).

[0078] The sequence coding layer uses a single-layer bidirectional GRU network with 128 hidden units. Figure 5 As shown, each GRU unit receives the input features at the current time step and the hidden state at the previous time step, and outputs the hidden state at the current time step. This layer serves as the foundational temporal dependency pattern for the user behavior sequence. During training, the dropout rate of the GRU layer is set to 0.2 to prevent overfitting.

[0079] Near-symmetric attention layer: This layer computes all hidden states output by the GRU layer. Its core function is to automatically identify and focus on paired manipulation behaviors with short time intervals. In real life, after a user turns on the bathroom light, barring external factors, they will definitely turn it off at some point in the future. For example, the pair of actions "turning on the bathroom light" at 7:00 AM and "turning off the bathroom light" at 7:05 AM, with only a 5-minute interval, will be identified as near-symmetric behavior by this layer and given a high association weight when calculating the final representation.

[0080] This layer is designed to be sensitive to time interval features, and its attention mechanism tends to assign higher weights to associated operations where ΔT is less than a preset threshold (set to 30 minutes in this embodiment).

[0081] The far-symmetric attention layer takes the weighted context vector from the previous layer as input and its core function is to capture periodic, recurring behaviors across long time intervals or numerous intermediate operations. For example, a user might "turn on the kitchen light" at 6:00 PM, then "turn off the kitchen light" at 8:00 PM after a series of actions such as cooking and eating. This behavior occurs at intervals of up to two hours, and the far-symmetric attention layer can effectively establish a correlation between these behaviors, thereby capturing such far-symmetric behavioral patterns.

[0082] The layer is designed to capture operation pairs with logical closed-loop characteristics whose time intervals exceed the aforementioned 30-minute threshold.

[0083] Predicting the output layer and model training: The output layer is a fully connected neural network with the number of neurons equal to the total number of possible manipulation actions in the system, K (K is approximately 20 in this example). This layer maps the comprehensive representation of the output of the far-symmetric attention layer to a K-dimensional probability distribution vector P=(p1,p2,...,pK), where pi represents the probability that the model predicts the user will perform the i-th action next.

[0084] Training configuration: The Adam optimizer was used with an initial learning rate of 0.001; the loss function was classification cross-entropy; and the batch size was set to 32. Training was performed on a server equipped with an NVIDIA GTX 1080 Ti graphics card (11GB VRAM) for 100 epochs until the loss function converged.

[0085] Step S4: Real-time intent prediction. After the model is trained, it is integrated into the prediction service of the cloud platform.

[0086] Real-time prediction: The system monitors user behavior in real time. After accumulating the latest consecutive L(15) records, it automatically calls the preprocessing process (S2) that is completely consistent with the training phase to generate a standardized input sequence and input it into the loaded model for inference.

[0087] The model outputs a predicted probability distribution P.

[0088] S5: Generate corresponding smart home device control commands based on the user intent prediction results.

[0089] Based on the predicted probability distribution P output by the model, the system or a user sets an execution threshold (e.g., 0.75). If the predicted probability of a certain action exceeds this threshold, it is considered that the user has a strong intention to respond. The system then generates standard device control commands (e.g., {"device": "living_room_ac", "command": "set_temperature", "value": 26}) and publishes them to the corresponding device control topic via the MQTT protocol. The corresponding home devices that have subscribed to this topic receive and execute the commands, thereby achieving proactive and personalized smart home automation.

[0090] Example 2

[0091] The above scheme was rigorously evaluated in the test environment built in Example 1:

[0092] Prediction accuracy: Using a reserved test dataset for evaluation, the top-1 intent prediction accuracy of our method (GRU + two-layer attention) reached 86.5%. In comparison, under the same data and training conditions, the accuracy of the baseline model using only a single GRU layer (without hierarchical attention) was 78.1%. The accuracy of the model in this invention is improved by approximately 8.4 percentage points, effectively validating the effectiveness of hierarchical modeling of behavioral symmetric features.

[0093] System real-time performance: The average end-to-end response latency from sensor data acquisition, uploading via gateway, cloud model inference, to command transmission back to the device is 280ms. Specifically, the critical network transmission components: Wi-Fi uplink / downlink latency ≤ 50ms, and ZigBee network internal transmission latency ≤ 100ms, fully meeting the performance requirements for real-time sensing and control in smart home scenarios.

[0094] System reliability: During a 7-day continuous stability test, the system achieved a 99.2% success rate in acquiring environmental data and a 99.8% predicted service availability, demonstrating the stability and robustness of the solution in actual deployment.

[0095] Example 3

[0096] This embodiment also provides a system platform for implementing Embodiment 1, specifically a smart home user intent prediction system based on multimodal behavioral symmetry features, such as... Figure 1 As shown, the system mainly includes the following four functional modules:

[0097] Data Acquisition Module: Responsible for acquiring multimodal raw data from the physical environment in real time. This module consists of various smart home terminal devices deployed indoors (such as smart lights and air conditioners), environmental sensors (such as temperature and humidity sensors), and user interaction terminals (such as mobile apps and smart speakers). These terminals continuously report data such as device operating status, environmental readings, and direct user control commands via a wireless network.

[0098] Data preprocessing module: As the core link connecting data acquisition and intelligent analysis, it is typically deployed at the home edge gateway or cloud platform entry point. This module receives the raw data stream and organizes and stores it by day and by user. Its core function is to execute the standardized preprocessing flow detailed in Example 1 (corresponding to step S2), specifically including: calculating the time interval between adjacent operations, tagging and encoding devices and states, and performing min-max normalization on all features. The processed standardized sequence data is stored in a database or directly pushed to downstream modules.

[0099] Prediction Model Module: Serving as the system's "intelligent brain," this module is deployed on a cloud server. It loads a pre-trained time-series prediction model, and its architecture is constructed as in Example 1 (corresponding to step S3), including:

[0100] Sequence coding unit: Composed of a GRU neural network, responsible for extracting basic temporal features from the input sequence.

[0101] Hierarchical attention unit: It includes a near-symmetric attention computation unit and a far-symmetric attention computation unit. The two are connected in series and work together to extract near-symmetric behavior patterns within a short time window and far-symmetric behavior patterns over a long span from temporal features.

[0102] Predictive output unit: Employs a fully connected neural network to map the comprehensive features output by the attention unit into a probability distribution of future manipulation actions.

[0103] Control and Execution Module: Deployed within a cloud platform or home gateway, this module is responsible for decision-making and execution. It receives the probability distribution output by the prediction model module and makes judgments based on a preset probability threshold (e.g., 0.75). For predicted intentions exceeding the threshold, this module automatically generates corresponding device control commands and reliably sends them to the relevant smart home devices via message protocols such as MQTT, thus completing a closed loop from "intention prediction" to "automatic execution."

[0104] In a further optimized embodiment, the core entity of the data acquisition module in this system is a multi-protocol smart gateway. This gateway, based on an ARM Cortex-A9 or higher-performance processor, integrates wireless modules supporting Wi-Fi, ZigBee, and 433MHz communication protocols through hardware expansion. Its network topology is as follows: Figure 4 As shown.

[0105] For Wi-Fi devices (such as smart TVs and refrigerators), the gateway acts as a wireless access point (AP) or client, directly establishing a connection with them to achieve high-speed data and control command transmission.

[0106] For ZigBee devices and sensors (such as temperature and humidity sensors), the gateway uses its ZigBee coordinator to build a low-power self-organizing network to stably collect environmental data at a cycle of once every 5 minutes.

[0107] For devices that use the 433MHz protocol (such as some older or specific brand remote control devices), the gateway uses its 433MHz module to achieve long-distance, wall-penetrating signal reception and control.

[0108] This multi-protocol smart gateway serves as a unified hardware entry point, resolving the interoperability issues between smart devices from different brands and using different protocols. It performs initial aggregation and protocol conversion of all heterogeneous raw data, then uploads the unified format data stream to subsequent modules on the cloud platform for processing via a home router. The software flow responsible for wireless communication management within the gateway can be found in [link to relevant documentation]. Figure 2 .

[0109] The above description is a detailed description of the preferred embodiments of the present invention. However, the embodiments are not intended to limit the scope of the patent application of the present invention. All equivalent changes or modifications made under the technical spirit of the present invention should fall within the patent scope covered by the present invention.

Claims

1. A method for predicting user intent in smart homes based on multimodal behavioral symmetric features, characterized in that, include: S1: Acquire multimodal user behavior data, which includes at least: operating status data of home appliances; sensor detection data related to the environment; and control command data from the server; wherein each data record includes the identification of the control device, the control status, and the control timestamp; S2: Preprocess the multimodal user behavior data, the preprocessing including: The control timestamp is converted into a feature representing the time interval between adjacent control actions; the control device identifier and control status are encoded. The time interval, device code, and status code are normalized to eliminate the dimensional differences between different modal data. The preprocessed sequence data is divided into multiple consecutive subsequences using a sliding time window method, which serve as training samples for the model. S3: Input the preprocessed training samples into the time series prediction model for training. The time series prediction model includes, in sequence: Sequence coding layers are used to extract temporal features from input sequence data; A near-symmetric attention layer is used to apply first-level attention calculation to the temporal features in order to focus on and extract the manipulation behavior features with corresponding state change relationships formed within a preset time window; The far-symmetric attention layer is used to apply a second-level attention calculation to the features processed by the near-symmetric attention layer in order to associate and extract the control behavior features with corresponding state change relationships formed across time periods or multiple control steps. A prediction output layer is used to predict the user's subsequent control intentions based on the features output by the far-symmetric attention layer. S4: After preprocessing the real-time acquired user behavior sequence in step S2, input it into the trained time-series prediction model to obtain the user intent prediction result; S5: Generate corresponding smart home device control commands based on the user intent prediction results.

2. The method according to claim 1, characterized in that, The sequence coding layer adopts a gated recurrent unit network structure. The input of each GRU unit includes the input features at the current time and the hidden state passed by the previous GRU unit.

3. The method according to claim 1, characterized in that, The environmental sensor data includes indoor environmental data and outdoor environmental data. The indoor environmental data is collected by temperature sensors, humidity sensors, and light intensity sensors installed in the home, while the outdoor environmental data is obtained through network communication with a meteorological service platform.

4. The method according to claim 1, characterized in that, The conversion of the time interval feature specifically involves calculating the difference between the timestamps of two adjacent manipulation records in the sequence to obtain the time interval value in minutes.

5. The method according to claim 1, characterized in that, The normalization process employs a minimum-maximum normalization method to linearly transform the original data to the [0,1] interval. The transformation formula is as follows: Where x is the original data, VAL_min represents the minimum value of the same type of data, VAL_max represents the maximum value of the same type of data, and y is the normalized result.

6. The method according to claim 1, characterized in that, The near-symmetric attention layer is configured to capture paired manipulation behaviors with a time interval within a preset threshold, and the far-symmetric attention layer is configured to capture paired manipulation behaviors with a time interval exceeding the preset threshold or interspersed with other manipulation behaviors.

7. The method according to claim 1, characterized in that, The prediction output layer adopts a fully connected neural network structure to map the features output by the far-symmetric attention layer to the probability distribution of user control over at least one smart home device.

8. The method according to claim 1, characterized in that, The generation of the control command includes: selecting control intentions with probability values ​​exceeding a preset threshold based on the predicted control probability distribution, generating corresponding device control commands, and sending them to the corresponding smart home devices through a message publish-subscribe communication protocol.

9. A smart home user intent prediction system based on multimodal behavioral symmetry features, characterized in that, The system for implementing the method according to any one of claims 1-8 comprises: The data acquisition module is used to collect operating status data, environmental sensor data, and user control data of smart home devices; A data preprocessing module, connected to the data acquisition module, is used to preprocess the acquired data, including time interval calculation, encoding, and normalization. The predictive model module, deployed on a cloud platform, includes: The sequence coding unit adopts a GRU network structure; the hierarchical attention unit includes a near-symmetric attention computation unit and a far-symmetric attention computation unit; the prediction output unit adopts a fully connected network structure. The control execution module, connected to the prediction model module, is used to generate and issue device control commands based on the prediction results.

10. The system according to claim 9, characterized in that, The data acquisition module is implemented through a multi-protocol smart gateway, which simultaneously supports Wi-Fi, ZigBee and 433MHz communication protocols, and is used to uniformly access different types of smart home devices and sensors.

Images