Smart home device control method, apparatus, and smart home control system

Abstract

The application provides a control method and device of a smart home device and a smart home control system. The method comprises: obtaining multiple multi-dimensional data related to the operation of the smart home device, wherein the multiple multi-dimensional data comprises device data, user characteristic data and user instruction data, the device data comprises the operating state of the smart home device and the environmental parameters of the environment where the smart home device is located, the user characteristic data comprises the data of pre-configured user attributes and the data of user behavior preferences, and the user instruction data comprises voice instructions and / or application program instructions; performing machine learning according to the multiple multi-dimensional data, identifying the relationship between the multiple multi-dimensional data, so as to identify the user intention; and controlling the operation of the smart home device according to the user intention. The scheme solves the problem that the prior art is difficult to meet the personalized needs of users for smart home.

Description

Technical Field

[0001] This application relates to the field of smart home control technology, and more specifically, to a control method for smart home devices, a control device for smart home devices, a computer program product, and a smart home control system. Background Technology

[0002] With the development of IoT and AI technologies, smart home systems have gradually become widespread. In smart home scenarios, the generation and execution of device control strategies are key to achieving home automation and improving user comfort and convenience.

[0003] Currently, solutions rely on preset, fixed rules (such as rigid "If-Then" logic) or simple user behavior statistics. These solutions collect user commands and match them according to pre-defined conditions to generate fixed device control strategies. For example, when the light intensity is detected to be below a certain threshold, the lights are automatically turned on; or when the user says "turn on the air conditioner," a standard startup procedure is executed. This approach cannot accurately capture the underlying real needs behind user commands, has poor understanding of user intent, and struggles to meet users' personalized needs for smart homes. Summary of the Invention

[0004] The main objective of this application is to provide a control method, control device, computer program product, and control system for smart home devices, so as to at least solve the problem that the prior art is unable to meet users' personalized needs for smart homes.

[0005] To achieve the above objectives, according to one aspect of this application, a control method for a smart home device is provided, comprising: acquiring data from multiple dimensions related to the operation of the smart home device to obtain multiple multidimensional data, wherein the multiple multidimensional data includes device data, user feature data, and user instruction data, wherein the device data includes the operating status of the smart home device and environmental parameters of the environment in which the smart home device is located, the user feature data includes pre-configured user attribute data and user behavior preference data, and the user instruction data includes voice commands and / or application commands; performing machine learning based on the multiple multidimensional data to identify the relationships between the multiple multidimensional data, so as to identify user intentions; and controlling the operation of the smart home device according to the user intentions.

[0006] Optionally, machine learning is performed on multiple sets of multidimensional data to identify relationships between them, thereby enabling the identification of user intent. This includes: using a contrastive learning framework to perform machine learning on historical multidimensional data to construct multiple learning samples; and clustering based on the relationships between the multiple learning samples to obtain multiple scenarios, wherein the scenarios and historical user intents are correlated. Figure 1 One-to-one correspondence; the scenario with the highest similarity to the multiple multidimensional data is determined as the target scenario, and the historical user intent corresponding to the target scenario is determined as the user intent corresponding to the multiple multidimensional data.

[0007] Optionally, a contrastive learning framework is used to perform machine learning based on historical multidimensional data to construct multiple learning samples, including: obtaining a contrastive learning framework, wherein the contrastive learning framework is the SimCLR framework; constructing a feature extraction model using the encoder of the contrastive learning framework, wherein the feature extraction model is a CNN-LSTM model; adding a lightweight multi-scale extraction module after the convolutional layer and before the sequence layer of the feature extraction model, such that the output of the convolutional layer is used as the input of the added lightweight multi-scale extraction module, and the output of the added lightweight multi-scale extraction module is used as the input of the sequence layer, in the concatenation of the feature extraction model. A guided centroid regression module is added after the concatenation layer and before the loss function layer of the feature extraction model. The output of the concatenation layer serves as the input to the guided centroid regression module, and the output of the guided centroid regression module serves as the input to the loss function layer, resulting in an improved contrastive learning framework. The historical multidimensional data and corresponding feature labels are combined to form a training set. The improved contrastive learning framework is trained using this training set to obtain a sample extraction framework. The feature labels are the historical sample features of the historical multidimensional data in the training set. The historical multidimensional data is then input into the sample extraction framework to obtain the learning samples corresponding to the historical multidimensional data.

[0008] Optionally, clustering is performed based on the association between multiple learning samples to obtain multiple scenarios, including: calculating the similarity between multiple learning samples using the InfoNCE loss function; and using a clustering algorithm to cluster based on the similarity between the learning samples to obtain multiple sample clusters, wherein the similarity between the learning samples in each sample cluster is not zero, and the sample clusters correspond one-to-one with the scenarios.

[0009] Optionally, controlling the operation of the smart home device according to the user's intent includes: when the similarity between the multidimensional data and the scene is greater than or equal to a first similarity threshold, determining the scene as the target scene and executing the control steps corresponding to the target scene, wherein the target scene corresponds to the user's intent, and the control steps are pre-configured parameters for controlling the operation of the smart home device; when the similarity between the multidimensional data and the scene is less than the first similarity threshold and the similarity between the multidimensional data and the scene is greater than or equal to a second similarity threshold, determining the scene as the target scene, adjusting the parameters of the smart home device operation according to a first preset step size until they are the same as the parameters of the smart home device operation corresponding to the control steps, wherein the first similarity threshold is greater than the second similarity threshold; when switching from one scene to another, adjusting the parameters of the smart home device operation corresponding to one scene to the parameters of the smart home device operation corresponding to the other scene according to a second preset step size.

[0010] Optionally, controlling the operation of the smart home device according to the user's intent includes: generating initial personalized control steps using a Q-learning reinforcement learning algorithm when the similarity between the multidimensional data and all the scenarios is less than a third similarity threshold; adjusting the initial personalized control steps based on the modification request upon receiving a modification request to obtain personalized control steps, wherein the modification request is a request from the user to modify the parameters of the smart home device's operation; using the personalized control steps as the new scenario, and controlling the operation of the smart home device using the personalized control steps.

[0011] Optionally, performing machine learning based on multiple sets of multidimensional data to identify relationships between the multiple sets of multidimensional data in order to identify user intent includes: preprocessing the multidimensional data to obtain preprocessed multidimensional data, wherein the preprocessing method includes one or more of outlier removal, missing value imputation, data normalization, and timestamp alignment; and performing machine learning based on the preprocessed multidimensional data to identify relationships between the preprocessed multidimensional data in order to identify user intent.

[0012] According to another aspect of this application, a control device for a smart home device is provided, comprising: an acquisition unit, configured to acquire data related to the operation of the smart home device in multiple dimensions, obtaining multiple multidimensional data, wherein the multiple multidimensional data includes device data, user feature data, and user instruction data, wherein the device data includes the operating status of the smart home device and environmental parameters of the environment in which the smart home device is located, the user feature data includes pre-configured user attribute data and user behavior preference data, and the user instruction data includes voice commands and / or application commands; an identification unit, configured to perform machine learning based on the multiple multidimensional data to identify the relationship between the multiple multidimensional data, so as to identify user intentions; and a control unit, configured to control the operation of the smart home device according to the user intentions.

[0013] According to another aspect of this application, a computer program product is provided, comprising a computer program that, when executed by a processor, implements the steps of any of the control methods for the smart home device.

[0014] According to another aspect of this application, a smart home control system is provided, comprising: one or more processors, a memory, and one or more programs, wherein the one or more programs are stored in the memory and configured to be executed by the one or more processors, the one or more programs including a control method for performing any of the smart home devices described above.

[0015] By applying the technical solution of this application, device data (reflecting the environment and device status), user characteristic data (reflecting the user's static attributes and dynamic preferences), and user command data (reflecting the user's explicit needs) are acquired. By introducing user characteristic data (including pre-configured user attributes and user behavior preferences), individual differences among different users can be distinguished. By introducing device data, the correlation between user commands, user characteristics, environment, and device data can be analyzed through machine learning algorithms in conjunction with the real-time environmental background. This machine learning process based on relationship recognition can capture the implicit logic and fine-grained features behind the data, thereby accurately inferring the user's true intention from multi-dimensional data, rather than simply responding to literal commands. This allows for a more accurate understanding of the user's personalized needs in specific scenarios. (For different user characteristics or different environmental parameters, the identified intention may correspond to different temperature settings or mode selections.) Furthermore, based on this accurately identified user intention, corresponding controls are generated and executed, realizing the response to the user's personalized needs. Attached Figure Description

[0016] The accompanying drawings, which form part of this application, are used to provide a further understanding of this application. The illustrative embodiments and descriptions of this application are used to explain this application and do not constitute an undue limitation of this application. In the drawings:

[0017] Figure 1 A hardware structure block diagram of a mobile terminal for implementing a control method for a smart home device, according to an embodiment of this application, is shown.

[0018] Figure 2 A flowchart illustrating a control method for a smart home device according to an embodiment of this application is shown.

[0019] Figure 3 A flowchart illustrating a specific method for controlling a smart home device is shown.

[0020] Figure 4 A structural block diagram of a control device for a smart home device according to an embodiment of this application is shown.

[0021] The above figures include the following reference numerals:

[0022] 102. Processor; 104. Memory; 106. Transmission device; 108. Input / output device. Detailed Implementation

[0023] It should be noted that, unless otherwise specified, the embodiments and features described in this application can be combined with each other. This application will now be described in detail with reference to the accompanying drawings and embodiments.

[0024] To enable those skilled in the art to better understand the present application, the technical solutions in the embodiments of the present application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present application, and not all embodiments. Based on the embodiments in the present application, all other embodiments obtained by those of ordinary skill in the art without creative effort should fall within the scope of protection of the present application.

[0025] It should be noted that the terms "first," "second," etc., in the specification, claims, and accompanying drawings of this application are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that such data can be interchanged where appropriate for the embodiments of this application described herein. Furthermore, the terms "comprising" and "having," and any variations thereof, are intended to cover non-exclusive inclusion; for example, a process, method, system, product, or apparatus that comprises a series of steps or units is not necessarily limited to those steps or units explicitly listed, but may include other steps or units not explicitly listed or inherent to such processes, methods, products, or apparatus.

[0026] Current solutions generate fixed device control strategies by pre-setting fixed rules (such as rigid "If-Then" logic) or simple user behavior statistics, combined with collected device operation data and user commands. Some solutions introduce basic machine learning algorithms for simple scenario matching to achieve automated device control. However, existing solutions mostly rely on manually pre-setting rules or single-dimensional data statistics, without in-depth mining of multi-dimensional data, and the control strategies are difficult to dynamically adjust according to scenarios and user needs after generation.

[0027] The existing solutions have the following problems: inaccurate understanding of users' true intentions, poor targeting of control strategies, inability to accurately match users' personalized needs, easy to misjudgment of control, and impact on user experience; rigid control strategies, unable to adapt to changes in smart home scenarios (such as sudden changes in environmental parameters or changes in user behavior habits), slow control response speed, and high complexity of manual settings for users.

[0028] Existing solutions rely on simple environmental data collection or single-dimensional user status recognition to determine user needs. Some solutions only use scene classifiers for scene matching, failing to integrate user characteristic data with deep correlations between user commands and user features. Because they lack multi-dimensional fusion analysis of user commands, device data, and user feature data, and fail to uncover fine-grained relationships between different data samples, they cannot capture the underlying needs behind user commands. This leads to discrepancies between control strategies and the user's true intentions, resulting in issues such as false triggers and unreasonable control parameters. Therefore, they struggle to meet the personalized and precise device control needs of smart home users. Existing solutions generate device control strategies through preset fixed scene modes or static rule bases. While some solutions allow for sub-model fine-tuning, they lack dynamic iterative optimization mechanisms and heavily rely on cloud-based decision processing. The lack of real-time adaptive adjustment capabilities in these control strategies prevents rapid responses to dynamic changes in home environmental parameters (temperature, humidity, light, etc.) and user behavior. Furthermore, cloud processing suffers from data transmission delays, and rule base updates require manual intervention, resulting in rigid and sluggish control strategies. Users must manually adjust rules or parameters to adapt to new scenarios. Therefore, it is difficult to meet the dynamic changes in smart home scenarios and users' needs for seamless intelligent control.

[0029] As described in the background section, existing technologies are insufficient to meet users' personalized needs for smart homes. To address these issues, embodiments of this application provide a control method for smart home devices, a control device for smart home devices, a computer program product, and a smart home control system.

[0030] The technical solutions of the present invention will be clearly and completely described below with reference to the accompanying drawings in the embodiments of the present invention.

[0031] The methods and embodiments provided in this application can be executed on a mobile terminal, computer terminal, or similar computing device. Taking running on a mobile terminal as an example, Figure 1 This is a hardware structure block diagram of a mobile terminal for a control method of a smart home device according to an embodiment of the present invention. Figure 1 As shown, a mobile terminal may include one or more ( Figure 1 Only one is shown in the diagram. A processor 102 (which may include, but is not limited to, a microprocessor MCU or a programmable logic device FPGA, etc.) and a memory 104 for storing data are also shown. The mobile terminal may further include a transmission device 106 for communication functions and an input / output device 108. Those skilled in the art will understand that... Figure 1 The structure shown is for illustrative purposes only and does not limit the structure of the mobile terminal described above. For example, the mobile terminal may also include components that are more... Figure 1 The more or fewer components shown, or having the same Figure 1The different configurations shown.

[0032] The memory 104 can be used to store computer programs, such as application software programs and modules, like the computer program corresponding to the control method of the smart home device in this embodiment of the invention. The processor 102 executes various functional applications and data processing by running the computer program stored in the memory 104, thereby implementing the above-described method. The memory 104 may include high-speed random access memory and may also include non-volatile memory, such as one or more magnetic storage devices, flash memory, or other non-volatile solid-state memory. In some instances, the memory 104 may further include memory remotely located relative to the processor 102, and these remote memories can be connected to the mobile terminal via a network. Examples of the aforementioned networks include, but are not limited to, the Internet, corporate intranets, local area networks, mobile communication networks, and combinations thereof. The transmission device 106 is used to receive or send data via a network. Specific examples of the aforementioned networks may include wireless networks provided by the mobile terminal's communication provider. In one example, the transmission device 106 includes a network interface controller (NIC), which can be connected to other network devices via a base station to communicate with the Internet. In one example, the transmission device 106 may be a radio frequency (RF) module, which is used to communicate with the Internet wirelessly.

[0033] This embodiment provides a control method for a smart home device that runs on a mobile terminal, computer terminal, or similar computing device. It should be noted that the steps shown in the flowchart in the accompanying drawings can be executed in a computer system such as a set of computer-executable instructions. Also, although a logical order is shown in the flowchart, in some cases, the steps shown or described may be executed in a different order than that shown here.

[0034] Figure 2 This is a flowchart illustrating a control method for a smart home device according to an embodiment of this application. Figure 2 As shown, the method includes the following steps:

[0035] Step S201: Obtain data related to the operation of smart home devices from multiple dimensions to obtain multiple multidimensional data. Among them, the multiple multidimensional data include device data, user characteristic data and user command data. The device data includes the operating status of the smart home devices and the environmental parameters of the environment in which the smart home devices are located. The user characteristic data includes pre-configured user attribute data and user behavior preference data. The user command data includes voice commands and / or application commands.

[0036] Specifically, device data can be collected via communication protocols such as ZigBee or Wi-Fi. For example, air conditioners collect temperature, fan speed, and operating status every 3 seconds; lights collect brightness, color temperature, and on / off status every 2 seconds; and curtains, air purifiers, and humidifiers collect operating parameters and environmental sensing data (such as temperature, humidity, and PM2.5 concentration) every 30 seconds. User characteristic data is collected through a combination of active input via the APP and sensor data. Static characteristics (i.e., user attribute data) (such as age, gender, physical condition, and daily habits) are pre-configured and entered by the user, while dynamic characteristics (i.e., user behavior preferences) (such as operation records, dwell areas, and preference parameters) are collected in real time through human body sensors and device operation logs. Privacy data is encrypted using AES, and only behavioral characteristics are extracted. User command data is collected through the terminal voice module. Application commands can be collected via the APP. After voice-to-text processing, voice commands are standardized into a "device + operation + parameter" format (such as "air conditioner + raise + temperature 2℃").

[0037] Step S202: Perform machine learning based on the above-mentioned multidimensional data to identify the relationship between the above-mentioned multidimensional data, so as to identify user intent;

[0038] Specifically, by using machine learning, the correlation between these multidimensional data in the feature space is analyzed to distinguish the sample distribution of different intentions, thereby accurately identifying the user's true control intentions, rather than relying solely on literal instructions.

[0039] Step S203: Control the operation of the smart home devices according to the user's intent.

[0040] Specifically, executing control strategies based on identified user intent can effectively reduce the misjudgment rate of device control and accurately match the personalized preferences of different users.

[0041] This embodiment acquires device data (reflecting the environment and device status), user characteristic data (reflecting the user's static attributes and dynamic preferences), and user command data (reflecting the user's explicit needs). By introducing user characteristic data (including pre-configured user attributes and user behavior preferences), it is possible to distinguish the individual differences of different users. By introducing device data, it is possible to combine the real-time environmental background and analyze the relationship between user commands, user characteristics, environment, and device data through machine learning algorithms. This machine learning process based on relationship recognition can capture the implicit logic and fine-grained features behind the data, thereby accurately inferring the user's true intention from multi-dimensional data, rather than simply responding to literal commands. This allows for a more accurate understanding of the user's personalized needs in specific scenarios. For different user characteristics or different environmental parameters, the identified intention may correspond to different temperature settings or mode selections. Based on this accurately identified user intention, corresponding controls are generated and executed, realizing the response to the user's personalized needs.

[0042] Specifically, the multidimensional data acquired in this solution includes three types: device data, user feature data, and user command data. This is because relying solely on commands to identify intent is unreliable—the same sentence can elicit completely different appropriate execution strategies in different environments and from different people. User commands refer to what was said (the surface expression of intent), device data refers to the environment at the time (verifying whether the command fits the current context and filtering out false triggers), and user features refer to which user said it (personalized differentiation; the same sentence can have different execution parameters for different people). The definition of a positive sample requires all three categories to simultaneously meet thresholds (command semantics ≥ 90%, device similarity ≥ 80%, user features ≥ 85%). If any category fails to meet the threshold, it becomes a negative sample. In this way, the model can learn to distinguish between situations where "the surface is similar but the intent is substantially different."

[0043] Specifically, such as Figure 3 As shown, this scheme mainly includes five steps: Step 1, multi-dimensional data collection and preprocessing. Step 2, definition of positive and negative samples. Step 3, feature vector extraction and similarity matrix calculation. Step 4, fine-grained relationship analysis. Step 5, dynamic generation and optimization of control strategies.

[0044] The first step involves multi-dimensional data collection and preprocessing. Three types of data are collected using the device's built-in sensors and external acquisition modules: Device data is collected via a ZigBee module; air conditioners collect temperature, fan speed, and operating status every 3 seconds; lights collect brightness, color temperature, and on / off status every 2 seconds; curtains, air purifiers, and humidifiers collect operating parameters and environmental perception data (temperature, humidity, PM2.5 concentration) every 30 seconds. User characteristic data is collected through a combination of active input via the app and sensor data. Static characteristics (i.e., user attribute data) (age, gender, physical condition, daily routine) are entered by the user; dynamic characteristics (i.e., user behavior preferences) (operation records, areas visited, preference parameters) are collected through human body sensors and device operation logs. Privacy data is encrypted using AES, and only behavioral characteristics are extracted. User command data is collected via the terminal's voice module. Application commands can be collected via the app, and voice commands are processed into text and standardized into a unified format of "device + operation + parameter" (e.g., "air conditioner + raise + temperature 2℃").

[0045] In the specific implementation process, machine learning is performed based on multiple sets of the aforementioned multidimensional data to identify the relationships between them, thereby enabling the recognition of user intent. This can be achieved through the following steps: A contrastive learning framework is used to perform machine learning on historical multidimensional data to construct multiple learning samples; clustering is performed based on the relationships between these learning samples to obtain multiple scenarios, where the scenarios and historical user intent are considered. Figure 1One-to-one correspondence; the scenario with the highest similarity to the above multidimensional data is identified as the target scenario, and the historical user intent corresponding to the target scenario is identified as the user intent corresponding to the above multidimensional data.

[0046] This scheme employs a contrastive learning framework for training, which can accurately distinguish the sample distribution of different user intentions in a high-dimensional space, bringing samples with similar intentions closer together and pushing samples with different intentions further apart. By clustering high-dimensional feature vectors into specific scene categories and establishing a one-to-one correspondence between each scene and historical user intentions, it achieves a mapping from data features to specific user intentions. This clustering-based scene segmentation can structure complex user behavior patterns and quickly locate potential user intentions based on the scene to which the current data belongs. By matching the similarity between the currently collected multidimensional data and typical scenes obtained from historical clustering, it can quickly determine the most likely user intention, thus ensuring a high accuracy rate in user intention recognition.

[0047] In some embodiments, a contrastive learning framework is used to perform machine learning based on historical multidimensional data to construct multiple learning samples. This can be achieved through the following steps: obtaining the contrastive learning framework, wherein the contrastive learning framework is the SimCLR framework; constructing a feature extraction model using the encoder of the contrastive learning framework, wherein the feature extraction model is a CNN-LSTM model; adding a lightweight multi-scale extraction module after the convolutional layer and before the sequence layer of the feature extraction model, such that the output of the convolutional layer serves as the input to the added lightweight multi-scale extraction module, and the output of the added lightweight multi-scale extraction module serves as the input to the sequence layer, in the aforementioned feature... A guided centroid regression module is added after the concatenation layer of the extraction model and before the loss function layer of the feature extraction model. The output of the concatenation layer serves as the input to the guided centroid regression module, and the output of the guided centroid regression module serves as the input to the loss function layer, resulting in an improved contrastive learning framework. The historical multidimensional data and corresponding feature labels are combined to form a training set. This training set is used to train the improved contrastive learning framework, resulting in a sample extraction framework. The feature labels are the historical sample features of the historical multidimensional data in the training set. The historical multidimensional data is then input into the sample extraction framework to obtain the learning samples corresponding to the historical multidimensional data.

[0048] In this scheme, by introducing a lightweight multi-scale extraction module and a guided center point regression module, the improved SimCLR framework can more effectively extract local details and multi-scale features from multi-dimensional data. The preprocessed historical multi-dimensional data is input into the trained sample extraction framework, which outputs a high-dimensional feature vector for each data sample. These feature vectors are called "learning samples". These learning samples retain the key semantics and correlation information in the original data, thereby ensuring that the accurate user intent can be learned from the samples later.

[0049] Specifically, this solution builds a contrastive learning framework based on the SimCLR architecture, introduces a region-guided centroid regression module and a lightweight multi-scale extraction module, collects smart home device data, user feature data, and user command data, and performs standardized preprocessing. Based on three types of data, positive samples (user commands, device data, and user feature similarity all meet preset thresholds) and three types of negative samples (command negative samples, environment negative samples, and user negative samples) are defined. The CNN-LSTM hybrid feature encoder extracts sample feature vectors, calculates a similarity matrix, and uses cluster analysis and association rule mining to analyze fine-grained relationships between samples and uncover core user needs and potential intentions.

[0050] This solution is the first to apply a contrastive learning framework to the generation of control strategies for smart home devices, breaking through the limitations of single-dimensional data analysis in existing technologies. Through multi-dimensional sample comparison and fine-grained relationship mining, it achieves accurate identification of user intent. Unlike existing patents that generate strategies solely through environmental data or simple user state analysis, this solution eliminates the need for manually preset intent recognition rules, enabling adaptive mining of user intent.

[0051] As stated above, the solution of this application can accurately capture the real needs behind user commands, effectively reduce the misjudgment rate of device control (the misjudgment rate is reduced by more than 80%), accurately match the personalized preferences of different users (such as different users' air conditioning temperature preferences and lighting brightness requirements), improve the accuracy of device control, improve the user experience, and solve the problem of inaccurate understanding of user intentions in the prior art.

[0052] SimCLR is the overall training framework (responsible for defining positive and negative samples, calculating the InfoNCE contrastive loss, and driving model optimization). CNN-LSTM is the feature encoder embedded within the SimCLR framework, specifically the implementation of the "basic encoder" role in SimCLR. CNN extracts local features of device parameters and command semantics, while LSTM extracts temporal features of user behavior, concatenating them to output a 128-dimensional vector. The two have a nested relationship of "framework + encoder component."

[0053] The lightweight multi-scale extraction module (located between the convolutional layer and the sequence layer) means that features are extracted in parallel using convolutional kernels of different sizes (such as 1×1, 3×3, 5×5), with different kernels corresponding to different receptive fields. The lightweight multi-scale extraction module captures both fine-grained (local details) and coarse-grained (global semantic) information at the same time, achieving multi-scale fusion with low computational cost. The lightweight multi-scale extraction module has the following hierarchy: input layer (receives CNN output) → multi-branch parallel convolutional layer (processes different kernels in parallel) → feature fusion layer (concatenation or weighted fusion) → output layer (feeds to LSTM).

[0054] The Region-Guided Centroid Regression Module (located between the concatenation layer and the loss function layer) divides the 128-dimensional feature vector into several sub-regions according to dimension or semantics, and regresses a learnable centroid vector for each sub-region. The function of the Region-Guided Centroid Regression Module is to make features of similar samples cluster towards the center of their respective regions and dissipate features of dissimilar samples, thereby enhancing feature discrimination. The Region-Guided Centroid Regression Module hierarchy is: Input layer (receives 128-dimensional vector) → Region partitioning layer (grouping) → Centroid regression layer (calculates the center of each region) → Output layer (outputs enhanced features).

[0055] The standard SimCLR structure is: random data augmentation → basic encoder (ResNet) → nonlinear projection head (MLP) → contrastive loss function. This scheme makes two structural changes: a lightweight multi-scale extraction module is inserted between the CNN and LSTM, i.e., inside the encoder. The standard SimCLR encoder is a single network; this scheme breaks it down into a CNN + multi-scale module + LSTM, representing an improvement to the encoder's internal structure. A region-guided center point regression module is inserted between the concatenation layer (encoder output) and the loss function; this position roughly corresponds to the projection head in the standard SimCLR. However, the standard projection head is simply an MLP mapping; this scheme uses region center point regression to replace the projection head's function.

[0056] In this architecture, the guided center point regression module is added after the feature encoder (CNN-LSTM) and before the contrastive loss function (InfoNCE). The hierarchical relationship is as follows: ① Input layer → ② Normalization layer → ③ Sample generation layer → ④ Convolutional layer → ⑤ Sequence layer → ⑥ Concatenation layer → ⑦ New module (Region guided center point regression module: divides the 128-dimensional vector into regions, regresses the center points of each region, and enhances feature discrimination) → ⑧ Loss function layer → ⑨ Output layer.

[0057] This scheme's architecture is an improved version of SimCLR. The core reason is that the learning paradigm remains contrastive learning (positive and negative sample comparison + InfoNCE loss), and the overall process still follows SimCLR's "sample generation → encoding → projection → contrastive loss" paradigm. The 9-layer structure has a clear mapping relationship with the four major components of the standard SimCLR: ③ The sample generation layer corresponds to SimCLR's random data augmentation module. The difference is that SimCLR performs data augmentation on a single sample to generate positive pairs, while this scheme defines positive and negative sample pairs according to a similarity threshold. The mechanisms are different, but the effects are the same; ④⑤⑥ (convolution + sequence + concatenation) correspond to SimCLR's basic encoder network. SimCLR uses ResNet for overall encoding, while this scheme uses CNN + LSTM for split encoding, which is more granular; ⑦ Region-guided centroid regression corresponds to SimCLR's non-linear projection head. SimCLR's projection head is an MLP linear mapping, while this scheme uses region centroid regression instead. The mechanisms are different, but the positions are the same; ⑧ The loss function layer is exactly the same as SimCLR's contrastive loss function, which is InfoNCE.

[0058] Functions and calculation steps of each level:

[0059] ① Input layer: Receives three types of raw data (device data, user characteristics, and user instructions);

[0060] ② Normalization layer: 3σ criterion to remove outliers, mean to fill missing values, min-max normalization to [0,1], and alignment of the three types of data by timestamp;

[0061] ③ Sample generation layer: The similarity of sample pairs is calculated using cosine similarity. Positive samples are defined according to the threshold (instruction ≥90%, device ≥80%, feature ≥85%), and negative samples are defined below 50%. The ratio of positive to negative samples is 1:3.

[0062] ④ Convolutional layer: CNN performs convolution operations on device parameters and instruction semantics to extract local spatial features;

[0063] ⑤ Sequence layer: LSTM performs gating operations on user behavior time-series data to extract time-dependent features;

[0064] ⑥ Concatenation layer: The CNN output and the LSTM output are concatenated along the feature dimension to obtain a 128-dimensional vector;

[0065] ⑦ Region-guided centroid regression module: Divide the 128-dimensional vector into K sub-regions according to the dimensions, set a learnable centroid vector for each sub-region, calculate the distance from the sample feature to the centroid of its region, and use regression loss to make similar samples cluster towards the center and dissimilar samples move away, and output the enhanced features.

[0066] ⑧ Loss function layer: Calculate the InfoNCE contrastive loss to bring positive sample pairs closer together and push negative sample pairs further apart;

[0067] ⑨ Output layer: Outputs the 128-dimensional feature vector after training and optimization.

[0068] In this architecture, the lightweight multi-scale extraction module is added after the CNN convolutional layer and before the LSTM sequence layer, embedded within the feature encoder. The hierarchical relationship is as follows: ① Input layer → ② Normalization layer → ③ Sample generation layer → ④ Convolutional layer → ⑦ New module (lightweight multi-scale extraction module: simultaneously extracts fine-grained (local details) and coarse-grained (global semantic) features using different convolutional kernel receptive fields, performs multi-scale fusion, and has lightweight computational cost) → ⑤ Sequence layer → ⑥ Concatenation layer → ⑧ Loss function layer → ⑨ Output layer.

[0069] The above solution is also an improvement on SimCLR. This module is embedded inside the encoder and represents a refinement of the basic SimCLR encoder. The encoder of the standard SimCLR is a monolithic network (such as ResNet), while this solution breaks it down into four parts: CNN → multi-scale module → LSTM → concatenation. The multi-scale module enhances the feature extraction stage within the encoder without changing the overall paradigm of SimCLR.

[0070] Functions and calculation steps of each level:

[0071] ① Input layer: Receives three types of raw data (same as above);

[0072] ② Normalization layer: Data preprocessing (same as above);

[0073] ③ Sample generation layer: Definition of positive and negative sample pairs (same as above);

[0074] ④ Convolutional layer: CNN extracts local spatial features of device parameters and instruction semantics, and outputs feature maps;

[0075] ⑦ Lightweight multi-scale extraction module: The feature maps output by the convolutional layer are fed into convolutional kernels of different sizes (such as 1×1, 3×3, 5×5) for parallel processing. 1×1 captures point-by-point channel relationships (fine-grained), while 3×3 and 5×5 capture local context of different ranges (coarse-grained). The outputs of each branch are spliced ​​and fused along the channel dimension. Then, 1×1 convolution is used to reduce the number of channels to control the amount of computation and output a multi-scale fused feature map.

[0076] ⑤ Sequence layer: LSTM processes the fused features step by step to extract temporal dependencies;

[0077] ⑥ Concatenation layer: Concatenates the features into a 128-dimensional feature vector;

[0078] ⑧ Loss function layer: InfoNCE contrastive loss calculation;

[0079] ⑨ Output layer: Outputs feature vectors;

[0080] Neither architecture is entirely new; both are essentially based on the SimCLR contrastive learning paradigm, with three components (data augmentation → sample generation, encoder → CNN-LSTM splitting + multi-scale, projector → center point regression) being replaced or refined. The core contrastive learning mechanism (InfoNCE loss driving positive and negative sample comparison) remains unchanged.

[0081] Specifically, for the second step, positive and negative samples are defined. A contrastive learning framework based on SimCLR is built, introducing a region-guided centroid regression module and a lightweight multi-scale extraction module. The feature encoder uses a CNN-LSTM hybrid network, and the contrastive loss function uses the InfoNCE loss function. Positive samples are defined as sample pairs with "user command semantic similarity ≥ 90%, device data similarity ≥ 80%, and user feature data similarity ≥ 85%". Semantic and data similarity are calculated using a cosine similarity algorithm, with static feature weights of 0.4 and dynamic feature weights of 0.6. Negative samples are divided into three categories: command negative samples (different command semantics, device or user feature similarity < 50%), environment negative samples (identical commands, device data similarity < 50%, user feature similarity ≥ 85%), and user negative samples (identical commands, device data similarity ≥ 80%, user feature similarity < 50%). The ratio of positive to negative sample pairs is controlled at 1:3, generating a total of 3000 positive sample pairs and 9000 negative sample pairs for model training.

[0082] Specifically, for the third step, feature vector extraction and similarity matrix calculation, the generated positive and negative samples are input into the feature encoder. The CNN network extracts local features of device parameters and instruction semantics, and the LSTM network extracts temporal features of user behavior. These are then concatenated to obtain a 128-dimensional feature vector. The cosine similarity algorithm is used to calculate the similarity of the feature vectors of 10,000 samples, generating a 10,000×10,000 similarity matrix. The matrix elements take values ​​in the range [0,1]. The closer the similarity is to 1, the more similar the sample features are.

[0083] In the specific implementation process, clustering is performed based on the association between multiple learning samples to obtain multiple scenarios. This can be achieved through the following steps: calculating the similarity between multiple learning samples using the InfoNCE loss function; using a clustering algorithm to cluster based on the similarity between the learning samples to obtain multiple sample clusters, wherein the similarity between the learning samples in each sample cluster is not zero, and each sample cluster corresponds one-to-one with the scenario.

[0084] In this scheme, the training objective of the learning samples is to make the extracted feature vectors have a higher similarity to similar intentions in the semantic space. The cosine similarity of these feature vectors is used to quantify the association between samples. Samples within a cluster represent similar user behavior patterns or intentions. By mapping each cluster to a specific scenario, an accurate mapping relationship between user intentions and scenarios can be obtained, so that user intentions can be determined more accurately in the future.

[0085] Specifically, the above scheme is fine-grained relationship analysis. For the fourth step, fine-grained relationship analysis, the K-means clustering algorithm is used, and the elbow rule is used to determine the optimal number of clusters as 8, clustering the samples into 8 classes, corresponding to 8 scenarios such as "waking up in the morning," "working during the day," and "sleeping at night." In intra-class analysis, the core feature vector of each class of samples is calculated, and samples with a similarity deviation ≤10% are considered typical samples. Common features of typical samples are extracted (e.g., in the nighttime sleep scenario: air conditioning at 26℃, lights off, curtains closed). In inter-class analysis, the similarity of core features for each class is calculated, categorized into associated categories (similarity ≥70%), weakly associated categories (30%-70%), and unassociated categories (<30%). Switching conditions for associated categories are extracted (e.g., the conditions for switching from the evening leisure scenario to the nighttime sleep scenario are light intensity <500 lux and time ≥22:00). Abnormal samples (similarity <20% with all samples) are identified and individually labeled for use in special strategy generation.

[0086] Specifically, the eight scenarios were automatically clustered by the algorithm, not user-defined or manufacturer-preset. The optimal number of clusters was determined to be eight using the elbow rule. The K-means algorithm automatically calculates the optimal number of clusters based on the distribution characteristics of the sample data, then clusters the samples into eight classes. Subsequent names such as "Waking Up in the Morning," "Working During the Day," and "Sleeping at Night" are simply manual labels assigned after the clustering results are generated, based on the common characteristics of each class of samples (time, device status, etc.)—they correspond to a specific scenario, not eight pre-set categories into which data is added.

[0087] Within-cluster analysis refers to comparing samples within the same cluster. For example, the cluster "Nighttime Sleep" contains 500 samples, and the analysis focuses on the relationships between these 500 samples. Between-cluster analysis refers to comparing samples between different clusters. For example, comparing the "Nighttime Sleep" cluster with the "Morning Wake-up" cluster.

[0088] The following are examples:

[0089] Samples A, B, and C all come from data from the same smart home system. A: 23:00, air conditioner 26℃, lights off, curtains closed; B: 22:45, air conditioner 25℃, lights dim, curtains closed; C: 07:00, air conditioner 24℃, lights on full, curtains open.

[0090] After MAIN clustering, A and B are placed in the "Nighttime Sleep" category, and C is placed in the "Morning Wake-up" category. Within-class analysis compares the similarity between A and B. Between-class analysis compares the overall characteristics of the "Nighttime Sleep" category with those of the "Morning Wake-up" category.

[0091] Intra-class computation is used to confirm the consistency of samples of the same class and to find the user's "standard intent pattern" in the scenario (such as the standard intent of the nighttime sleep scenario is: to be quiet, to be dark, and to have a constant temperature of 26°C); inter-class computation is used to discover the relationship between scenarios and to determine which scenarios may switch between each other and which are completely independent.

[0092] The relationship between the above analysis and intent recognition is as follows: within a class, it determines "what the user wants to do in the current scenario," and between classes, it predicts "what the user might want to do next." The former ensures accurate control, while the latter enables proactive service (preparing transition strategies in advance without waiting for user instructions).

[0093] Samples with a similarity deviation of ≤10% are considered typical samples. These highly similar samples are extracted, and common features are discovered (such as the commonalities of typical samples in a nighttime sleep scenario: air conditioning at 26°C, lights off, and curtains closed). These features are then used directly to generate the basic control strategy for that scenario, which is the standard pattern.

[0094] Edge samples with large similarity deviations are not discarded but retained within the class and marked for reference when dynamically adjusting parameters later. As shown in Table 1.

[0095] Table 1

[0096]

[0097] The underlying logic is as follows: Since we know that "evening leisure" and "nighttime sleep" are strongly correlated scenarios, we look at actual data. In a large sample size, we analyze the environmental data changes at the point in time when "evening leisure" samples transition to "nighttime sleep" samples. Statistical analysis reveals that when light intensity is <500 lux and time is ≥22:00, users are highly likely to transition from leisure to sleep. This is the switching condition. Similarity tells the model "who is associated with whom," while the switching condition tells the model "under what circumstances will the user switch from A to B."

[0098] In some embodiments, controlling the operation of the smart home device according to the user's intent can be achieved through the following steps: If the similarity between the multidimensional data and the scene is greater than or equal to a first similarity threshold, the scene is determined as the target scene, and control steps corresponding to the target scene are executed. The target scene corresponds to the user intent, and the control steps are pre-configured parameters for controlling the operation of the smart home device. If the similarity between the multidimensional data and the scene is less than the first similarity threshold, and the similarity between the multidimensional data and the scene is greater than or equal to a second similarity threshold, the scene is determined as the target scene, and the parameters for the operation of the smart home device are adjusted according to a first preset step size until they are the same as the parameters for the operation of the smart home device corresponding to the control steps. The first similarity threshold is greater than the second similarity threshold. When switching from one scene to another, the parameters for the operation of the smart home device corresponding to one scene are adjusted to the parameters for the operation of the smart home device corresponding to the other scene according to a second preset step size.

[0099] This solution achieves a balance between flexibility and comfort in control strategies by setting two levels of similarity thresholds and a smooth scene switching mechanism. When user behavior highly matches typical scenarios, the strategy is executed directly to ensure response speed; when deviations exist, parameters are fine-tuned to adapt to personalized needs, improving control accuracy and user satisfaction; during scene switching, a smooth transition avoids discomfort caused by sudden changes in device status, enhancing the intelligence level of smart home control and the user experience.

[0100] Specifically, the first similarity threshold can be 90%, the second similarity threshold can be 70%, or any other feasible value. The device parameters are smoothly adjusted according to a first preset step size (e.g., temperature adjusted by 0.5℃ every 2 seconds, brightness adjusted by 5% every 1 second) until the standard control parameters for the scene are reached. Alternatively, a smooth transition can be performed according to a second preset step size (e.g., light brightness changes linearly over 3 seconds, air conditioning temperature changes slowly over 5 seconds).

[0101] Specifically, for the fifth step, the control strategy is dynamically generated and optimized. Basic control strategies are generated based on clustering scenarios, such as a children's learning scenario: 70% light intensity, 25℃ air conditioning, and 50% curtain opening / closing. Current data is collected in real-time, feature vectors are extracted and compared with various core feature vectors. If the similarity is ≥90%, the basic strategy is executed; if it's 70%-90%, parameters are dynamically adjusted. A smooth transition strategy is implemented when switching between related categories (e.g., gradual change in light intensity with a 3-second transition time). For scenarios corresponding to abnormal samples, a Q-learning algorithm is used to generate personalized strategies, and parameters are adjusted based on user feedback. A feedback mechanism is established to collect user satisfaction and manual adjustment data. The sample set is updated every 7 days, the comparative learning model is retrained, and the control strategy is iteratively optimized. Simultaneously, a real-time strategy generation and adjustment module is deployed on the edge gateway, while the cloud server is responsible for sample updates and model training. The control response latency is controlled within 80ms, and the false positive rate is less than 5%.

[0102] In one application scenario, the currently collected data is 22:50, in a bedroom, with a temperature of 25.5℃, dim lighting, and curtains closed. After extracting the feature vector and comparing it with 8 core vectors, the similarity to the "nighttime sleep" category is 94% (≥90%). The basic strategy for this scenario is then directly executed: set the air conditioner to 26℃, turn off the lights, and close the curtains.

[0103] When the similarity is ≥90%, the intent is clear, and the basic strategy for that scenario is executed directly. For example, in a nighttime sleep scenario, the lights are immediately turned off, the temperature is set to 26°C, and the curtains are closed. When the similarity is between 70% and 90%, the intent is less clear, and instead of a full switch, a gradual transition in parameters is implemented. For example, the lights are not turned off immediately but dimmed slowly over 3 seconds. It's better to be conservative and avoid misjudgments that might cause user discomfort. When a switch from the current scenario to a strongly related scenario is detected (e.g., from evening leisure to nighttime sleep), the strategy transitions smoothly, not instantly. Light brightness changes gradually, and the temperature is adjusted progressively to make the user feel seamless. For anomalous samples with less than 20% similarity to any scenario, it indicates a situation the system has never encountered before, and no existing strategy is available. In this case, Q-learning reinforcement learning is used to try different combinations of device parameters. Based on subsequent manual adjustments and user satisfaction feedback, the system gradually learns a personalized strategy for this new scenario.

[0104] The edge gateway is responsible for real-time inference, receiving sensor data, extracting features, comparing with eight scenes, and issuing control commands. It operates locally to ensure millisecond-level response and does not rely on the network. The cloud server is responsible for offline training, summarizing new samples every seven days, retraining the comparative learning model, updating parameters, and pushing the updated data to the edge gateway. This is because the training computation is large and cannot be run locally, and it does not need to be completed in real time.

[0105] In the specific implementation process, controlling the operation of the smart home devices according to the user's intent can be achieved through the following steps: when the similarity between the multidimensional data and all the scenarios is less than the third similarity threshold, an initial personalized control step is generated using the Q-learning reinforcement learning algorithm; upon receiving a modification request, the initial personalized control step is adjusted based on the modification request to obtain a personalized control step, wherein the modification request is a request from the user to modify the operating parameters of the smart home devices; the personalized control step is used as the new scenario, and the operation of the smart home devices is controlled using the personalized control step.

[0106] In this solution, when user behavior exceeds the preset scenario range, Q-learning reinforcement learning and user feedback mechanisms are introduced to actively explore and generate personalized strategies, and continuous optimization is carried out through user feedback, thereby improving the intelligence level of smart home control and user experience.

[0107] Specifically, the third similarity threshold can be 20%, and the third similarity threshold is less than the second similarity threshold.

[0108] Specifically, this solution generates basic control strategies for various scenarios based on fine-grained relationship analysis results, and dynamically adjusts strategy parameters by combining real-time collected multi-dimensional data; for special scenarios, it uses Q-learning reinforcement learning algorithm to generate personalized strategies, establishes user feedback mechanism and sample iterative update mechanism to achieve continuous optimization of control strategies; it adopts edge computing and cloud collaborative deployment, with the edge responsible for real-time strategy generation and adjustment, and the cloud responsible for sample updates and model training.

[0109] Specifically, this solution breaks through the limitations of existing technologies with fixed rules or static models, and constructs a three-level strategy generation mechanism of "basic strategy + dynamic adjustment + iterative optimization" to achieve adaptive adaptation of control strategies to scene changes. Combined with edge computing to reduce response latency, unlike existing solutions that rely on cloud processing or static fine-tuning, strategy updates can be completed without manual intervention, thereby improving the level of control intelligence.

[0110] Specifically, the control strategy of this solution can adapt to the dynamic changes in the home environment and user needs in real time, with a response latency of less than 100ms. Users do not need to manually set control rules or parameters, reducing the complexity of user operation. Through continuous iterative optimization, it ensures that the control strategy always meets user needs, solving the problems of rigid control strategies and slow response speed in existing technologies. At the same time, it can reduce home energy consumption by 15%-20%, combining practicality and economy.

[0111] In some embodiments, machine learning is performed on multiple sets of the aforementioned multidimensional data to identify the relationships between them, thereby enabling the identification of user intent. This can be achieved through the following steps: preprocessing the multidimensional data to obtain preprocessed multidimensional data, wherein the preprocessing methods include one or more of outlier removal, missing value imputation, data normalization, and timestamp alignment; and performing machine learning on the preprocessed multidimensional data to identify the relationships between them, thereby enabling the identification of user intent.

[0112] In this scheme, preprocessing of multidimensional data, such as removing outliers, imputing missing values, normalizing data, and aligning timestamps, eliminates noise and dimensional differences in the data, thus ensuring the effectiveness of the multidimensional data.

[0113] Specifically, for the first step, multi-dimensional data collection and preprocessing. In the preprocessing stage, the 3σ criterion is used to remove outliers (such as air conditioning temperature >35℃ or <16℃), the mean imputation method is used to fill in missing humidity data, the min-max normalization algorithm is used to map numerical data to the [0,1] interval, and the three types of data are aligned according to the timestamp to form a data sample every 5 seconds, for a total of 10,000 samples, which are then stored in the local data storage module.

[0114] The method of using preprocessed multidimensional data for machine learning to identify the relationships between the preprocessed multidimensional data, so as to identify the user intent, is the same as the method of using multiple multidimensional data for machine learning to identify the relationships between multiple multidimensional data, so as to identify the user intent. It will not be elaborated here.

[0115] In summary, the solution proposed in this application can accurately identify the user's true intentions, dynamically adapt to changes in the scenario, eliminate the need for users to manually set rules, significantly improve the intelligence level of device control and user experience, and reduce household energy consumption, thus verifying the practicality and effectiveness of the present invention.

[0116] This application also provides a control device for a smart home device. It should be noted that the control device for the smart home device in this application can be used to execute the control method for smart home devices provided in this application. This device is used to implement the above embodiments and preferred embodiments; details already described will not be repeated. As used below, the term "module" can refer to a combination of software and / or hardware that implements a predetermined function. Although the device described in the following embodiments is preferably implemented in software, hardware implementation, or a combination of software and hardware, is also possible and contemplated.

[0117] The following describes the control device for smart home devices provided in the embodiments of this application.

[0118] Figure 4 This is a structural block diagram of the control device for a smart home device according to an embodiment of this application. Figure 4 As shown, the device includes:

[0119] The acquisition unit 10 is used to acquire data related to the operation of smart home devices in multiple dimensions, and obtain multiple multidimensional data. The multiple multidimensional data include device data, user characteristic data and user command data. The device data includes the operating status of the smart home device and the environmental parameters of the environment in which the smart home device is located. The user characteristic data includes pre-configured user attribute data and user behavior preference data. The user command data includes voice commands and / or application commands.

[0120] The identification unit 20 is used to perform machine learning based on multiple of the aforementioned multidimensional data to identify the relationship between the multiple of the aforementioned multidimensional data, so as to identify the user's intent.

[0121] The control unit 30 is used to control the operation of the smart home devices according to the user's intention.

[0122] This embodiment acquires device data (reflecting the environment and device status), user characteristic data (reflecting the user's static attributes and dynamic preferences), and user command data (reflecting the user's explicit needs). By introducing user characteristic data (including pre-configured user attributes and user behavior preferences), it is possible to distinguish the individual differences of different users. By introducing device data, it is possible to combine the real-time environmental background and analyze the relationship between user commands, user characteristics, environment, and device data through machine learning algorithms. This machine learning process based on relationship recognition can capture the implicit logic and fine-grained features behind the data, thereby accurately inferring the user's true intention from multi-dimensional data, rather than simply responding to literal commands. This allows for a more accurate understanding of the user's personalized needs in specific scenarios. For different user characteristics or different environmental parameters, the identified intention may correspond to different temperature settings or mode selections. Based on this accurately identified user intention, corresponding controls are generated and executed, realizing the response to the user's personalized needs.

[0123] In its specific implementation, the identification unit includes a learning module, a clustering module, and a determination module. The learning module uses a contrastive learning framework to perform machine learning based on historical multidimensional data, constructing multiple learning samples. The clustering module clusters these learning samples based on their relationships to obtain multiple scenarios, where each scenario is related to historical user intent. Figure 1 One-to-one correspondence; the determination module is used to determine the scenario with the highest similarity to the above-mentioned multidimensional data as the target scenario, and to determine the above-mentioned historical user intent corresponding to the above-mentioned target scenario as the above-mentioned user intent corresponding to the above-mentioned multidimensional data.

[0124] This scheme employs a contrastive learning framework for training, which can accurately distinguish the sample distribution of different user intentions in a high-dimensional space, bringing samples with similar intentions closer together and pushing samples with different intentions further apart. By clustering high-dimensional feature vectors into specific scene categories and establishing a one-to-one correspondence between each scene and historical user intentions, it achieves a mapping from data features to specific user intentions. This clustering-based scene segmentation can structure complex user behavior patterns and quickly locate potential user intentions based on the scene to which the current data belongs. By matching the similarity between the currently collected multidimensional data and typical scenes obtained from historical clustering, it can quickly determine the most likely user intention, thus ensuring a high accuracy rate in user intention recognition.

[0125] In some embodiments, the learning module includes an acquisition submodule, a first construction submodule, a second construction submodule, a second acquisition submodule, a training submodule, and a processing submodule. The acquisition submodule is used to acquire a contrastive learning framework, wherein the contrastive learning framework is the SimCLR framework. The first construction submodule is used to construct a feature extraction model from the encoder of the contrastive learning framework, wherein the feature extraction model is a CNN-LSTM model. The second construction submodule is used to add a lightweight multi-scale extraction module after the convolutional layer of the feature extraction model and before the sequence layer of the feature extraction model, such that the output of the convolutional layer is used as the input of the added lightweight multi-scale extraction module, and the output of the added lightweight multi-scale extraction module is used as the output of the sequence layer. A guided centroid regression module is added after the concatenation layer and before the loss function layer of the feature extraction model. The output of the concatenation layer serves as the input to the guided centroid regression module, and the output of the guided centroid regression module serves as the input to the loss function layer, resulting in an improved contrastive learning framework. A training submodule is used to form a training set from the historical multidimensional data and corresponding feature labels. This training set is then used to train the improved contrastive learning framework, resulting in a sample extraction framework. The feature labels are the historical sample features of the historical multidimensional data in the training set. A processing submodule inputs the historical multidimensional data into the sample extraction framework to obtain the learning samples corresponding to the historical multidimensional data.

[0126] In this scheme, by introducing a lightweight multi-scale extraction module and a guided center point regression module, the improved SimCLR framework can more effectively extract local details and multi-scale features from multi-dimensional data. The preprocessed historical multi-dimensional data is input into the trained sample extraction framework, which outputs a high-dimensional feature vector for each data sample. These feature vectors are called "learning samples". These learning samples retain the key semantics and correlation information in the original data, thereby ensuring that the accurate user intent can be learned from the samples later.

[0127] In the specific implementation process, the clustering module includes a calculation submodule and a clustering submodule. The calculation submodule is used to calculate the similarity between multiple learning samples using the InfoNCE loss function. The clustering submodule is used to cluster the learning samples according to the similarity between them using a clustering algorithm to obtain multiple sample clusters. The similarity between the learning samples in each sample cluster is not zero, and each sample cluster corresponds to the scenario.

[0128] In this scheme, the training objective of the learning samples is to make the extracted feature vectors have a higher similarity to similar intentions in the semantic space. The cosine similarity of these feature vectors is used to quantify the association between samples. Samples within a cluster represent similar user behavior patterns or intentions. By mapping each cluster to a specific scenario, an accurate mapping relationship between user intentions and scenarios can be obtained, so that user intentions can be determined more accurately in the future.

[0129] In some embodiments, the control unit includes a first control module, a second control module, and a third control module. The first control module is used to determine the scene as the target scene when the similarity between the multidimensional data and the scene is greater than or equal to a first similarity threshold, and execute the control steps corresponding to the target scene. The target scene corresponds to the user intent, and the control steps are pre-configured parameters for controlling the operation of the smart home device. The second control module is used to determine the scene as the target scene when the similarity between the multidimensional data and the scene is less than the first similarity threshold and the similarity between the multidimensional data and the scene is greater than or equal to a second similarity threshold. It then adjusts the operating parameters of the smart home device by a first preset step size until they are the same as the operating parameters of the smart home device corresponding to the control steps. The first similarity threshold is greater than the second similarity threshold. The third control module is used to adjust the operating parameters of the smart home device corresponding to one scene to the operating parameters of the smart home device corresponding to the other scene by a second preset step size when switching from one scene to another.

[0130] This solution achieves a balance between flexibility and comfort in control strategies by setting two levels of similarity thresholds and a smooth scene switching mechanism. When user behavior highly matches typical scenarios, the strategy is executed directly to ensure response speed; when deviations exist, parameters are fine-tuned to adapt to personalized needs, improving control accuracy and user satisfaction; during scene switching, a smooth transition avoids discomfort caused by sudden changes in device status, enhancing the intelligence level of smart home control and the user experience.

[0131] In the specific implementation process, the control unit includes a generation module, an adjustment module, and a fourth control module. The generation module is used to generate initial personalized control steps using a Q-learning reinforcement learning algorithm when the similarity between the multidimensional data and all the aforementioned scenarios is less than a third similarity threshold. The adjustment module is used to adjust the initial personalized control steps based on the modification request received to obtain personalized control steps, wherein the modification request is a request from the user to modify the operating parameters of the smart home device. The fourth control module is used to treat the personalized control steps as a new scenario and control the operation of the smart home device using the personalized control steps.

[0132] In this solution, when user behavior exceeds the preset scenario range, Q-learning reinforcement learning and user feedback mechanisms are introduced to actively explore and generate personalized strategies, and continuous optimization is carried out through user feedback, thereby improving the intelligence level of smart home control and user experience.

[0133] In some embodiments, the identification unit includes a preprocessing module and an identification module. The preprocessing module is used to preprocess the multidimensional data to obtain preprocessed multidimensional data. The preprocessing method includes one or more of outlier removal, missing value imputation, data normalization, and timestamp alignment. The identification module is used to perform machine learning based on the preprocessed multidimensional data to identify the relationship between the preprocessed multidimensional data, so as to identify the user intent.

[0134] In this scheme, preprocessing of multidimensional data, such as removing outliers, imputing missing values, normalizing data, and aligning timestamps, eliminates noise and dimensional differences in the data, thus ensuring the effectiveness of the multidimensional data.

[0135] The control device for the aforementioned smart home device includes a processor and a memory. The acquisition unit, identification unit, and control unit are all stored as program units in the memory, and the processor executes these program units to achieve the corresponding functions. All of the above modules reside in the same processor; alternatively, the modules may be located in different processors in any combination.

[0136] The processor contains a kernel, which retrieves the corresponding program units from memory. One or more kernels can be configured, and adjusting kernel parameters can address the challenge of meeting users' personalized needs for smart homes in existing technologies.

[0137] The memory may include non-permanent memory in computer-readable media, such as random access memory (RAM) and / or non-volatile memory, such as read-only memory (ROM) or flash RAM, and the memory includes at least one memory chip.

[0138] This invention provides a computer-readable storage medium including a stored program, wherein, when the program is executed, it controls the device containing the computer-readable storage medium to perform the control method of the smart home device.

[0139] This invention provides a processor for running a program, wherein the program executes the control method for the smart home device.

[0140] This invention provides a device including a processor, a memory, and a program stored in the memory and executable on the processor. When the processor executes the program, it implements a method for controlling a smart home device. The device described herein can be a server, PC, tablet, mobile phone, etc.

[0141] This application also provides a computer program product that, when executed on a data processing device, is suitable for executing a program that initializes control method steps for smart home devices.

[0142] This application also provides a smart home control system, including: one or more processors, a memory, and one or more programs, wherein the one or more programs are stored in the memory and configured to be executed by the one or more processors, and the one or more programs include a control method for executing any of the above-described smart home devices.

[0143] It is obvious to those skilled in the art that the modules or steps of the present invention described above can be implemented using general-purpose computing devices. They can be centralized on a single computing device or distributed across a network of multiple computing devices. They can be implemented using computer-executable program code, and thus can be stored in a storage device for execution by a computing device. In some cases, the steps shown or described can be performed in a different order than those described herein, or they can be fabricated as separate integrated circuit modules, or multiple modules or steps can be fabricated as a single integrated circuit module. Thus, the present invention is not limited to any particular combination of hardware and software.

[0144] Those skilled in the art will understand that embodiments of this application can be provided as methods, systems, or computer program products. Therefore, this application can take the form of a completely hardware embodiment, a completely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, this application can take the form of a computer program product embodied on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0145] This application is described with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of this application. It will be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, special-purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, generate instructions for implementing the flowchart... Figure 1 One or more processes and / or boxes Figure 1 A device that provides the functions specified in one or more boxes.

[0146] These computer program instructions may also be stored in a computer-readable storage medium that can direct a computer or other programmable data processing device to function in a particular manner, such that the instructions stored in the computer-readable storage medium produce an article of manufacture including instruction means, which are implemented in a process Figure 1 One or more processes and / or boxes Figure 1 The function specified in one or more boxes.

[0147] These computer program instructions may also be loaded onto a computer or other programmable data processing equipment to cause a series of operational steps to be performed on the computer or other programmable equipment to produce a computer-implemented process, thereby providing instructions that execute on the computer or other programmable equipment for implementing the process. Figure 1 One or more processes and / or boxes Figure 1 The steps of the function specified in one or more boxes.

[0148] In a typical configuration, a computing device includes one or more processors (CPU), input / output interfaces, network interfaces, and memory.

[0149] Memory may include non-persistent memory in computer-readable media, such as random access memory (RAM) and / or non-volatile memory, such as read-only memory (ROM) or flash RAM. Memory is an example of computer-readable media.

[0150] Computer-readable media includes both permanent and non-permanent, removable and non-removable media that can store information using any method or technology. Information can be computer-readable instructions, data structures, modules of programs, or other data. Examples of computer storage media include, but are not limited to, phase-change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technologies, CD-ROM, digital versatile optical disc (DVD) or other optical storage, magnetic tape, disk storage or other magnetic storage devices, or any other non-transferable medium that can be used to store information accessible by a computing device. As defined herein, computer-readable media does not include transient computer-readable media, such as modulated data signals and carrier waves.

[0151] The technical features of the above embodiments can be combined in any way. For the sake of brevity, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0152] It should also be noted that the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such process, method, article, or apparatus. Unless otherwise specified, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes that element.

[0153] The above description is merely a preferred embodiment of this application and is not intended to limit this application. Various modifications and variations can be made to this application by those skilled in the art. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of this application should be included within the protection scope of this application.

Claims

1. A control method for a smart home device, characterized in that, include: Data related to the operation of smart home devices is acquired from multiple dimensions to obtain multiple multidimensional data. The multiple multidimensional data include device data, user characteristic data, and user command data. The device data includes the operating status of the smart home device and the environmental parameters of the environment in which the smart home device is located. The user characteristic data includes pre-configured user attribute data and user behavior preference data. The user command data includes voice commands and / or application commands. Machine learning is performed on multiple sets of multidimensional data to identify the relationships between them, thereby enabling the identification of user intent; The user controls the operation of the smart home devices according to their intent.

2. The method according to claim 1, characterized in that, Machine learning is performed on multiple sets of multidimensional data to identify relationships between them, thereby enabling the identification of user intent, including: A contrastive learning framework is used to perform machine learning based on historical multidimensional data to construct multiple learning samples; Clustering is performed based on the correlation between multiple learning samples to obtain multiple scenarios, wherein each scenario corresponds one-to-one with a historical user intent; The scenario with the highest similarity to the multiple multidimensional data is identified as the target scenario, and the historical user intent corresponding to the target scenario is identified as the user intent corresponding to the multiple multidimensional data.

3. The method according to claim 2, characterized in that, A contrastive learning framework is used to perform machine learning based on historical multidimensional data, constructing multiple learning samples, including: Obtain a contrastive learning framework, wherein the contrastive learning framework is the SimCLR framework; The encoder of the contrastive learning framework is used to construct a feature extraction model, wherein the feature extraction model is a CNN-LSTM model; A lightweight multi-scale extraction module is added after the convolutional layer and before the sequence layer of the feature extraction model, so that the output of the convolutional layer serves as the input of the lightweight multi-scale extraction module and the output of the lightweight multi-scale extraction module serves as the input of the sequence layer. A guided centroid regression module is added after the concatenation layer and before the loss function layer of the feature extraction model, so that the output of the concatenation layer serves as the input of the guided centroid regression module and the output of the guided centroid regression module serves as the input of the loss function layer, thus obtaining an improved contrastive learning framework. The historical multidimensional data and corresponding feature labels are combined to form a training set. The improved contrastive learning framework is trained using the training set to obtain a sample extraction framework. The feature labels are the historical sample features of the historical multidimensional data in the training set. The historical multidimensional data is input into the sample extraction framework to obtain the learning samples corresponding to the historical multidimensional data.

4. The method according to claim 2, characterized in that, Clustering is performed based on the relationships between multiple learning samples to obtain multiple scenarios, including: The similarity between multiple learning samples is calculated using the InfoNCE loss function; A clustering algorithm is used to cluster the learning samples based on their similarity to obtain multiple sample clusters. The similarity between the learning samples in each sample cluster is not zero, and each sample cluster corresponds one-to-one with the scene.

5. The method according to claim 2, characterized in that, Controlling the operation of the smart home devices according to the user's intent includes: If the similarity between the multidimensional data and the scene is greater than or equal to a first similarity threshold, the scene is determined as the target scene, and the control steps corresponding to the target scene are executed. The target scene corresponds to the user intent, and the control steps are pre-configured parameters for controlling the operation of the smart home device. If the similarity between the multidimensional data and the scene is less than the first similarity threshold and the similarity between the multidimensional data and the scene is greater than or equal to the second similarity threshold, the scene is determined as the target scene, and the operating parameters of the smart home device are adjusted according to the first preset step size until they are the same as the operating parameters of the smart home device corresponding to the control step, wherein the first similarity threshold is greater than the second similarity threshold. When switching from one scenario to another, the operating parameters of the smart home device corresponding to one scenario are adjusted to the operating parameters of the smart home device corresponding to the other scenario according to a second preset step size.

6. The method according to claim 5, characterized in that, Controlling the operation of the smart home devices according to the user's intent includes: When the similarity between the multidimensional data and all the scenarios is less than the third similarity threshold, the Q-learning reinforcement learning algorithm is used to generate the initial personalized control steps. Upon receiving a modification request, the initial personalized control steps are adjusted based on the modification request to obtain personalized control steps, wherein the modification request is a request from the user to modify the operating parameters of the smart home device; The personalized control steps are used as the new scenario, and the personalized control steps are used to control the operation of the smart home devices.

7. The method according to any one of claims 1 to 6, characterized in that, Machine learning is performed on multiple sets of multidimensional data to identify relationships between them, thereby enabling the identification of user intent, including: The multidimensional data is preprocessed to obtain preprocessed multidimensional data. The preprocessing method includes one or more of outlier removal, missing value imputation, data normalization, and timestamp alignment. Machine learning is performed on the preprocessed multidimensional data to identify the relationships between the preprocessed multidimensional data, thereby enabling the identification of the user's intent.

8. A control device for a smart home device, characterized in that, include: The acquisition unit is used to acquire data related to the operation of smart home devices from multiple dimensions, thereby obtaining multiple multidimensional data. The multiple multidimensional data include device data, user characteristic data, and user command data. The device data includes the operating status of the smart home device and the environmental parameters of the environment in which the smart home device is located. The user characteristic data includes pre-configured user attribute data and user behavior preference data. The user command data includes voice commands and / or application commands. The identification unit is used to perform machine learning based on multiple sets of multidimensional data to identify the relationship between the multiple sets of multidimensional data, so as to identify the user's intent; A control unit for controlling the operation of the smart home devices according to the user's intent.

9. A computer program product, comprising a computer program, characterized in that, When the computer program is executed by the processor, it implements the steps of the control method for the smart home device according to any one of claims 1 to 7.

10. A smart home control system, characterized in that, include: One or more processors, a memory, and one or more programs, wherein the one or more programs are stored in the memory and configured to be executed by the one or more processors, the one or more programs including a control method for performing any one of claims 1 to 7 of a smart home device.

Images