A smart home decision-making system based on cognitive perception and intention prediction

Abstract

This invention discloses a smart home decision-making system based on cognitive perception and intent prediction, relating to the field of smart home control. It includes an instruction acquisition module for acquiring control instructions issued by multiple users to smart home devices within the same time period; a conflict identification module for detecting control conflicts between two or more control instructions and constructing a conflict state input when conflicts exist; a control strategy generation module for inputting the conflict state into a control strategy generation model trained based on historical control data; and an execution control module for receiving the overall control scheme output by the control strategy generation model and controlling multiple target devices to operate according to the control scheme. This invention solves the device control problem when multiple users simultaneously issue control instructions and control conflicts exist.

Description

Technical Field

[0001] This invention relates to the field of smart home control, and more specifically, to a smart home decision-making system based on cognitive perception and intention prediction. Background Technology

[0002] With the increasing prevalence of smart home systems, it is becoming more common for users to issue control commands simultaneously within the same home space. This is especially true in multi-user shared environments, where different users may simultaneously issue conflicting control intentions towards the same or related devices. For example, one user might want to turn off the lights to rest, while another wants to turn them on to read. These conflicts stem not only from overlapping operational goals but also from differences in user roles, current cognitive states (such as fatigue or stress), situational urgency, and subjective preferences regarding the environment. Existing systems typically handle conflicts using simplistic methods such as "newest command priority" or "priority to the primary user," lacking the ability to perceive, understand, and reconcile the true nature of the conflict.

[0003] In practical use, the optimal solution to the same conflicting command combination may vary drastically depending on the user, the time of day, or the environmental conditions. For example, a child's request to turn off the lights at night might have higher priority than a father's need to watch a movie, while the opposite might be true in the early morning or at a social gathering. Therefore, existing technologies struggle to dynamically determine the appropriate coordination and control scheme based on the conflict context, and cannot summarize the optimal strategy from historical behavior. There is an urgent need for a smart home decision-making system with conflict identification, classification, modeling, and strategy generation capabilities to improve intelligent response levels and user experience in multi-user environments. Summary of the Invention

[0004] The technical problem to be solved by the present invention is to provide a smart home decision-making system based on cognitive perception and intention prediction, so as to solve the problems mentioned in the background art.

[0005] To achieve the above objectives, the present invention adopts the following technical solution:

[0006] A smart home decision-making system based on cognitive perception and intention prediction includes:

[0007] The command acquisition module is used to collect control commands issued by multiple users to smart home devices within the same time period;

[0008] The conflict identification module is used to detect whether there is a control conflict between two or more control commands, and to construct a conflict state input when a control conflict exists;

[0009] The control strategy generation module is used to input the conflict state into a control strategy generation model trained based on historical control data;

[0010] The execution control module is used to receive the overall control scheme output by the control strategy generation model and control multiple target devices to operate according to the control scheme.

[0011] In some embodiments, the control command includes a user identifier, a target device to be controlled, a control operation type, and a command issuance time.

[0012] In some embodiments, the conflict state includes the following information:

[0013] User control intent related to the conflict, current environmental status information, and current status information of each target device.

[0014] In some embodiments, the control strategy generation model is a model trained through supervised learning, and the training process uses historical conflict states and corresponding device control combinations as sample pairs.

[0015] In some embodiments, the training process of the control policy generation model includes:

[0016] Collect a data sample set containing historical conflict control scenarios. Each sample includes: conflicting control commands issued by two or more users within the same time period, environmental status information and equipment status information corresponding to the control commands, and the equipment control combination actually executed by the users after negotiation in the historical data under the conflict scenario, which serves as a supervision label.

[0017] The data samples are structured and encoded to transform user intent, environmental state, and device state into input vectors that the model can process, and the device control combination is transformed into multi-objective output labels.

[0018] Training is performed using a multilayer perceptron model or a tree model based on ensemble learning to minimize the discrepancy between the predicted output and the supervised label.

[0019] In some embodiments, the control policy generation model is a multilayer perceptron model, including:

[0020] An input layer is used to receive a structured conflict state input vector, which is composed of user intent encoding, environment state encoding and device state encoding concatenated together;

[0021] It has at least two hidden layers, each containing 64 to 512 neurons, with ReLU activation function and fully connected layers;

[0022] An output layer contains multiple output node groups corresponding to the number of required control devices, and each output node group corresponds to the control operation classification result of a target device.

[0023] The multilayer perceptron model is trained under supervised supervision using the cross-entropy loss function.

[0024] In some embodiments, the process of transforming user intent, environmental state, and device state into input vectors that the model can process includes:

[0025] Each user's control intent is encoded, and the encoding includes a unique hot code for the user identifier, a category code for the target device being controlled, a unique hot code for the type of control operation, and a time period classification code for the time when the intent is issued.

[0026] The current environmental state is numerically processed, including time, light intensity, noise level, and temperature sensor data, and continuous variables are normalized.

[0027] The current device status is enumerated and encoded, including the device on / off status, current set value, and position status;

[0028] The encoded results are concatenated to form a fixed-length structured input vector, which is used as the input to the control strategy generation model.

[0029] In some embodiments, the instruction acquisition module includes:

[0030] The speech recognition submodule is used to receive the user's natural language commands through a microphone array and convert them into structured control commands;

[0031] The touch input submodule is used to receive operation input from mobile terminals or wall control panels.

[0032] In some embodiments, the instruction acquisition module further includes:

[0033] The timestamp synchronization submodule is used to perform time-series normalization processing on multiple user commands received within the same time period.

[0034] In some embodiments, the conflict identification module uses the following criteria to determine whether two or more control commands conflict:

[0035] Whether the control command applies to the same device or the same group of devices;

[0036] Control whether operation types are mutually exclusive;

[0037] Whether control commands occur simultaneously within a set time window, wherein the time window is less than a preset threshold;

[0038] Control whether the target device is currently in an incompatible state transition.

[0039] The advantages of this invention over existing technologies lie in that it provides a smart home decision-making system based on cognitive perception and intent prediction. This system can accurately identify conflict relationships and construct structured conflict states when multiple users simultaneously issue control commands and control conflicts exist. Furthermore, it generates the most suitable device control combination scheme for the current scenario based on a strategy model trained from historical control data. Compared to traditional methods that process commands according to their order or the primary user's priority, this invention comprehensively considers user role differences, current environmental conditions, and device operating states, dynamically generating more collaborative and rational control responses. Attached Figure Description

[0040] Figure 1 This is a schematic diagram of the overall system of the present invention;

[0041] Figure 2 This is a schematic diagram of conflict identification in this invention;

[0042] Figure 3 This is a schematic diagram illustrating the generation of the control strategy of the present invention. Detailed Implementation

[0043] The specific embodiments of the present invention will now be described with reference to the accompanying drawings.

[0044] In the field of smart homes, device control in multi-user scenarios often faces challenges of complexity and conflict. To address this issue, this invention proposes a smart home decision-making system based on cognitive perception and intent prediction. This system collects user commands, identifies conflicts, generates control strategies, and executes device control, achieving efficient management and coordination of smart home devices in a multi-user environment. The following embodiments will describe in detail the system architecture, working principle, and specific implementation scheme, covering the functional implementation of each module, model training, and data processing.

[0045] like Figure 1 As shown, the system of this invention aims to solve the problem of how to intelligently decide and execute the optimal control scheme when multiple users issue conflicting control commands to smart home devices within the same time period. The system consists of four core modules: a command acquisition module, a conflict identification module, a control strategy generation module, and an execution control module. These modules work together to form a complete closed-loop decision-making process from command acquisition to final device control.

[0046] The command acquisition module is responsible for acquiring control commands issued by multiple users in real time through various methods. These commands may originate from voice input, touch operations on mobile applications, or manual input from wall control panels. To achieve this function, the command acquisition module is subdivided into three sub-modules, each with its own function, ensuring accurate command acquisition and timing processing.

[0047] The first submodule is the speech recognition submodule. It receives natural language commands from the user, such as "turn on the bedroom light," via a built-in microphone array. The received speech signal is processed by Natural Language Processing (NLP) technology and converted into structured control commands. Specifically, the system uses a pre-trained speech-to-text model (such as a deep learning-based speech recognition algorithm) to parse the user's speech content and extract key information, including the user's identity, the target device (e.g., the bedroom light), the operation type (e.g., turning it on), and the specific time the command was issued (accurate to the second). To improve recognition accuracy, the microphone array may employ beamforming technology to enhance the target sound signal and suppress background noise.

[0048] The second submodule is the touch input submodule, which handles input from mobile devices (such as smartphones or tablets) and wall-mounted control panels. For example, when a user taps the "turn off the air conditioner" button via a mobile app or slides on a wall panel to adjust the brightness of lights, these actions are recorded as control commands. The touch input submodule acquires user operation data through communication protocols with the devices (such as Wi-Fi or Zigbee) and converts it into a structured format consistent with voice commands.

[0049] The third submodule is the timestamp synchronization submodule. Since commands may be issued intensively within a short period in multi-user scenarios, the system needs to ensure the accuracy of command timing information. This submodule assigns a precise timestamp to each command and performs timing normalization on multiple commands within the same time period (e.g., a 5-second window). The normalization process includes converting the timestamp into a relative time offset (e.g., the difference in seconds based on the earliest command) so that subsequent modules can determine the order or simultaneity of commands. For example, if user A issues the command "turn on the TV" at 14:30:02 and user B issues the command "turn off the TV" at 14:30:04, the timestamp synchronization submodule will record the time difference between the two as 2 seconds and mark them as belonging to the same time period.

[0050] Each control command includes the following information: user identifier (such as the unique ID of user A or B), target device (such as television or lighting fixture), control operation type (such as turn on, turn off, adjust), and command issuance time. This information provides the basic data for subsequent conflict identification and strategy generation.

[0051] like Figure 2 As shown, the conflict identification module's task is to analyze the acquired control commands, determine whether a conflict exists, and construct a structured conflict state input when a conflict is detected. The implementation of this module relies on explicit conflict judgment rules and comprehensive state information acquisition.

[0052] Conflict determination is based on the following criteria. First, check whether the control commands apply to the same device or the same group of devices. For example, living room lights might be defined as a group of devices; if user A commands "turn on the living room light" and user B commands "turn off the living room light," then both are targeting the same object. Second, analyze whether the control operation types are mutually exclusive. For example, "turn on" and "turn off" are typical mutually exclusive operations and cannot be executed simultaneously. Third, determine whether the commands occur simultaneously within a set time window. The time window is usually set to a small threshold (e.g., 3 to 5 seconds) to capture high-frequency commands within a short period. Finally, check whether the target device is currently in an incompatible state transition. For example, an air conditioner might be switching from off to on and cannot immediately respond to a new off command.

[0053] When the system detects a conflict, the conflict identification module generates a conflict status input containing the following: The first part is the user's control intent related to the conflict, such as user A wanting to turn on the light and user B wanting to turn it off. The second part is the current environmental status information, reflecting the real-time environment through data collected by sensors, including time (e.g., 14:30), light intensity (e.g., 50 lux), noise level (e.g., 40 dB), and temperature (e.g., 25°C). The third part is the current status of the target device, such as the light fixture's on / off status (off), current brightness setting (0%), and location (living room).

[0054] For example, suppose user A says "Turn on the air conditioner" via voice at 14:30:01, and user B clicks "Turn off the air conditioner" via a mobile app at 14:30:03. The system detects that the two commands are directed at the same device (air conditioner), the operation types are mutually exclusive, the time difference is within a 5-second window, and the air conditioner is currently off. The conflict detection module will construct a conflict state input, recording the intentions of user A and user B, the current ambient temperature (26℃), and the air conditioner status (off).

[0055] like Figure 3 As shown, the control strategy generation module is the decision-making core of the system of this invention. It receives conflict state input and generates an overall control scheme through a model trained based on historical data. The design of this module is divided into two stages: model training and inference.

[0056] The model is trained using supervised learning, with training data derived from a sample set of historical conflict control scenarios. Each sample contains three parts: first, conflicting control commands issued by multiple users within the same time period, such as "turn on the light" and "turn off the light"; second, the corresponding environmental state information, such as light intensity and temperature; third, the device state information, such as the on / off status of the light; and fourth, the actual combination of device control executed by the system in that scenario, which represents the actual operation obtained through negotiation by users in the corresponding environment in the past, serving as a supervision label.

[0057] The first step in data processing is to perform structured encoding on the samples. User intent is encoded into multiple parts, including one-hot encoding of the user identifier (e.g., user A is [1,0], user B is [0,1]), encoding of the category of the controlled target device (0 for lights, 1 for air conditioners), one-hot encoding of the control operation type (on is [1,0], off is [0,1]), and time period classification encoding of the command issuance time (e.g., 14:30 is classified as the afternoon). Environmental conditions are numerically processed; for example, continuous variables such as light intensity of 50 lux and temperature of 25°C are normalized to x. normalized Mapped to the interval [0,1], the formula is:

[0058]

[0059] Where x is the original value, x min and x max These represent the historical minimum and maximum values ​​of the variable (e.g., the range of light intensity might be 0 to 1000 lux). This formula ensures that variables with different dimensions have the same scale, facilitating model processing. Device status is represented by enumerated codes, such as on / off status (0 for off, 1 for on) and setpoints (e.g., 50% brightness is encoded as 0.5).

[0060] All encoded results are concatenated into a fixed-length input vector, such as a vector containing 20 elements. The first 5 elements represent the user intent, the next 10 represent the environmental state, and the last 5 represent the device state. The output labels are in a multi-target format, with each target device corresponding to a control operation classification result (such as on, off, or unchanged).

[0061] During training, the system can choose either a Multilayer Perceptron (MLP) model or a tree model based on ensemble learning (such as a random forest). The objective is to minimize the difference between the predicted output and the supervision label. A commonly used loss function is the cross-entropy loss, whose formula is:

[0062]

[0063] Where N is the number of samples, y i For real labels, This formula predicts probabilities for the model. It measures the difference between the predicted distribution and the true distribution and is applicable to classification tasks. During training, parameters are optimized using gradient descent. The learning rate is typically between 0.001 and 0.01, and the number of iterations can be set from 100 to 500, depending on the data size.

[0064] In this invention system, the control strategy generation model adopts a multilayer perceptron structure. The input layer receives the aforementioned structured input vector, with a fixed length (e.g., 20 dimensions). The model contains at least two hidden layers, with each layer containing between 64 and 512 neurons, the specific number adjusted according to the number of devices and conflict complexity. For example, for controlling 5 devices, the hidden layer can be set to 256 neurons. The activation function chosen is ReLU (Rectified Linear Unit), whose expression is:

[0065] f(x) = max(0,x);

[0066] The advantage of ReLU lies in its ability to accelerate convergence and alleviate the vanishing gradient problem. Fully connected layers ensure sufficient information transfer. The output layer contains multiple groups of output nodes, each corresponding to a classification result of the control operation of a target device. For example, if controlling 3 devices, each with 3 operations (on, off, hold), the output layer would have 9 nodes. Cross-entropy loss is still used for supervised optimization during training.

[0067] In actual operation, conflict state inputs are fed into a trained model, and the model outputs the control operation for each device. For example, given conflict commands from user A and user B (turning the light on and off), light intensity (low), and light status (off), the model may predict, based on historical scenarios, that "turning the light on" is more consistent with the predicted state given the conflict and the corresponding environmental conditions.

[0068] The execution control module receives the control scheme output by the model and sends instructions to the target device via a device communication protocol (such as MQTT). For example, if the model decides to "turn on the light," the module sends the instruction to the light fixture's controller, and the light fixture immediately turns on. This module also needs to handle execution feedback to ensure that the instructions are implemented correctly.

[0069] This invention significantly improves the control efficiency of smart home devices in multi-user environments through cognitive perception and intent prediction technologies. Its flexible command acquisition methods, history-based intelligent decision-making, and efficient conflict resolution capabilities make it suitable for multi-user scenarios such as homes and offices. For example, in a multi-user conference room, the system can coordinate the control needs of different users for lighting and air conditioning, ensuring both environmental comfort and energy efficiency.

[0070] The above description is only a preferred embodiment of the present invention, but the scope of protection of the present invention is not limited thereto. Any equivalent substitutions or modifications made by those skilled in the art within the scope of the technology disclosed in the present invention, based on the technical solution and inventive concept of the present invention, should be covered within the scope of protection of the present invention.

Claims

1. A smart home decision-making system based on cognitive perception and intention prediction, characterized in that, include: The command acquisition module is used to collect control commands issued by multiple users to smart home devices within the same time period; The conflict identification module is used to detect whether there is a control conflict between two or more control commands, and to construct a conflict state input when a control conflict exists; The control strategy generation module is used to input the conflict state into a control strategy generation model trained based on historical control data; The execution control module is used to receive the overall control scheme output by the control strategy generation model and control multiple target devices to operate according to the control scheme. The control strategy generation model is a model trained through supervised learning. The training process uses historical conflict states and corresponding equipment control combinations as sample pairs. The training process of the control strategy generation model includes: Collect a data sample set containing historical conflict control scenarios. Each sample includes: conflicting control commands issued by two or more users within the same time period, environmental status information and equipment status information corresponding to the control commands, and the equipment control combination actually executed by the users after negotiation in the historical data under the conflict scenario, which serves as a supervision label. The data samples are structured and encoded to transform user intent, environmental state, and device state into input vectors that the model can process, and the device control combination is transformed into multi-objective output labels. Training is performed using a multilayer perceptron model or a tree model based on ensemble learning to minimize the discrepancy between the predicted output and the supervised label.

2. The smart home decision-making system based on cognitive perception and intention prediction according to claim 1, characterized in that, The control command includes the user identifier, the target device to be controlled, the type of control operation, and the time when the command is issued.

3. The smart home decision-making system based on cognitive perception and intention prediction according to claim 1, characterized in that, The conflict status includes the following information: User control intent related to the conflict, current environmental status information, and current status information of each target device.

4. The smart home decision-making system based on cognitive perception and intention prediction according to claim 1, characterized in that, The control strategy generation model is a multilayer perceptron model, including: An input layer is used to receive a structured conflict state input vector, which is composed of user intent encoding, environment state encoding and device state encoding concatenated together; It has at least two hidden layers, each containing 64 to 512 neurons, with ReLU activation function and fully connected layers; An output layer contains multiple output node groups corresponding to the number of required control devices, and each output node group corresponds to the control operation classification result of a target device. The multilayer perceptron model is trained under supervised supervision using the cross-entropy loss function.

5. The smart home decision-making system based on cognitive perception and intention prediction according to claim 1, characterized in that, The process of transforming user intent, environmental state, and device state into input vectors that the model can process includes: Each user's control intent is encoded, and the encoding includes a unique hot code for the user identifier, a category code for the target device being controlled, a unique hot code for the type of control operation, and a time period classification code for the time when the intent is issued. The current environmental state is numerically processed, including time, light intensity, noise level, and temperature sensor data, and continuous variables are normalized. The current device status is enumerated and encoded, including the device on / off status, current set value, and position status; The encoded results are concatenated to form a fixed-length structured input vector, which is used as the input to the control strategy generation model.

6. The smart home decision-making system based on cognitive perception and intention prediction according to claim 1, characterized in that, The instruction acquisition module includes: The speech recognition submodule is used to receive the user's natural language commands through a microphone array and convert them into structured control commands; The touch input submodule is used to receive operation input from mobile terminals or wall control panels.

7. The smart home decision-making system based on cognitive perception and intention prediction according to claim 6, characterized in that, The instruction acquisition module also includes: The timestamp synchronization submodule is used to perform time-series normalization processing on multiple user commands received within the same time period.

8. The smart home decision-making system based on cognitive perception and intention prediction according to claim 1, characterized in that, The conflict identification module uses the following criteria to determine whether two or more control commands conflict: Whether the control command applies to the same device or the same group of devices; Control whether operation types are mutually exclusive; Whether control commands occur simultaneously within a set time window, wherein the time window is less than a preset threshold; Control whether the target device is currently in an incompatible state transition.

Images