Smart home decision-making system based on cognitive perception and intention prediction

Abstract

The invention discloses a smart home decision-making system based on cognitive perception and intention prediction, and relates to the field of smart home control, and the system comprises an instruction collection module which is used for collecting control instructions sent by a plurality of users to smart home equipment in the same time period; the conflict identification module is used for detecting whether a control conflict between two or more control instructions exists or not, and constructing conflict state input when the control conflict exists; the control strategy generation module is used for inputting the conflict state into a control strategy generation model obtained based on historical control data training; and the execution control module is used for receiving the overall control scheme output by the control strategy generation model and controlling a plurality of target devices to operate according to the control scheme. According to the invention, the problem of equipment control under the condition that multiple users simultaneously send out control instructions and control conflicts exist is solved.

Description

Technical Field

[0001] The present invention relates to the field of smart home control, and more specifically, to a smart home decision-making system based on cognitive perception and intention prediction. Background Art

[0002] With the prevalence of smart home systems, it's increasingly common for users to issue control commands simultaneously within the same home space. Especially in multi-user shared environments, different users may issue conflicting control intentions for the same device or associated devices within the same timeframe, for example, one user wanting to turn off the lights for a restful night, while another wants to turn them on for reading. These conflicts arise not only from overlapping operational objectives but also from differences in user roles, current cognitive states (such as fatigue and stress), situational urgency, and subjective preferences for the environment. Existing systems typically handle conflicts with simplistic approaches such as "latest command first" or "fixed primary user first," lacking the ability to perceive, understand, and mediate the nature of the conflict.

[0003] In practice, the optimal resolution strategy for the same conflicting command combination may differ significantly depending on the user, time, or environment. For example, a child's request to turn off the lights at night may take precedence over their father's request to watch a movie, while the opposite may be true in the early morning or at a social gathering. Therefore, existing technologies struggle to dynamically determine the appropriate coordinated control solution based on the conflict context, nor can they identify the optimal strategy from historical behavior. A smart home decision-making system with conflict identification, classification, modeling, and strategy generation capabilities is urgently needed to improve intelligent response and user experience in multi-user environments. Summary of the Invention

[0004] The technical problem to be solved by the present invention is to provide a smart home decision-making system based on cognitive perception and intention prediction to solve the problems mentioned in the background technology.

[0005] In order to achieve the above object, the present invention adopts the following technical solutions:

[0006] A smart home decision-making system based on cognitive perception and intention prediction, comprising:

[0007] The command collection module is used to collect control commands issued by multiple users to smart home devices in the same time period;

[0008] a conflict identification module for detecting whether there is a control conflict between two or more control instructions and constructing a conflict state input when a control conflict exists;

[0009] A control strategy generation module, configured to input the conflict state into a control strategy generation model trained based on historical control data;

[0010] The execution control module is used to receive the overall control plan output by the control strategy generation model and control multiple target devices to operate according to the control plan.

[0011] In some embodiments, the control instruction includes a user identifier, a control target device, a control operation type, and the time when the instruction is issued.

[0012] In some embodiments, the conflict status includes the following information:

[0013] The user control intention related to the conflict, the current environment status information, and the current status information of each target device.

[0014] In some embodiments, the control strategy generation model is a model trained by supervised learning, wherein the training process uses historical conflict states and corresponding device control combinations as sample pairs.

[0015] In some embodiments, the training process of the control strategy generation model includes:

[0016] Collect a data sample set containing historical conflict control situations. Each sample includes: conflicting control instructions issued by two or more users in the same time period, the environmental status information and device status information corresponding to the control instructions, and the device control combination actually executed after user negotiation in the historical data of the conflict situation as a supervision label;

[0017] Performing structured encoding on the data samples, converting user intent, environment state, and device state into input vectors that can be processed by the model, and converting device control combinations into multi-target output labels;

[0018] Train using a multi-layer perceptron model or a tree model based on ensemble learning to minimize the difference between the predicted output and the supervision label.

[0019] In some embodiments, the control strategy generation model is a multi-layer perceptron model, including:

[0020] An input layer, configured to receive a structured conflict state input vector, the input vector comprising a concatenation of a user intent code, an environment state code, and a device state code;

[0021] At least two hidden layers, each containing 64 to 512 neurons, with ReLU activation function and full connections between layers;

[0022] an output layer comprising a plurality of output node groups corresponding to the number of required control devices, each output node group corresponding to a control operation classification result of a target device;

[0023] The multi-layer perceptron model uses a cross entropy loss function for supervised training.

[0024] In some embodiments, the process of converting user intent, environment state, and device state into an input vector that can be processed by the model includes:

[0025] Encode each user's control intention, including a one-hot code of the user identifier, a category code of the control target device, a one-hot code of the control operation type, and a time period classification code of the time when the intention was issued;

[0026] Numerically processing the current environmental state, including time, light intensity, noise level, and temperature sensor data, and normalizing continuous variables;

[0027] Enumerate and encode the current device status, including the device switch status, current setting value, and position status;

[0028] The encoding results are concatenated to form a structured input vector of fixed length, which serves as the input of the control strategy generation model.

[0029] In some embodiments, the instruction collection module includes:

[0030] A speech recognition submodule is used to receive the user's natural language instructions through the microphone array and convert them into structured control instructions;

[0031] The touch input submodule is used to receive operation input from a mobile terminal or a wall control panel.

[0032] In some embodiments, the instruction collection module further includes:

[0033] The timestamp synchronization submodule is used to perform timing normalization processing on multiple user instructions received in the same time period.

[0034] In some embodiments, the basis for the conflict identification module to determine whether two or more control instructions conflict includes:

[0035] Whether the control command acts on the same device or the same group of devices;

[0036] Control whether operation types are mutually exclusive;

[0037] Whether the control instructions occur simultaneously within a set time window, and the time window is less than a preset threshold;

[0038] Controls whether the target device is currently in an incompatible state transition.

[0039] The advantages of this invention over existing technologies lie in its provision of a smart home decision-making system based on cognitive perception and intention prediction. This system can accurately identify conflict relationships and construct structured conflict states when multiple users issue control commands simultaneously and control conflicts exist. Furthermore, it generates the most appropriate device control combination for the current scenario based on a policy model trained with historical control data. Compared to traditional methods that prioritize commands based on command sequence or primary user priority, this invention comprehensively considers user role differences, the current environmental state, and the operating status of devices, dynamically generating more coordinated and rational control responses. BRIEF DESCRIPTION OF THE DRAWINGS

[0040] Figure 1 It is an overall schematic diagram of the system of the present invention;

[0041] Figure 2 It is a schematic diagram of conflict identification of the present invention;

[0042] Figure 3 Schematic diagram of control strategy generation of the present invention. DETAILED DESCRIPTION

[0043] The specific embodiments of the present invention will be described below with reference to the accompanying drawings.

[0044] In the smart home sector, device control in multi-user scenarios often faces the challenges of complexity and conflict. To address this issue, this paper proposes a smart home decision-making system based on cognitive perception and intent prediction. By collecting user commands, identifying conflicts, generating control strategies, and executing device controls, this system enables efficient management and coordination of smart home devices in multi-user environments. The following examples will describe in detail the architecture, operating principles, and specific implementation plans of the system, covering the functional implementation of each system module, model training, and data processing.

[0045] like Figure 1 As shown, the system of the present invention aims to solve the problem of intelligently determining and executing the optimal control solution when multiple users issue conflicting control commands to smart home devices within the same time period. The system consists of four core modules: a command acquisition module, a conflict identification module, a control strategy generation module, and an execution control module. These modules work together to form a complete closed-loop decision-making process, from command acquisition to final device control.

[0046] The command acquisition module is responsible for acquiring control commands issued by multiple users in real time through various means. These commands may originate from voice input, touch operations via mobile apps, or manual input via wall-mounted control panels. To achieve this function, the command acquisition module is divided into three submodules, each with its own specific function, ensuring accurate command acquisition and timely processing.

[0047] The first submodule is the speech recognition submodule. It receives natural language commands issued by the user, such as "turn on the bedroom light", through the built-in microphone array. The received voice signal is processed by natural language processing (NLP) technology and converted into structured control instructions. Specifically, the system uses a pre-trained speech-to-text model (such as a deep learning-based speech recognition algorithm) to parse the user's voice content and extract key information, including the identity of the user issuing the command, the target device (such as the bedroom light), the type of operation (such as turn on), and the specific time when the command was issued (accurate to seconds). In order to improve recognition accuracy, the microphone array may use beamforming technology to enhance the target sound signal and suppress background noise.

[0048] The second submodule is the touch input submodule, which is responsible for processing operational input from mobile devices (such as smartphones or tablets) and wall control panels. For example, if a user clicks the "turn off the air conditioner" button on a mobile app or slides on a wall panel to adjust the light brightness, these actions are recorded as control commands. The touch input submodule obtains user operation data through communication protocols with the device (such as Wi-Fi or Zigbee) and converts it into a structured format consistent with voice commands.

[0049] The third submodule is the timestamp synchronization submodule. Since instructions may be issued intensively in a short period of time in a multi-user scenario, the system needs to ensure that the timing information of the instructions is accurate. This submodule assigns a precise timestamp to each instruction and performs timing normalization on multiple instructions within the same time period (for example, a 5-second window). The normalization process involves converting the timestamp into a relative time offset (such as the difference in seconds based on the earliest instruction) so that subsequent modules can determine the order or simultaneity of the instructions. For example, user A issues the "turn on the TV" command at 14:30:02, and user B issues the "turn off the TV" command at 14:30:04. The timestamp synchronization submodule will record the time difference between the two as 2 seconds and mark them as belonging to the same time period.

[0050] Each control command contains the following information: user identification (e.g., unique IDs for users A and B), target device (e.g., TV, light), control operation type (e.g., on, off, adjust), and command issuance time. This information provides foundational data for subsequent conflict identification and policy generation.

[0051] like Figure 2 As shown in Figure 2, the conflict identification module is responsible for analyzing the collected control instructions, determining whether there is a conflict, and constructing a structured conflict status input when a conflict is detected. The implementation of this module relies on clear conflict judgment rules and comprehensive status information collection.

[0052] The judgment of conflicts is based on the following criteria. First, check whether the control instructions act on the same device or the same device group. For example, the lamps in the living room may be defined as a device group. If user A instructs "turn on the living room lights" and user B instructs "turn off the living room lights", then both are targeting the same target. Secondly, analyze whether the control operation types are mutually exclusive, such as "turn on" and "turn off" are typical mutually exclusive operations and cannot be executed at the same time. Thirdly, determine whether the instructions occur simultaneously within the set time window. The time window is usually set to a smaller threshold (such as 3 to 5 seconds) to capture high-frequency instructions in a short period of time. Finally, check whether the current state of the target device is in an incompatible state transition. For example, the air conditioner is in the process of switching from off to on and cannot immediately respond to the new off instruction.

[0053] When the system detects a conflict, the conflict identification module generates a conflict state input, which contains the following content. The first part is the user control intention related to the conflict, for example, user A wants to turn on the light, and user B wants to turn off the light. The second part is the current environmental state information, which reflects the real-time environment through data collected by sensors, including time (such as 14:30), light intensity (such as 50 lux), noise level (such as 40 decibels) and temperature (such as 25°C). The third part is the current state of the target device, such as the switch state of the lamp (off), the current brightness setting value (0%) and the location state (living room).

[0054] For example, suppose user A says "Turn on the air conditioner" via voice at 2:30:01 PM, and user B clicks "Turn off the air conditioner" via the mobile app at 2:30:03 PM. The system detects that the two commands target the same device (the air conditioner), their operation types are mutually exclusive, the time difference is within a 5-second window, and the air conditioner is currently off. The conflict recognition module constructs a conflict state input, recording the intentions of user A and user B, the current ambient temperature (26°C), and the air conditioner status (off).

[0055] like Figure 3 As shown, the control strategy generation module is the decision-making core of the system of the present invention. It receives conflict state input and generates the overall control plan through a model trained based on historical data. The design of this module is divided into two stages: model training and reasoning.

[0056] The model is trained using supervised learning, with training data derived from historical examples of conflicting control situations. Each example contains three components: first, conflicting control commands issued by multiple users within the same time period, such as "turn on the light" and "turn off the light"; second, corresponding environmental state information, such as light intensity and temperature; third, device state information, such as whether a light was on or off; and fourth, the actual device control combination executed by the system in that situation. This combination, representing the actual actions achieved through historical negotiation by users in the corresponding environment, serves as a supervisory label.

[0057] The first step in data processing is to perform structured encoding on the samples. User intentions are encoded into multiple parts, including the unique hot encoding of the user identifier (for example, user A is [1,0], user B is [0,1]), the category encoding of the target device to be controlled (light is 0, air conditioner is 1), the unique hot encoding of the control operation type (open is [1,0], close is [0,1]), and the time period classification encoding of the instruction issuance time (for example, 14:30 is classified as the afternoon period). The environmental status is digitized, for example, continuous variables such as light intensity 50 lux and temperature 25℃ are normalized to x normalized , mapped to the interval [0,1], the formula is:

[0058]

[0059] Among them, x is the original value, x min and x max where is the historical minimum and maximum value of the variable (for example, light intensity might range from 0 to 1000 lux). This formula ensures that variables of different dimensions have the same scale, making them easier to process in the model. Device states are represented by enumerations, such as switch states (0 for off, 1 for on) and setpoints (for example, 50% brightness is encoded as 0.5).

[0060] All encoding results are concatenated into a fixed-length input vector, for example, a 20-element vector, where the first 5 elements represent the user's intent, the next 10 represent the environment state, and the last 5 represent the device state. The output labels are multi-target, with each target device corresponding to a control action classification result (e.g., open, close, or remain unchanged).

[0061] During training, the system can choose a multi-layer perceptron (MLP) model or a tree model based on ensemble learning (such as random forest). The goal is to minimize the difference between the predicted output and the supervised label. The commonly used loss function is cross entropy loss, which is formulated as:

[0062]

[0063] Where N is the number of samples, y i is the true label, = is the model's predicted probability. This formula measures the difference between the predicted distribution and the true distribution and is suitable for classification tasks. During training, parameter adjustment is optimized using gradient descent. The learning rate typically ranges from 0.001 to 0.01, and the number of iterations can be set from 100 to 500, depending on the data size.

[0064] In the system of the present invention, the control strategy generation model adopts a multi-layer perceptron structure. The input layer receives the above-mentioned structured input vector with a fixed length (e.g., 20 dimensions). The model contains at least two hidden layers, with the number of neurons in each layer ranging from 64 to 512, and the specific number is adjusted according to the number of devices and the complexity of the conflict. For example, for the control of 5 devices, the hidden layer can be set to 256 neurons. The activation function is ReLU (Rectified Linear Unit), which is expressed as:

[0065] f(x)=max(0,x);

[0066] The advantages of ReLU are accelerated convergence and mitigation of the vanishing gradient problem. Fully connected layers ensure sufficient information transfer. The output layer contains multiple groups of output nodes, each corresponding to the classification results of the control operation of a target device. For example, if three devices are controlled, each with three operations (on, off, hold), the output layer would have nine nodes. The cross-entropy loss function is still used for supervised optimization during training.

[0067] In actual operation, conflicting state inputs are fed into a trained model, which then outputs control actions for each device. For example, given conflicting commands from user A and user B (turning the light on and off), along with light intensity (low) and light state (off), the model might predict, based on historical scenarios, that "turning the light on" is more consistent with the predicted state given the conflict and the corresponding environmental conditions.

[0068] The execution control module receives the control plan output by the model and sends the command to the target device via a device communication protocol such as MQTT. For example, if the model determines to "turn on the light," the module sends the command to the light's controller, which then illuminates the light. This module also processes execution feedback to ensure the correct execution of the command.

[0069] The system, utilizing cognitive perception and intent prediction technologies, significantly improves the control efficiency of smart home devices in multi-user environments. Its flexible command acquisition, history-based intelligent decision-making, and efficient conflict resolution make it suitable for multi-user scenarios such as homes and offices. For example, in a multi-user conference room, the system can coordinate lighting and air conditioning control needs of different users, ensuring a comfortable environment while balancing energy savings.

[0070] The above description is only a preferred specific embodiment of the present invention, but the scope of protection of the present invention is not limited thereto. Any technician familiar with the technical field, within the technical scope disclosed by the present invention, who makes equivalent replacements or changes based on the technical solution and inventive concept of the present invention, should be covered by the scope of protection of the present invention.

Claims

1. A smart home decision-making system based on cognitive perception and intention prediction, characterized by: include: The command collection module is used to collect control commands issued by multiple users to smart home devices in the same time period; a conflict identification module for detecting whether there is a control conflict between two or more control instructions and constructing a conflict state input when a control conflict exists; A control strategy generation module, configured to input the conflict state into a control strategy generation model trained based on historical control data; The execution control module is used to receive the overall control plan output by the control strategy generation model and control multiple target devices to operate according to the control plan.

2. The smart home decision-making system based on cognitive perception and intention prediction according to claim 1 is characterized in that: The control instruction includes a user identifier, a control target device, a control operation type, and an instruction issuing time.

3. The smart home decision-making system based on cognitive perception and intention prediction according to claim 1 is characterized in that: The conflict status includes the following information: The user control intention related to the conflict, the current environment status information, and the current status information of each target device.

4. The smart home decision-making system based on cognitive perception and intention prediction according to claim 1 is characterized in that: The control strategy generation model is a model obtained by training in a supervised learning manner, wherein the training process uses historical conflict states and corresponding device control combinations as sample pairs.

5. The smart home decision-making system based on cognitive perception and intention prediction according to claim 4 is characterized in that: The training process of the control strategy generation model includes: Collect a data sample set containing historical conflict control situations. Each sample includes: conflicting control instructions issued by two or more users in the same time period, the environmental status information and device status information corresponding to the control instructions, and the device control combination actually executed after user negotiation in the historical data of the conflict situation as a supervision label; Performing structured encoding on the data samples, converting user intent, environment state, and device state into input vectors that can be processed by the model, and converting device control combinations into multi-target output labels; Train using a multi-layer perceptron model or a tree model based on ensemble learning to minimize the difference between the predicted output and the supervision label.

6. The smart home decision-making system based on cognitive perception and intention prediction according to claim 5 is characterized in that: The control strategy generation model is a multi-layer perceptron model, including: An input layer, configured to receive a structured conflict state input vector, the input vector comprising a concatenation of a user intent code, an environment state code, and a device state code; At least two hidden layers, each containing 64 to 512 neurons, with ReLU activation function and full connections between layers; an output layer comprising a plurality of output node groups corresponding to the number of required control devices, each output node group corresponding to a control operation classification result of a target device; The multi-layer perceptron model uses a cross entropy loss function for supervised training.

7. The smart home decision-making system based on cognitive perception and intention prediction according to claim 5 is characterized in that: The process of converting user intent, environment state, and device state into input vectors that the model can process includes: Encode each user's control intention, including a one-hot code of the user identifier, a category code of the control target device, a one-hot code of the control operation type, and a time period classification code of the time when the intention was issued; Numerically processing the current environmental state, including time, light intensity, noise level, and temperature sensor data, and normalizing continuous variables; Enumerate and encode the current device status, including the device switch status, current setting value, and position status; The encoding results are concatenated to form a structured input vector of fixed length, which serves as the input of the control strategy generation model.

8. The smart home decision-making system based on cognitive perception and intention prediction according to claim 1 is characterized in that: The instruction acquisition module includes: A speech recognition submodule is used to receive the user's natural language instructions through the microphone array and convert them into structured control instructions; The touch input submodule is used to receive operation input from a mobile terminal or a wall control panel.

9. The smart home decision-making system based on cognitive perception and intention prediction according to claim 8, characterized in that: The instruction acquisition module also includes: The timestamp synchronization submodule is used to perform timing normalization processing on multiple user instructions received in the same time period.

10. The smart home decision-making system based on cognitive perception and intention prediction according to claim 1, characterized in that: The basis for the conflict identification module to determine whether two or more control instructions conflict includes: Whether the control command acts on the same device or the same group of devices; Control whether operation types are mutually exclusive; Whether the control instructions occur simultaneously within a set time window, and the time window is less than a preset threshold; Controls whether the target device is currently in an incompatible state transition.

Images