Smart home control system of single-chip microcomputer based on 5G Internet of Things

Abstract

The invention provides a smart home control system of a single-chip microcomputer based on the 5G Internet of Things, and the system comprises a sensor module which collects environment data, a data processing module which integrates the data, and a deep Q network control module which is internally provided with a DQN algorithm model, learns and trains home environment data and user behavior data, generates a control strategy, and achieves the intelligent control of home equipment. According to the invention, the automation and intelligence level of the home environment can be effectively improved, and efficient, energy-saving and user behavior self-adaptive home equipment management is realized.

Description

Technical Field

[0001] The present invention relates to the field of smart home technology, and more specifically, to a smart home control system. Background Art

[0002] As an important development direction of modern home life, smart home systems aim to improve the comfort, safety and energy efficiency of living environments through technical means. Existing smart home systems mostly use sensors to collect environmental data and combine simple control logic or preset rules to control home appliances. Although these systems can achieve automatic control to a certain extent, they often lack the ability to learn and adapt to user behavior habits, resulting in inflexible and individuated control strategies. In addition, due to environmental changes and the diversity of user needs, static control strategies are difficult to meet the needs of dynamic changes, which limits the intelligence of the system.

[0003] In the process of implementing the embodiments of the present invention, the inventors found that there are at least the following problems or defects in the prior art: the existing smart home control system is often unable to effectively integrate and process a large amount of environmental data, lacks the ability to deeply learn and predict user behavior, and lacks the ability to optimize control strategies in a dynamic environment. These problems limit the performance and user experience of the smart home system. Summary of the invention

[0004] The present invention provides a smart home control system, comprising: The sensor module is used to collect home environment data, including temperature, humidity, and light sensor units, and transmit the collected home environment data to the data processing module; The data processing module integrates and denoises the home environment data collected by the sensor module, and converts the processed home environment data into the format required for the input of the deep Q network control module. The data processing module is also used to coordinate the data interaction between modules; The deep Q network control module receives data from the data processing module and generates corresponding control strategies through learning and training of home environment data and pre-collected user behavior data. The control strategies include control actions to achieve intelligent control of home appliances.

[0005] As a further improvement of the present application, the deep Q network control module has two built-in DQN algorithm models, each of which includes: The input layer is used to receive the normalized home environment status data; Multiple hidden layers, each hidden layer consists of multiple neurons, and the output of each neuron is determined by the ReLU activation function; The output layer is used to output the expected reward value of each control action on the home device.

[0006] As a further improvement of the present application, the training process of the deep Q network control module includes the following steps: Initialize the Q network parameters of the DQN algorithm model; Get the current smart home status from home environment data; Select a control action based on the current smart home state and the Q network parameters of the DQN algorithm model; Execute control actions and observe the resulting next smart home state and immediate rewards; Temporal difference learning updates the DQN algorithm model parameters, the formula is: ] in, is at the time step The state of smart homes and control actions The corresponding Q value is used to represent the expected utility of the state-action pair; It is the immediate reward obtained after performing a control action; is the discount factor used to adjust the current value of future rewards; is the learning rate; Is the next smart furniture state The maximum Q value of all control actions in; Regularly update the Q network parameters of the DQN algorithm model.

[0007] As a further improvement of the present application, the deep Q network control module includes an experience playback unit, and the experience playback unit performs the following steps: Storing each experience tuple in an experience playback buffer, the experience tuple including a current smart home state, a control action, an immediate reward, and a next smart home state; Randomly extract experience tuples from the experience replay buffer for training; The target Q value is calculated using the following formula:

[0008] in, is the target Q value used to update the current Q value, It is an instant reward. is the discount factor, It is the Q value prediction of the target DQN algorithm model for the next smart home state and control action.

[0009] As a further improvement of the present application, among the two DQN algorithm models, one DQN algorithm model is used to select a control action, and the other DQN algorithm model is used to calculate a target Q value.

[0010] As a further improvement of the present application, the deep Q network control module includes a control strategy generation unit, which performs the following steps as an intelligent agent: Initialize the DQN algorithm model parameters, set the number of input layer neurons to match the dimensions of the home environment status data collected by the sensor, determine the number of hidden layers and the number of neurons in each hidden layer based on the system complexity and training effect, and the number of output layer neurons corresponds to the number of controllable home device actions; As an agent, it enters a loop operation until the preset termination conditions are met, including: Based on the current smart home state, a control action is selected according to the ε-greedy strategy; The smart home environment updates its state based on the control actions selected by the agent; The agent receives immediate rewards from the new smart home state; The agent takes the new smart home state and immediate reward as input, inputs them into the DQN algorithm model, and obtains the new Q value through forward propagation.

[0011] As a further improvement of the present application, the deep Q network control module further includes an execution control unit, which performs the following steps based on the control action instructions received from the DQN algorithm model: The relay control subunit is used to control the power on and off of the home appliance according to the control action instructions output by the DQN algorithm model; The motor drive control subunit is used to drive the home appliances according to the control action instructions output by the DQN algorithm model.

[0012] As a further improvement of the present application, the deep Q network control module further includes a communication control unit, which performs the following steps: The 5G communication control subunit uses the n78 frequency band for data transmission, supports high-speed data upload and download, and is used to achieve real-time data transmission of the smart home system; The Bluetooth communication control subunit is used to achieve pairing connection with short-range smart devices.

[0013] As a further improvement of the present application, the deep Q network control module further includes a user interaction control unit, including: A mobile phone application control subunit is used to receive and display data collected by the sensor module, and remotely control home appliances through the mobile phone application control subunit; The intelligent control panel control subunit is used to receive and display the data collected by the sensor module and receive control instructions from the user.

[0014] As a further improvement of the present application, the deep Q network control module further includes a policy optimization unit, including: Automatically adjust control strategies based on user behavior and environmental changes; Using the policy iteration method, including policy evaluation and policy optimization, the policy evaluation uses the following formula:

[0015] in, Smart state and control actions The corresponding Q value is, Indicates expected value; Indicates the smart home status Next action Then transfer to smart home state probability; Indicates cumulative rewards, is the discount factor; Strategy optimization uses the following formula:

[0016] in, Is in the smart furniture state Select Control Action The probability of is an exponential function used to adjust the sensitivity of action selection. is a temperature parameter. The above-mentioned embodiments of the present invention have at least the following beneficial effects: the smart home control system can perform deep learning and training on home environment data and user behavior data by integrating a deep Q network control module to generate a more accurate and personalized control strategy. This intelligent control mechanism can significantly improve the response speed and control accuracy of home appliances, and achieve a more energy-saving and comfortable living environment. At the same time, the system's experience playback mechanism and dual DQN mechanism can improve learning efficiency and decision-making quality, so that the system can quickly adapt and optimize control strategies when facing a complex and changeable home environment, thereby enhancing the robustness and adaptability of the system.

[0017] In addition, the system can ensure high-speed and stable connection with smart devices through the combined use of 5G communication units and Bluetooth communication units, and realize seamless collaboration of remote and near-field devices. The introduction of user interaction modules, such as mobile phone applications and smart control panels, enables users to intuitively monitor the status of the home environment and conveniently perform control operations, thereby improving the user's operational convenience and the user-friendliness of the system. These beneficial effects work together to provide users with a more intelligent, convenient and comfortable smart home experience. BRIEF DESCRIPTION OF THE DRAWINGS

[0018] The above and other objects, features and advantages of the exemplary embodiments of the present invention will become readily understood by reading the detailed description below with reference to the accompanying drawings. In the accompanying drawings, several embodiments of the present invention are shown in an exemplary and non-limiting manner, in which: Figure 1 A schematic diagram of the structure of a smart home control system provided by one embodiment of the present invention. DETAILED DESCRIPTION

[0019] The principles and spirit of the present invention will be described below with reference to several exemplary embodiments. It should be understood that these embodiments are provided only to enable those skilled in the art to better understand and implement the present invention, and are not intended to limit the scope of the present invention in any way. On the contrary, these embodiments are provided to make the present invention more thorough and complete, and to fully convey the scope of the present invention to those skilled in the art.

[0020] Those skilled in the art know that the embodiments of the present invention can be implemented as a system, device, apparatus, method or computer program product. Therefore, the present invention can be specifically implemented in the following forms, namely: complete hardware, complete software (including firmware, resident software, microcode, etc.), or a combination of hardware and software.

[0021] It should be noted that any number of elements in the drawings is for illustration rather than limitation, and any naming is only for distinction and does not have any limiting meaning.

[0022] Reference below Figure 1 , Figure 1 This is a schematic diagram of the structure of a smart home control system based on a 5G Internet of Things single-chip microcomputer provided by an embodiment of the present invention. Figure 1 As shown, a smart home control system 100 includes: The sensor module 101 is used to collect home environment data, including temperature, humidity, light and other sensor units, and transmit the collected physical quantity data to the data processing module; The data processing module 102 integrates and denoises the data collected by the sensor module and converts the processed data into a format suitable for input to the algorithm model. This module is also responsible for coordinating the data interaction between the modules. The deep Q network control module 103 has a built-in DQN algorithm model, receives data from the data processing module, and generates corresponding control strategies through learning and training of home environment data and user behavior data to achieve intelligent control of home appliances.

[0023] It should be noted that the smart home control system includes a sensor module, a data processing module and a deep Q network control module. The sensor module is responsible for collecting home environment data, such as temperature, humidity, light, etc. These data are the basis for the smart home system to make intelligent control decisions. The data processing module integrates and denoises the collected data to ensure the accuracy and availability of the data. The deep Q network control module has a built-in DQN algorithm model, which generates corresponding control strategies by learning home environment data and user behavior data to achieve intelligent control of home appliances.

[0024] Specifically, the sensor module consists of multiple sensor units, each of which is responsible for collecting specific physical quantity data, such as temperature sensors collecting temperature data and humidity sensors collecting humidity data. These sensor units transmit the collected data to the data processing module. When integrating data, the data processing module uses a specific algorithm to denoise the data to improve data quality. The DQN algorithm model of the deep Q network control module will learn user preferences and environmental changes through training based on the input data to generate a control strategy.

[0025] Preferably, the sensor module may include but is not limited to temperature sensors, humidity sensors and light sensors, which can be selected and configured according to the specific needs of the home environment. The data processing module can use a filtering algorithm, such as Kalman filtering, to improve the stability and reliability of the data during denoising. The DQN algorithm model of the deep Q network control module can use an experience replay mechanism and a dual DQN mechanism during training to improve the efficiency and stability of learning. These mechanisms help the system to quickly adapt and optimize control strategies when facing a complex and changeable home environment.

[0026] In some embodiments, the deep Q network control module includes: The input layer is used to receive the normalized home environment status data; Multiple hidden layers, each hidden layer consists of multiple neurons, and the output of each neuron is determined by an activation function; The output layer is used to output the expected reward value for each possible action.

[0027] It should be noted that the deep Q network control module includes an input layer, multiple hidden layers, and an output layer. The function of the input layer is to receive the normalized home environment status data, that is, to convert the original data into a unified scale so that the algorithm model can process it more effectively. The hidden layer consists of multiple neurons, and the output of each neuron is determined by the activation function. These layers are responsible for extracting complex features and patterns in the data. The output layer is used to output the expected reward value of each possible action, providing a basis for the action selection of smart home devices.

[0028] Specifically, the number of neurons in the input layer corresponds to the dimension of the home environment status data collected by the sensor. For example, if the sensor collects three parameters: temperature, humidity, and light, then the input layer will have three neurons. The number of hidden layers and the number of neurons in each hidden layer can be determined based on the complexity of the system and the training effect. Common activation functions include ReLU, Sigmoid, or Tanh. The number of neurons in the output layer is the same as the number of controllable home device actions. For example, if there are three different device actions, the output layer will have three neurons, each corresponding to the expected reward value of an action.

[0029] Preferably, the normalization of the input layer can be achieved by methods such as min-max scaling or Z-score normalization to ensure that the data is on the same scale. The specific configuration of the hidden layer, such as the number of neurons and connection weights, can be optimized by techniques such as cross-validation. For the activation function, different functions can be selected according to the different characteristics of the hidden layer to improve the expressiveness of the model. The expected reward value of the output layer can be adjusted by the policy network during training to reflect the long-term utility of different actions. In addition, it is possible to consider introducing regularization techniques, such as L1 or L2 regularization, to prevent the model from overfitting and improve the generalization ability of the model.

[0030] In some embodiments, the training process of the deep Q network control module includes the following steps: Initialize Q network parameters; Get the current state from the environment; Select an action based on the current state and the Q network; Perform actions, observe the next state and the reward obtained; Update the Q network using the following formula:

[0031] in, is at the time step Status and actions The corresponding Q value is used to represent the expected utility of the state-action pair; It is the immediate reward obtained after performing an action, which is used to feedback the effect of the action; is the discount factor used to adjust the current value of future rewards; is the learning rate, which controls how quickly new information updates old knowledge; is in the next state The maximum Q value of all possible actions guides the selection of the best action; The target Q network parameters are periodically updated as target Q network parameters.

[0032] It should be noted that the training process of the deep Q network control module includes initializing the Q network parameters, obtaining the current state from the environment, selecting an action based on the current state and the Q network, executing the action to observe the next state and the reward obtained, updating the Q network using a specific update formula, and periodically updating the target Q network parameters to the target Q network parameters. This process is the core mechanism for the deep Q network to learn how to make the best action based on the environmental state. The Q network parameters refer to the set of all weights and biases in the network, while the environmental state refers to the specific conditions of the current environment, such as temperature, humidity, etc.

[0033] Specifically, initializing the Q network parameters can be done by random initialization or using pre-trained weights. Obtaining the current state from the environment involves data acquisition by the sensor module and pre-processing by the data processing module. When selecting an action, an ε-greedy strategy can be adopted, that is, the current optimal action is selected with a certain probability, and the action is randomly selected with a certain probability to balance exploration and utilization. After executing the action, the system observes the next state of the environment and the reward obtained, and this information will be used to update the Q network. The formula for updating the Q network involves the immediate reward, discount factor, learning rate, and the maximum Q value of the next state. These parameters together determine how the Q value is updated.

[0034] Preferably, when initializing the Q network parameters, a specific initialization method, such as He initialization or Xavier initialization, can be selected to reduce the gradient problem in the early stage of training. The ε value in the ε-greedy strategy can be gradually reduced as the training progresses to gradually shift from exploration to exploitation. The discount factor (γ) can be set between 0 and 1 to reflect the current value of future rewards. The learning rate (α) can use an adaptive learning rate algorithm, such as the Adam optimizer, to dynamically adjust the learning rate. When updating the Q network, it is possible to consider introducing an experience replay mechanism to store and randomly extract experience tuples for training to improve sample efficiency and reduce the correlation between data samples. In addition, the update of the target Q network parameters can adopt a soft update strategy, that is, gradually bringing the parameters of the target network closer to the parameters of the current network to stabilize the training process.

[0035] In some embodiments, the deep Q network control module includes an experience playback mechanism, which includes the following steps: Store each experience tuple (state, action, reward, next state) into the experience replay buffer; Randomly extract a batch of experience tuples from the experience replay buffer for training; The target Q value is calculated using the following formula:

[0036] in, is the target Q value used to update the current Q value, It is an instant reward. is the discount factor, is the target Q network for the next state and actions Q-value prediction.

[0037] It should be noted that the experience replay mechanism in the deep Q network control module includes the steps of storing each experience tuple (state, action, reward, next state) in the experience replay buffer and randomly extracting a batch of experience tuples from the experience replay buffer for training. An experience tuple refers to a set of data generated during the interaction between the agent and the environment, which contains the current state, the action taken, the reward obtained, and the subsequent state. The experience replay buffer is a container for storing these tuples for subsequent training processes.

[0038] Specifically, the experience replay buffer can be an array of fixed size or a dynamic data structure, such as a queue, to store recent experience tuples. When the buffer is full, new experience tuples replace the oldest tuples. Randomly sampling a batch of experience tuples for training means that during the training process, instead of directly using the latest data, multiple tuples are randomly selected from the buffer. This can reduce the correlation between data and improve the stability of training. The formula for calculating the target Q value involves the immediate reward, the discount factor, and the target Q network's prediction of the Q value for the next state and action.

[0039] Preferably, the size of the experience replay buffer can be set according to the system's memory capacity and training requirements to balance storage efficiency and training effect. The number of randomly selected experience tuples is also an important parameter, which can be determined according to the batch size of training. When calculating the target Q value, the discount factor (γ) can be set between 0 and 1 to reflect the current value of future rewards.

[0040] Furthermore, a priority experience replay mechanism can be introduced to adjust the probability of the experience tuple being extracted according to its importance (such as the difference from the expected reward), so that important experience can be used for training more effectively. Batch normalization techniques can also be considered to stabilize the training process and accelerate convergence by reducing the offset of internal covariates.

[0041] In some embodiments, the deep Q network control module includes a dual DQN mechanism, comprising the following steps: Use two Q networks, one for selecting actions and the other for calculating target Q values; When calculating the target Q value, the following formula is used:

[0042] in, is the target Q value, It is an instant reward. is the discount factor, is in the next state The action that maximizes the Q value .

[0043] It should be noted that the dual DQN mechanism in the deep Q network control module involves the use of two Q networks, one for selecting actions and the other for calculating the target Q value. The core of this mechanism is to separate the process of action selection and target value evaluation to reduce the overestimation problem and improve the stability of learning. The dual in the dual DQN mechanism refers to the use of two different networks in the decision-making process, and DQN is the abbreviation of deep Q network, which is an algorithm that uses deep learning to learn strategies.

[0044] Specifically, when using the dual DQN mechanism, the first Q network (also called the behavior network) is responsible for selecting an action based on the current state. This action is based on the action with the largest current Q value. The second Q network (also called the target network) is used to evaluate the target Q value of this action, that is, to predict the maximum future reward that may be obtained after performing this action. When calculating the target Q value, the target network's prediction of the Q value of the next state is used instead of the prediction of the behavior network, which can reduce estimation bias. The key to this mechanism lies in the way the parameters of the two networks are updated: the behavior network is updated after each training, while the target network is updated at a slower rate to maintain stability.

[0045] Preferably, the two Q networks can share the same network structure, but remain independent during parameter update. The parameters of the behavior network can be updated using a gradient descent method, and the parameters of the target network can be gradually approached to the parameters of the behavior network by a soft update method. For example, at regular intervals, the parameters of the target network will approach the parameters of the behavior network by a certain proportion.

[0046] Furthermore, in order to further improve the efficiency and effectiveness of learning, some advanced technologies can be introduced, such as Double Experience Replay, which combines the experience of the behavior network and the target network to enhance the diversity and robustness of learning. You can also consider using some regularization techniques, such as dropout or weight decay, to prevent overfitting and improve the generalization ability of the model.

[0047] In some embodiments, the specific steps of generating a control strategy by the deep Q network control module include: Initialize the neural network parameters, set the number of input layer neurons to correspond to the dimensions of the home environment status data collected by the sensor, the number of hidden layers and the number of neurons in each hidden layer are determined based on the system complexity and training effect, and the number of output layer neurons is the same as the number of controllable home device actions; Enter the loop operation until the preset termination conditions are met, including: The agent selects an action based on the current state according to the ε greedy strategy; The environment updates its state based on the action chosen by the agent; The agent receives a reward from the new state; The agent takes the new state and reward as input, inputs it into the neural network, and obtains the new Q value through forward propagation; The agent updates the Q-value function using the following formula:

[0048] in, is at the time step Status and actions The corresponding Q value is, is the immediate reward obtained after performing an action, is the discount factor, is the learning rate, is in the next state The maximum Q value of all possible actions in ; The target Q value is calculated using the following formula:

[0049] in, is the target Q value, It is an instant reward. is the discount factor, is the target network's next state and actions Q-value prediction.

[0050] It should be noted that the specific steps of the deep Q network control module to generate the control strategy include initializing the neural network parameters, setting the number of input layer neurons to correspond to the dimensions of the home environment status data collected by the sensor, the number of hidden layers and the number of neurons in each hidden layer are determined based on the system complexity and training effect, and the number of output layer neurons is the same as the number of controllable home appliance actions. What is involved here is the design of the neural network structure in deep learning, in which the input layer, hidden layer and output layer are the basic units that constitute the neural network, and the number of neurons determines the complexity and processing power of the network.

[0051] Specifically, initializing the neural network parameters usually involves assigning initial values ​​to the weights and biases in the network, which can be random or based on some pre-training strategy. The number of neurons in the input layer directly corresponds to the data dimension collected by the sensor module. For example, if the sensor collects three environmental parameters, the input layer will have three neurons. The number of hidden layers and the number of neurons can be adjusted according to the specific needs of the system and experimental results to achieve the best training effect. The number of neurons in the output layer corresponds to the number of actions of the home device, and each neuron represents a possible action.

[0052] Preferably, when initializing the neural network parameters, methods such as Xavier initialization or He initialization can be used to reduce the problem of gradient vanishing or exploding. The number of neurons and the number of layers in the hidden layer can be adjusted according to the complexity of the home environment. For example, for a simple home environment, only one or two hidden layers may be required, while for a more complex environment, more layers and neurons may be required. The design of the output layer can also be adjusted according to the actual control requirements. For example, if the action space of the home device is large, more neurons may be required to represent these actions.

[0053] Furthermore, we can consider introducing regularization techniques such as dropout to prevent overfitting and improve the generalization ability of the model. During the training process, we can use early stopping to avoid overfitting, that is, stop training when the performance on the validation set no longer improves.

[0054] In some embodiments, the execution module includes: Relay unit, used to control the power on and off of household equipment; The motor drive unit is used to drive the operation of curtain motors, smart door lock motors and other equipment.

[0055] It should be noted that the execution module includes a relay unit and a motor drive unit, which are used to control the power on and off of home appliances and drive the operation of related equipment. The relay unit is an electronic switch that can connect or disconnect the circuit according to the control signal, while the motor drive unit is responsible for providing power so that the motor and other equipment can operate in a predetermined manner. These two units are key components for implementing specific operations in the smart home system.

[0056] Specifically, the relay unit can receive the corresponding command signal according to the control strategy generated by the deep Q network control module to control the power on and off of the home appliance. For example, when the system needs to turn off an appliance, the relay unit will receive a signal and then cut off the current flowing to the appliance. The motor drive unit is responsible for controlling the operation of equipment such as curtain motors and smart door lock motors. It realizes the automatic operation of the equipment by receiving control signals and converting them into motor rotation. The specific parameter settings of these units, such as current capacity, voltage level and drive frequency, need to be determined according to the specific requirements of the controlled equipment.

[0057] Preferably, the relay unit can select high reliability and low energy consumption products to ensure long-term stable operation and reduce energy consumption. The motor drive unit can use a precisely controlled stepper motor or servo motor to achieve precise position and speed control. When designing, you can consider using a modular design so that the relay unit and the motor drive unit can be flexibly configured according to different home appliance requirements.

[0058] Furthermore, fault detection and protection mechanisms, such as overload protection and short-circuit protection, can be introduced to improve the safety and reliability of the system. In the control strategy, feedback control links can be added to monitor the device status in real time and adjust the control signal to ensure that the device operates accurately as expected.

[0059] In some embodiments, the communication module includes: 5G communication unit uses n78 frequency band for data transmission; The Bluetooth communication unit is used to achieve pairing connection with a short-range smart device.

[0060] It should be noted that the communication module includes a 5G communication unit and a Bluetooth communication unit, which are used to realize data transmission and pairing connection between the smart home system and smart devices. The 5G communication unit refers to a module that uses the fifth generation of mobile communication technology for data transmission, while the Bluetooth communication unit is a short-range wireless communication technology used for pairing and connecting with close-range smart devices. These two units together constitute the basic architecture of the system's external communication.

[0061] Specifically, the 5G communication unit uses the n78 frequency band for data transmission. The n78 frequency band is one of the frequency bands used for urban and suburban coverage in the 5G network, with a high data transmission rate and low latency. The Bluetooth communication unit is used for pairing and data exchange with close-range devices such as smart watches and headphones. The specific parameter settings of these communication units, such as transmission power, frequency range, and data encoding method, need to be determined based on actual communication needs and environmental conditions.

[0062] Preferably, the 5G communication unit can be configured to support multiple-input multiple-output (MIMO) technology to improve the efficiency and reliability of data transmission. The Bluetooth communication unit can choose products that support Bluetooth 5.0 or higher to achieve a longer transmission distance and faster data transmission rate. In actual deployment, dual-mode or multi-mode communication modules can be considered to be compatible with different communication protocols and frequency bands.

[0063] Furthermore, network slicing technology can be introduced to provide customized network environments for different smart home services to meet the performance requirements of specific services. In terms of security, end-to-end encryption technology can be used to ensure the security and privacy of data transmission.

[0064] In some embodiments, the user interaction module includes: A mobile phone application for receiving and displaying data collected by the sensor module; The intelligent control panel is used to receive and display the data collected by the sensor module and receive user control instructions.

[0065] It should be noted that the user interaction module includes a mobile application and a smart control panel, which are used to receive and display the data collected by the sensor module and receive control instructions from the user. The mobile application is a software that can be run on a smartphone or tablet, while the smart control panel is a physical device, usually mounted on the wall, for direct interaction with the user. These two components provide an interface between the user and the smart home system.

[0066] Specifically, mobile phone applications can connect to the smart home system through a wireless network, receive data collected by the sensor module in real time, such as indoor temperature and humidity, and display them to users in graphical or digital form. At the same time, users can send control instructions through the application, such as adjusting the air conditioning temperature or turning on and off the lights. The smart control panel can display key information and is equipped with buttons or touch screens, allowing users to operate directly on the panel. The specific parameter settings of these interactive modules, such as display resolution, layout of the operation interface, and response time, need to be optimized based on user experience and ease of operation.

[0067] Preferably, the mobile phone application can be designed with a highly customizable user interface, allowing users to adjust the layout and settings according to their preferences. The smart control panel can adopt touch screen technology to provide a more intuitive operation experience, and can be equipped with voice control function, allowing users to control home devices through voice commands.

[0068] Furthermore, biometric technologies such as fingerprint recognition or facial recognition can be introduced to increase security and convenience. In terms of software design, cross-platform frameworks can be considered to enable applications to run on devices with different operating systems. Artificial intelligence technology can also be integrated to provide personalized recommendations and automated controls based on user behavior patterns.

[0069] In some embodiments, the control strategy optimization of the deep Q network control module includes: Automatically adjust control strategies based on user behavior and environmental changes; Using the policy iteration method, including policy evaluation and policy optimization, the policy evaluation uses the following formula:

[0070] in, Indicates status and actions The corresponding Q value is, Represents expected value, used to calculate expected utility; Indicates in status Next action After transfer to state probability; Indicates cumulative rewards, is the discount factor; Strategy optimization uses the following formula:

[0071] in, Indicates in status Next select action The probability of represents an exponential function, which is used to adjust the sensitivity of action selection. is the temperature parameter used to control the exploration and exploitation balance of the strategy.

[0072] It should be noted that the control strategy optimization of the deep Q network control module includes automatically adjusting the control strategy according to user behavior and environmental changes, and using policy iteration methods, including policy evaluation and policy optimization. Policy iteration is a method in dynamic programming algorithms that is used to improve the efficiency and effectiveness of the decision-making process by iteratively evaluating and optimizing strategies. In this context, policy evaluation refers to evaluating the effectiveness of the current strategy, while policy optimization is to improve the strategy based on the evaluation results.

[0073] Specifically, during the control strategy optimization process, the system continuously monitors user behavior patterns and environmental changes, such as changes in user preferences for home temperature or seasonal environmental changes, and automatically adjusts the control strategy to adapt to these changes. Policy evaluation uses a specific formula to calculate the expected utility, which involves evaluating the Q value based on the state transition probability and the cumulative reward. Policy optimization uses an exponential function to adjust the sensitivity of action selection, which is a process involving a temperature parameter that controls the exploration and exploitation balance of the strategy.

[0074] Preferably, control strategy optimization can be further refined into techniques for real-time monitoring and prediction of user behavior, for example, by analyzing historical data of users through machine learning algorithms to predict their future behavior. Strategy evaluation can use advanced techniques such as Monte Carlo tree search to improve the accuracy of evaluation. Temperature parameters in strategy optimization can be dynamically adjusted based on user feedback to achieve more refined control.

[0075] Furthermore, it is possible to consider introducing multi-objective optimization techniques to simultaneously consider multiple objectives such as energy saving and user comfort. When implementing policy optimization, it is also possible to consider using other algorithms in reinforcement learning, such as the Actor-Critic method, to provide more comprehensive policy improvements.

[0076] The above-mentioned embodiments of the present invention have the following beneficial effects: the smart home control system collects home environment data through the sensor module, can monitor and respond to environmental changes in real time, and the denoising and data integration capabilities of the data processing module can improve the accuracy and availability of data. The deep Q network control module has a built-in DQN algorithm model, which can learn user behavior and environmental changes, automatically generate and optimize control strategies, and realize intelligent control of home appliances. This integrated intelligent control mechanism can improve the automation and intelligence level of the home environment, and realize efficient energy saving and user behavior adaptive home appliance management.

[0077] In addition, the system includes an execution module, a communication module, and a user interaction module, which can fully control and monitor home appliances. The execution module directly controls the power supply and mechanical action of home appliances through relay units and motor drive units, the communication module achieves high-speed connection with smart devices through 5G and Bluetooth technology, and the user interaction module provides an intuitive user interface, allowing users to easily monitor and control the home environment. The collaborative work of these modules can enhance the functionality and user experience of the system, making the smart home system more flexible, responsive, and easy to operate.

[0078] Furthermore, the storage medium of the embodiment of the present application stores program instructions that can implement all the above methods, wherein the program instructions can be stored in the above storage medium in the form of a software product, including several instructions for enabling a computer device (which can be a personal computer, server, or network device, etc.) or a processor to execute all or part of the steps of the methods described in each embodiment of the present application. The aforementioned storage medium includes: various media that can store program codes, such as a USB flash drive, a mobile hard disk, a read-only memory (ROM), a random access memory (RAM), a magnetic disk or an optical disk, or terminal devices such as a computer, a server, a mobile phone, and a tablet.

[0079] The above descriptions are only some preferred embodiments of the present invention and an explanation of the technical principles used. Those skilled in the art should understand that the scope of the invention involved in the embodiments of the present invention is not limited to the technical solutions formed by a specific combination of the above technical features, but should also cover other technical solutions formed by any combination of the above technical features or their equivalent features without departing from the above inventive concept. For example, the above features are replaced with (but not limited to) technical features with similar functions disclosed in the embodiments of the present invention.

Claims

1. A smart home control system based on a single-chip microcomputer of 5G Internet of Things, characterized in that: include: The sensor module is used to collect home environment data, including temperature, humidity, and light sensor units, and transmit the collected home environment data to the data processing module; The data processing module integrates and denoises the home environment data collected by the sensor module, and converts the processed home environment data into the format required for the input of the deep Q network control module. The data processing module is also used to coordinate the data interaction between modules; The deep Q network control module receives data from the data processing module and generates corresponding control strategies through learning and training of home environment data and pre-collected user behavior data. The control strategies include control actions to achieve intelligent control of home appliances.

2. The smart home control system according to claim 1, characterized in that: The deep Q network control module has two built-in DQN algorithm models, each of which includes: The input layer is used to receive the normalized home environment status data; Multiple hidden layers, each hidden layer consists of multiple neurons, and the output of each neuron is determined by the ReLU activation function; The output layer is used to output the expected reward value of each control action on the home device.

3. The smart home control system according to claim 2, characterized in that: The training process of the deep Q network control module includes the following steps: Initialize the Q network parameters of the DQN algorithm model; Get the current smart home status from home environment data; Select a control action based on the current smart home state and the Q network parameters of the DQN algorithm model; Execute control actions and observe the resulting next smart home state and immediate rewards; Temporal difference learning updates the DQN algorithm model parameters, the formula is: ] in, is at the time step The state of smart homes and control actions The corresponding Q value is used to represent the expected utility of the state-action pair; It is the immediate reward obtained after performing a control action; is the discount factor used to adjust the current value of future rewards; is the learning rate; Is the next smart furniture state The maximum Q value of all control actions in; Regularly update the Q network parameters of the DQN algorithm model.

4. The smart home control system according to claim 3, characterized in that: The deep Q network control module includes an experience playback unit, which performs the following steps: Storing each experience tuple in an experience playback buffer, the experience tuple including a current smart home state, a control action, an immediate reward, and a next smart home state; Randomly extract experience tuples from the experience replay buffer for training; The target Q value is calculated using the following formula: in, is the target Q value used to update the current Q value, It is an instant reward. is the discount factor, It is the Q value prediction of the target DQN algorithm model for the next smart home state and control action.

5. The smart home control system according to claim 2, characterized in that: Among the two DQN algorithm models, one DQN algorithm model is used to select a control action, and the other DQN algorithm model is used to calculate a target Q value.

6. The smart home control system according to claim 4, characterized in that: The deep Q network control module includes a control strategy generation unit, which acts as an agent to perform the following steps: Initialize the DQN algorithm model parameters, set the number of input layer neurons to match the dimensions of the home environment status data collected by the sensor, determine the number of hidden layers and the number of neurons in each hidden layer based on the system complexity and training effect, and the number of output layer neurons corresponds to the number of controllable home device actions; As an agent, it enters a loop operation until the preset termination conditions are met, including: Based on the current smart home state, a control action is selected according to the ε-greedy strategy; The smart home environment updates its state based on the control actions selected by the agent; The agent receives immediate rewards from the new smart home state; The agent takes the new smart home state and immediate reward as input, inputs them into the DQN algorithm model, and obtains the new Q value through forward propagation.

7. The smart home control system according to claim 1, characterized in that: The deep Q network control module also includes an execution control unit, which performs the following steps based on the control action instructions received from the DQN algorithm model: The relay control subunit is used to control the power on and off of the home appliance according to the control action instructions output by the DQN algorithm model; The motor drive control subunit is used to drive the home appliances according to the control action instructions output by the DQN algorithm model.

8. The smart home control system according to claim 1, characterized in that: The deep Q network control module also includes a communication control unit, which performs the following steps: The 5G communication control subunit uses the n78 frequency band for data transmission, supports high-speed data upload and download, and is used to achieve real-time data transmission of the smart home system; The Bluetooth communication control subunit is used to achieve pairing connection with short-range smart devices.

9. The smart home control system according to claim 1, characterized in that: The deep Q network control module also includes a user interaction control unit, including: A mobile phone application control subunit is used to receive and display data collected by the sensor module, and remotely control home appliances through the mobile phone application control subunit; The intelligent control panel control subunit is used to receive and display the data collected by the sensor module and receive control instructions from the user.

10. The smart home control system according to claim 1, characterized in that: The deep Q network control module also includes a strategy optimization unit, including: Automatically adjust control strategies based on user behavior and environmental changes; Using the policy iteration method, including policy evaluation and policy optimization, the policy evaluation uses the following formula: in, Smart state and control actions The corresponding Q value is, Indicates expected value; Indicates the smart home status Next action Then transfer to smart home state probability; Indicates cumulative rewards, is the discount factor; Strategy optimization uses the following formula: in, Is in the smart furniture state Select Control Action The probability of is an exponential function used to adjust the sensitivity of action selection. is the temperature parameter.

Images