Home panel adaptive control method based on reinforcement learning

Abstract

The invention discloses a home panel self-adaptive control method based on reinforcement learning, and aims to solve the problem that a traditional home panel is insufficient in intelligence and self-adaptive capability. Comprising the following steps: S1, acquiring environmental parameters and user operation data, and constructing a time sequence data pair; s2, preprocessing the time sequence data pair to generate a feature vector; s3, training by adopting a reinforcement learning algorithm based on the feature vector and obtaining an optimal control strategy; s4, applying the optimal control strategy to the home panel to realize automatic parameter adjustment; s5, continuously collecting user feedback and environment change information, and optimizing a model state and a control strategy in real time; and S6, dynamically adjusting the output of the home panel according to the optimization strategy to realize adaptive control. According to the invention, the intelligent and self-adaptive capabilities of the home panel can be significantly improved, efficient and energy-saving management is realized, and the comfort of the home environment and the user satisfaction are effectively improved.

Description

Technical Field

[0001] This invention relates to the fields of smart home control and artificial intelligence technology, and in particular to a home panel adaptive control method based on reinforcement learning, which is applicable to application scenarios such as the collection of multi-source environmental parameters and user behavior data in the home environment, intelligent control strategy learning, adaptive parameter optimization, and home energy efficiency management. Background Technology

[0002] With the continuous development of IoT technology and artificial intelligence, smart home systems are widely used in modern homes and offices. Home control panels, as the core interaction and control terminal of smart home systems, handle the centralized management and adjustment of various devices such as lights, air conditioners, and curtains. Traditional home control panel systems mainly rely on preset rules, timed tasks, scene modes, or simple sensor linkage mechanisms. While they achieve automation and convenience to a certain extent, their level of intelligence and adaptability remain limited. In practical applications, users' living habits and preferences vary significantly, and external conditions of the home environment, such as temperature, humidity, and lighting, also change dynamically, making it difficult for traditional control methods to meet users' personalized, intelligent, and comfort needs.

[0003] Existing home panel control systems generally employ static configurations and simple logic, failing to deeply mine and learn from long-term user behavior data. Users often need to frequently manually adjust device parameters or pre-set multiple complex scene modes, making it impossible for the system to automatically adapt to changes in user preferences or lifestyles. Furthermore, current systems are largely reactive in energy efficiency management, lacking intelligent optimization and energy consumption assessment mechanisms, making it difficult to balance user comfort with intelligent management of household energy consumption. In addition, existing home panel systems often only provide limited personalized services when facing user needs and environmental changes, unable to differentiate responses based on different user habits, resulting in difficulty in improving user satisfaction and system stickiness.

[0004] To enhance the intelligence and adaptability of smart home systems, some research and products in recent years have attempted to apply artificial intelligence (AI) technology to home control, such as using decision trees and neural networks to model user behavior or introducing simple self-learning mechanisms for scene optimization. However, most of these methods remain at the offline training or semi-automation stage. The systems struggle to achieve real-time perception and online optimization of user feedback and environmental changes, resulting in limited speed for updating and adjusting control strategies and insufficient self-optimization and dynamic adaptability. Furthermore, existing methods have low integration in energy efficiency management and multi-objective optimization, failing to effectively integrate multi-dimensional indicators such as user satisfaction and energy consumption evaluation, leaving room for improvement in both energy conservation, emission reduction, and user experience. How to continuously collect and utilize user feedback information and optimize control strategies in real time to meet personalized user needs and environmental changes remains a crucial problem that current technologies urgently need to address.

[0005] To address the shortcomings of existing technologies, this invention proposes a reinforcement learning-based adaptive control method for home control panels. This method utilizes real-time acquisition and processing of multi-source environmental parameters and user operation behavior data, employing a reinforcement learning algorithm to train and optimize the control strategy for the home control panel. The system constructs a state space, action space, and reward function, incorporating multi-dimensional objectives such as user satisfaction and energy consumption into the decision-making process. The model is dynamically updated during actual operation, enabling online optimization and self-adjustment of the control strategy. Compared to traditional preset rules or static scene modes, this invention's method can proactively learn and adapt to users' lifestyles and preferences, reducing the frequency of manual operation, improving the comfort and personalization of the home environment, and optimizing energy efficiency management, effectively overcoming the deficiencies of existing technologies in terms of intelligence, adaptability, and energy efficiency. Summary of the Invention

[0006] One objective of this invention is to propose an adaptive control method for home panels based on reinforcement learning. This invention fully utilizes artificial intelligence reinforcement learning algorithms and smart home control systems, and describes in detail the training and self-optimization process of intelligent control strategies based on multi-source environmental parameters and user behavior data. It has the advantages of high intelligence, strong adaptability and excellent energy efficiency management.

[0007] An adaptive control method for home panels based on reinforcement learning according to an embodiment of the present invention includes the following steps:

[0008] S1. Collect environmental parameter data and user operation behavior data of the home panel, and match the collected data with the collection time to form a preliminary time series data pair;

[0009] S2. Preprocess the initial time series data pairs to generate feature vectors. These feature vectors serve as input to the reinforcement learning model and can comprehensively reflect the environmental state and user behavior characteristics of the home panel at each moment.

[0010] S3. Based on the feature vector, the control strategy of the home panel is trained using a reinforcement learning algorithm to obtain the optimal control strategy. The training process includes constructing a state space, an action space, and a reward function.

[0011] S4. Integrate the obtained optimal control strategy and apply it to the home panel control system. Automatically select and execute the optimal action based on the current real-time status, and dynamically adjust various control parameters of the home panel.

[0012] S5. During the home control process, continuously collect user feedback information and environmental change information, update the status of the reinforcement learning model in real time, and optimize the control strategy online.

[0013] S6. Based on the optimized control strategy, dynamically adjust the output of the home panel to achieve adaptive control of the home panel in response to user needs and environmental changes.

[0014] Optionally, step S1 specifically includes:

[0015] S11. Real-time collection of environmental parameter data through the sensor module of the home panel. The environmental parameter data includes the current ambient temperature T, humidity H, light intensity I, and the current working status S of each device. The working status includes the on / off status, working mode, and operating parameters of each device.

[0016] S12. Collect user operation behavior data A through the human-computer interaction interface of the home panel. The operation behavior data includes user button input, touch input, voice commands and remote control commands on the panel.

[0017] S13. Match the collected environmental parameter data {T,H,I,S} with the user operation behavior data A based on the collection time to form a preliminary time-series data pair {(T,H,I,S,A)}. t , where t represents the time index of each acquisition moment.

[0018] Optionally, step S2 specifically includes:

[0019] S21. For the initial time series data pair {(T,H,I,S,A)} t Further integration based on time indexes forms a structured original data sequence X. t =[T,H,I,S,A] t ;

[0020] S22, regarding the original data sequence X t Each data item in the sequence is normalized to obtain the preprocessed data sequence X. t ′=[T′,H′,I′,S′,A′] t Where T′, H′, I′, S′, and A′ are the normalized results of temperature, humidity, light intensity, equipment status, and user operation behavior, respectively, and the zero-mean normalization formula used is:

[0021]

[0022] Where, x i,t μ is the original value of the i-th item in the original data sequence at time t. i Let σ be the mean of the i-th data item. i Let x′ be the standard deviation of the i-th data point. i,t The value is the normalized value;

[0023] For example, suppose that at a certain time t, the original temperature data T is 28℃, the mean of the temperature data at all sampling times is 25℃, and the standard deviation is 2℃, then the normalized temperature data is calculated as follows:

[0024]

[0025] That is, the normalized result of the temperature data at that moment is 1.5;

[0026] S23, preprocess the data sequence X t ′=[T′,H′,I′,S′,A′] t It is used as a feature vector input into the reinforcement learning model.

[0027] Optionally, step S3 specifically includes:

[0028] S31. Input feature vector X t ′=[T′,H′,I′,S′,A′] t Used to construct the state space S = {X' t}, where each state s∈S corresponds to the environmental and user operation characteristics of the home panel at time t;

[0029] S32. Set the action space A, which includes several control commands, wherein each action a represents the adjustment of various control parameters of the home panel. The control commands include light brightness adjustment, air conditioner temperature adjustment, curtain opening and closing, and other related commands.

[0030] S33. Establish a reward function R(s,a), where s and a are the aforementioned state and action, respectively. The reward value is calculated based on user feedback F and energy consumption evaluation E. The specific reward function is as follows:

[0031] R(s,a)=αF-βE,

[0032] Where α is the user feedback weight, F is the user's satisfaction with the current control effect, β is the energy consumption weight, and E is the current energy consumption value;

[0033] S34. The control policy π is trained based on the Q-learning reinforcement learning algorithm. The control policy π is the mapping relationship between state s and action a. The policy is optimized using the following Q-value iterative formula:

[0034]

[0035] Among them, Q t (s,a) represents the Q-values ​​of state s and action a at time t, η is the learning rate, and γ is the discount factor. ′ The new state after action a is executed, a ′ This is an optional action;

[0036] S35. Obtain the optimal control strategy π through strategy optimization. * The optimal control strategy π * This is the mapping from state space S to action space A, i.e.:

[0037]

[0038] Where, π * (s) represents the optimal action in state s.

[0039] Optionally, step S4 specifically includes:

[0040] S41, Apply the optimal control strategy π * Integrated and applied to home panel control systems;

[0041] S42. During the actual operation of the home panel, the state s at the current time t is collected in real time. t =X′ t And through the optimal control strategy π * Obtain the optimal action Right now:

[0042]

[0043] S43. Based on the selected optimal action The system automatically adjusts various control parameters of the home appliance control panel, including the light brightness L at the current time t. t Air conditioning temperature T t Curtain opening angle θ t The adjustment relationship is as follows:

[0044]

[0045] Where f1, f2, and f3 are the optimal actions. The corresponding device control functions;

[0046] Through the above steps, the home panel can make dynamic decisions and adaptive adjustments based on the real-time status at any given moment, thereby effectively meeting user needs and environmental changes.

[0047] Optionally, step S5 specifically includes:

[0048] S51. During the actual control process of the home panel, real-time user feedback F at the current moment t is collected. t And environmental change data {T,H,I,S} t ;

[0049] S52, User feedback F t And environmental change data {T,H,I,S} t By associating with time information, feedback and environmental change time-series data pairs are formed {(F t ,T,H,I,S)} t ;

[0050] S53, Based on real-time user feedback F t The reward function R(s,a) is modified using the following formula:

[0051] R ′ (s,a)=R(s,a)+λF t ,

[0052] Among them, R ′ (s,a) is the corrected reward value, R(s,a) is the original reward value, and λ is the user feedback weight coefficient.

[0053] S54, with the corrected reward value R ′ Based on (s,a), the state Q(s,a) of the reinforcement learning model is updated in real time, and online optimization is performed using the following Q-value update formula:

[0054]

[0055] Among them, Q t (s,a) represents the Q-values ​​of state s and action a at time t, η is the learning rate, and γ is the discount factor. ′ The new state after action a is executed, a ′ This is an optional action;

[0056] Through the above steps, the home panel control system can achieve continuous self-optimization and personalized response, enabling the system to continuously adjust and optimize control strategies based on real-time user feedback and environmental changes. This better adapts to users' personalized needs and lifestyles, improves the level of intelligence in control and user experience, and effectively enhances the energy efficiency management capabilities of the home environment.

[0057] Optionally, step S6 specifically includes:

[0058] S61. After each state update, based on the real-time optimized control strategy π * Get the current state s t The optimal action Right now:

[0059]

[0060] S62, Optimal Action The various control outputs applied to the home appliance panel, including the light brightness L... t Air conditioning temperature T t and the opening angle θ of the curtains t The dynamic adjustment relationship is as follows:

[0061]

[0062] Where f1, f2, and f3 are based on the optimal action Defined device control functions;

[0063] S63. After adjusting the output of the home panel, continuously monitor the environmental parameter data {T,H,I,S}. t+1 With user operation behavior data A, new environmental and user data are used as input to enter the next round of state update and strategy optimization, so as to realize the dynamic adaptive control of the home panel to user needs and environmental changes.

[0064] Optionally, the frequency of collecting the environmental parameter data can be set according to actual needs, preferably at least once per minute, to ensure that the system can promptly sense and respond to environmental changes.

[0065] Optionally, the user feedback includes a satisfaction score and the number of manual adjustments. The system adjusts the reward function based on a weighted average of the satisfaction score and the number of manual adjustments, using the following formula:

[0066] R ″ (s,a)=R(s,a)+λ1F t +λ2N,

[0067] Among them, R ″(s,a) represents the final corrected reward value, λ1 and λ2 are the weights of the satisfaction score and the number of manual adjustments, respectively, and N is the number of manual adjustments.

[0068] Beneficial effects

[0069] The adaptive control method for home panels based on reinforcement learning proposed in this invention breaks through the limitations of existing technologies in terms of intelligence, adaptability, and energy efficiency management.

[0070] First, by introducing reinforcement learning algorithms, the system can collect and comprehensively analyze multi-source environmental parameters and user operation behavior data in real time, continuously improving and optimizing control strategies. This effectively solves the problems of slow response and rigid control methods of traditional home panels to user behavior and environmental changes. Compared with traditional methods that rely on preset rules or static scene modes, this invention can proactively learn and adapt to users' personalized preferences and lifestyles, continuously improving the automation and intelligence level of the home system in dynamic environments, significantly reducing the frequency of manual intervention by users, and enhancing the convenience and user-friendly experience of home environment management.

[0071] Secondly, this invention constructs a reward function encompassing multiple dimensions such as user satisfaction and energy consumption evaluation, closely integrating user experience with energy efficiency to achieve optimal control decisions under multiple objectives. The system can dynamically optimize the operating parameters of devices such as lights, air conditioners, and curtains while ensuring user comfort, scientifically managing household energy consumption, effectively improving energy efficiency, and contributing to the achievement of energy conservation and emission reduction goals. Compared to traditional home control panel methods that only focus on a single objective, this invention provides a more scientific and efficient energy efficiency management method for smart homes, promoting the intelligent upgrade of home energy management.

[0072] Furthermore, the method proposed in this invention possesses continuous self-optimization and online learning capabilities, enabling it to adjust control strategies in real time based on user feedback and environmental changes during actual operation. This allows it to continuously adapt to the changing needs of different users and environments, thereby maintaining the system's high level of intelligence and personalized service capabilities. This not only enhances the long-term use value of home furnishing panels but also strengthens user stickiness and the system's market competitiveness.

[0073] In summary, this invention, through a reinforcement learning-driven adaptive control method, achieves comprehensive improvements in the intelligence, adaptability, energy efficiency management, and personalized services of smart home systems. It significantly improves the comfort and management efficiency of the home environment, providing solid technical support and broad application prospects for the sustainable development and technological advancement of the smart home industry. Attached Figure Description

[0074] The accompanying drawings are provided to further illustrate the invention and form part of the specification. They are used in conjunction with embodiments of the invention to explain the invention and do not constitute a limitation thereof. In the drawings:

[0075] Figure 1 This is a schematic diagram of the adaptive control method for home panels based on reinforcement learning proposed in this invention.

[0076] Figure 2 This is a schematic diagram of the reinforcement learning control module processing flow in this invention;

[0077] Figure 3 This is a structural block diagram of the smart control system for home panels in this invention. Detailed Implementation

[0078] The present invention will now be described in further detail with reference to the accompanying drawings. These drawings are simplified schematic diagrams, illustrating only the basic structure of the invention, and therefore only show the components relevant to the invention.

[0079] refer to Figure 1-3 A reinforcement learning-based adaptive control method for home panels includes:

[0080] S1. Collect environmental parameter data and user operation behavior data of the home panel, and match the collected data with the collection time to form a preliminary time series data pair;

[0081] S2. Preprocess the initial time series data pairs to generate feature vectors. These feature vectors serve as input to the reinforcement learning model and can comprehensively reflect the environmental state and user behavior characteristics of the home panel at each moment.

[0082] S3. Based on the feature vector, the control strategy of the home panel is trained using a reinforcement learning algorithm to obtain the optimal control strategy. The training process includes constructing a state space, an action space, and a reward function.

[0083] S4. Integrate the obtained optimal control strategy and apply it to the home panel control system. Automatically select and execute the optimal action based on the current real-time status, and dynamically adjust various control parameters of the home panel.

[0084] S5. During the home control process, continuously collect user feedback information and environmental change information, update the status of the reinforcement learning model in real time, and optimize the control strategy online.

[0085] S6. Based on the optimized control strategy, dynamically adjust the output of the home panel to achieve adaptive control of the home panel in response to user needs and environmental changes.

[0086] In this embodiment, step S1 specifically includes:

[0087] S11. Real-time collection of environmental parameter data through the sensor module of the home panel. The environmental parameter data includes the current ambient temperature T, humidity H, light intensity I, and the current working status S of each device. The working status includes the on / off status, working mode, and operating parameters of each device.

[0088] S12. Collect user operation behavior data A through the human-computer interaction interface of the home panel. The operation behavior data includes user button input, touch input, voice commands and remote control commands on the panel.

[0089] S13. Match the collected environmental parameter data {T,H,I,S} with the user operation behavior data A based on the collection time to form a preliminary time-series data pair {(T,H,I,S,A)}. t , where t represents the time index of each acquisition moment.

[0090] In this embodiment, step S2 specifically includes:

[0091] S21. For the initial time series data pair {(T,H,I,S,A)} t Further integration based on time indexes forms a structured original data sequence X. t =[T,H,I,S,A] t ;

[0092] S22, regarding the original data sequence X t Each data item in the sequence is normalized to obtain the preprocessed data sequence X. t ′ =[T ′ H ′ ,I ′ ,S ′ A ′ ] t , among which, T ′ H ′ ,I ′ ,S ′ A ′ The normalized results for temperature, humidity, light intensity, equipment status, and user operation behavior are respectively, and the zero-mean normalization formula used is:

[0093]

[0094] Where, x i,t μ is the original value of the i-th item in the original data sequence at time t. i Let σ be the mean of the i-th data item. i Let x′ be the standard deviation of the i-th data point. i,tThe value is the normalized value;

[0095] For example, suppose that at a certain time t, the original temperature data T is 28℃, the mean of the temperature data at all sampling times is 25℃, and the standard deviation is 2℃, then the normalized temperature data is calculated as follows:

[0096]

[0097] That is, the normalized result of the temperature data at that moment is 1.5;

[0098] S23. The preprocessed data sequence X′ t =[T′,H′,I′,S′,A′] t It is used as a feature vector input into the reinforcement learning model.

[0099] In this embodiment, step S3 specifically includes:

[0100] S31. Input the feature vector X′ t =[T′,H′,I′,S′,A′] t Used to construct the state space S = {X} t}, where each state s∈S corresponds to the environmental and user operation characteristics of the home panel at time t;

[0101] S32. Set the action space A, which includes several control commands, wherein each action a represents the adjustment of various control parameters of the home panel. The control commands include light brightness adjustment, air conditioner temperature adjustment, curtain opening and closing, and other related commands.

[0102] S33. Establish a reward function R(s,a), where s and a are the aforementioned state and action, respectively. The reward value is calculated based on user feedback F and energy consumption evaluation E. The specific reward function is as follows:

[0103] R(s,a)=αF-βE,

[0104] Where α is the user feedback weight, F is the user's satisfaction with the current control effect, β is the energy consumption weight, and E is the current energy consumption value;

[0105] S34. The control policy π is trained based on the Q-learning reinforcement learning algorithm. The control policy π is the mapping relationship between state s and action a. The policy is optimized using the following Q-value iterative formula:

[0106]

[0107] Among them, Q t (s,a) represents the Q-values ​​of state s and action a at time t, η is the learning rate, and γ is the discount factor. ′The new state after action a is executed, a ′ This is an optional action;

[0108] S35. Obtain the optimal control strategy π through strategy optimization. * The optimal control strategy π * This is the mapping from state space S to action space A, i.e.:

[0109]

[0110] Where, π * (s) represents the optimal action in state s.

[0111] In this embodiment, step S4 specifically includes:

[0112] S41, Apply the optimal control strategy π * Integrated and applied to home panel control systems;

[0113] S42. During the actual operation of the home panel, the state s at the current time t is collected in real time. t =X′ t And through the optimal control strategy π * Obtain the optimal action Right now:

[0114]

[0115] S43. Based on the selected optimal action The system automatically adjusts various control parameters of the home appliance control panel, including the light brightness L at the current time t. t Air conditioning temperature T t Curtain opening angle θ t The adjustment relationship is as follows:

[0116]

[0117] Where f1, f2, and f3 are the optimal actions. The corresponding device control functions;

[0118] Through the above steps, the home panel can make dynamic decisions and adaptive adjustments based on the real-time status at any given moment, thereby effectively meeting user needs and environmental changes.

[0119] In this embodiment, step S5 specifically includes:

[0120] S51. During the actual control process of the home panel, real-time user feedback F at the current moment t is collected. t And environmental change data {T,H,I,S} t ;

[0121] S52, User feedback F t And environmental change data {T,H,I,S} t By associating with time information, feedback and environmental change time-series data pairs are formed {(F t ,T,H,I,S)} t ;

[0122] S53, Based on real-time user feedback F t The reward function R(s,a) is modified using the following formula:

[0123] R ′ (s,a)=R(s,a)+λF t ,

[0124] Among them, R ′ (s,a) is the corrected reward value, R(s,a) is the original reward value, and λ is the user feedback weight coefficient.

[0125] S54, with the corrected reward value R ′ Based on (s,a), the state Q(s,a) of the reinforcement learning model is updated in real time, and online optimization is performed using the following Q-value update formula:

[0126]

[0127] Among them, Q t (s,a) represents the Q-values ​​of state s and action a at time t, η is the learning rate, and γ is the discount factor. ′ The new state after action a is executed, a ′ This is an optional action;

[0128] Through the above steps, the home panel control system can achieve continuous self-optimization and personalized response, enabling the system to continuously adjust and optimize control strategies based on real-time user feedback and environmental changes. This better adapts to users' personalized needs and lifestyles, improves the level of intelligence in control and user experience, and effectively enhances the energy efficiency management capabilities of the home environment.

[0129] In this embodiment, step S6 specifically includes:

[0130] S61. After each state update, based on the real-time optimized control strategy π * Get the current state s t The optimal action Right now:

[0131]

[0132] S62, Optimal Action The various control outputs applied to the home appliance panel, including the light brightness L... t Air conditioning temperature T t and the opening angle θ of the curtains t The dynamic adjustment relationship is as follows:

[0133]

[0134] Where f1, f2, and f3 are based on the optimal action Defined device control functions;

[0135] S63. After adjusting the output of the home panel, continuously monitor the environmental parameter data {T,H,I,S}. t+1 With user operation behavior data A, new environmental and user data are used as input to enter the next round of state update and strategy optimization, so as to realize the dynamic adaptive control of the home panel to user needs and environmental changes.

[0136] In this embodiment, the frequency of environmental parameter data collection can be set according to actual needs, preferably at least once per minute, to ensure that the system can promptly sense and respond to environmental changes.

[0137] In this embodiment, the user feedback includes a satisfaction score and the number of manual adjustments. The system adjusts the reward function based on a weighted average of the satisfaction score and the number of manual adjustments. The adjustment formula is as follows:

[0138] R ″ (s,a)=R(s,a)+λ1F t +λ2N,

[0139] Among them, R ″ (s,a) represents the final corrected reward value, λ1 and λ2 are the weights of the satisfaction score and the number of manual adjustments, respectively, and N is the number of manual adjustments.

[0140] Example 1:

[0141] To verify the feasibility and effectiveness of this invention in practice, it was applied to a high-end smart home project in a large urban residential community. A 200-square-meter, three-bedroom, two-living-room apartment was selected as a pilot unit. This apartment includes a master bedroom, a secondary bedroom, a living room, a dining room, a kitchen, and a bathroom, each equipped with smart lighting, smart air conditioning, smart curtains, and other devices that can be centrally controlled via a home control panel. The experiment lasted from March to May 2024, involving a family of four: two working parents, a middle school student, and an elderly person. The existing home control panel system in the apartment used conventional preset scenes and timer control modes, which users consistently reported were cumbersome to operate, lacked flexibility in device adjustments, and consumed a lot of energy.

[0142] During the testing process, data was first collected using traditional home control panel methods (timed and scene modes) within the same residence, user group, and testing period. Subsequently, the home system was switched to the reinforcement learning-based adaptive control method for home panels proposed in this invention, and data for various indicators were collected again under the same period and environment. All data were obtained under a unified testing scenario to ensure the objectivity and scientific validity of the comparison results.

[0143] In practice, the team integrated the reinforcement learning-based adaptive control method for home panels into the existing smart home system. The system automatically collects data including temperature, humidity, light intensity, and the on / off status and operating parameters of each device through the panel's built-in environmental sensors and user operation logs. Simultaneously, the system records the manual actions of each family member, such as turning lights on and off, adjusting air conditioning temperature, and opening or closing curtains. All data is recorded and managed synchronously according to the collection time.

[0144] The collected data, after preprocessing, is converted into feature vectors and input into the reinforcement learning model. The sampling period is 30 seconds, and the system analyzes the current environmental state and user behavior in real time, dynamically constructing the state space and action space. The system sets a reward function based on each user's feedback and actual energy consumption, and continuously optimizes the control strategy. For example, if a user enters the master bedroom after 10 PM, the system learns that they habitually set the air conditioner temperature to 24℃, dim the lights to 30%, and partially close the curtains; in subsequent similar scenarios, it can automatically perform the same adjustments. When encountering seasonal changes and fluctuations in external temperature and humidity, the reinforcement learning model proposed in this invention will automatically adjust the optimal control strategy based on recent user feedback and environmental changes.

[0145] In actual operation, the system continuously collects user feedback and environmental change data, and adjusts the reward function and optimizes the Q-value in real time to ensure that each equipment adjustment meets the latest user needs. For example, if an elderly person repeatedly adjusts the curtain angle manually during their afternoon nap, the system will detect this behavior and proactively adjust the curtains to the elderly person's preferred angle under the same environmental conditions the next time. Every night, the system automatically adjusts the equipment operation strategy for the next day based on the day's energy consumption data and user feedback, thereby reducing ineffective energy consumption.

[0146] Through three months of field testing, the team systematically compared the actual performance of the method of this invention with traditional home panel control methods in terms of user operation convenience, environmental comfort, and energy consumption. Specific data are shown in Table 1.

[0147] Table 1: Performance Comparison Results of Traditional Home Panel Control Methods and the Method of the Invention

[0148]

[0149] During the testing period, traditional home control panels required an average of 22 manual adjustments per household per day, often resulting in family members frequently adjusting air conditioning temperatures, repeatedly switching lights on and off, and manually drawing curtains. However, after adopting the method of this invention, the average number of manual adjustments per day was significantly reduced to 6, greatly alleviating the user's workload. Regarding user satisfaction ratings, the system regularly collects user questionnaires and panel scores for statistical analysis. Results show that the traditional system averaged 6.7 points, while the average score after adopting the method of this invention increased to 9.2 points. Users generally reported that the system better meets their personalized needs and responds more intelligently and promptly. In terms of daily power consumption, this invention, through continuous optimization of device runtime and energy consumption, reduced daily power consumption from 28.5 kWh to 22.3 kWh, significantly improving energy efficiency. Furthermore, compared to traditional home control panels, the method of this invention improved nighttime air conditioning temperature stability and curtain automatic adjustment accuracy by 16.5% and 47.8%, respectively, effectively addressing the problems of large nighttime temperature fluctuations and inaccurate curtain automatic control. The improvement in the proportion of lighting energy consumption to total energy consumption was -5.3%, further enhancing overall energy efficiency. The improvement in the number of user-initiated feedbacks was -73.3%, indicating that the system performs better in adapting to user needs and effectively reduces the frequency of users making improvement suggestions or complaints.

[0150] In summary, this embodiment fully verifies the application effect of the present invention in actual smart home scenarios, significantly improves the intelligence level and user experience of home panel systems, realizes adaptive environmental regulation and intelligent optimization of energy efficiency management, and effectively solves the prominent problems of traditional home panels in terms of intelligence, adaptive capabilities and energy efficiency management.

[0151] The above description is only a preferred embodiment of the present invention, but the scope of protection of the present invention is not limited thereto. Any equivalent substitutions or modifications made by those skilled in the art within the scope of the technology disclosed in the present invention, based on the technical solution and inventive concept of the present invention, should be covered within the scope of protection of the present invention.

Claims

1. A reinforcement learning based adaptive control method for home panels, characterized in that, Comprise the following steps: S1, collect the environmental parameter data and user operation behavior data of the home panel, and correspond the collected data based on the collection time to form a preliminary time series data pair; S2, pre-process the preliminary time series data pair to generate a feature vector, which is used as the input of the reinforcement learning model and can comprehensively reflect the environmental state and user behavior characteristics of the home panel at each time; S3, based on the feature vector, the control strategy of the home panel is trained using a reinforcement learning algorithm to obtain an optimal control strategy, and the training process includes constructing a state space, an action space and a reward function; S4, integrate and apply the obtained optimal control strategy to the home panel control system, automatically select and execute the optimal action according to the current real-time state, and dynamically adjust the control parameters of the home panel; S5, in the control process of the home panel, continuously collect user feedback information and environmental change information, real-time update the state of the reinforcement learning model and online optimize the control strategy; S6, according to the optimized control strategy, dynamically adjust the output of the home panel to realize the adaptive control of the home panel to user demand and environmental change.

2. The reinforcement learning based home panel adaptive control method according to claim 1, characterized in that, The step S1 specifically comprises: S11, real-time collect environmental parameter data through the sensor module of the home panel, the environmental parameter data including the temperature T, humidity H, light intensity I of the current environment and the working state S of the current devices, the working state including the on-off state, working mode and running parameters of each device; S12, collect user's operation behavior data A each time through the man-machine interface of the home panel, the operation behavior data including the user's key input, touch input, voice instruction and remote control instruction to the panel; S13, correspond the collected environmental parameter data {T, H, I, S} and the user operation behavior data A based on the collection time to form a preliminary time sequence data pair {(T, H, I, S, A)} t wherein t represents the time index of each collection time. 3.The reinforcement learning based home panel adaptive control method according to claim 1, wherein, The step S2 specifically comprises: S21, pair {(T, H, I, S, A)} of preliminary timing data t Further integration according to time index, resulting in structured raw data sequence X t = [T, H, I, S, A] t ; S22, normalizing each item of data in the original data sequence X t to obtain a preprocessed data sequence X' t = [T', H', I', S', A'] t , wherein T', H', I', S', and A' are normalization results of temperature, humidity, light intensity, device status, and user operation behavior, respectively. S23, the pre-processed data sequence X' t = [T', H', I', S', A'] t as a feature vector to the reinforcement learning model.

4. The reinforcement learning based home panel adaptive control method according to claim 1, wherein, The step S3 specifically comprises: S31, input the feature vector X' t = [T', H', I', S', A'] t for constructing a state space S = {X' t}, where each state s e S corresponds to the environmental and user operation features of the home panel at time t; S32, set the action space A, including a plurality of control instructions, wherein each action a represents the adjustment of the control parameters of the home panel, and the control instructions include light brightness adjustment, air conditioner temperature adjustment, curtain opening and other related instructions; S33, establish a reward function R(s, a), wherein s and a are the aforementioned state and action, and the reward value is calculated according to the user feedback F and the energy consumption evaluation E; S34, train the control strategy π based on the Q-learning reinforcement learning algorithm, the control strategy π being the mapping relationship between the state s and the action a, and the strategy optimization is performed using the following Q value iteration formula: wherein Q t (s, a) is the Q-value of state s and action a at time t, η is the learning rate, γ is the discount factor, s' is the new state after action a is performed, and a' is a selectable action. S35、Through policy optimization, an optimal control policy π is obtained * , the optimal control policy π * is a mapping from the state space S to the action space A.

5. The reinforcement learning based home panel adaptive control method according to claim 1, wherein, The step S4 specifically comprises: S41, the optimal control strategy π * integrated and applied to a home panel control system; S42, in the actual operation process of the home panel, the state s of the current time t is collected in real time t = X' t and the optimal control strategy π * obtains the optimal action S43、According to the selected optimal action automatically adjusting various control parameters of the home panel, including the light brightness L at the current time t t , air conditioning temperature T t , curtain opening angle θ t .

6. The reinforcement learning based home panel adaptive control method according to claim 1, wherein, The step S5 specifically comprises: S51, in the actual control process of the home panel, real-time collection of user feedback F of the current time t t and environmental change data {T, H, I, S} t ; S52, feedback F from user t and environmental change data {T, H, I, S} t associated with time information, forming feedback and environmental change timing data pairs {(F t , T, H, I, S)} t ; S53、According to the real-time collected user feedback F t , the user feedback weight coefficient is introduced to modify the reward function R(s, a), and the modified reward value R'(s, a) is obtained. S54, based on the corrected reward value R'(s, a), real-time update the state Q(s, a) of the reinforcement learning model, and online optimize using the following Q value update formula: where Q t (s, a) is the Q-value of state s and action a at time t, η is the learning rate, γ is the discount factor, s' is the new state after action a is performed, and a' is the optional action.

7. The reinforcement learning based home panel adaptive control method according to claim 1, wherein, The step S6 specifically comprises: S61, after each state update, the optimal action a* under the current state s according to the real-time optimized control policy π * t ​​ S62, the optimal action application to the various control outputs of the home panel, including light brightness L t , air conditioning temperature T t and curtain opening angle; S63, after the home panel output adjustment is completed, the environmental parameter data {T, H, I, S} is continuously monitored t+1 and the user operation behavior data A, the new environmental and user data are taken as inputs to enter the next round of state updating and policy optimization.

8. The reinforcement learning based home panel adaptive control method according to claim 1, wherein, The collection frequency of the environmental parameter data can be set according to actual needs, preferably at least once per minute, to ensure that the system can timely perceive and respond to environmental changes. 9.The reinforcement learning based home panel adaptive control method according to claim 1, wherein, The user feedback includes satisfaction score and manual adjustment times, and the system corrects the reward function according to the weighted results of the satisfaction score and the manual adjustment times.

Images