Deep reinforcement learning fogging habit optimization system
By using a deep reinforcement learning-based atomization habit optimization system, combined with multi-dimensional data collection and modular design, the system solves the adaptability problem of traditional methods in complex scenarios, and achieves accurate and efficient optimization of user habits.
Patent Information
- Application Number
- CN202511104880.1
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2025-08-07
- Publication Date
- 2025-11-21
AI Technical Summary
Traditional methods are ill-suited to the complex needs and dynamic changes of individuals in different situations, and existing deep reinforcement learning lacks targeted design in habit optimization, making it impossible to accurately and efficiently optimize user habits.
The system employs deep reinforcement learning to optimize habits, including a state awareness module, an experience generation module, an experience playback module, a deep reinforcement learning model, and a behavior decision-making module. Through multi-dimensional data collection and preprocessing, combined with policy networks and value networks, it optimizes action selection and uses a modular design to achieve collaborative operation of the system.
It achieves comprehensive perception and dynamic capture of user status, improves the adaptability and flexibility of habit optimization, solves the adaptability problem of traditional methods in complex scenarios, and improves learning efficiency and the scientific nature of decision-making.
Smart Images
Figure CN120997006A_ABST
Abstract
Description
TECHNICAL FIELD
[0001] The present application relates to the field of artificial intelligence habit optimization, in particular to a deep reinforcement learning atomization habit optimization system. BACKGROUND
[0002] In modern life, people often want to cultivate or change certain habits to improve the quality of life or achieve specific goals, such as health management, skill improvement, etc. The formation and change of habits is a complex process, and traditional methods usually rely on simple reminder mechanisms or fixed plans, which are difficult to adapt to the complex needs and dynamic changes of individuals in different situations. With the development of artificial intelligence technology, reinforcement learning, as a method of learning optimal behavior strategy through interaction between agent and environment and according to reward signal, is gradually applied to various fields. However, in the context of habit optimization, traditional reinforcement learning faces many challenges. On the one hand, habit formation involves a very complex state space, including individual physiological state, psychological state, environmental factors, time factors and other dimensions. Traditional methods are difficult to effectively process such high-dimensional state information. On the other hand, the reward signal in the habit change process often has sparsity and delay, for example, the formation of healthy habits may take a long time to observe obvious physical indicators to improve, which makes it difficult for the agent to quickly and accurately learn effective behavior strategies. Deep reinforcement learning combines the powerful feature extraction capability of deep learning with the decision-making capability of reinforcement learning, bringing new opportunities to solve habit optimization problems. However, existing deep reinforcement learning applied to habit optimization mostly lacks in-depth understanding and targeted design of habit formation mechanism, and cannot fully tap the potential rules in individual behavior data, making it difficult to achieve precise and efficient habit optimization. Therefore, a deep reinforcement learning atomization habit optimization system is proposed. SUMMARY
[0003] In view of the deficiencies of the prior art, the present application provides a deep reinforcement learning atomization habit optimization system to solve the problems raised in the background art.
[0004] To achieve the above purpose, the present application provides the following technical solution: a deep reinforcement learning atomization habit optimization system, comprising: a state perception module, an experience generation module, an experience replay module, a deep reinforcement learning model and a behavior decision module; The state perception module is used to collect physiological data, psychological state data, environmental data and time data, preprocess to generate feature vectors, and obtain data through sensors, input devices and third party interfaces. The output end of the state perception module is electrically connected with the experience generation module; The experience generation module generates experience samples based on state information and behavior actions, and assigns reward values according to a reward rule, and an output end of the experience generation module is electrically connected with the experience replay module; The experience replay module stores samples through an experience pool, randomly extracts samples for model training to break time correlation, and an output end of the experience replay module is electrically connected with the deep reinforcement learning model; The deep reinforcement learning model includes a policy network and a value network, the policy network outputs an action probability distribution, the value network evaluates an action value, parameters are optimized through learning samples, and an output end is electrically connected with the behavior decision module; The behavior decision module determines specific actions in combination with an exploration strategy, feeds back to a user and outputs to the experience generation module, and an output end is electrically connected with the experience generation module; The deep reinforcement learning habit optimization system comprehensively collects physiological, psychological, environmental and time data of a user through a state perception module, provides a solid foundation for subsequent analysis, the combination of the experience generation module and the experience replay module enables the system to efficiently generate and utilize experience samples for model training, effectively breaks time correlation, improves learning efficiency, the policy network and the value network in the deep reinforcement learning model work cooperatively, optimize action selection, and ensure the scientificity and rationality of decision-making, and the behavior decision module combines an exploration strategy to ensure decision quality while encouraging the system to explore new actions to improve adaptability and flexibility.
[0005] Preferably, in the state perception module, physiological data is collected by wearable devices to collect heart rate, blood pressure and sleep quality data, psychological state data is collected by psychological questionnaires and voice emotion recognition, environmental data is obtained by sensors and positioning devices, and time data is obtained by a system clock; Physiological data can be collected by smart bracelets or watches integrated with heart rate and blood pressure monitoring functions, sleep quality is analyzed by built-in algorithms of these devices, psychological state is comprehensively judged by regular psychological questionnaires and real-time voice emotion recognition algorithms (such as based on voiceprint and tone analysis), environmental data is obtained by temperature and humidity sensors, air quality sensors and GPS positioning devices, and time data is directly provided by a system built-in clock.
[0006] Preferably, in the experience generation module, a reward rule is set according to target achievement, behavior persistence and state adaptability, positive reward is given to behaviors close to the target, negative reward is given to deviated behaviors, incremental reward is given to persistent beneficial behaviors, and additional reward is given to optimal behaviors in specific states; In the experience generation module, the specific implementation of the reward rule can be realized by setting quantitative indicators, for example, the target achievement degree can be scored according to the comparison between the user's set atomization habit improvement target, such as reducing the number of daily atomization, and the actual achievement; the behavior persistence is measured by the number of consecutive days of meeting the target; and the state adaptability can be evaluated according to the matching degree of the user's current physiological and psychological state and the environment, and a positive score is given when approaching the target, and a score is deducted when deviating, and the reward is increased when continuously meeting the target, and an additional score is given for the optimal behavior in a specific state.
[0007] Preferably, in the experience replay module, the experience pool adopts a circular buffer structure, and when the pool is full, the new sample replaces the earliest sample; when extracting, the priority is allocated according to the TD error, and the high-priority sample is extracted first. In the experience replay module, the experience pool is realized by adopting a circular buffer structure, and a fixed capacity is set. When the experience pool is full, the new sample automatically replaces the earliest sample, thereby maintaining the freshness of the data in the pool. When extracting the sample, the priority is calculated according to the TD error time difference of each sample. The greater the TD error, the higher the potential value of the sample for model training, and therefore, the high-TD-error sample is extracted first for training, so as to accelerate the convergence of the model and improve the learning efficiency.
[0008] Preferably, in the deep reinforcement learning model, the policy network adopts a multi-layer perceptron structure, the input feature vector is transformed by the hidden layer to output an action probability distribution; and the value network is isomorphic, and inputs the state information to output a value estimate. When the policy network adopts a multi-layer perceptron structure, the hidden layer can adopt an activation function such as ReLU to enhance the non-linear expression capability, and the output layer converts the feature vector into an action probability distribution through a Softmax function. The value network is isomorphic to the policy network, and also utilizes a multi-layer perceptron structure to extract features from the state information through the hidden layer, and finally outputs a value estimate to provide a reference for policy selection.
[0009] Preferably, the deep reinforcement learning model adopts a proximal policy optimization algorithm to update parameters, and limits the policy update amplitude by clipping the objective function to improve the training stability. A target function is defined, which combines the advantage estimate of the current policy and a clipping mechanism to limit the amplitude of policy update. By comparing the action probability ratio of the new policy and the old policy in the same state, the ratio exceeding the threshold is clipped, thereby ensuring the stability of policy update and avoiding training instability or collapse caused by excessive update amplitude.
[0010] Preferably, the behavior decision module adopts an epsilon-greedy strategy to randomly explore actions with an epsilon probability and select optimal actions with a -epsilon probability, and the value of epsilon decreases with training to balance exploration and utilization. In implementing the epsilon-greedy strategy, the behavior decision module can set an initial epsilon value, randomly explore actions with a higher probability at the initial stage of training to increase model diversity, while selecting the current optimal action with a lower probability to ensure certain effectiveness, and gradually decrease the epsilon value as the training progresses to reduce the proportion of random exploration and increase the proportion of optimal action selection, balancing exploration and utilization.
[0011] Preferably, the model evaluation module is further included for periodically evaluating the model performance, the indicators including target achievement rate, action rationality and user satisfaction, and adjusting the training parameters accordingly; The model evaluation module periodically collects system operation data, including user target achievement, behavior rationality score and user satisfaction feedback, calculates various indicators using a preset evaluation algorithm, and then automatically adjusts the training parameters of the deep reinforcement learning model, such as learning rate and discount factor, to optimize the model performance and ensure that the system continuously adapts to changes in user demand.
[0012] Preferably, in the model evaluation module, the target achievement rate is calculated by state difference, the action rationality is evaluated by state target matching degree, and the satisfaction is collected by user feedback questionnaire, and the output end of the model evaluation module is electrically connected with the deep reinforcement learning model and the behavior decision module; When calculating the target achievement rate, the system can compare the difference between the current state and the expected state of the user to quantitatively evaluate the target achievement; the action rationality is measured by comparing the matching degree between the actual action taken by the user and the optimal action or the expected action under the current state; the satisfaction is collected by periodically sending feedback questionnaires to the user to collect the subjective evaluation of the user on the optimization effect of the system.
[0013] Preferably, the course learning module is further included for formulating personalized courses according to the optimization progress and current state, containing difficulty-increasing sub-tasks, and the output end of the course learning module is electrically connected with the state perception module and the behavior decision module; The course learning module generates a personalized course plan containing a series of difficulty-increasing sub-tasks by analyzing the user's optimization progress data and current state, each sub-task aiming to gradually guide the user to improve their atomization habits, and the course learning module dynamically adjusts the difficulty of subsequent courses according to the user's completion of sub-tasks, ensuring the continuity and effectiveness of the learning process, and it interacts with the state perception module and the behavior decision module through electrical connection to realize real-time adjustment of the learning plan.
[0014] In summary, compared with the prior art, the deep reinforcement learning atomization habit optimization system provided by the present application has the following advantages: The application realizes comprehensive perception of the user state through multi-dimensional data acquisition and preprocessing, has the benefits of dynamically capturing changes in individual physiology, psychology, environment and time factors, and solves the problem that traditional habit optimization methods are difficult to adapt to complex scenes and individual differences; the effective processing of high-dimensional state information is realized through a deep reinforcement learning model combined with a policy network and a value network, which has the benefit of accurately mining the potential laws of behavior data, and solves the problems of complex state space, sparse and delayed reward signals and the like faced by traditional reinforcement learning in habit optimization; The optimization adjustment of behavior decision is realized through experience replay and exploration strategy, which has the benefits of improving the adaptability and flexibility of the habit optimization scheme, and solves the problem that traditional methods rely on fixed plans and cannot be dynamically adjusted; the collaborative work of the system components is realized through modular design, which has the benefits of scalability and easy maintenance, and solves the problem that existing deep reinforcement learning applications lack targeted design. BRIEF DESCRIPTION OF DRAWINGS
[0015] Figure 1 is a schematic diagram of the deep reinforcement learning atomization habit optimization system of the application: 1, state perception module; 2, experience generation module; 3, experience replay module; 4, deep reinforcement learning model; 5, behavior decision module; 6, model evaluation module; 7, course learning module. DETAILED DESCRIPTION
[0016] The application provides a technical solution, a deep reinforcement learning atomization habit optimization system, please refer to Figure 1 , comprising: a state perception module 1, an experience generation module 2, an experience replay module 3, a deep reinforcement learning model 4 and a behavior decision module 5; The state perception module 1 is used for collecting physiological data, psychological state data, environmental data and time data, preprocessing to generate a feature vector, and obtaining data through sensors, input devices and third-party interfaces, and the output end of the state perception module 1 is electrically connected with the experience generation module 2; The experience generation module 2 generates experience samples based on state information and behavior actions, and assigns reward values according to reward rules, and the output end of the experience generation module 2 is electrically connected with the experience replay module 3; The experience replay module 3 stores samples through an experience pool, and randomly extracts them for model training to break the time correlation, and the output end of the experience replay module 3 is electrically connected with the deep reinforcement learning model 4; The deep reinforcement learning model 4 includes a policy network and a value network, the policy network outputs an action probability distribution, the value network evaluates the value of the action, and the output end is electrically connected with the behavior decision module 5 through learning samples to optimize parameters; The behavior decision module 5 determines specific actions in combination with the exploration strategy, feeds back to the user and outputs to the experience generation module 2, and the output end is electrically connected with the experience generation module 2; The deep reinforcement learning atomization habit optimization system comprehensively collects physiological, psychological, environmental and time data of the user through the state perception module 1, providing a solid foundation for subsequent analysis. The combination of the experience generation module 2 and the experience replay module 3 enables the system to efficiently generate and utilize experience samples for model training, effectively breaking time correlation and improving learning efficiency. The policy network and value network in the deep reinforcement learning model 4 work cooperatively to optimize action selection, ensuring the scientificity and rationality of decision-making. The behavior decision module 5 combines exploration strategies to ensure decision quality while encouraging the system to explore new actions, enhancing adaptability and flexibility. Overall, the system achieves precise optimization of the user's atomization habit through modular design.
[0017] Please refer to Figure 1 In the state perception module 1, physiological data is collected through wearable devices for heart rate, blood pressure and sleep quality data; psychological state data is collected through psychological questionnaires and voice emotion recognition; environmental data is obtained through sensors and positioning devices; and time data is obtained from the system clock. Physiological data can be collected through smart bracelets or watches integrated with heart rate and blood pressure monitoring functions. Sleep quality is analyzed using built-in algorithms in these devices. Psychological state is comprehensively judged through regular psychological questionnaires and real-time voice emotion recognition algorithms (such as based on voiceprint and tone analysis). Environmental data is obtained using temperature and humidity sensors, air quality sensors and GPS positioning devices. Time data is provided synchronously by the system's built-in clock. Through multi-source data collection, including physiological, psychological, environmental and time dimensions, comprehensive monitoring of the user's state is achieved, providing a solid foundation for subsequent optimization. Secondly, real-time collection and analysis of physiological data enable users to immediately understand their health status and adjust their lifestyle. Psychological state assessment helps users manage their emotions and improve their psychological health. The acquisition of environmental data allows users to make adaptive adjustments based on environmental changes, such as adjusting indoor temperature and humidity. The introduction of time data enables the system to provide personalized optimization suggestions based on the characteristics of different time periods. Overall, this module provides strong support for subsequent deep reinforcement learning and behavior decision-making through comprehensive and accurate state perception, effectively improving user experience and system efficiency.
[0018] Please refer to Figure 1 In the experience generation module 2, reward rules are set based on target achievement, behavior persistence and state adaptability. Positive rewards are given for behaviors close to the target, negative rewards for deviating behaviors, incremental rewards for persistent beneficial behaviors, and additional rewards for optimal behaviors in specific states. In the experience generation module 2, the specific implementation of the reward rule can be realized by setting quantitative indicators, for example, the target achievement degree can be scored according to the comparison between the user's set nebulization habit improvement target, such as reducing the number of daily nebulization, and the actual achievement; the behavior persistence is measured by the number of consecutive days of meeting the target; and the state adaptability can be evaluated according to the matching degree of the user's current physiological and psychological state and the environment, and a positive score is given when approaching the target, and a score is deducted when deviating, the incremental reward for continuous compliance stimulates the user's long-term adherence, and the additional reward for the optimal behavior in a specific state further improves the personalized adaptability of the system and the user's satisfaction. The reward rule design in the experience generation module 2 realizes the fine management of user behavior by comprehensively considering the target achievement degree, behavior persistence and state adaptability. This design not only encourages users to approach and achieve the target, but also provides timely feedback on the effect of behavior through positive and negative reward mechanisms, enhances user motivation, and the incremental reward for continuous beneficial behavior stimulates the user's long-term adherence, and the additional reward for the optimal behavior in a specific state further improves the personalized adaptability of the system and the user's satisfaction.
[0019] Please refer to Figure 1 In the experience replay module 3, the experience pool adopts a circular buffer structure, and when the pool is full, the new sample covers the earliest sample; when extracting, the priority is allocated according to the TD error, and the high-priority sample is extracted first; In the experience replay module 3, the experience pool is implemented by a circular buffer structure, and a fixed capacity is set. When the experience pool is full, the new sample automatically covers the earliest sample, maintaining the freshness of the data in the pool. When extracting samples, the priority is calculated according to the TD error time difference error of each sample. The larger the TD error, the higher the potential value of the sample for model training, so high-TD-error samples are extracted first for training to speed up model convergence and improve learning efficiency; The design of the experience replay module 3 provides an efficient and stable learning mechanism for the deep reinforcement learning model 4. The circular buffer structure of the experience pool effectively manages the storage and update of sample data, avoids data redundancy, and ensures that the data used for model training is the latest, which helps the model to adapt to the changing environment. The extraction strategy based on TD error priority further improves the learning efficiency, enabling the model to capture key information more quickly and speed up the convergence process. This design not only improves the training speed, but also enhances the generalization ability of the model.
[0020] Please refer to Figure 1 In the deep reinforcement learning model 4, the policy network adopts a multi-layer perceptron structure, and the input feature vector is transformed by the hidden layer to output the action probability distribution; the value network is isomorphic, and the input state information outputs the value estimate; When the policy network adopts a multi-layer perceptron structure, the hidden layer can use an activation function such as ReLU to enhance the non-linear representation capability, and the output layer converts the feature vector into an action probability distribution through a Softmax function. The value network is isomorphic to the policy network and also uses a multi-layer perceptron structure to extract features from the state information through the hidden layer, and finally outputs a value estimate to provide a reference for policy selection. In the deep reinforcement learning model 4, the cooperation between the policy network and the value network brings significant advantages to the system. The policy network effectively captures the complex relationship between state features and actions through a multi-layer perceptron structure, and outputs an action probability distribution to provide flexible and diverse choices for behavior decision-making. The value network evaluates the value of actions in different states to provide an objective basis for policy selection, ensuring that the system can make optimal decisions when faced with complex environments. This structure not only improves the decision-making ability of the system, but also enhances its adaptability and robustness.
[0021] Please refer to Figure 1 The deep reinforcement learning model 4 uses a proximal policy optimization algorithm to update parameters and limits the policy update amplitude through a clipping objective function to improve training stability. A target function is defined, which combines the advantage estimate of the current policy and the clipping mechanism to limit the amplitude of policy updates. By comparing the action probability ratio of the new policy and the old policy in the same state, and clipping the ratio that exceeds the threshold, the stability of policy updates is ensured, avoiding training instability or collapse caused by excessive update amplitude. The deep reinforcement learning model 4 uses a proximal policy optimization algorithm to update parameters and limits the policy update amplitude through a clipping objective function, which significantly improves the stability of training. The proximal policy optimization algorithm combines advantage estimation and clipping mechanisms to ensure effective policy updates while avoiding training instability caused by excessive policy changes. This stability is particularly important for long-term learning in complex environments, as it allows the model to find a better balance between exploration and exploitation, thereby more efficiently learning the optimal policy. In addition, the proximal policy optimization algorithm also has good convergence and sample efficiency, which can achieve good performance with limited training data, which is of great significance for resource constraints and computational cost considerations in practical applications.
[0022] Please refer to Figure 1 The behavior decision-making module 5 uses an ε-greedy strategy to randomly explore actions with an ε probability and select optimal actions with a 1-ε probability. The value of ε decreases with training to balance exploration and exploitation. In implementing the epsilon-greedy strategy, the behavior decision module 5 can set an initial epsilon value, such as 0.9, and in the early stage of training, randomly explore actions with a high probability of 90% to increase model diversity, while selecting the current optimal action with a low probability of 10% to ensure certain effects. As the training progresses, the epsilon value gradually decreases, such as decreasing by 0.01 per round, to reduce the proportion of random exploration and increase the proportion of optimal action selection, balancing exploration and utilization. The behavior decision module 5 adopts the epsilon-greedy strategy, effectively balancing the relationship between exploration and utilization. In the early stage of training, it randomly explores with a high probability, which can increase action diversity and help the model discover more potential effective strategies, avoiding falling into local optimization. As the training progresses, the epsilon value decreases and the proportion of optimal action selection increases, ensuring that the model further optimizes based on the effective strategies already explored, improving the overall decision-making quality. This strategy not only ensures the model's exploration ability in unknown environments, but also ensures its efficient decision-making ability in known environments, helping the system continuously learn and improve in complex environments and providing more optimized atomization habit suggestions for users.
[0023] Please refer to Figure 1 It also includes a model evaluation module 6, which is used to periodically evaluate model performance, including target achievement rate, action rationality, and user satisfaction, and adjust training parameters accordingly; The model evaluation module 6 periodically collects system operation data, including user target achievement, behavior rationality score, and user satisfaction feedback, calculates various indicators using a pre-set evaluation algorithm, and then automatically adjusts the training parameters of the deep reinforcement learning model 4, such as learning rate and discount factor, to optimize model performance and ensure the system continuously adapts to changes in user demand; This module provides a comprehensive feedback mechanism for the system by periodically evaluating model performance using multi-dimensional indicators such as target achievement rate, action rationality, and user satisfaction. This design not only ensures that the system can continuously adapt and optimize user behavior, but also improves the model's adaptability and accuracy by automatically adjusting training parameters. At the same time, the inclusion of user satisfaction makes the system more user-friendly, better meeting individual user needs and enhancing user experience. Overall, the introduction of the model evaluation module 6 significantly improves the system's intelligence level and user satisfaction.
[0024] Please refer to Figure 1 In the model evaluation module 6, the target achievement rate is calculated by state difference, the action rationality is evaluated by state target matching degree, and the satisfaction is collected through user feedback questionnaires. The output of the model evaluation module 6 is electrically connected to the deep reinforcement learning model 4 and the behavior decision module 5; In calculating the target achievement rate, the system can compare the difference between the current state and the expected state of the user, quantify the evaluation of the target achievement; the action rationality is measured by comparing the matching degree of the actual action taken by the user and the optimal action or expected action under the current state; the satisfaction is measured by sending feedback questionnaires to the user regularly, collecting the subjective evaluation of the user on the optimization effect of the system; By calculating the target achievement rate through the state difference, the system can objectively and quantitatively evaluate the optimization effect, ensuring that the user gradually approaches and achieves the expected health state, secondly, by evaluating the action rationality through the state target matching degree, the system can identify and adjust unreasonable behaviors, ensuring that the actions taken by the user conform to the health optimization target, finally, the introduction of user feedback questionnaire enables the system to collect the subjective satisfaction of the user, further optimizing the system performance and improving the user experience, the output end of the model evaluation module 6 is electrically connected with the deep reinforcement learning model 4 and the behavior decision module 5, realizing the real-time adjustment of the evaluation result to the system parameters, ensuring the continuous optimization of the system.
[0025] Please refer to Figure 1 It also includes a course learning module 7, which formulates personalized courses according to the optimization progress and current state, containing difficulty-increasing sub-tasks, the output end of the course learning module 7 is electrically connected with the state perception module 1 and the behavior decision module 5; The course learning module 7 generates a personalized course plan by analyzing the optimization progress data and current state of the user, which contains a series of difficulty-increasing sub-tasks, each sub-task aims to gradually guide the user to improve their vaping habits, the course learning module 7 dynamically adjusts the difficulty of subsequent courses according to the user's completion of sub-tasks, ensuring the continuity and effectiveness of the learning process, it interacts with the state perception module 1 and the behavior decision module 5 through electrical connection to realize real-time adjustment of the learning plan; The introduction of the course learning module 7 makes the deep reinforcement learning vaping habit optimization system more personalized and intelligent in guiding users to improve their habits, by formulating personalized courses according to the optimization progress and current state, this module can ensure that users at different stages can receive the most suitable learning content for themselves, thereby improving learning efficiency and effectiveness, containing difficulty-increasing sub-tasks, which helps to gradually improve the user's ability and confidence, enabling users to continuously progress in continuous learning and practice, in addition, the electrical connection of the course learning module 7 with the state perception module 1 and the behavior decision module 5 enables the system to perceive the user's state in real time and make corresponding adjustments, further enhancing the adaptability and flexibility of the system.
[0026] It is to be understood that the terminology used herein is for the purpose of describing particular embodiments only and is not intended to be limiting; it is not intended to exclude myriad other embodiments of the present application that other inventors can develop based on the same general inventive concepts embodied by the described embodiments. That is, although the present application is described in terms of particular embodiments and illustrative figures, it should be apparent that the scope of the present application is not limited to these specific embodiments.
[0027] While the embodiments of the application have been shown and described herein, it is to be understood that the scope of the application, jointly pointed out in the appended claims, is not limited to the details of the embodiments shown, and that various changes can be made and equivalents employed without departing from the intended spirit and scope of the application.
Claims
1. A deep reinforcement learning misting habit optimization system, characterized in that, The application relates to a state-aware module (1), an experience generation module (2), an experience replay module (3), a deep reinforcement learning model (4) and a behavior decision module (5). The state-aware module (1) is used for collecting physiological data, psychological state data, environmental data and time data, preprocessing to generate a feature vector, acquiring data through sensors, input devices and third-party interfaces, and electrically connecting the output end of the state-aware module (1) with the experience generation module (2). The experience generation module (2) generates experience samples based on state information and behavior actions, and distributes reward values according to a reward rule, and the output end of the experience generation module (2) is electrically connected with the experience replay module (3). The experience replay module (3) stores samples through an experience pool, randomly extracts samples for model training to break time correlation, and the output end of the experience replay module (3) is electrically connected with the deep reinforcement learning model (4). The deep reinforcement learning model (4) comprises a policy network and a value network, the policy network outputs an action probability distribution, the value network evaluates the value of an action, parameters are optimized through learning samples, and the output end is electrically connected with the behavior decision module (5). The behavior decision module (5) determines specific actions in combination with an exploration strategy, feeds back to the user and outputs to the experience generation module (2), and the output end is electrically connected with the experience generation module (2). In the state-aware module (1), physiological data is collected through wearable devices to obtain heart rate, blood pressure and sleep quality data; psychological state data is collected through psychological questionnaires and voice emotion recognition; environmental data is obtained through sensors and positioning devices; and time data is obtained from a system clock.
2. The deep reinforcement learning atomization habit optimization system of claim 1, wherein: In the experience generation module (2), a reward rule is set according to target achievement, behavior persistence and state adaptability, positive rewards are given to behaviors close to the target, negative rewards are given to deviated behaviors, rewards are increased for continuous beneficial behaviors, and additional rewards are given to optimal behaviors in specific states.
3. The deep reinforcement learning atomization habit optimization system of claim 1, wherein: In the experience replay module (3), the experience pool adopts a circular buffer structure, and when the pool is full, new samples replace the earliest samples; when samples are extracted, priorities are allocated according to TD errors, and high-priority samples are extracted preferentially.
4. The deep reinforcement learning atomization habit optimization system of claim 1, wherein: In the deep reinforcement learning model (4), the policy network adopts a multilayer perceptron structure, input feature vectors are transformed through a hidden layer to output an action probability distribution; the value network is isomorphic, and input state information is output to estimate the value.
5. The deep reinforcement learning atomization habit optimization system of claim 1, wherein: The deep reinforcement learning model (4) adopts a proximal policy optimization algorithm to update parameters, and a target function is cut to limit the updating amplitude of the policy.
6. The deep reinforcement learning atomization habit optimization system of claim 1, wherein: The behavior decision module (5) adopts an epsilon-greedy strategy, randomly explores actions with an epsilon probability, selects optimal actions with a 1-epsilon probability, and decreases the epsilon value with training to balance exploration and utilization.
7. The deep reinforcement learning vaping habit optimization system of claim 1, wherein: The application further comprises a model evaluation module (6) which is used for periodically evaluating the performance of the model, indexes include target achievement rate, action rationality and user satisfaction, and training parameters are adjusted accordingly.
8. The deep reinforcement learning vaping habit optimization system of claim 1, wherein: 9. The deep reinforcement learning atomization habit optimization system of claim 8, wherein: In the model evaluation module (6), the target achievement rate is calculated by state difference, the action rationality is evaluated by state target matching degree, and the satisfaction degree is collected by user feedback questionnaire. The output end of the model evaluation module (6) is electrically connected with the deep reinforcement learning model (4) and the behavior decision module (5) respectively.
10. The deep reinforcement learning atomization habit optimization system of claim 1, wherein: A course learning module (7) is also included. The course learning module (7) formulates a personalized course according to the optimization progress and the current state, and the course includes difficulty-increasing subtasks. The output end of the course learning module (7) is electrically connected with the state perception module (1) and the behavior decision module (5).