Employee health dynamic intervention strategy method and system based on deep reinforcement learning
By employing deep reinforcement learning methods and utilizing multilayer perceptrons and deep Q-networks to dynamically select health intervention strategies, this approach addresses the lack of specificity and real-time capability in traditional health management methods. It enables personalized and automated health interventions, thereby improving the efficiency and effectiveness of health management.
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-04-02
- Publication Date
- 2026-04-03
AI Technical Summary
Traditional health management methods lack specificity and real-time updates, failing to effectively address the diversity and dynamic changes in individual employee health conditions. This leads to health problems becoming obstacles to personal development and corporate productivity.
We employ a deep reinforcement learning-based approach, using a multilayer perceptron model and a deep Q-network to predict future health status based on employees' historical health data, dynamically select the optimal intervention strategy, and combine heuristic exploration strategies to optimize intervention measures.
It enables personalized and automated health interventions, dynamically adjusting intervention strategies based on employees' real-time health status, improving decision-making efficiency and accuracy, and responding promptly to changes and risks in health issues.
Smart Images
Figure CN121789949A_ABST
Abstract
Description
Technical Field
[0001] This invention relates to the fields of health technology and artificial intelligence, and in particular to a method and system for dynamic intervention strategies for employee health based on deep reinforcement learning. Background Technology
[0002] In today's society, with economic development and technological progress, work environments and lifestyles have undergone tremendous changes. While these changes have improved convenience to some extent, they have also brought about a series of health problems. Especially for corporate employees, long working hours, high-intensity pressure, and irregular lifestyles have combined to make health issues a significant obstacle to personal development and corporate productivity. Problems such as chronic fatigue, excessive psychological stress, and neck, shoulder, back, and leg pain have become commonplace for many employees.
[0003] Traditionally, companies have primarily adopted static, one-size-fits-all interventions for employee health management, such as regularly organizing physical examinations and providing gym facilities. However, these measures often overlook the diversity and dynamic changes in employees' health conditions, lacking specificity and real-time relevance. For example, the health challenges and support measures needed by a software engineer who spends long hours sitting in an office and a sales representative who frequently travels for work are clearly different. Similarly, even for the same person, their health condition and required interventions will vary at different times due to factors such as work stress and life events.
[0004] Therefore, there is an urgent need to develop a new health management approach that utilizes advanced data collection and analysis technologies to monitor employees' health status in real time, including both physical and psychological data. Based on this data, the approach should employ intelligent analytical methods, such as machine learning or deep learning, to identify health risks and potential for improvement, thereby tailoring dynamically adjusted intervention plans for each employee. Such plans can not only be more personalized but also adjusted according to real-time changes in employees' health status, thus more effectively promoting employee health and improving work efficiency and quality of life.
[0005] Furthermore, through continuous health management and intervention, companies can collect valuable health data. This data not only helps optimize current interventions but also provides data support for long-term health management strategies. Ultimately, this dynamic, data-driven approach to health intervention will become an important tool for companies to improve employee well-being and build a healthy corporate culture, as well as a key factor in enhancing corporate competitiveness and sustainable development capabilities. Summary of the Invention
[0006] This invention addresses the shortcomings of existing technologies by providing a method and system for dynamic intervention strategies for employee health based on deep reinforcement learning.
[0007] To achieve the above-mentioned objectives, the technical solution adopted by the present invention is as follows:
[0008] A method for dynamic intervention strategy of employee health based on deep reinforcement learning includes the following steps:
[0009] S1: Develop a dataset of employee health intervention strategies, including: health education and promotion,
[0010] Health checkups and screenings, disease management and treatment, behavioral interventions and lifestyle changes, mental health support, work environment improvement, social support and welfare;
[0011] S2: Develop employee health status classification standards;
[0012] Collect employees' historical health data, including medical history, health indicators, and lifestyle.
[0013] Based on historical health data and expert opinions, we developed a classification standard for employee health status, including healthy, sub-healthy, and chronic diseases.
[0014] S3: Based on the Multilayer Perceptron (MLP) model, it uses historical employee data to predict employees' future health status and the benefits of implementing intervention strategies;
[0015] Use an MLP model and input the employee's historical health data.
[0016] The model is trained to predict employees’ future health status using a supervised learning method.
[0017] Analyze the benefits of implementing intervention strategies and evaluate their effectiveness by comparing the predicted results of different intervention strategies.
[0018] S4: Construct a dynamic intervention strategy model for employee health using deep Q-networks and heuristic exploration strategies. Construct a deep Q-network to learn the mapping relationship between employee health status and intervention strategies.
[0019] Design heuristic exploration strategies to effectively search for optimal intervention strategies in complex environments.
[0020] Train the DQN model so that it can dynamically select the optimal intervention strategy based on the employee's current health status.
[0021] S5: Obtain the optimal intervention strategy for each health state.
[0022] After training, the trained DQN model is used to select the optimal intervention strategy for employees in different health states.
[0023] The effectiveness and feasibility of the selected intervention strategies are verified through simulation experiments or practical applications.
[0024] Continuously update and optimize the model to adapt to changes in employee health status and new intervention strategies.
[0025] Furthermore, the sub-steps of S3 are as follows:
[0026] S31: Data Preprocessing:
[0027] The historical health data of employees is cleaned and preprocessed, including missing value handling, outlier detection and handling, and feature normalization.
[0028] S32: Feature Extraction:
[0029] Feature extraction and selection of health data, including: vital signs, lifestyle habits, and disease diagnosis history.
[0030] S33: MLP Model Design:
[0031] Design the structure of a multi-MLP model, including the number of neurons and connection methods in the input layer, hidden layer, and output layer.
[0032] Determine the activation function, loss function, and key parameters of the optimization algorithm for the model.
[0033] S33: Model Training:
[0034] The preprocessed historical health data is input into the MLP model for training.
[0035] The backpropagation algorithm is used to optimize the model parameters by minimizing the loss function.
[0036] S34: Prediction of Future Health Status:
[0037] A trained multilayer perceptron model is used to predict the future health status of employees.
[0038] Input the employee's current health data, and the MLP model outputs the prediction results, which are either a probability distribution of the health status or specific predicted values.
[0039] S35: Benefit assessment of intervention strategy implementation:
[0040] Based on the prediction results and the intervention strategy dataset, the benefits of implementing different intervention strategies are evaluated.
[0041] Compare the predicted results under different intervention strategies, including indicators of the degree of improvement in health status and the degree of risk reduction.
[0042] Furthermore, the MLP model described in S33 includes: an input layer with 50 input nodes, corresponding to 50 features of the dataset.
[0043] First hidden layer: Contains 128 nodes, using the ReLU activation function.
[0044] The second hidden layer contains 64 nodes, followed by a ReLU activation function, and then a Dropout layer with a dropout rate of 0.2.
[0045] The third hidden layer contains 32 nodes and uses the ReLU activation function, followed by a Dropout layer with a dropout rate of 0.1.
[0046] Output layer: has 3 nodes, uses the softmax activation function, and is suitable for multi-class classification.
[0047] Each layer is fully connected. Dropout layers randomly "drop" a portion of neurons during training to reduce overfitting. The ReLU activation function is used to increase the network's non-linearity.
[0048] The loss function uses Categorical Crossentropy, which enables the model to learn the correct probability distribution across the entire output layer, rather than just predicting the correct category.
[0049] The optimization algorithm used is Adam, with a learning rate of 0.01.
[0050] Furthermore, the sub-steps of S4 are as follows:
[0051] S41: Constructing a Deep Q-Network (DQN) model:
[0052] Design the structure of a deep Q-network, including an input layer, hidden layers, and an output layer.
[0053] The input layer receives the health status of employees as input, and the output layer outputs the Q value of each intervention strategy.
[0054] S42: Define the reward function:
[0055] Design a reward function to evaluate the effectiveness of each intervention step.
[0056] The reward function takes into account the degree of improvement in employee health, the degree of risk reduction, and the cost of intervention strategies.
[0057] S43: Design Heuristic Exploration Strategies:
[0058] Design a heuristic algorithm to efficiently search for the optimal policy in the intervention policy space. The heuristic algorithm employs a greedy strategy.
[0059] Define the priority of intervention strategies:
[0060] Each intervention strategy is assigned a priority, which is determined based on indicators such as its contribution to improving health status and cost-effectiveness.
[0061] Select intervention strategies based on priority:
[0062] Based on the employee's current health status and projected future health status, as well as the priority of each intervention strategy, the intervention strategy with the highest potential benefit is selected.
[0063] Prioritize intervention strategies that have the greatest impact on current health status and are the least costly.
[0064] Considering the effects of historical interventions:
[0065] Based on the historical intervention effects on employees, evaluate the effectiveness of various intervention strategies implemented in the past.
[0066] Learn from past experience and avoid choosing intervention strategies that have not worked well in the past.
[0067] Flexible adjustment of intervention strategies:
[0068] The intervention strategy is flexibly adjusted based on changes in employee health status and real-time feedback on the intervention's effectiveness.
[0069] Intervention strategies should be reassessed and updated promptly based on the latest developments to maintain their effectiveness and adaptability.
[0070] S44: Training the DQN model:
[0071] Historical health data and intervention records were used as training datasets to train a deep Q-network model.
[0072] Use an experience replay mechanism to train the model to improve training efficiency and stability.
[0073] Predict the Q-value for each sample using the current DQN model.
[0074] Using the Q-learning algorithm or an improved version, the target Q value for each sample is calculated, and the error between the predicted Q value and the target Q value is used as the loss function to update the DQN model parameters through the backpropagation algorithm.
[0075] S45: Selection of Dynamic Intervention Strategies
[0076] The trained DQN model dynamically selects the optimal intervention strategy based on the employee's current health status.
[0077] At each time step, the DQN model is input based on the current health status, and the intervention strategy with the highest Q value is selected as the optimal strategy.
[0078] Furthermore, the Q-network structure in S41 is an MLP model.
[0079] The first layer is a mapping from the input layer to the hidden layer. The input layer represents the healthy state, and the hidden layer has 64 nodes.
[0080] The second layer is a mapping from one hidden layer to another, containing 32 nodes.
[0081] The third layer is a mapping from the last hidden layer to the output layer, and the nodes in the output layer represent the intervention strategy type.
[0082] This invention also discloses a deep reinforcement learning-based dynamic intervention strategy system for employee health. This system can be used to implement the aforementioned deep reinforcement learning-based dynamic intervention strategy method for employee health, specifically including:
[0083] Intervention strategy data module: Includes storage of strategy data for health education and promotion, health check-ups and screenings, disease management and treatment, behavioral interventions and lifestyle changes, mental health support, work environment improvement, social support and welfare.
[0084] Health Status Classification Module: Input employees' historical health data and formulate employee health status classification standards based on historical health data and expert data, including: healthy, sub-healthy, and chronic diseases.
[0085] Health Status Prediction and Intervention Strategy Benefit Analysis Module: This module is based on the Multilayer Perceptron (MLP) model, uses historical employee data to predict employees' future health status, analyzes the benefits of implementing intervention strategies, and evaluates the effectiveness of different intervention strategies.
[0086] Optimal Intervention Strategy Analysis Module:
[0087] This module, based on a deep Q-network, learns the mapping relationship between employee health status and intervention strategies. It also designs heuristic exploration strategies to effectively search for the optimal intervention strategy in complex environments and trains the deep Q-network to dynamically select the optimal intervention strategy.
[0088] Using a pre-trained DQN model, the optimal intervention strategy is selected for employees in different health states. The effectiveness and feasibility of the selected intervention strategies are verified through simulation experiments or real-world applications, and the model is continuously updated and optimized to adapt to changes in employee health states and new intervention strategies.
[0089] Results Presentation Module: Presents the analysis results from the Optimal Intervention Strategy Analysis Module to the user.
[0090] The present invention also discloses a computer device, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the program to implement the above-mentioned deep reinforcement learning-based dynamic intervention strategy method for employee health.
[0091] The present invention also discloses a computer-readable storage medium storing a computer program thereon, which, when executed by a processor, implements the above-described method for dynamic intervention strategy of employee health based on deep reinforcement learning.
[0092] Compared with the prior art, the advantages of the present invention are as follows:
[0093] Personalized intervention: By modeling and predicting the health status of employees through deep learning models, personalized intervention strategies can be implemented, and the optimal intervention plan can be developed for each employee's health status and needs.
[0094] Automated decision-making: Deep reinforcement learning-based methods can automatically learn and adjust intervention strategies, reducing the burden of manual strategy formulation and improving the efficiency and accuracy of decision-making.
[0095] Dynamic adjustment: This method can dynamically adjust intervention strategies based on employees' real-time health status and environmental changes, and respond promptly to changes in health problems and the emergence of risks.
[0096] Comprehensive consideration: By taking into account various health intervention strategies and the individual characteristics of employees, the optimal health intervention plan can be formulated by comprehensively considering factors such as intervention effectiveness and cost-effectiveness.
[0097] Real-time feedback: Deep reinforcement learning-based methods can monitor employees' health status and intervention effects in real time, and adjust intervention strategies in a timely manner, enabling timely feedback and adjustment. Attached Figure Description
[0098] Figure 1 This is a flowchart of the employee health dynamic intervention strategy method based on deep reinforcement learning in an embodiment of the present invention. Detailed Implementation
[0099] To make the objectives, technical solutions, and advantages of the present invention clearer, the present invention will be further described in detail below with reference to the accompanying drawings and examples.
[0100] This invention provides a method for dynamic intervention strategy of employee health based on deep reinforcement learning, comprising the following steps:
[0101] S1: Develop employee health intervention strategy dataset based on interviews with medical experts and literature review.
[0102] Conduct interviews with medical experts to gather their opinions and experiences.
[0103] Conduct a literature review, including medical and health intervention strategies and best practices.
[0104] Based on the opinions of medical and nursing experts and literature, a dataset of employee health intervention strategies was developed.
[0105] The employee health intervention strategy dataset includes:
[0106] Health education and promotion:
[0107] Provide health education activities, including nutrition guidance, exercise, and stress management.
[0108] Encourage employees to participate in health promotion activities, such as fitness classes and health lectures.
[0109] Health check-ups and screenings:
[0110] Regularly organize health checkups, including physical examinations and vital sign monitoring.
[0111] Implement health risk assessments and disease screenings to detect health problems at an early stage.
[0112] Disease management and treatment:
[0113] Provide disease management plans for common chronic diseases such as hypertension and diabetes.
[0114] Provide medical services and treatment options, including medication and physical therapy.
[0115] Behavioral interventions and lifestyle changes:
[0116] Provide lifestyle intervention programs such as smoking cessation and alcohol reduction.
[0117] Encourage employees to make behavioral changes such as improving their eating habits and increasing their physical activity.
[0118] Mental health support:
[0119] Provide mental health counseling services, including psychotherapy and psychological counseling.
[0120] Organize mental health training and activities to raise employees' awareness of mental health.
[0121] Improved work environment:
[0122] Adjust the work environment to reduce occupational health risks, such as reducing sitting time and providing ergonomic chairs.
[0123] Provide occupational disease prevention facilities and training to ensure the occupational health and safety of employees.
[0124] Social support and welfare:
[0125] Provide social support networks, including employee mutual aid organizations and community service resources.
[0126] Offer benefits such as flexible working hours and health insurance.
[0127] S2: Develop employee health status classification standards based on employees' historical health data and interviews with medical experts;
[0128] Collect employees' historical health data, including medical history, health indicators, and lifestyle.
[0129] Conduct interviews with medical experts to understand their views and suggestions on the health status of employees.
[0130] Based on historical health data and expert opinions, we developed a classification standard for employee health status, including healthy, sub-healthy, and chronic diseases.
[0131] The employee health status classification criteria specifically include:
[0132] Health status:
[0133] Employees without chronic diseases have healthy lifestyles and normal physiological indicators.
[0134] Body mass index (BMI) is within the normal range, and physiological indicators such as blood pressure, blood sugar, and blood lipids are normal.
[0135] There are no long-term health problems, such as chronic pain or fatigue.
[0136] Sub-health state:
[0137] There are some health issues, but they have not yet developed into a clear chronic disease.
[0138] Mild physical discomfort may occur, such as occasional headaches or indigestion.
[0139] Lifestyle may be unhealthy to some extent, such as an unbalanced diet and lack of exercise.
[0140] Chronic disease status:
[0141] They have been diagnosed with one or more chronic diseases, such as hypertension, diabetes, and cardiovascular disease.
[0142] Long-term medication or other interventions are required to control the progression of the disease.
[0143] Lifestyle factors, such as smoking cessation and dietary control, can influence disease control.
[0144] Other states (can be added as needed):
[0145] This includes acute illness states and recovery periods.
[0146] Depending on the actual situation and needs, other specific health statuses can be added.
[0147] S3: Based on the multilayer perceptron model (MLP), it uses historical employee data to predict the future health status of employees and the benefits of implementing intervention strategies;
[0148] Use a multilayer perceptron (MLP) model to input historical health data of employees.
[0149] The model is trained to predict employees’ future health status using a supervised learning method.
[0150] Analyze the benefits of implementing intervention strategies and evaluate their effectiveness by comparing the predicted results of different intervention strategies.
[0151] The sub-steps of S3 are as follows:
[0152] S31: Data Preprocessing:
[0153] The historical health data of employees is cleaned and preprocessed, including missing value handling, outlier detection and handling, and feature normalization.
[0154] S32: Feature Extraction:
[0155] Based on the problem requirements and model characteristics, features are extracted and selected from the health data.
[0156] Use domain knowledge and data analytics techniques to build meaningful features to improve model performance.
[0157] S33: MLP Model Design:
[0158] Design the structure of a multilayer perceptron (MLP) model, including the number of neurons and their connections in the input, hidden, and output layers.
[0159] Determine the activation function, loss function, and key parameters of the optimization algorithm for the model.
[0160] Input layer: There are 50 input nodes, corresponding to 50 features of the dataset.
[0161] First hidden layer: Contains 128 nodes, using the ReLU activation function.
[0162] The second hidden layer contains 64 nodes, followed by a ReLU activation function, and then a Dropout layer with a dropout rate of 0.2.
[0163] The third hidden layer contains 32 nodes and uses the ReLU activation function, followed by a Dropout layer with a dropout rate of 0.1.
[0164] Output layer: has 3 nodes, uses the softmax activation function, and is suitable for multi-class classification.
[0165] Each layer is fully connected. Dropout layers randomly "drop" a portion of neurons during training to reduce overfitting. The ReLU activation function is used to increase the network's non-linearity.
[0166] The loss function uses Categorical Crossentropy, which enables the model to learn the correct probability distribution across the entire output layer, rather than just predicting the correct category.
[0167] The optimization algorithm used is Adam, with a learning rate of 0.01.
[0168] S33: Model Training:
[0169] The preprocessed historical health data is input into the MLP model for training.
[0170] Supervised learning methods, such as backpropagation, are used to optimize model parameters by minimizing the loss function.
[0171] Cross-validation is used to evaluate the model's performance and perform parameter tuning and optimization.
[0172] S34: Prediction of Future Health Status:
[0173] A trained multilayer perceptron model is used to predict the future health status of employees.
[0174] Input the employee's current health data, and the MLP model outputs the prediction results, which are either a probability distribution of the health status or specific predicted values.
[0175] For example, the employee health data entered is as follows:
[0176] Blood pressure: 120 mmHg;
[0177] Blood glucose: 5.8 mmol / L;
[0178] BMI Index: 23;
[0179] Exercise time: 1 hour;
[0180] The model outputs a probability distribution of health status predictions:
[0181] Probability of being in good health: 0.75;
[0182] Probability of being in a sub-healthy state: 0.20;
[0183] Probability of chronic disease state: 0.05.
[0184] This prediction result can be interpreted as follows: the highest probability of an employee's health status being healthy (0.75), followed by a sub-healthy state (0.20), and the lowest probability being a chronic disease state (0.05). This suggests that the employee's current health status is good and may not require much intervention.
[0185] S35: Benefit assessment of intervention strategy implementation:
[0186] Based on the prediction results and the intervention strategy dataset, the benefits of implementing different intervention strategies are evaluated.
[0187] Compare the predicted results under different intervention strategies, including indicators such as the degree of improvement in health status and the degree of risk reduction.
[0188] Taking into account the cost, effectiveness, and feasibility of intervention strategies, the optimal intervention strategy should be selected.
[0189] For example, the intervention strategy dataset contains two intervention strategies: health education and promotion, and regular health check-ups. The dataset evaluates the benefits of implementing these two strategies.
[0190] Intervention Strategy 1: Health Education and Promotion
[0191] Health education and promotion activities include providing health education courses, health lectures, and promoting healthy eating and lifestyles.
[0192] Expected results: Improve employees' health awareness, improve their lifestyles, and reduce the risk of chronic diseases.
[0193] Intervention Strategy Two: Regular Health Checkups
[0194] Regularly organize health checkups, including physical examinations, vital sign monitoring, and health assessments.
[0195] Expected results: Early detection of health problems, timely intervention, and reduction of the risk of developing chronic diseases.
[0196] Based on the prediction results of the MLP model and the prediction results after implementing these two intervention strategies, the following aspects are considered to select the optimal intervention strategy:
[0197] Improvement in health status: Compare the degree of improvement in employee health status before and after the intervention. If one strategy significantly improves the predictive probability of employee health status, then that strategy is likely more effective.
[0198] Risk reduction degree: Compare the degree of risk reduction for employees developing chronic diseases before and after the intervention. If a strategy can reduce the risk of employees developing chronic diseases, then that strategy is likely to be more advantageous.
[0199] Cost and Feasibility: Consider the cost and feasibility of implementing each strategy. If a strategy is less costly, easier to implement, and more effective, then that strategy is likely to be preferred.
[0200] S4: Construct a dynamic intervention strategy model for employee health using deep Q-networks (DQN) and heuristic exploration strategies.
[0201] Construct a deep Q-network (DQN) to learn the mapping relationship between employee health status and intervention strategies.
[0202] Design heuristic exploration strategies to effectively search for optimal intervention strategies in complex environments.
[0203] Train the DQN model so that it can dynamically select the optimal intervention strategy based on the employee's current health status.
[0204] The sub-steps of S4 are as follows:
[0205] S41: Constructing a Deep Q-Network (DQN) model:
[0206] Design the structure of a deep Q-network, including an input layer, hidden layers, and an output layer.
[0207] The input layer receives the health status of employees as input, and the output layer outputs the Q value of each intervention strategy.
[0208] The Q network structure includes:
[0209] The first layer is a mapping from the input layer to the hidden layer. The input layer represents the healthy state, and the hidden layer has 64 nodes.
[0210] The second layer is a mapping from one hidden layer to another, containing 32 nodes.
[0211] The third layer is a mapping from the last hidden layer to the output layer, and the nodes in the output layer represent the intervention strategy type.
[0212] S42: Define the reward function:
[0213] Design a reward function to evaluate the effectiveness of each intervention step.
[0214] The reward function should take into account factors such as the degree of improvement in employee health, the degree of risk reduction, and the cost of intervention strategies.
[0215] For example, the reward function: R(s,a,s′)=α·health_improvement+β·risk_reduction-γ·
[0216] intervention_cost;
[0217] R(s,a,s′) is the reward value, representing the reward for taking action a in state s and reaching state s′.
[0218] health_improvementhealth represents the degree of improvement in health status, calculated based on the changes in health status s and s'.
[0219] risk_reductionrisk represents the degree of risk reduction, which can be calculated based on changes in the health status of s and s'.
[0220] intervention_cost represents the cost of taking action 'a', which can be a direct cost or an indicator to measure the effectiveness of the intervention strategy.
[0221] α, β, and γ are weighting coefficients used to balance the contributions of improved health, reduced risk, and intervention costs to the reward value.
[0222] S43: Design Heuristic Exploration Strategies:
[0223] Design a heuristic algorithm to efficiently search for the optimal policy in the intervention policy space.
[0224] Define the priority of intervention strategies:
[0225] Each intervention strategy is assigned a priority, which is determined based on indicators such as its contribution to improving health status and cost-effectiveness.
[0226] Select intervention strategies based on priority:
[0227] Based on the employee's current health status and projected future health status, as well as the priority of each intervention strategy, the intervention strategy with the highest potential benefit is selected.
[0228] Prioritize intervention strategies that have the greatest impact on current health status and are the least costly.
[0229] Considering the effects of historical interventions:
[0230] Based on the historical intervention effects on employees, evaluate the effectiveness of various intervention strategies implemented in the past.
[0231] Learn from past experience and avoid choosing intervention strategies that have not worked well in the past.
[0232] Flexible adjustment of intervention strategies:
[0233] The intervention strategy is flexibly adjusted based on changes in employee health status and real-time feedback on the intervention's effectiveness.
[0234] Intervention strategies should be reassessed and updated promptly based on the latest developments to maintain their effectiveness and adaptability.
[0235] S44: Training the DQN model:
[0236] Historical health data and intervention records were used as training datasets to train a deep Q-network model.
[0237] The experience replay mechanism is used to train the model to improve training efficiency and stability.
[0238] Predict the Q-value for each sample using the current DQN model.
[0239] Using the Q-learning algorithm or its improved versions (such as Double Q-learning, Dueling Q-learning, etc.), the target Q value for each sample is calculated, and the error between the predicted Q value and the target Q value is used as the loss function to update the DQN model parameters through the backpropagation algorithm.
[0240] S45: Selection of Dynamic Intervention Strategies
[0241] The trained DQN model dynamically selects the optimal intervention strategy based on the employee's current health status.
[0242] At each time step, the DQN model is input based on the current health status, and the intervention strategy with the highest Q value is selected as the optimal strategy.
[0243] S5: Obtain the optimal intervention strategy for each health state.
[0244] After training, the trained DQN model is used to select the optimal intervention strategy for employees in different health states.
[0245] The effectiveness and feasibility of the selected intervention strategies are verified through simulation experiments or practical applications.
[0246] Continuously update and optimize the model to adapt to changes in employee health status and new intervention strategies.
[0247] In another embodiment of the present invention, a dynamic intervention strategy system for employee health based on deep reinforcement learning is provided. This system can be used to implement the above-mentioned dynamic intervention strategy method for employee health based on deep reinforcement learning, specifically including:
[0248] Intervention strategy data module: Includes storage of strategy data for health education and promotion, health check-ups and screenings, disease management and treatment, behavioral interventions and lifestyle changes, mental health support, work environment improvement, social support and welfare.
[0249] Health Status Classification Module: Input employees' historical health data and formulate employee health status classification standards based on historical health data and expert data, including: healthy, sub-healthy, and chronic diseases.
[0250] Health Status Prediction and Intervention Strategy Benefit Analysis Module: This module is based on the Multilayer Perceptron (MLP) model, uses historical employee data to predict employees' future health status, analyzes the benefits of implementing intervention strategies, and evaluates the effectiveness of different intervention strategies.
[0251] Optimal Intervention Strategy Analysis Module:
[0252] This module, based on a deep Q-network, learns the mapping relationship between employee health status and intervention strategies. It also designs heuristic exploration strategies to effectively search for the optimal intervention strategy in complex environments and trains the deep Q-network to dynamically select the optimal intervention strategy.
[0253] Using a pre-trained DQN model, the optimal intervention strategy is selected for employees in different health states. The effectiveness and feasibility of the selected intervention strategies are verified through simulation experiments or real-world applications, and the model is continuously updated and optimized to adapt to changes in employee health states and new intervention strategies.
[0254] Results Presentation Module: Presents the analysis results from the Optimal Intervention Strategy Analysis Module to the user;
[0255] This module specifically includes the following functions:
[0256] Recommended intervention strategies:
[0257] The optimal intervention strategies recommended for employees with different health conditions are presented in the form of lists or graphs, so that users can clearly understand the specific content and recommendation level of each intervention strategy.
[0258] Intervention effect evaluation presentation:
[0259] The presentation showcases the predicted and actual effects of each intervention strategy, including indicators such as the predicted degree of improvement in health status and the degree of risk reduction, as well as changes in employee health status after actual application.
[0260] Decision support information display:
[0261] It provides comprehensive evaluation and comparison of different intervention strategies, including cost-benefit analysis and feasibility assessment of the intervention strategies, to provide decision support for users or decision-makers.
[0262] User interface design:
[0263] The system should be designed with a user-friendly interface to allow users to easily browse and understand the system's output, while also providing interactive functions such as filtering intervention strategies and adjusting parameters.
[0264] Applications of visualization technology:
[0265] Using visualization techniques such as charts and graphs to display data and results enables users to intuitively understand and analyze the information output by the system.
[0266] Reporting and summary functions:
[0267] It provides report generation and summary functions, outputting the system results in the form of reports, including detailed descriptions of intervention strategies, effect evaluations, decision recommendations, etc., to facilitate users to conduct further analysis and share.
[0268] In another embodiment of the present invention, a terminal device is provided, comprising a processor and a memory. The memory stores a computer program, which includes program instructions. The processor executes the program instructions stored in the computer storage medium. The processor may be a Central Processing Unit (CPU), or other general-purpose processors, digital signal processors (DSPs), application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. It is the computing and control core of the terminal, suitable for implementing one or more instructions, specifically suitable for loading and executing one or more instructions to achieve corresponding method flows or corresponding functions. The processor described in this embodiment of the present invention can be used for the operation of a method for dynamic intervention strategies for employee health based on deep reinforcement learning, including the following steps:
[0269] S1: Develop a dataset of employee health intervention strategies, including: health education and promotion,
[0270] Health checkups and screenings, disease management and treatment, behavioral interventions and lifestyle changes, mental health support, work environment improvement, social support and welfare;
[0271] S2: Develop employee health status classification standards;
[0272] Collect employees' historical health data, including medical history, health indicators, and lifestyle.
[0273] Based on historical health data and expert opinions, we developed a classification standard for employee health status, including healthy, sub-healthy, and chronic diseases.
[0274] S3: Based on the Multilayer Perceptron (MLP) model, it uses historical employee data to predict employees' future health status and the benefits of implementing intervention strategies;
[0275] Use an MLP model and input the employee's historical health data.
[0276] The model is trained to predict employees’ future health status using a supervised learning method.
[0277] Analyze the benefits of implementing intervention strategies and evaluate their effectiveness by comparing the predicted results of different intervention strategies.
[0278] S4: Construct a dynamic intervention strategy model for employee health using deep Q-networks and heuristic exploration strategies. Construct a deep Q-network to learn the mapping relationship between employee health status and intervention strategies.
[0279] Design heuristic exploration strategies to effectively search for optimal intervention strategies in complex environments.
[0280] Train the DQN model so that it can dynamically select the optimal intervention strategy based on the employee's current health status.
[0281] S5: Obtain the optimal intervention strategy for each health state.
[0282] After training, the trained DQN model is used to select the optimal intervention strategy for employees in different health states.
[0283] The effectiveness and feasibility of the selected intervention strategies are verified through simulation experiments or practical applications.
[0284] Continuously update and optimize the model to adapt to changes in employee health status and new intervention strategies.
[0285] In another embodiment of the present invention, a storage medium is provided, specifically a computer-readable storage medium (Memory). This computer-readable storage medium is a memory device in a terminal device used to store programs and data. It is understood that the computer-readable storage medium here can include both the built-in storage medium in the terminal device and extended storage media supported by the terminal device. The computer-readable storage medium provides storage space that stores the terminal's operating system. Furthermore, this storage space also stores one or more instructions suitable for loading and execution by a processor. These instructions can be one or more computer programs (including program code). It should be noted that the computer-readable storage medium here can be high-speed RAM or non-volatile memory, such as at least one disk storage device.
[0286] One or more instructions stored in a computer-readable storage medium can be loaded and executed by a processor to implement the corresponding steps of the method for dynamic intervention strategy of employee health based on deep reinforcement learning in the above embodiments; one or more instructions in the computer-readable storage medium are loaded and executed by the processor in the following steps:
[0287] S1: Develop a dataset of employee health intervention strategies, including: health education and promotion,
[0288] Health checkups and screenings, disease management and treatment, behavioral interventions and lifestyle changes, mental health support, work environment improvement, social support and welfare;
[0289] S2: Develop employee health status classification standards;
[0290] Collect employees' historical health data, including medical history, health indicators, and lifestyle.
[0291] Based on historical health data and expert opinions, we developed a classification standard for employee health status, including healthy, sub-healthy, and chronic diseases.
[0292] S3: Based on the Multilayer Perceptron (MLP) model, it uses historical employee data to predict employees' future health status and the benefits of implementing intervention strategies;
[0293] Use an MLP model and input the employee's historical health data.
[0294] The model is trained to predict employees’ future health status using a supervised learning method.
[0295] Analyze the benefits of implementing intervention strategies and evaluate their effectiveness by comparing the predicted results of different intervention strategies.
[0296] S4: Construct a dynamic intervention strategy model for employee health using deep Q-networks and heuristic exploration strategies. Construct a deep Q-network to learn the mapping relationship between employee health status and intervention strategies.
[0297] Design heuristic exploration strategies to effectively search for optimal intervention strategies in complex environments.
[0298] Train the DQN model so that it can dynamically select the optimal intervention strategy based on the employee's current health status.
[0299] S5: Obtain the optimal intervention strategy for each health state.
[0300] After training, the trained DQN model is used to select the optimal intervention strategy for employees in different health states.
[0301] The effectiveness and feasibility of the selected intervention strategies are verified through simulation experiments or practical applications.
[0302] Continuously update and optimize the model to adapt to changes in employee health status and new intervention strategies.
[0303] Those skilled in the art will understand that embodiments of the present invention can be provided as methods, systems, or computer program products. Therefore, the present invention can take the form of a completely hardware embodiment, a completely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, the present invention can take the form of a computer program product embodied on one or more computer-usable storage media (including, but not limited to, disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.
[0304] This invention is described with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of the invention. It will be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, special-purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, generate instructions for implementing the flowchart illustrations and / or block diagrams. Figure 1 One or more processes and / or boxes Figure 1 A device that provides the functions specified in one or more boxes.
[0305] These computer program instructions may also be stored in a computer-readable storage medium that can direct a computer or other programmable data processing device to function in a particular manner, such that the instructions stored in the computer-readable storage medium produce an article of manufacture including instruction means, which are implemented in a process Figure 1 One or more processes and / or boxes Figure 1 The function specified in one or more boxes.
[0306] These computer program instructions may also be loaded onto a computer or other programmable data processing equipment to cause a series of operational steps to be performed on the computer or other programmable equipment to produce a computer-implemented process, thereby providing instructions that execute on the computer or other programmable equipment for implementing the process. Figure 1 One or more processes and / or boxes Figure 1 The steps of the function specified in one or more boxes.
[0307] Those skilled in the art will recognize that the embodiments described herein are intended to help the reader understand the implementation methods of the present invention, and should be understood that the scope of protection of the present invention is not limited to such specific statements and embodiments. Those skilled in the art can make various other specific modifications and combinations based on the technical teachings disclosed in this invention without departing from the spirit of the invention, and these modifications and combinations are still within the scope of protection of the present invention.
Claims
1. A method for dynamic intervention strategy of employee health based on deep reinforcement learning, characterized in that, Includes the following steps: S1: Develop a dataset of employee health intervention strategies, including: health education and promotion, Health checkups and screenings, disease management and treatment, behavioral interventions and lifestyle changes, mental health support, work environment improvement, social support and welfare; S2: Develop employee health status classification standards; Collect employees' historical health data, including medical history, health indicators, and lifestyle. Based on historical health data and expert opinions, we developed a classification standard for employee health status, including healthy, sub-healthy, and chronic diseases. S3: Based on the multilayer perceptron (MLP) model, use historical employee data to predict future employee health status and the benefits of intervention strategies. Use an MLP model and input historical health data of employees; The model is trained to predict employees’ future health status using a supervised learning method. Analyze the benefits of implementing intervention strategies and evaluate their effectiveness by comparing the predicted results of different intervention strategies; S4: Construct a dynamic intervention strategy model for employee health using deep Q-networks and heuristic exploration strategies. Construct a deep Q-network to learn the mapping relationship between employee health status and intervention strategies. Design heuristic exploration strategies to effectively search for optimal intervention strategies in complex environments; Train the DQN model so that it can dynamically select the optimal intervention strategy based on the employee's current health status; S5: Obtain the optimal intervention strategy for each health state. After training, the trained DQN model is used to select the optimal intervention strategy for employees in different health states. The effectiveness and feasibility of the selected intervention strategies are verified through simulation experiments or practical applications; Continuously update and optimize the model to adapt to changes in employee health status and new intervention strategies.
2. The employee health dynamic intervention strategy method according to claim 1, characterized in that: The sub-steps of S3 are as follows: S31: Data Preprocessing: Clean and preprocess the employees' historical health data, including missing value handling, outlier detection and handling, and feature normalization; S32: Feature Extraction: Feature extraction and selection of health data, including: vital signs, lifestyle habits and disease diagnosis history; S33: MLP Model Design: Design the structure of a multi-MLP model, including the number of neurons and connection methods in the input layer, hidden layer, and output layer; Determine the activation function, loss function, and key parameters of the optimization algorithm for the model; S33: Model Training: The preprocessed historical health data is input into the MLP model for training; The backpropagation algorithm is used to optimize the model parameters by minimizing the loss function; S34: Prediction of Future Health Status: Using a trained multilayer perceptron model, predict the future health status of employees; Input the employee's current health data, and the MLP model outputs the prediction results, which are either a probability distribution of the health status or specific predicted values. S35: Benefit assessment of intervention strategy implementation: Based on the prediction results and intervention strategy dataset, evaluate the benefits of implementing different intervention strategies; Compare the predicted results under different intervention strategies, including indicators of the degree of improvement in health status and the degree of risk reduction.
3. The employee health dynamic intervention strategy method according to claim 2, characterized in that: The MLP model described in S33 includes: an input layer with 50 input nodes, corresponding to 50 features of the dataset; First hidden layer: Contains 128 nodes, using the ReLU activation function; The second hidden layer contains 64 nodes, followed by a ReLU activation function, and then a Dropout layer with a dropout rate of 0.
2. The third hidden layer contains 32 nodes and uses the ReLU activation function, followed by a Dropout layer with a dropout rate of 0.
1. Output layer: has 3 nodes, uses the softmax activation function, and is suitable for multi-class classification; Each layer is fully connected; the Dropout layer randomly "drops" a portion of neurons during training to reduce overfitting; the ReLU activation function is used to increase the network's non-linearity. The loss function uses Categorical Crossentropy, which enables the model to learn the correct probability distribution across the entire output layer, rather than just predicting the correct category. The optimization algorithm used is Adam, with a learning rate of 0.
01.
4. The employee health dynamic intervention strategy method according to claim 1, characterized in that: The sub-steps of S4 are as follows: S41: Constructing a Deep Q-Network (DQN) model: Design the structure of a deep Q-network, including an input layer, hidden layers, and an output layer; The input layer receives the health status of employees as input, and the output layer outputs the Q value of each intervention strategy; S42: Define the reward function: Design a reward function to evaluate the effectiveness of each intervention step; The reward function considers the degree of improvement in employee health, the degree of risk reduction, and the cost of intervention strategies; S43: Design Heuristic Exploration Strategies: Design a heuristic algorithm to efficiently search for the optimal policy in the intervention policy space. The heuristic algorithm employs a greedy strategy. Define the priority of intervention strategies: Each intervention strategy is assigned a priority, which is determined based on indicators such as its contribution to improving health status and cost-effectiveness. Select intervention strategies based on priority: Based on the employee's current health status and predicted future health status, and the priority of each intervention strategy, select the intervention strategy with the highest potential benefit. Prioritize intervention strategies that have the greatest impact on current health status and are the lowest cost; Considering the effects of historical interventions: Based on the historical intervention effects on employees, evaluate the effectiveness of various intervention strategies implemented in the past. Drawing on past experience, avoid choosing intervention strategies that have not been effective in the past; Flexible adjustment of intervention strategies: The intervention strategy is flexibly adjusted based on changes in employee health status and real-time feedback on the intervention effect. Intervention strategies should be reassessed and updated promptly based on the latest developments to maintain their effectiveness and adaptability. S44: Training the DQN model: Historical health data and intervention records were used as training datasets to train the deep Q-network model. Use an experience replay mechanism to train the model to improve training efficiency and stability; Predict the Q-value for each sample using the current DQN model. Using the Q-learning algorithm or an improved version, the target Q value for each sample is calculated, and the error between the predicted Q value and the target Q value is used as the loss function to update the DQN model parameters through the backpropagation algorithm. S45: Selection of Dynamic Intervention Strategies The trained DQN model dynamically selects the optimal intervention strategy based on the employee's current health status; At each time step, the DQN model is input based on the current health status, and the intervention strategy with the highest Q value is selected as the optimal strategy.
5. The employee health dynamic intervention strategy method according to claim 4, characterized in that: The Q-network structure in S41 is an MLP model; The first layer is a mapping from the input layer to the hidden layer. The input layer represents the healthy state, and the hidden layer has 64 nodes. The second layer is a mapping from one hidden layer to another, containing 32 nodes. The third layer is a mapping from the last hidden layer to the output layer, and the nodes in the output layer represent the intervention strategy type.
6. A dynamic intervention strategy system for employee health based on deep reinforcement learning, characterized in that: This system can be used to implement the employee health dynamic intervention strategy method according to any one of claims 1 to 5, specifically including: Intervention strategy data module: This module includes data on strategies for health education and promotion, health check-ups and screenings, disease management and treatment, behavioral interventions and lifestyle changes, mental health support, work environment improvement, social support and welfare. Health Status Classification Module: Input employees' historical health data and formulate employee health status classification standards based on historical health data and expert data, including: healthy, sub-healthy, and chronic diseases; Health Status Prediction and Intervention Strategy Benefit Analysis Module: This module is based on the Multilayer Perceptron (MLP) model, uses historical employee data to predict employees' future health status, analyzes the benefits of implementing intervention strategies, and evaluates the effectiveness of different intervention strategies. Optimal Intervention Strategy Analysis Module: This module is based on a deep Q-network to learn the mapping relationship between employee health status and intervention strategies. At the same time, it designs a heuristic exploration strategy to effectively search for the optimal intervention strategy in complex environments and trains the deep Q-network to dynamically select the optimal intervention strategy. Using a trained DQN model, the optimal intervention strategy is selected for employees in different health states. The effectiveness and feasibility of the selected intervention strategy are verified through simulation experiments or real-world applications. The model is continuously updated and optimized to adapt to changes in employee health states and new intervention strategies. Results Presentation Module: Presents the analysis results from the Optimal Intervention Strategy Analysis Module to the user.
7. A computer device, characterized in that: It includes a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor, when executing the program, implements the employee health dynamic intervention strategy method according to any one of claims 1 to 5.
8. A computer-readable storage medium, characterized in that: It stores a computer program that, when executed by a processor, implements the employee health dynamic intervention strategy method as described in any one of claims 1 to 5.