Closed-loop personalized healthy drink management system and method based on AIoT
By constructing an AIoT closed-loop health beverage management system, the problems of incomplete data, open-loop architecture, and rigid models in existing systems have been solved. This system enables accurate perception of users' health status and personalized recommendations, has self-learning capabilities, and provides continuously evolving health beverage services.
Patent Information
- Application Number
- CN202511765221.2
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2025-11-27
- Publication Date
- 2026-02-27
AI Technical Summary
Existing health drink recommendation systems suffer from limitations in data dimensions, non-closed-loop system architecture, and shallow knowledge models, resulting in an inability to accurately match users' health needs and an inability to learn and adjust themselves.
A closed-loop personalized health beverage management system based on AIoT is constructed, including a multi-dimensional data acquisition and fusion module, a personalized recommendation engine module, an intelligent execution and feedback module, and a self-learning optimization module. Deep learning and reinforcement learning algorithms are used to realize a complete closed loop of perception-decision-execution-feedback.
It achieves a comprehensive understanding of users' health status, providing personalized, actionable, and continuously evolving health drink recommendations, thus improving the accuracy of recommendations and the system's adaptability.
Smart Images
Figure CN121581964A_ABST
Abstract
Description
TECHNICAL FIELD
[0001] The application belongs to the field of digital health and artificial intelligence Internet of Things, and particularly relates to an AIoT-based closed-loop personalized health beverage management system and method. BACKGROUND
[0002] With the improvement of people's health awareness, digital health management has become a trend. However, the existing health beverage (such as tea, coffee, functional beverage, etc.) recommendation and management system generally has the following three structural defects, which seriously restricts its application effect and intelligent level:
[0003] Defect one: limitation of data dimension (data defectiveness).
[0004] Most of the existing technologies only rely on a single data source (such as historical purchase records or simple drinking frequency) for recommendation. For example, traditional e-commerce recommendation algorithms are only based on analyzing the purchase preferences of users, and cannot perceive the current health status of users and external environment (such as temperature changes and seasonal changes). The lack of data dimension leads to the fact that the recommended results are often "what users like in the past", rather than "what users need now", and cannot accurately match the current health needs of users;
[0005] Defect two: non-closed-loop of system architecture (open-loop architecture).
[0006] Existing systems are usually one-way and present an "open-loop" structure. One type is a software system that only makes recommendations (such as various health apps), which cannot know whether the user has adopted the recommendations. Another type is a hardware system that only records (such as ordinary smart water cups), which can record the amount of drinking water but lack a decision-making brain and cannot provide recommendations. The lack of a complete closed loop that connects "recommendation decisions" with "physical execution" and "effect feedback" leads to the fact that the system cannot perceive the execution effect and the algorithm cannot verify its accuracy, so it cannot be effectively iterated;
[0007] Defect three: shallow knowledge model (model rigidity).
[0008] Most of the existing technologies rely on fixed rule engines (such as "if you have a fever, recommend green tea"), and this simple IF-THEN logic has two problems: one is that it cannot handle complex and composite health states (such as "qi deficiency and damp heat" or "yang deficiency but fever"); two is that once the rules are set, they are difficult to change, and the model cannot be self-learned and parameter-adjusted according to user feedback data. The system cannot dynamically adjust the strategy as the user's physical condition changes, making it difficult to achieve "one size fits all" dynamic service. SUMMARY
[0009] The technical problem to be solved by this invention is to address the shortcomings of the prior art by providing a closed-loop personalized health beverage management system and method based on AIoT. It achieves adaptive evolution of recommendation strategies by constructing a complete closed loop of "perception-decision-execution-feedback" and introducing deep learning and reinforcement learning algorithms.
[0010] The technical solution adopted in this invention is: a closed-loop personalized health beverage management system based on AIoT, characterized in that: it includes a multi-dimensional data acquisition and fusion module, a personalized recommendation engine module, an intelligent execution and feedback module, and a self-learning optimization module;
[0011] The multidimensional data acquisition and fusion module is used to acquire user health data from multiple dimensions, and to obtain the user's health status demand vector based on a dynamic weighting algorithm according to the acquired multidimensional health data.
[0012] The personalized recommendation engine module is used to generate personalized health intervention plans based on the user's health status demand vector and a preset bimodal knowledge graph.
[0013] The intelligent execution and feedback module includes at least one intelligent terminal device, which receives and assists the user in executing the health intervention plan, and collects objective behavioral data and subjective feedback information during the user's execution process.
[0014] The self-learning optimization module is used to optimize the parameters of the personalized recommendation engine module based on the objective behavioral data and subjective feedback information through a preset machine learning algorithm, so as to improve the accuracy of subsequent health intervention plans.
[0015] Preferably, the multidimensional data acquisition and fusion module includes a static data acquisition unit, a dynamic data acquisition unit, an environmental data acquisition unit, and a data fusion unit;
[0016] The static data acquisition unit is used to obtain users' baseline health data through an interactive health assessment questionnaire;
[0017] The dynamic data acquisition unit communicates with the smart terminal to continuously acquire user behavior data in real time; the environmental data acquisition unit is used to collect environmental factors of the user's geographical location.
[0018] The data fusion unit is used to integrate three layers of heterogeneous data—user baseline health data, behavioral data, and environmental factors—in real time using a dynamic weighted fusion algorithm to obtain a comprehensive demand vector for user health status.
[0019] Preferably, the dynamic weighted fusion algorithm includes:
[0020] The collected multi-source heterogeneous data is normalized and mapped to a unified numerical range;
[0021] Preset the baseline weights for each dimension of data;
[0022] Based on the current time point and user status conflict, the baseline weight is adjusted through a preset dynamic correction function to generate dynamic weight;
[0023] The normalized data is weighted and summed based on dynamic weights to output the demand vector.
[0024] Preferably, the personalized recommendation engine module includes a dual-modal retrieval unit, a collaborative filtering and sorting unit, and an intelligent recommendation unit;
[0025] The bimodal retrieval unit retrieves beverages that match the user's demand vector in the bimodal knowledge graph.
[0026] The collaborative filtering sorting unit sorts the search results based on the collaborative filtering algorithm;
[0027] The recommendation unit outputs a structured drinking plan based on the ranking results, including the recommended beverage, drinking time, drinking dosage, and brewing parameters of the recommended beverage.
[0028] Preferably, the bimodal knowledge graph includes a first modal attribute set and a second modal attribute set;
[0029] The first modality attribute set is a quantitative attribute set analyzed by modern science, including the chemical composition content of each beverage and its physiological effects verified by evidence-based medicine.
[0030] The second modal attribute set is a qualitative attribute set based on traditional empirical theories, including the properties, flavors, meridian tropism, and traditional conditioning effects of each beverage.
[0031] Preferably, the collaborative filtering algorithm includes:
[0032] Calculate the cosine similarity between the current user and other users;
[0033] Select the K neighboring users with the highest similarity;
[0034] Based on the historical positive feedback rate of neighboring users for a specific beverage, predict the current user's preference rating for that beverage and adjust the ranking accordingly.
[0035] Preferably, the self-learning optimization module includes a reinforcement learning unit, which employs a dual-ring Bayesian potential field model, including:
[0036] The inner loop submodule is used to infer the posterior probability distribution of a user's current health status based on a Bayesian belief network, using multidimensional input data as evidence nodes.
[0037] The outer ring submodule is used to select recommended actions based on the policy network and the posterior probability distribution, and to calculate reward values based on feedback data.
[0038] The parameter update submodule is used to update the parameters of the recommendation unit based on the reward value using a policy gradient algorithm.
[0039] Preferably, when updating parameters, the parameter update submodule introduces a regularization constraint term based on the prior knowledge of the bimodal knowledge graph, and calculates the distance between the attribute vector of the current recommendation result and the prior knowledge vector as a penalty term to restrict the recommendation result to conform to preset scientific rules.
[0040] A closed-loop personalized health beverage management method based on AIoT includes the following steps:
[0041] S1: Obtain the user's multidimensional health data and fuse the multidimensional heterogeneous health data to generate a user health demand vector;
[0042] S2: Based on the user's health needs vector, retrieve the beverage with the highest matching degree with the needs vector in the bimodal knowledge graph, and generate a health intervention plan including drinking time, dosage and brewing parameters.
[0043] S3: Receive and guide the execution of health intervention plans through smart terminal devices, and collect actual execution data and user feedback data simultaneously during the execution process;
[0044] S4: Quantify actual execution data and user feedback data into reward signals, and use machine learning algorithms to update the parameters of the recommendation model.
[0045] Preferably, step S4, which uses a machine learning algorithm to update the parameters of the recommendation model, includes collecting user subjective ratings and objective consumption amounts, calculating reward values, and updating the recommendation model parameters according to the policy gradient formula, which is:
[0046]
[0047] in, For model parameters, For learning rate, For policy networks, For recommended actions, It indicates a healthy state.
[0048] The beneficial effects of this invention are as follows:
[0049] (1) Solved the problem of incomplete data: Through the dynamic fusion of multidimensional heterogeneous data, the system can perceive the user's "physical condition + status + environment" in an all-round way, and construct a three-dimensional and accurate user health profile;
[0050] (2) Solved the open-loop architecture problem: Through deep integration of smart terminals and cloud algorithms, a complete AIoT closed loop was constructed. Every drinking behavior of the user becomes the data basis for the next recommendation, ensuring the feasibility and verifiability of the intervention plan;
[0051] (3) Solved the problem of model rigidity: By introducing a bimodal knowledge graph and a bicyclic Bayesian potential field model, the system not only integrates traditional experience with modern science, but also has a strong self-learning ability. It can continuously evolve with the user and provide personalized services that become more and more accurate with use.
[0052] This invention is based on AIoT to realize personalized tea recommendations based on user health data, and constructs a closed-loop healthy tea management system with self-learning and self-optimization capabilities based on the execution and feedback of smart terminals. Attached Figure Description
[0053] Figure 1 This is a flowchart of the system of the present invention;
[0054] Figure 2 This is a flowchart of the multi-dimensional data acquisition and fusion module of the present invention;
[0055] Figure 3 This is a block diagram of the bimodal knowledge graph of the present invention;
[0056] Figure 4 This is a flowchart of the personalized recommendation engine module of the present invention;
[0057] Figure 5 This is a flowchart of the self-learning optimization module of the present invention;
[0058] Figure 6 This is a schematic diagram of the dual-ring Bayesian potential field model architecture of the present invention. Detailed Implementation
[0059] The present invention will now be described in further detail with reference to the accompanying drawings and specific embodiments.
[0060] Example
[0061] like Figure 1 , Figure 2 , Figure 3 , Figure 4 , Figure 5 and Figure 6As shown, the AIoT-based closed-loop personalized health beverage management system provided in this embodiment includes a multi-dimensional data acquisition and fusion module, a personalized recommendation engine module, an intelligent execution and feedback module, and a self-learning optimization module.
[0062] The healthy beverage described in this embodiment is tea;
[0063] The multidimensional data acquisition and fusion module is used to acquire user health data from multiple dimensions, and to obtain the user's health status demand vector based on a dynamic weighting algorithm according to the acquired multidimensional health data.
[0064] The multidimensional data acquisition and fusion module includes a static data acquisition unit, a dynamic data acquisition unit, an environmental data acquisition unit, and a data fusion unit;
[0065] The static data acquisition unit is used to obtain user baseline health data through an interactive health assessment questionnaire. Specifically, the health assessment questionnaire based on a decision tree algorithm can guide users to input baseline data that constitutes their health record. User baseline health data includes demographic information (such as gender, age, height, weight, etc.), lifestyle information (sleep patterns, exercise habits, dietary structure, smoking and drinking history, etc.), and constitution identification information (based on the national standard "Classification and Determination of Traditional Chinese Medicine Constitutions", the user's TCM constitution type is preliminarily determined by answering questions about physical signs and feelings, such as balanced constitution, yang deficiency constitution, etc.).
[0066] The dynamic data acquisition unit communicates with the smart terminal to continuously acquire user behavior data in real time; specifically, it can communicate with the smart teacup to continuously acquire user tea drinking behavior data in real time, including tea drinking frequency, total daily tea consumption, tea consumption per serving, brewing water temperature, and brewing time.
[0067] The environmental data acquisition unit is used to collect environmental factors of the user's geographical location; specifically, it can collect data by calling the location service of the mobile device and third-party meteorological services; the environmental factors include temperature, humidity, air pressure, season and solar term;
[0068] The data fusion unit is used to integrate three layers of heterogeneous data—user baseline health data, behavioral data, and environmental factors—in real time using a dynamic weighted fusion algorithm to obtain a comprehensive demand vector for user health status; specifically, it includes the following steps:
[0069] Step A: Normalize the three types of heterogeneous data collected and map them to a unified numerical range; specifically, perform Min-Max standardization on user baseline health data, user behavior data, and environmental data.
[0070] Step B: Preset the baseline weights for the three types of heterogeneous data;
[0071] Step C: Based on the current time point and user status conflict, adjust the baseline weights using a preset dynamic correction function to generate dynamic weights; specifically, based on the dynamic correction function... Adjust the benchmark weights;
[0072] Step D: The normalized data is weighted and summed based on dynamic weights to obtain the final user's current health need vector; the weighted summation specifically employs... ;
[0073] The personalized recommendation engine module is used to generate personalized health intervention plans based on the user's health status demand vector and a preset bimodal knowledge graph; specifically, the personalized recommendation engine module includes a bimodal retrieval unit, a collaborative filtering ranking unit, and an intelligent recommendation unit;
[0074] The bimodal retrieval unit retrieves beverages that match the user's demand vector in the bimodal knowledge graph.
[0075] The bimodal knowledge graph extracts entities and relationships such as "chemical components" and "traditional efficacy" from structured databases and unstructured texts through an automated pipeline, constructing a cross-modal aligned knowledge network to provide underlying data support for the personalized recommendation engine module.
[0076] The bimodal knowledge graph includes a first modality attribute set and a second modality attribute set;
[0077] The first modality attribute set is a quantitative attribute set analyzed by modern science, including the chemical composition content of each beverage and its physiological effects verified by evidence-based medicine.
[0078] The second modal attribute set is a qualitative attribute set based on traditional empirical theory, including the properties, flavors, meridian tropism, and traditional conditioning effects of each beverage.
[0079] Based on the demand vector, a graph query is performed in the bimodal knowledge graph to retrieve all tea entities with the highest cosine similarity to the demand vector. Specifically, if the demand vector has a high score on a certain dimension, tea entities that possess both the quantitative attribute set of modern scientific analysis and the qualitative attribute set of traditional empirical theory will be retrieved first, forming an initial list.
[0080] The collaborative filtering sorting unit sorts the search results based on a collaborative filtering algorithm; specifically:
[0081] According to the cosine similarity formula
[0082]
[0083] Calculate the cosine similarity between the current user and other users;
[0084] Select the K neighboring users with the highest similarity;
[0085] Based on the historical positive feedback rate of neighboring users for a specific beverage, predict the current user's preference rating for that beverage, and re-rank the search results in the initial list according to the rating, with the tea with the highest ranking being the primary recommendation;
[0086] The recommendation unit outputs a structured drinking plan based on the ranking results. Specifically, it selects the top three teas with the highest ranking and extracts their best brewing parameters and drinking suggestions from the knowledge graph. The drinking suggestions include drinking time and dosage.
[0087] The intelligent execution and feedback module includes at least one intelligent terminal device, which receives and assists the user in executing the health intervention plan, and collects objective behavioral data and subjective feedback information during the user's execution process.
[0088] The smart terminal device is a smart teacup or a smart brewing machine;
[0089] Each smart terminal includes a communication unit, a sensing unit, and an interaction unit;
[0090] A communication unit, used to establish a data connection with a cloud server or mobile terminal;
[0091] The sensing unit includes a temperature sensor for monitoring the temperature of the liquid and a motion sensor for monitoring actions such as picking up, pouring, or shaking by the user.
[0092] The interactive unit is used to display the health intervention plan and provide audio-visual guidance;
[0093] The smart terminal device is configured to guide the user to complete the preparation process according to the brewing parameters in the health intervention plan, and upload the actual execution data collected by the sensor to the self-learning optimization module;
[0094] The self-learning optimization module is used to optimize the parameters of the personalized recommendation engine module based on the objective behavioral data and subjective feedback information, using a preset machine learning algorithm to improve the accuracy of subsequent health intervention plans. The machine learning algorithm specifically adopts a dual-ring Bayesian potential field model, which includes:
[0095] The inner loop submodule is used to infer the posterior probability distribution of a user's current health status based on a Bayesian belief network, using multidimensional input data as evidence nodes.
[0096] The outer ring submodule is used to select recommended actions based on the policy network and the posterior probability distribution, and to calculate reward values based on feedback data.
[0097] The parameter update submodule is used to update the parameters of the recommendation model based on the reward value using a policy gradient algorithm. When updating the parameters, the parameter update submodule introduces a regularization constraint term based on the prior knowledge of the bimodal knowledge graph, and calculates the distance between the attribute vector of the current recommendation result and the prior knowledge vector as a penalty term to restrict the recommendation result to conform to preset scientific rules. The policy gradient formula is:
[0098]
[0099] in, For model parameters, For learning rate, For policy networks, For recommended actions, In good health;
[0100] This is achieved through the following steps:
[0101] A1: Definition of the model's topology;
[0102] The inner loop submodule constructs a directed acyclic graph (DAG). In this system, the node set V is divided into two layers:
[0103] Level 1: Evidence Node Layer (E), corresponding to multi-dimensional input data, including: (Body type) (Recent diet) (Environmental factors) (Overt symptoms);
[0104] Level Two: Latent Variable Inference Layer (H), corresponding to core health indicators, including: (Heat Index) (Humidity Index) (Energy level);
[0105] A2: Construction of conditional probability tables;
[0106] The system has a pre-defined conditional probability table (CPT), for example, for The CPT structure is shown in the table below (partial examples):
[0107]
[0108] A3: Detailed inference algorithm flow:
[0109] Step A301 (Evidence Vectorization): The system receives real-time data and constructs evidence vectors. ;
[0110] Step A302 (Joint Probability Distribution Calculation): Calculate according to the chain rule:
[0111]
[0112] Step A303 (Marginalization): Calculate the known evidence Posterior probability distribution ;
[0113] Step A303 (Output): Output a standardized probability vector. This quantifies the user's current health status;
[0114] A4: Outer Loop Submodule: Action Selection and Update Based on Policy Gradient;
[0115] This stage implements the self-evolution of the recommendation strategy, with the inner loop submodule outputting... As the input to the outer-loop policy network, specifically:
[0116] B1: Action Selection: The deep neural network selects actions based on the input. Output all available drinks (action) The system selects the action with the highest probability of being recommended and then issues it.
[0117] B2: Reward Calculation: After the user executes the command, the system collects feedback and calculates the reward value. ;
[0118] B3: Parameter Update: The parameters are updated using the policy gradient algorithm, specifically:
[0119]
[0120] in, The physical meaning of this formula, where the learning rate is: if the user feedback is positive, the parameters are adjusted to increase the probability of recommending the beverage in the current state.
[0121] B5: Regularization Constraints: To ensure the scientific rigor of AI recommendations, the system introduces prior constraints based on a bimodal knowledge graph during computation and updates. The Euclidean distance between the attribute vector of the current recommended action and the standard attribute vector in the knowledge graph is calculated and added as a penalty term to the objective function to prevent the model from converging to an unreasonable strategy.
[0122] The above description is only a preferred embodiment of the present invention, but the scope of protection of the present invention is not limited thereto. Any modifications and substitutions based on the technical solutions and inventive concepts provided by the present invention should be covered within the scope of protection of the present invention.
Claims
1. A closed-loop personalized health beverage management system based on AIoT, characterized in that: It includes a multi-dimensional data acquisition and fusion module, a personalized recommendation engine module, an intelligent execution and feedback module, and a self-learning optimization module; The multidimensional data acquisition and fusion module is used to acquire user health data from multiple dimensions, and to obtain the user's health status demand vector based on a dynamic weighting algorithm according to the acquired multidimensional health data. The personalized recommendation engine module is used to generate personalized health intervention plans based on the user's health status demand vector and a preset bimodal knowledge graph. The intelligent execution and feedback module includes at least one intelligent terminal device, which receives and assists the user in executing the health intervention plan, and collects objective behavioral data and subjective feedback information during the user's execution process. The self-learning optimization module is used to optimize the parameters of the personalized recommendation engine module based on the objective behavioral data and subjective feedback information through a preset machine learning algorithm, so as to improve the accuracy of subsequent health intervention plans.
2. The AIoT-based closed-loop personalized health beverage management system according to claim 1, characterized in that: The multidimensional data acquisition and fusion module includes a static data acquisition unit, a dynamic data acquisition unit, an environmental data acquisition unit, and a data fusion unit; The static data acquisition unit is used to obtain users' baseline health data through an interactive health assessment questionnaire; The dynamic data acquisition unit acquires user behavior data in real time and continuously by communicating with the smart terminal. The environmental data acquisition unit is used to collect environmental factors at the user's geographical location; The data fusion unit is used to integrate three layers of heterogeneous data—user baseline health data, behavioral data, and environmental factors—in real time using a dynamic weighted fusion algorithm to obtain a comprehensive demand vector for user health status.
3. The AIoT-based closed-loop personalized health beverage management system according to claim 1 or 2, characterized in that: The dynamic weighted fusion algorithm includes: The collected multi-source heterogeneous data is normalized and mapped to a unified numerical range; Preset the baseline weights for each dimension of data; Based on the current time point and user status conflict, the baseline weight is adjusted through a preset dynamic correction function to generate dynamic weight; The normalized data is weighted and summed based on dynamic weights to output the demand vector.
4. The closed-loop personalized health beverage management system and method based on AIoT according to claim 1, characterized in that: The personalized recommendation engine module includes a dual-modal retrieval unit, a collaborative filtering and sorting unit, and an intelligent recommendation unit; The bimodal retrieval unit retrieves beverages that match the user's demand vector in the bimodal knowledge graph. The collaborative filtering sorting unit sorts the search results based on the collaborative filtering algorithm; The recommendation unit outputs a structured drinking plan based on the ranking results, including the recommended beverage, drinking time, drinking dosage, and brewing parameters of the recommended beverage.
5. The AIoT-based closed-loop personalized health beverage management system according to claim 1 or 4, characterized in that: The bimodal knowledge graph includes a first modality attribute set and a second modality attribute set; The first modality attribute set is a quantitative attribute set analyzed by modern science, including the chemical composition content of each beverage and its physiological effects verified by evidence-based medicine. The second modal attribute set is a qualitative attribute set based on traditional empirical theories, including the properties, flavors, meridian tropism, and traditional conditioning effects of each beverage.
6. The AIoT-based closed-loop personalized health beverage management system according to claim 4, characterized in that: The collaborative filtering algorithm includes: Calculate the cosine similarity between the current user and other users; Select the K neighboring users with the highest similarity; Based on the historical positive feedback rate of neighboring users for a specific beverage, predict the current user's preference rating for that beverage and adjust the ranking accordingly.
7. The AIoT-based closed-loop personalized health beverage management system according to claim 4, characterized in that: The self-learning optimization module includes a reinforcement learning unit, which employs a dual-ring Bayesian potential field model, including: The inner loop submodule is used to infer the posterior probability distribution of a user's current health status based on a Bayesian belief network, using multidimensional input data as evidence nodes. The outer ring submodule is used to select recommended actions based on the policy network and the posterior probability distribution, and to calculate reward values based on feedback data. The parameter update submodule is used to update the parameters of the recommendation unit based on the reward value using a policy gradient algorithm.
8. The AIoT-based closed-loop personalized health beverage management system according to claim 7, characterized in that: When updating parameters, the parameter update submodule introduces a regularization constraint term based on the prior knowledge of the bimodal knowledge graph. It calculates the distance between the attribute vector of the current recommendation result and the prior knowledge vector as a penalty term to restrict the recommendation result from conforming to preset scientific rules.
9. A closed-loop personalized health beverage management method based on AIoT, characterized in that: Includes the following steps: S1: Obtain the user's multidimensional health data and fuse the multidimensional heterogeneous health data to generate a user health demand vector; S2: Based on the user's health needs vector, retrieve the beverage with the highest matching degree with the needs vector in the bimodal knowledge graph, and generate a health intervention plan including drinking time, dosage and brewing parameters. S3: Receive and guide the execution of health intervention plans through smart terminal devices, and collect actual execution data and user feedback data simultaneously during the execution process; S4: Quantify actual execution data and user feedback data into reward signals, and use machine learning algorithms to update the parameters of the recommendation model.
10. The closed-loop personalized health drink management method based on AIoT according to claim 9, characterized in that: Step S4 involves updating the parameters of the recommendation model using machine learning algorithms. This includes collecting user subjective ratings and objective consumption amounts, calculating reward values, and updating the recommendation model parameters according to the policy gradient formula. The policy gradient formula is as follows: ； in, For model parameters, For learning rate, For policy networks, For recommended actions, It indicates a healthy state.