Diabetic diet recommendation method and system based on deep reinforcement learning and oriented to multi-objective optimization

Through deep reinforcement learning and Q-MOPSO algorithm, combined with BERT and VIT models, we balance the health goals, diet preferences and patterns in diabetic diet recommendations, solving the problem of insufficient personalized recommendations in the existing technology, and achieving more accurate and flexible Diabetes diet plan.

CN119092051APending Publication Date: 2024-12-06INNER MONGOLIA UNIV OF TECH
View PDF 0 Cites 5 Cited by

Patent Information

Application Number
CN202411205742.8
Authority / Receiving Office
CN · China
Patent Type
Applications(China)
Current Assignee / Owner
Filing Date
2024-08-30
Publication Date
2024-12-06

AI Technical Summary

Technical Problem

The existing diabetic diet recommendations focus more on the balance of nutrients, ignoring the user's health goals, diet preferences and dietary patterns, resulting in a lack of personalization of recommendations and being unable to adapt to the needs of different groups of people, reducing user satisfaction and acceptance of the plan.

Method used

Using a multi-objective optimization method based on deep reinforcement learning, the Q-MOPSO algorithm balances user health goals, diet preferences and diet patterns, combines BERT and VIT models to extract features, and use deep reinforcement learning models to personalize diet recommendations, and dynamically adjust the recommendation results. To adapt to user changes.

Benefits of technology

A more accurate and personalized diet recommendation for diabetes is achieved, which can comprehensively consider users' health goals, diet preferences and dietary patterns, improve the adaptability and flexibility of recommendations, and adapt to changes in demand during different time periods.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN119092051A_ABST
    Figure CN119092051A_ABST
Patent Text Reader

Abstract

The invention discloses a multi-objective optimization-oriented diabetic diet recommendation method and system based on deep reinforcement learning. The method comprises the steps of obtaining a diet data set; extracting user health target features and user diet mode features in the diet data set by using a BERT model; extracting user diet preference characteristics in the user diet interaction data by using a VIT model; constructing a fitness function about the user health target, the user diet preference and the user diet mode, and obtaining an initial diet recommendation scheme by using a Q-MOPSO algorithm in combination with the extracted user health target features, the extracted user diet preference features and the extracted user diet mode features; and taking the initial diet recommendation scheme as a state function of deep reinforcement learning, and performing diabetic diet personalized recommendation through a deep reinforcement learning model, the deep reinforcement learning model comprising a user health prediction module. According to the method, the accuracy, individuation, real-time performance and dynamic performance of diabetic diet recommendation are improved.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present invention relates to the field of artificial intelligence technology, and in particular to a diabetic diet recommendation method and system based on deep reinforcement learning and multi-objective optimization. Background Art

[0002] Mumini et al. [Omisore, Olatunji Mumini and Ojokoh, Bolanle Adefowoke, et al. An affective learning-based system for diagnosis and personalized management of diabetes mellitus [J]. FUTURE GENERATION COMPUTER SYSTEMS-THE INTERNATIONAL JOURNAL OF ESCIENCE. 2021, 117: 273-290.] proposed a multimodal adaptive neuro-fuzzy inference model for diabetes diagnosis and a knowledge-based diet recommendation model. The integrated system consists of a multimodal adaptive neuro-fuzzy inference model designed for diabetes diagnosis and a knowledge-based diet recommendation model designed for personalized diabetes management. The recommendation model combines the user's diagnosis results with their diet formula to determine the user's daily food consumption and generates a weekly personalized diet plan from an expert-designed template. Although the study achieved personalized diet recommendations, it ignored the balance between user health goals, user dietary preferences and user dietary patterns, which would reduce user compliance and cause the user's diet plan to lack flexibility and be unable to adapt to changes in user needs at different times, resulting in poor long-term health management results.

[0003] Hu Xiaoya [Hu Xiaoya. Research on the application of improved fast non-dominated sorting genetic algorithm in multi-objective recipe recommendation problem [D]. Jilin University, 2023.] designed a multi-objective optimization model suitable for diabetic recipe recommendation. Under the premise that the intake of each food index does not exceed the recommended value, the model maximizes the total energy of food, maximizes the total mass of food, and minimizes the error between the three nutrients and the recommended value. The study introduced the firefly algorithm into the improved NSGA-Ⅱ algorithm to form a hybrid algorithm. After the improved NSGA-Ⅱ algorithm generates the offspring population, the study uses the attraction operator and position update mechanism of the firefly algorithm to update the position of individuals in the offspring population, so that the inferior individuals are close to the excellent individuals. The population with updated position relationship is merged with the original parent population as the new offspring population, and the new parent population is selected using the improved elite retention strategy, which effectively improves the solution accuracy of the algorithm. Although the study achieved the maximization of total food energy, the maximization of total food mass, and the minimization of the errors between the three nutrients and the recommended values, it ignored personalized dietary recommendations, which would lead to an inability to adapt to the needs of different groups of people. Users would feel that the dietary plan was not considerate enough or difficult to adapt to their lifestyles, which would reduce user satisfaction and acceptance of the plan.

[0004] In summary, the existing diabetic diet recommendations have the following problems:

[0005] Existing diabetes diet recommendations focus more on the balance of various nutrients, while ignoring the balance between user health goals, user dietary preferences and user dietary patterns. Current diabetes diet recommendation research focuses more on the balance between various nutrients in the diet, such as protein, carbohydrates, fat, vitamins and minerals. This focus is reasonable because a balanced diet is essential for the health of diabetic patients. However, user health goals, dietary preferences and dietary patterns also play a key role in diet recommendations, but current research lacks in-depth exploration of these factors.

[0006] Existing diabetes diet recommendations often focus on the accuracy and novelty of the recommendations, but ignore the personalization of the recommendations. Current diabetes diet recommendation research strives to provide accurate and scientific guidance. Researchers are more focused on ensuring that the recommended diet can meet the needs and health standards of the general population, emphasizing the completeness and balance of nutrition. However, current research has the problem of ignoring attention to personalized factors, resulting in the lack of personalization in most diet plans. Summary of the invention

[0007] In response to the above problems, the present invention proposes a diabetic diet recommendation method and system based on deep reinforcement learning for multi-objective optimization. The Q-MOPSO algorithm is used to balance the three factors of user health goals, user dietary patterns and user dietary preferences, and the deep reinforcement learning model is used to continuously adjust the recommendation results according to user feedback, ultimately achieving personalized diabetic diet recommendations.

[0008] In order to achieve the above object, the present invention adopts the following technical solutions:

[0009] On the one hand, the present invention proposes a diabetes diet recommendation method based on deep reinforcement learning for multi-objective optimization, comprising:

[0010] Step 1: Get the diet dataset;

[0011] Step 2: Use the BERT model to extract user health goal features and user dietary pattern features from the dietary dataset; use the VIT model to extract user dietary preference features from user dietary interaction data;

[0012] Step 3: Construct a fitness function about the user's health goals, user dietary preferences, and user dietary patterns. Combine the user's health goal features, user dietary preference features, and user dietary pattern features extracted in step 2, and use the Q-MOPSO algorithm to obtain the initial dietary recommendation plan.

[0013] Step 4: Use the initial diet recommendation plan as the state function of deep reinforcement learning, and perform personalized diabetic diet recommendations through a deep reinforcement learning model, wherein the deep reinforcement learning model includes a user health prediction module.

[0014] Furthermore, the diet data set includes a user basic indicator file, a health goal file, a diet pattern data file and a diet interaction data file; the user basic indicator file includes the user's weight, body mass index and basal metabolic rate.

[0015] Furthermore, in step 3, the fitness function is:

[0016] Fitness=w1*F health +w2*F preference +w3*F lifestyle ,

[0017] Among them, F health 、F preference and F lifestyle They represent the user's health goal fitness, user's dietary preference fitness, and user's dietary pattern fitness, respectively. w1, w2, and w3 are F health 、F preference and F lifestyle The corresponding weight.

[0018] Furthermore, the user's health goal fitness is calculated as follows:

[0019]

[0020] Among them, F health Indicates the user's health goal fitness, carbs_actual is the actual carbohydrate intake in the diet plan; carbs_target is the target carbohydrate intake in the diet plan.

[0021] Further, the user's dietary preference adaptability is as follows:

[0022]

[0023] NPI represents the number of foods that the user likes, and TNI represents the total number of foods.

[0024] Furthermore, the user's dietary pattern fitness is calculated as follows:

[0025] F lifestyle =ω veg *V+ω act *A+ω age *G,

[0026] where ω veg ,ω act ,ω age are all weight parameters, which respectively represent the impact of the diet plan, the user's daily activity level, and the user's age on the fitness of the diet pattern; V represents whether the diet plan is completely in line with the user's diet pattern, V=1 represents that the diet plan is completely in line with the user's diet pattern, V∈(0,1) represents that the diet plan is generally in line with the user's diet pattern, and V=0 represents that the diet plan is completely inconsistent with the user's diet pattern; A represents a value reflecting the user's daily activity level; G is a value related to the user's age.

[0027] Furthermore, the Q-MOPSO algorithm includes:

[0028] Step 3.1: Set decision variables; the decision variables include total calories, carbohydrates, protein and fat; in the algorithm, one particle represents a specific recommended recipe, and the position vector of each particle represents the specific values ​​of total calories, carbohydrates, protein and fat respectively;

[0029] Step 3.2: Randomly initialize particle positions; initialize particle velocities; set fitness function; determine individual and global optimal solutions of the particle swarm;

[0030] Step 3.3: Apply Q-Learing to dynamically adjust the particle search strategy in the multi-objective particle swarm algorithm; including: defining a state space, where the state is the current position of the particle; defining an action space, including changing the proportion of each nutrient in the diet plan; setting the reward function to the improvement of the fitness of the new diet plan for the health goal; learning the optimal strategy by updating the Q value, where the Q value represents the expected reward for taking a specific action under a specific state; in each iteration step, selecting the optimal action based on the current Q value, and then updating the position and velocity of the particle based on the result of the action;

[0031] Step 3.4: Evaluate the fitness of each solution; determine the Pareto optimal solution, and in each iteration, update the individual optimal solution and the global optimal solution of the particle swarm; in the iteration process, collect all non-dominated solutions to form the Pareto frontier; select solutions from the Pareto frontier according to the specific application scenario and user preferences;

[0032] Step 3.5: Obtain the final global optimal solution, including the specific values ​​of total calories, carbohydrates, proteins, and fats, and convert the final global optimal solution into the user's initial diet recommendation plan.

[0033] Furthermore, the user health prediction module is composed of CNN and GRU. CNN is used to analyze the user's daily dietary data to extract food type characteristics, nutrient composition characteristics and intake characteristics; GRU is used to process the user's food type characteristics, nutrient composition characteristics and intake characteristics for several consecutive days, learn the dependencies in the time series, and predict the user's future body mass index changes.

[0034] Furthermore, the deep reinforcement learning model also includes a strategy network and a value network;

[0035] The input of the strategy network is the initial diet recommendation output by the Q-MOPSO algorithm, the user's weight, body mass index and basal metabolic rate, and the user's body mass index value predicted by the user health prediction module, and the output of the strategy network is the probability distribution of the recipes that may be selected;

[0036] The input of the value network is the diet action a selected by the strategy network, and the output of the value network is the approximate action value function;

[0037] The value network scores the action a of the strategy network in the current state and returns the result with the highest score to the strategy network, which then outputs the final diet recommendation result.

[0038] Another aspect of the present invention provides a diabetes diet recommendation system based on deep reinforcement learning and multi-objective optimization, comprising:

[0039] A diet data set acquisition unit, used for acquiring a diet data set;

[0040] A feature extraction unit is used to extract the user health goal features and user diet pattern features in the diet data set using the BERT model; and to extract the user diet preference features in the user diet interaction data using the VIT model;

[0041] The initial diet recommendation scheme obtaining unit is used to construct a fitness function about the user's health goal, the user's diet preference and the user's diet pattern, and to obtain the initial diet recommendation scheme by using the Q-MOPSO algorithm in combination with the user's health goal features, the user's diet preference features and the user's diet pattern features extracted by the feature extraction unit;

[0042] The diabetic diet personalized recommendation unit is used to use the initial diet recommendation plan as the state function of deep reinforcement learning, and perform personalized diabetic diet recommendations through a deep reinforcement learning model, wherein the deep reinforcement learning model includes a user health prediction module.

[0043] Compared with the prior art, the present invention has the following beneficial effects:

[0044] Existing research on diabetes diet focuses on the balance between nutritional elements, but ignores the balance between user health goals, user dietary preferences and user dietary patterns. The present invention uses the Q-MOPSO algorithm to balance the characteristics of user health goals, user dietary patterns and user dietary preferences. Specifically, the Q-Learning algorithm is added to the MOPSO algorithm to dynamically adjust the update direction and step size of the particles, thereby reducing the search time and computing cost, and improving the efficiency and performance of the algorithm.

[0045] The existing dietary recommendations focus more on the accuracy of recommendations, while ignoring the problem of personalized recommendations. The present invention uses a deep reinforcement learning model to improve the personalization of diabetic dietary recommendations. Specifically, the output of the user health prediction module is integrated into the value network of the DDPG model to timely correct future dietary recommendation results.

[0046] By balancing the user's health goals, user dietary preferences and user dietary patterns through the Q-MOPSO algorithm, the user's needs and preferences can be more accurately portrayed to make more accurate recommendations. The present invention combines the user's health goals, dietary preferences and dietary patterns, and can comprehensively consider multiple factors when recommending dietary plans, making the recommendations more comprehensive and integrated. The Q-MOPSO algorithm used in the present invention is real-time and dynamic, and can be adjusted as the user's health status and needs change. This flexibility can better adapt to the changing needs of diabetic patients in different time periods. The constructed deep reinforcement learning model can gradually improve the accuracy and adaptability of recommendations by continuously learning user feedback and adjusting recommendation strategies. BRIEF DESCRIPTION OF THE DRAWINGS

[0047] Figure 1 This is a flow chart of a diabetes diet recommendation method based on deep reinforcement learning for multi-objective optimization according to an embodiment of the present invention;

[0048] Figure 2 A schematic diagram of the Bert model structure provided by an embodiment of the present invention;

[0049] Figure 3 A schematic diagram of the VIT model structure provided by an embodiment of the present invention;

[0050] Figure 4 A flow chart of the Q-MOPSO algorithm provided by an embodiment of the present invention;

[0051] Figure 5 A schematic diagram of the structure of a user health prediction module provided by an embodiment of the present invention;

[0052] Figure 6 A schematic diagram of the Actor-Critic model structure provided by an embodiment of the present invention;

[0053] Figure 7 A schematic diagram of the Actor network structure provided by an embodiment of the present invention;

[0054] Figure 8 A schematic diagram of the Critic network structure provided by an embodiment of the present invention;

[0055] Fig. 9 This is a schematic diagram of the architecture of a diabetes diet recommendation system based on deep reinforcement learning for multi-objective optimization according to an embodiment of the present invention. DETAILED DESCRIPTION

[0056] The present invention will be further explained below with reference to the accompanying drawings and specific embodiments:

[0057] like Figure 1 As shown, a diabetes diet recommendation method based on deep reinforcement learning for multi-objective optimization of the present invention comprises:

[0058] S101: Obtain a diet dataset;

[0059] S102: Use the BERT model to extract the user health goal features and user diet pattern features in the diet dataset; use the VIT model to extract the user diet preference features in the user diet interaction data;

[0060] S103: construct a fitness function for the user's health goal, the user's dietary preference, and the user's dietary pattern, combine the user's health goal features, the user's dietary preference features, and the user's dietary pattern features extracted in S102, and use the Q-MOPSO algorithm to obtain an initial dietary recommendation plan;

[0061] S104: Using the initial diet recommendation plan as a state function of deep reinforcement learning, and performing personalized diabetic diet recommendations through a deep reinforcement learning model, wherein the deep reinforcement learning model includes a user health prediction module.

[0062] The method specifically includes:

[0063] 1 Dataset Introduction

[0064] The dietary dataset used in this paper was collected by the research team and combined with the classic Pima dataset, aiming to provide users with a personalized diet that is closer to their living habits. The dataset is divided into four parts, each containing information of different dimensions:

[0065] The user basic indicator file contains the user's basic information and health indicator data, including user ID, gender, age, height, weight, body mass index (BMI index), basal metabolic rate (BMR), fasting blood glucose value and occupation data, totaling 5912 records.

[0066] The health goal file contains the user's health goal data and records the user's health goal related information. The user's health goal data is associated with the user ID.

[0067] The dietary pattern data file records the dietary pattern data of the user. The dietary pattern data of the user is associated with the user ID and records the dietary habits of the user in detail.

[0068] The dietary interaction data file contains the user's dietary data and food pictures. It is also associated with the user ID, recording the user's dietary data and attached with relevant food pictures.

[0069] 2 Extracting User Features

[0070] 1) Extract the features of user health goals and user dietary pattern: The user's health goals are keywords such as "fat loss" and "maintaining weight", and their data source is the user's basic indicator file in the data set; the user's dietary pattern is the user's special eating habits, such as: vegetarian, etc., and its data source is the dietary pattern data file of the data set, so the BERT model is used to extract the user's health goal features and user dietary pattern features. The data is organized into text format and input into the BERT model. The text is converted into feature vector form through Embedding, and the vector representation is obtained through multiple transformer encoders. The BERT model structure is as follows: Figure 2 shown.

[0071] 2) Extracting user dietary preference features: For the image data in the dietary interaction data file, the VIT model is used to extract features. The original image is input into the VIT model, and the image is scaled to a uniform size. The image is then cut into patches of the same size through the convolution layer. Each patch is then stretched into a vector and the position vector is added to obtain the pos_embedding vector. The obtained pos_embedding vector is input into a network mainly composed of Transformer Encoder for training to obtain a feature vector. The VIT model structure is as follows: Figure 3 shown.

[0072] 3Q-MOPSO algorithm

[0073] The solution to balancing user health goals, user dietary preferences, and user dietary patterns can be viewed as a nonlinear multi-objective optimization problem. Compared to some traditional optimization methods, the multi-objective particle swarm algorithm (MOPSO) does not require prior gradient information, which makes it more applicable when dealing with engineering problems, especially for those complex, multi-variable systems. Its parallel computing characteristics mean that it can make full use of modern computing resources, such as multi-core processors or distributed systems, to speed up the calculation process. Its adaptability also enables a better balance between the convergence and diversity of searches in different engineering problems, thereby more effectively discovering a set of high-quality solutions. Based on the above advantages, the present invention selects MOPSO to solve the problem of solving user health goals, user dietary preferences, and user dietary patterns, and adds the Q-Learning algorithm to the particle search strategy, so that it can dynamically adjust the search position of particles, thereby reducing search time and computational costs. Figure 4 As shown, the Q-MOPSO execution steps include:

[0074] S201: Set decision variables and fitness function.

[0075] A particle in the algorithm represents a specific recommended recipe, which achieves a balance between the user's dietary preferences and health goals by optimizing the four decision variables of total calories, carbohydrates, protein, and fat.

[0076] Decision variables: In dietary recommendations for patients with diabetes, decision variables should include those dietary components that directly affect glycemic control and overall health:

[0077] 1) Total calories: set based on the individual's basal metabolic rate (BMR) and activity level.

[0078] 2) Carbohydrates: Considering that they have the greatest impact on blood sugar, the specific number of grams or percentage of total calories consumed should be set.

[0079] 3) Protein: Sets the daily required grams to support muscle maintenance and other body functions.

[0080] 4) Fat: Control your intake, especially saturated fat and trans fat, to reduce the risk of cardiovascular disease.

[0081] In the Q-MOPSO algorithm, the position vector of each particle represents a specific solution. In the diabetic diet recommendation scenario, each component of this vector represents the specific value of total calories, carbohydrates, protein, and fat. Through these values, a specific recipe can be directly determined.

[0082] The fitness function can be formalized as:

[0083] Fitness=w1*F health +w2*F preference +w3*F lifestyle Formula (1)

[0084] Among them: F health 、F preference and F lifestyle They represent health goal fitness, dietary preference fitness, and dietary pattern fitness (corresponding quantitative scores). w1, w2, and w3 are the weights of each goal, which are adjusted according to the personal needs of different users and the advice of doctors.

[0085] As an implementable method, setting weights w1=5, w2=3, w3=2 means:

[0086] w1=5 (health goal fitness): This weight is the highest, indicating the priority of health goals in dietary recommendations, especially blood sugar control is crucial for diabetic patients.

[0087] w2=3 (diet preference fitness): This indicates that the user's diet preference is also very important, but slightly less important than health goals. This ensures that the user is more likely to accept and like the recommended diet.

[0088] w3=2 (diet pattern adaptability): This weight is the lowest, ensuring that the diet recommendation is suitable for the user's diet pattern and habits.

[0089] S202: Fitness function design.

[0090] Fitness function: There are three main goals: health goal fitness F health ; Diet preference adaptability F preference ; Dietary pattern adaptability F lifestyleEach objective has a corresponding quantitative scoring method, and the final fitness function can be expressed as a weighted sum of these scores:

[0091] Fitness = 5*F health + 3*F preference +2*F lifestyle Formula (2)

[0092] F health (Health Goal Fitness): A measurement method to measure whether the carbohydrate control in the diet plan meets the doctor's recommendations

[0093]

[0094] Where carbs_actual is the actual carbohydrate intake in the diet; carbs_target is the target carbohydrate intake in the diet.

[0095] F preference (Diet preference fitness): Calculates the proportion of user-preferred foods included in the diet plan

[0096]

[0097] Among them, NPI (number of preferred items): the number of foods that users like;

[0098] TNI (total number of items): total number of food.

[0099] F lifestyle (Dietary pattern adaptability): calculated based on nutritional needs, blood sugar control and eating habits

[0100] F lifestyle =ω veg *V+ω act *A+ω age *G Formula (5)

[0101] where ω veg is a weight parameter, which indicates the effect of the diet on the fitness of the diet pattern; ω act is a weight parameter, which indicates the influence of the user's daily activity level on the fitness of the dietary pattern; ω ageis a weight parameter, which indicates the influence of user age on the fitness of diet pattern. V: whether the diet plan is completely in line with the user's diet pattern; V = 1: the diet plan is completely in line with the user's diet pattern; V∈(0,1): if the diet plan contains some foods that are inconsistent with the user's diet pattern, but most of them are still in line with the user's diet pattern, V can be set to a value between 0 and 1, indicating the fitness of the plan. V = 0: the diet plan is completely inconsistent with the user's diet pattern. A represents a value reflecting the user's daily activity level; A = 2 indicates a low activity level, the user's daily activity is very small, and basically in a sedentary state; A = 5 indicates a medium activity level, the user has a certain amount of daily activity, but not too high; A = 8 indicates a high activity level, the user's daily activity is large, or often performs moderate to high intensity exercise. G is a value related to the user's age, G = 2: refers to people aged between 18 and 35; G = 5: refers to people aged between 36 and 60; G = 8: refers to people aged over 60.

[0102] S203: Particle swarm initialization.

[0103] Determine the number of particles: Due to the high complexity of the problem and the large number of decision variables, the number of particles is set to between 30 and 50, and more particles will be added later as appropriate.

[0104] Initialize particle positions randomly:

[0105] Determine the range of decision variables: total calories: 1500-2500; carbohydrates: 0.4-0.5; protein: 0.2-0.3; fat: 0.1-0.2; randomly generate initial positions.

[0106] Initialize particle velocity:

[0107] Total calories: a random value in the range [-10,10]; Carbohydrates: a random value in the range [-5,5]; Protein: a random value in the range [-2,2]; Fat: a random value in the range [-1,1].

[0108] Set the fitness function: Fitness = 5*F health +3*F preference +2*F lifestyle .

[0109] The goal of the fitness function is to maximize the Fitness value to find the most suitable diet plan for the user.

[0110] Determine individual and global optimal solutions:

[0111] Determine the optimal solution for an individual:

[0112] During initialization: The individual optimal position of each particle is initially set to its initial position. The fitness value of each particle at its initial position is calculated.

[0113] After each iteration: calculate the fitness value of each particle in the new position. If the fitness value of the new position is better than the current individual optimal value, update the individual optimal position and fitness value of the particle.

[0114] Update rule: f(x i )>f(pbest i ), then pbest i =x i ;

[0115] Among them, f(x) is the fitness function, x i is the position of the ith particle, pbest i is the individual optimal position of the ith particle.

[0116] Determine the global optimal solution:

[0117] During initialization: select the one with the highest fitness value from the individual optimal solutions of all particles as the global optimal solution.

[0118] After each iteration: re-evaluate the individual optimal solutions of all particles and select the individual optimal solution with the highest fitness value as the new global optimal solution.

[0119] Update rule: gbest = argmax pbesti f(pbest i );

[0120] Among them, the argmax operation is to select the one with the highest fitness value among all individual optimal solutions, gbest i is the global optimal position.

[0121] S204: Particle swarm update rules.

[0122] Q-Learing is applied to dynamically adjust the particle search strategy in MOPSO.

[0123] Defining the state space: In Q-Learning, the state represents the current problem environment. For diabetes diet recommendations, the state is the position of the current particle, representing a specific diet plan, including the total calories, carbohydrates, protein, and fat ratios.

[0124] Define the action space: An action is an action that can be taken in a given state. In Q-MOPSO, actions include adjusting the velocity of a particle or directly modifying its position, i.e., changing the ratio of each nutrient in the diet plan.

[0125] Reward function: Reward is the feedback given based on the result of an action, usually based on the improvement of the fitness function. In the diabetic diet recommendation, the reward can be the improvement of the fitness of the new diet plan for health goals, such as blood sugar control, nutritional balance, etc.

[0126] R(s,a)=ΔHealth Index Formula (6)

[0127] Where ΔHealth Index represents the change in the health index after executing action a.

[0128] Q-value update: Q-Learning learns the optimal strategy by updating the Q-value (Q-table). The Q-value represents the expected reward for taking a specific action in a specific state. The update formula is:

[0129] Q(s,a)←Q(s,a)+α[R(s,a)+γmax a′ Q(s′,a′)-Q(s,a)] Formula (7)

[0130] Where α: learning rate, which controls the importance of newly acquired information; γ: discount factor, which determines the value of future rewards; S': the new state reached after executing action a; max a′ Q(s′,a′): The maximum expected reward for all possible actions in the new state.

[0131] Strategy execution: In each iteration step, the optimal action is selected based on the current Q value, and then the position and velocity of the particle are updated based on the action result. The particle update rule can integrate the output of Q-Learning to dynamically adjust the search strategy.

[0132] S205: Select a solution for multi-objective optimization.

[0133] Evaluate the fitness of each solution: When using the Q-MOPSO algorithm, each particle represents a specific recommended recipe, and its position vector contains the decision variables of the solution such as total calories, carbohydrates, protein and fat content. Fitness function Fitness = 5*F health +3*F preference +2*F lifestyle Used to calculate the fitness value of each solution, a higher fitness value indicates a better solution.

[0134] Determine the Pareto optimal solution: In multi-objective optimization, solutions are usually evaluated based on Pareto dominance. A solution is considered Pareto optimal if no other solution is better than it in all objectives (i.e., no other solution can make one objective better without making at least one objective worse). In each iteration, the individual best solution (pbest) and the global best solution (gbest) of the particle are updated. The individual best solution is the position with the highest fitness value in the particle's history. The global best solution is the one with the highest fitness among all the individual best solutions of the particles.

[0135] Applying the Pareto front: During the iteration process, all non-dominated solutions are collected to form the Pareto front. These are the sets of solutions that trade off between multiple objectives, from which the final solution can be selected.

[0136] Selecting solutions: When selecting solutions from the Pareto frontier, further filter them based on specific application scenarios and user preferences. For example, if a user is more focused on health goals rather than dietary preferences, a diet plan with a high health fitness score can be selected.

[0137] S206: Outputting initial diet recommendation results.

[0138] The global optimal solution obtained by the Q-MOPSO algorithm represents an optimized diet plan, including specific values ​​of total calories, carbohydrates, protein, and fat. In order to convert this optimal solution into a diet recommendation plan that can actually be executed by the user, these decision variables must first be standardized.

[0139] Specifically, the standardization process first distributes the total calories in the global optimal solution reasonably according to the user's basal metabolic rate and activity level. Then, the total calories are allocated to carbohydrates, proteins, and fats in proportion to ensure that the intake of these nutrients meets the requirements of diabetes management. Then, based on the preset food items and their basic intake (such as the standard intake of each food), the standardized nutrient requirements are converted into specific food intakes to generate an initial dietary recommendation plan.

[0140] 4 Deep Reinforcement Learning

[0141] The deep reinforcement learning model (HealthAware-DDPG) constructed by the present invention includes a user health prediction module, a policy network (Actor) and a value network (Critic).

[0142] 1) User health prediction module

[0143] Considering that the data set contains the user's current BMI value record and three days of diet records, we finally chose to combine the CNN and GRU hybrid model to implement the user health prediction module, such as Figure 5 shown.

[0144] CNN: Used to analyze daily dietary data and extract features about food type, nutrients, and intake.

[0145] Food type features: Food type is the basic category that distinguishes different foods, such as vegetables, fruits, meat, dairy products, grains, etc. Each type of food has different effects on the human body, and therefore has different effects on BMI. Accurate classification of food types helps the model learn the relationship between specific types of food and changes in BMI.

[0146] Nutritional composition features: including calorie intake, which is a key factor affecting body weight and BMI. By calculating the total calorie intake of each meal, CNN can learn the relationship between calorie intake and BMI changes.

[0147] Intake characteristics: including the number of meals per day, such as whether there is a habit of snacks or late-night snacks. The amount of food per meal can be quantified by the weight of the food or the estimated portion size.

[0148] GRU: Used to process time series data (diet records for several days), learn dependencies in time series, and predict future BMI changes.

[0149] 2) Actor and Critic

[0150] The Actor-Critic algorithm model structure is as follows Figure 6 As shown in the figure, the recommendation algorithm based on deep reinforcement learning first extracts training data from the original data file, then processes the data and performs related embedding, and then iterates the neural network. The training of the neural network consists of two parts: Actor and Critic. Deep learning is used to simulate the reinforcement learning process and update and retain parameters.

[0151] State value function: V π (s)=∑ a π(a|s)*Q π (s,a), V is the action value function Q π The expectation of π(a|s) is the policy function that controls the agent to move, Q π The (s,a) value function evaluates the quality of the action. Since the above two functions are difficult to solve, a neural network is used to approximate these two functions respectively, and then the Actor-Critic method is used to learn these two networks simultaneously. Specifically:

[0152] Policy network (Actor): Use the network π(s|a; θ) to approximate π(s|a), θ is the network parameter.

[0153] Value network (Critic): Use the network q(s,a;ω) to approximate Q π (s,a),ω are network parameters.

[0154] From the above steps, the state value function can be approximated as:

[0155] V π (s)=∑ a π(a|s)*Q π (s,a)≈∑ a π(a|s;θ)*q(s,a;ω) Formula (8)

[0156] a) Actor Network

[0157] Actor network structure is as follows Figure 7 shown.

[0158] (1) Input: the initial diet recommendation result obtained by the Q-MOPSO algorithm after balancing the user's health goal characteristics, user's dietary preference characteristics, and user's dietary pattern characteristics; the user's body data of weight, BMI, and BMR; the user's BMI value predicted by the user health prediction module;

[0159] (2) Output: Probability distribution of possible recipes.

[0160] The convolutional layer conv converts the state into a feature vector feature, and uses one or more fully connected layers dense to map the feature vector into a vector, and after normalization, the probability of each action is obtained.

[0161] b)Critic Network

[0162] The critic network structure is as follows Figure 8 shown.

[0163] (1) Input: the eating action a selected by the Actor network;

[0164] (2) Output: approximate action-value function.

[0165] The Critic network uses convolutional layers and fully connected layers to extract features from the input to obtain two feature vectors, then concatenates the two feature vectors to obtain a higher feature vector, and finally uses a fully connected layer to output a real number. That is, the Critic network scores the action a of the Actor network in the current state, and returns the highest score to the Actor network, which then outputs the final diet recommendation result.

[0166] The execution steps of HealthAware-DDPG are:

[0167] S301, observe the old state s t , according to the Actor network μ(s t |θ μ ) Randomly sample an action a t .

[0168] S302, Agent performs action a t ; The environment will feedback the new state s t+1 and reward r t .

[0169] S303, add a user health prediction module: use CNN to analyze daily dietary data and extract feature vector f i ; Input the feature vectors of multiple days into GRU to generate the predicted BMI of the user health prediction module i+1 ; Combine these features with the prediction results as part of the Actor network input.

[0170] S304, calculate new action: with new state s t+1 As input, the Actor network calculates new probabilities and randomly samples new actions: a t+1 =μ(s t+1 |θ μ ), this action is just a hypothetical action, the Agent will not execute it, it is only used to calculate the Q value.

[0171] S305, then calculate the output of the Critic network twice:

[0172] Calculate the Q value for the current state and action:

[0173] Q(s t ,a t |θ Q )

[0174] Calculate the target Q value for the new state and new action:

[0175] Q′(s t+1 ,μ′(s t+1 |θμ′ )|θ Q′ )

[0176] Among them, θ Q is the parameter vector of the Critic network; s t is the current state; t+1 Indicates execution of action a t After that, the new state of the environment feedback. t Indicates that in state s t Next, the action selected by the Actor network; t+1 To perform action a t The new state after the environmental feedback; μ(s t |θ μ ) indicates that the Actor network is in state s t The output action a t , given the parameter θ μ .

[0177] S306, calculate TD error:

[0178] Target Q value calculation:

[0179] y t =r t +γQ′(s t+1 ,μ′(s t+1 |θ μ′ )|θ Q′ )

[0180] TD Error:

[0181] δ t =y t -Q(s t ,a t |θ Q )

[0182] Among them, r t represents the reward of the current time step; Q′(s t+1 ,μ′(s t+1 |θ μ′ )|θ Q′ ) represents the target Critic network in the new state s t+1 and the Q value of the target Actor network μ′, given the parameter θ Q′ and θ μ′ ; Q(s t ,a t |θ Q ) indicates that the Critic network is in the current state s t and action a t The Q value under given parameter θ Q; γ represents the discount factor, usually between [0,1], which is used to weigh the relative importance of current rewards and future rewards.

[0183] S307, derive the Critic network:

[0184] The loss function of the Critic network is:

[0185]

[0186] The parameter θ of the Critic network Q Derivative:

[0187]

[0188] Where N is the batch size.

[0189] S308, TD algorithm updates the Critic network

[0190] Use gradient descent to update the parameters of the Critic network:

[0191]

[0192] Where α is the learning rate used to update the Critic network.

[0193] S309, derive the Actor network:

[0194] The goal of the Actor network is to maximize the Q-value function:

[0195]

[0196] Parameters θ for the Actor network μ Derivative:

[0197]

[0198] S310, use gradient ascent to update the Actor network:

[0199]

[0200] Among them, θ μ is the parameter vector of the Actor network; β is the learning rate used to update the Actor network.

[0201] Each round of iteration performs the above 10 steps, makes an action, observes a reward, and updates the neural network parameters.

[0202] Based on the above embodiments, Fig. 9As shown, the present invention also proposes a diabetes diet recommendation system based on deep reinforcement learning for multi-objective optimization, including:

[0203] A diet data set acquisition unit, used for acquiring a diet data set;

[0204] A feature extraction unit is used to extract the user health goal features and user diet pattern features in the diet data set using the BERT model; and to extract the user diet preference features in the user diet interaction data using the VIT model;

[0205] The initial diet recommendation scheme obtaining unit is used to construct a fitness function about the user's health goal, the user's diet preference and the user's diet pattern, and to obtain the initial diet recommendation scheme by using the Q-MOPSO algorithm in combination with the user's health goal features, the user's diet preference features and the user's diet pattern features extracted by the feature extraction unit;

[0206] The diabetic diet personalized recommendation unit is used to use the initial diet recommendation plan as the state function of deep reinforcement learning, and perform personalized diabetic diet recommendations through a deep reinforcement learning model, wherein the deep reinforcement learning model includes a user health prediction module.

[0207] In summary, existing research on diabetes diet focuses on the balance between nutritional elements, but ignores the balance between user health goals, user dietary preferences and user dietary patterns. The present invention uses the Q-MOPSO algorithm to balance the characteristics of user health goals, user dietary pattern characteristics and user dietary preference characteristics. Specifically, the Q-Learning algorithm is added to the MOPSO algorithm to dynamically adjust the update direction and step size of the particles, thereby reducing the search time and computing cost, and improving the efficiency and performance of the algorithm.

[0208] The existing dietary recommendations focus more on the accuracy of recommendations, while ignoring the problem of personalized recommendations. The present invention uses the HealthAware-DDPG model to improve the personalization of dietary recommendations. Specifically, the output of the user health prediction module is integrated in the Actor network of the DDPG model to timely correct the dietary recommendation results in the future.

[0209] The above is only a preferred embodiment of the present invention. It should be pointed out that a person skilled in the art can make several improvements and modifications without departing from the principle of the present invention. These improvements and modifications should also be regarded as within the scope of protection of the present invention.

Claims

1. A diabetes diet recommendation method based on deep reinforcement learning for multi-objective optimization, characterized in that: include: Step 1: Get the diet dataset; Step 2: Use the BERT model to extract user health goal features and user dietary pattern features in the diet dataset; Use the VIT model to extract user dietary preference features from user dietary interaction data; Step 3: Construct a fitness function about the user's health goals, user dietary preferences, and user dietary patterns. Combine the user's health goal features, user dietary preference features, and user dietary pattern features extracted in step 2, and use the Q-MOPSO algorithm to obtain the initial dietary recommendation plan. Step 4: Use the initial diet recommendation plan as the state function of deep reinforcement learning, and perform personalized diabetic diet recommendations through a deep reinforcement learning model, wherein the deep reinforcement learning model includes a user health prediction module.

2. The diabetes diet recommendation method based on deep reinforcement learning for multi-objective optimization according to claim 1, characterized in that: The dietary data set includes a user basic indicator file, a health goal file, a dietary pattern data file and a dietary interaction data file; the user basic indicator file includes the user's weight, body mass index and basal metabolic rate.

3. The diabetes diet recommendation method based on deep reinforcement learning for multi-objective optimization according to claim 1, characterized in that: In step 3, the fitness function is: Fitness=w1*F health +w2*F preference +w3*F lifestyle , Among them, F health 、F preference and F lifestyle They represent the user's health goal fitness, user's dietary preference fitness, and user's dietary pattern fitness, respectively. w1, w2, and w3 are F health 、F preference and F lifestyle The corresponding weight.

4. The diabetes diet recommendation method based on deep reinforcement learning for multi-objective optimization according to claim 3, characterized in that: The user's health goal fitness is calculated as follows: Among them, F health Indicates the user's health goal fitness, carbs_actual is the actual carbohydrate intake in the diet plan; carbs_target is the target carbohydrate intake in the diet plan.

5. The diabetes diet recommendation method based on deep reinforcement learning for multi-objective optimization according to claim 3, characterized in that: User dietary preferences are adapted as follows: NPI represents the number of foods that the user likes, and TNI represents the total number of foods.

6. The diabetes diet recommendation method based on deep reinforcement learning for multi-objective optimization according to claim 3, characterized in that: The fitness of a user's eating pattern is calculated as follows: F lifestyle =ω veg *V+ω act *A+ω age *G, where ω veg ,ω act ,ω age are weight parameters, which respectively represent the influence of diet plan, user's daily activity level and user's age on the fitness of diet pattern; V represents whether the diet plan is completely consistent with the user's diet pattern, V=1 represents that the diet plan is completely consistent with the user's diet pattern, V∈(0,1) represents that the diet plan is generally consistent with the user's diet pattern, and V=0 represents that the diet plan is completely inconsistent with the user's diet pattern; A represents a value that reflects the user's daily activity level; G is a value related to the user's age.

7. The diabetes diet recommendation method based on deep reinforcement learning for multi-objective optimization according to claim 1, characterized in that: The Q-MOPSO algorithm includes: Step 3.1: Set decision variables; the decision variables include total calories, carbohydrates, protein and fat; in the algorithm, one particle represents a specific recommended recipe, and the position vector of each particle represents the specific values ​​of total calories, carbohydrates, protein and fat respectively; Step 3.2: Randomly initialize particle positions; initialize particle velocities; set fitness function; determine individual and global optimal solutions of the particle swarm; Step 3.3: Apply Q-Learing to dynamically adjust the particle search strategy in the multi-objective particle swarm algorithm; including: defining a state space, where the state is the current position of the particle; defining an action space, including changing the proportion of each nutrient in the diet plan; setting the reward function to the improvement of the fitness of the new diet plan for the health goal; learning the optimal strategy by updating the Q value, where the Q value represents the expected reward for taking a specific action under a specific state; in each iteration step, selecting the optimal action based on the current Q value, and then updating the position and velocity of the particle based on the result of the action; Step 3.4: Evaluate the fitness of each solution; determine the Pareto optimal solution, and in each iteration, update the individual optimal solution and the global optimal solution of the particle swarm; in the iteration process, collect all non-dominated solutions to form the Pareto frontier; select solutions from the Pareto frontier according to the specific application scenario and user preferences; Step 3.5: Obtain the final global optimal solution, including the specific values ​​of total calories, carbohydrates, proteins, and fats, and convert the final global optimal solution into the user's initial diet recommendation plan.

8. The diabetes diet recommendation method based on deep reinforcement learning for multi-objective optimization according to claim 1, characterized in that: The user health prediction module is composed of CNN and GRU. CNN is used to analyze the user's daily diet data and extract food type features, nutrient composition features and intake features. The GRU is used to process the user's food type features, nutrient composition features, and intake features for consecutive days, learn the dependencies in the time series, and predict the user's future body mass index changes.

9. The diabetes diet recommendation method based on deep reinforcement learning for multi-objective optimization according to claim 1, characterized in that: The deep reinforcement learning model also includes a strategy network and a value network; The input of the strategy network is the initial diet recommendation output by the Q-MOPSO algorithm, the user's weight, body mass index and basal metabolic rate, and the user's body mass index value predicted by the user health prediction module, and the output of the strategy network is the probability distribution of the recipes that may be selected; The input of the value network is the diet action a selected by the strategy network, and the output of the value network is the approximate action value function; The value network scores the action a of the strategy network in the current state and returns the result with the highest score to the strategy network, which then outputs the final diet recommendation result.

10. A diabetes diet recommendation system based on deep reinforcement learning for multi-objective optimization, characterized in that: include: A diet data set acquisition unit, used for acquiring a diet data set; A feature extraction unit, used to extract user health goal features and user dietary pattern features from a dietary dataset using a BERT model; Use the VIT model to extract user dietary preference features from user dietary interaction data; The initial diet recommendation scheme obtaining unit is used to construct a fitness function about the user's health goal, the user's diet preference and the user's diet pattern, and to obtain the initial diet recommendation scheme by using the Q-MOPSO algorithm in combination with the user's health goal features, the user's diet preference features and the user's diet pattern features extracted by the feature extraction unit; The diabetic diet personalized recommendation unit is used to use the initial diet recommendation plan as the state function of deep reinforcement learning, and perform personalized diabetic diet recommendation through a deep reinforcement learning model, wherein the deep reinforcement learning model includes a user health prediction module.

Citation Information

Cited By

  • Personalized food-grade stomach-occupying capsule use recommendation system and method

    CN119517311A

  • Personalized food-grade stomach-suppressing capsule usage recommendation system and method

    CN119517311B

  • Agricultural water right distribution and transaction management method suitable for high-water areas

    CN119963297A

  • Active induced protein digestion efficiency optimization method and system

    CN121687217A

  • Diet health assessment method and system, and computer equipment

    CN121725993A