Personalized dietary recommendation method using differential constraint improved reinforcement Q-learning mechanism

Abstract

This invention relates to a personalized dietary recommendation method employing a differentiated constraint-based enhanced Q-learning mechanism, belonging to the field of nutrition control methods. It comprises two steps: data acquisition and preprocessing, and a reinforcement learning mechanism using a fusion linear programming algorithm. This method dynamically learns and adjusts recommendation strategies from user interactions through a Q-learning algorithm, while simultaneously utilizing a differentiated constraint mechanism to ensure nutritional balance in the recommendations. By satisfying dynamic user preferences and achieving nutritional balance, it significantly improves user satisfaction, contributing an innovative and efficient solution to the field of intelligent health management.

Description

Technical Field

[0001] This invention pertains to nutritional control methods, and in particular relates to a personalized dietary recommendation method that employs differentiated constraints to improve the reinforcement Q-learning mechanism. Background Technology

[0002] With the increasing health awareness and faster pace of life, personalized diet recommendation systems have become a key tool for meeting the health needs of modern society. Traditional diet recommendation methods, such as content-based filtering or collaborative filtering, perform well in providing static and universal advice. However, these methods have significant limitations in responding to dynamically changing user preferences, achieving multi-objective optimization (such as nutritional balance and user satisfaction), and learning to improve recommendation strategies from long-term interactions. User tastes, physiological needs, activity levels, and even emotions can change at any time, and traditional systems often struggle to capture and adapt to these subtle and continuous changes.

[0003] Currently, researchers have explored various methods to achieve personalized recommendations. Early work mainly focused on the application of traditional recommendation algorithms: such as content-based filtering methods, which recommend similar dishes to users by analyzing features such as ingredients and cuisine; and collaborative filtering techniques, which recommend dishes based on user groups with similar preferences. Although these methods meet the needs of personalization to some extent, they generally face challenges such as slow response to changes in user preferences, difficulty in simultaneously optimizing multiple dimensions such as nutrition and flavor diversity, and the cold start problem. When users' tastes change or they try new dishes, the system often struggles to adapt quickly.

[0004] In recent years, with the advancement of artificial intelligence technology, especially the development of reinforcement learning, its application in recommender systems has gradually become a new research hotspot. Reinforcement learning (RL), as an important branch of machine learning, has demonstrated powerful advantages in dynamic and sequential decision-making tasks. RL models can learn through continuous interaction with users and optimize their decision-making strategies based on the rewards obtained (i.e., user feedback) to maximize long-term returns. This learning paradigm aligns perfectly with the essence of dietary recommendations: recommender systems need to make sequential decisions based on constantly changing nutritional needs and user preferences, and learn from users' acceptance or rejection of recommendations.

[0005] Therefore, this application introduces reinforcement learning, which is expected to overcome the shortcomings of traditional methods and provide users with more adaptive and personalized dietary advice. Summary of the Invention

[0006] The present invention aims to overcome the shortcomings of the prior art and provide a personalized diet recommendation method that uses differentiated constraints to improve the reinforcement Q learning mechanism.

[0007] The personalized dietary recommendation method of the present invention, which employs differentiated constraints to improve the reinforcement Q-learning mechanism, is implemented through the following steps: S1. Data Acquisition and Preprocessing (1) Data cleaning and missing value handling We collected existing publicly available recipe data, removed duplicate recipes, and supplemented each recipe with nutritional indicator data. (2) Feature engineering and standardization Extract the ingredient quantity and cooking step length parameters from the recipe data in step (1); (3) User data and preference storage The SQLite database is used as the backend storage structure. The data in steps (1) and (2) are stored in the SQLite database, and a user basic information table, a recipe preference list Q table, and a reinforcement learning Q table are added to the database.

[0008] S2. Reinforcement learning mechanism employing fusion linear programming algorithm (1) Basic Recommendations Based on the user's basic information, for new users without preference parameters, a linear programming algorithm is used to recommend three meals a day while meeting the user's nutritional needs. (2) Update the user preference list Q table After receiving algorithmic recommendations, users choose to accept or reject them. If they accept, the recommendations stop. The learning mechanism updates the user's preference list Q; if the user does not accept the suggestion, a linear algorithm is used to continue making recommendations. The learning mechanism updates the user's preference list, Q table; (3) Reward calculation and preference list Q table update Based on the preference list from step (2), The learning mechanism learns each user's preferences and forms a reinforcement learning Q-table. In the next recommendation, a reinforcement learning mechanism that integrates linear programming algorithms is used to recommend recipes.

[0009] As a further improvement of the present invention, the basic recommendation in step (1) of S2 includes: obtaining user information, including user basic information, health goals and real-time status, constructing a multi-dimensional state vector, and standardizing all indicators to the [0,1] interval; constructing dietary actions, which cover five categories of food: staple food, protein, vegetables, fruits and oils, to ensure that the nutritional needs of the actions and user status are matched; and using linear function approximation. Initialize the weight vector Through feature mapping The cross-features of user state and eating behavior are transformed into a linearly computable form, initially... All values ​​are set to 0 to ensure that the algorithm learns adaptively from scratch.

[0010] As a further improvement of the present invention, step (2) of S2 adopts Recommended steps for the learning mechanism: (1) Action selection strategy: The learning mechanism adopts Greedy algorithms select action strategies when... At that time, the optimal action with the highest current Q value is selected with a 90% probability, and a new action is randomly selected with a 10% probability to avoid the recommendation result becoming fixed; (2) Through linear The learning mechanism outputs the top 3 recipes ranked by Q-value as the initial recommendation set.

[0011] As a further improvement of the present invention, step (3) of S2 adopts the following: The learning mechanism performs reward calculation and Q-value update steps, including: (1) Reward signal ( Design: Employs multi-dimensional, real-time feedback scoring to quantify user satisfaction with the recommended results and their nutritional suitability; reward value is... ; (2) Q-value update execution: Rewards based on user feedback and the next state ,pass Learning update rules adjust weight vector : (3) Adaptive adjustment logic: If the reward value of 3 consecutive recommendations is high... Automatically increase Up to 0.3; if Continue for 2 times, reduce The value was reduced to 0.05, enabling dynamic adaptation of the recommendation strategy.

[0012] This invention presents a personalized diet recommendation method that employs a differentiated constraint-enhanced Q-learning mechanism. Through the Q-learning algorithm, the method dynamically learns and adjusts the recommendation strategy from user interactions. At the same time, it utilizes a differentiated constraint mechanism to ensure the nutritional balance of the recommendations. By satisfying users' dynamic preferences and achieving nutritional balance, this method significantly improves user satisfaction and contributes an innovative and efficient solution to the field of intelligent health management. Attached Figure Description

[0013] Figure 1 Missing data values; Figure 2 Data distribution and outlier analysis; Figure 3 Reinforcement learning flowchart; Figure 4 Q-value variation graph; Figure 5 Absolute heat deviation distribution of the DC-RL model and the DC model lower limit target; Figure 6 Average number of times per meal refused. Detailed Implementation

[0014] The personalized dietary recommendation method of the present invention, which employs differentiated constraints to improve the reinforcement Q-learning mechanism, is implemented through the following steps:

[0015] S1. Data Acquisition and Preprocessing

[0016] This application performs systematic preprocessing operations on recipe data and user data before reinforcement learning recommendation to ensure the stability and personalization of the model training and recommendation process. The preprocessing work mainly includes data cleaning and missing value handling, feature engineering and standardization, and the construction of a storage mechanism for user data and preference information.

[0017] (1) Data cleaning and missing value handling The original recipe data mainly comes from public datasets, including structured recipe nutritional information (such as calories, protein, fat, sodium content, etc.) and unstructured text descriptions (such as ingredients and cooking instructions), totaling 35,628 data entries.

[0018] like Figure 1 , Figure 2 As shown, this application conducted statistical analysis on the data. To improve data quality, duplicate dishes were first removed, and missing values ​​were filled for key nutritional indicator fields (including calories, protein, fat, and sodium). The column median was used to mitigate the impact of extreme values. In addition, to prevent outliers from interfering with the recommendation results, the quantile truncation method was used to truncate the above nutritional fields and ratings to the upper limit of outliers (99th quantile).

[0019] (2) Feature engineering and standardization To enrich the input features, this application further extracts two auxiliary features based on the original JSON format recipe data: the number of ingredients (len_ingredients) and the length of cooking steps (len_directions). len_directions is obtained by counting the length of the merged instruction text, while len_ingredients is the element count of the ingredient list. All newly added features still have some missing values ​​after merging, which are uniformly filled using the median. In addition, the system uniformly adds a preference score field, preference_score, to the recipe data, with an initial value of 0, for dynamically updating user interest tendencies in subsequent reinforcement learning recommendations.

[0020] (3) Data and Preference Storage The system uses an SQLite database as its backend storage structure and features three core tables: `users`, `preferences`, and `q_tables`. These tables store basic user information, historical food preferences, and parameters for the Q-table used in reinforcement learning, respectively. During registration, users are required to provide information such as gender, age, height, weight, and activity level. The system calculates their daily nutritional needs based on the Mifflin-St Jeor formula and the PAL (Physical Activity Level) factor. Regarding preference recording, the system records `last_eaten_date` and `preference_score` for each recommendation and dynamically adjusts these scores based on whether the user accepts or rejects the recommendation. In reinforcement learning, the Q-table is serialized into a nested dictionary format as a JSON string and persisted to the `q_tables` table, enabling continuous learning of personalized recommendation strategies.

[0021] S2. Reinforcement learning mechanism employing fusion linear programming algorithm This application proposes an improved reinforcement Q-learning mechanism with differentiated constraints (see...). Figure 3 The core of this approach lies in integrating the precise nutritional matching capabilities under differentiated constraints with the dynamic strategy adjustment capabilities of reinforced Q-learning. Through this combination, the system aims to both meet users' personalized nutritional needs and adapt to their dynamic preferences, ultimately maximizing long-term user satisfaction. RL generates recommendation strategies by analyzing users' health goals, dynamic preferences, and historical behavioral data; users provide feedback during actual interactions, and the system converts this feedback into reward signals, which are then used to update the Q-table. This mechanism ensures that the recommendation strategy can continuously adapt and adjust as user needs change, ultimately improving recommendation acceptance rates while maintaining nutritional balance.

[0022] The algorithm first constructs a reinforcement learning state by combining user activity levels and recent recommendation acceptance data, and inputs it into the Q-Learning model. Then, it selects a recommendation strategy from the action space (including standard recommendation, preference-oriented recommendation, and new dish exploration) according to the corresponding strategy. Subsequently, it filters candidate recipes based on user nutritional needs and a cooling-off mechanism, and completes robust optimization recommendations that meet nutritional indicators such as calories and protein through a linear programming model. Based on whether the user accepts the recommendation results, the system adjusts the dish preference score and gives positive rewards or gradient penalties to the Q-Learning model, while updating the Q-table to complete a single recommendation learning cycle. This process supports up to 5 rounds of recommendation attempts and will clear the preference cache and reset the recommendation strategy when consecutive recommendations fail to ensure the robustness and personalization of the recommendations.

[0023] (1) Basic Recommendations The core of the recommendation process is to establish a mapping relationship between "user status - eating behavior - feedback reward". The preliminary preparation stage requires data standardization and definition of the status / action space. User status ( Quantification: Integrating basic user information, health goals, and real-time status to construct a multi-dimensional status vector. Basic information includes age, gender, height and weight (calculated as BMI), and dietary restrictions; health goals are clearly tagged with categories such as weight loss (low-calorie, high-fiber), muscle gain (high-protein, high-carbohydrate), and chronic disease management (sugar / salt control); real-time status includes daily exercise volume, percentage of nutrient intake, and feedback on physical discomfort (such as blood sugar fluctuations), with all indicators standardized to the [0,1] range. Dietary actions ( The dietary recommendation unit is defined as a "single meal recipe combination," covering five categories of ingredients: staple food, protein, vegetables, fruits, and oils. The action space contains 100+ preset recipes, each labeled with its core nutritional characteristics (calories, protein, carbohydrates, fat, and dietary fiber content) to ensure that actions match the nutritional needs of the user's state. Differentiated constraint value function initialization: using linear function approximation Avoid traditional The problem of exploding table dimensions. Initializing the weight vector. Through feature mapping The cross-features of user status and dietary actions (such as "weight loss goal + low-calorie diet" or "chronic disease management + blood sugar control diet") are transformed into a linearly computable form, initially... All values ​​are set to 0 to ensure that the algorithm learns adaptively from scratch.

[0024] (2) Update the user preference list Q table This stage improves the core logic of the personalized diet recommendation method by employing differentiated constraints to enhance the Q-learning mechanism, selecting the optimal diet plan from the action space. The specific steps are as follows: ①Action selection strategy: The learning mechanism adopts Greedy algorithms balance "exploration" and "exploitation". When At that time, the optimal action with the highest current Q value is selected with a 90% probability (utilizing existing learning results), and a new action is randomly selected with a 10% probability (exploring potential better recipes) to avoid the solidification of recommendation results; ② Real-time value calculation: based on linear The core formula for learning completes the value assessment, and the calculation process is as follows: a. Enter the current user status and candidate diet actions Through feature mapping Extract cross features (e.g., BMI ≥ 28 + weight loss goal + high-protein, low-calorie exercise); b. Substitute into linear... function Calculate the value of the state-action pair; c. Iterate through all candidate actions and output the top 3 recipes by Q-value as the initial recommendation set; ③ Constraint verification: The initial recommendation set is filtered for compliance to ensure that it meets the user's dietary restrictions and nutritional balance requirements, and finally 1-2 optimal solutions are retained as output results.

[0025] (3) Reward calculation and preference list Q table update Recommendations are not a one-time output; they need to be adaptively optimized based on user feedback to form a closed-loop iteration. ① Reward signal ( Design: Employing a multi-dimensional, real-time feedback rating system to quantify user satisfaction with the recommended results and nutritional suitability. User direct ratings (1-5 points) account for 60%, nutritional achievement rate (the degree to which actual intake matches target needs) accounts for 30%, and dietary diversity (to avoid duplicate recommendations) accounts for 10%. The reward value is calculated by combining these factors. .

[0026] ② Q-value update execution: Rewards are updated based on user feedback. and the next state (The user's physical condition and remaining nutritional needs after consuming the recommended diet) through Learning update rules adjust weight vector : a. Core update formula:

[0027] b. Parameter settings: Learning rate (Control update step size to avoid oscillations), discount factor (Focusing on long-term future rewards); c. Differentiated Constraint Weight Optimization: Minimizing the prediction using gradient descent Values ​​and Objectives Error in value, making Gradually adapt to user preferences.

[0028] ③ Adaptive adjustment logic: If the reward value of 3 consecutive recommendations is... Automatically increase Up to 0.3 (increase the proportion of new recipe exploration); if Continue for 2 times, reduce Up to 0.05 (enhancing the utilization of optimal recipes) to achieve dynamic adaptation of recommendation strategies.

[0029] This invention employs a personalized dietary recommendation method that improves upon the reinforcement Q-learning mechanism using differentiated constraints. It deeply integrates differentiated constraints with a Q-learning reinforcement learning model: differentiated constraints ensure precise fulfillment of nutritional constraints under the current strategy, while reinforcement learning adaptively selects the optimal recommendation strategy in response to dynamic user feedback and preferences. A multi-dimensional reward function is designed: this function not only considers whether the user accepts the recommendation but also additionally rewards behaviors that align with the user's long-term preferences and successful exploration of new dishes, thus more precisely guiding the learning direction of the reinforcement learning model. A closed-loop evaluation environment is constructed: by simulating user behavior, a closed-loop evaluation environment is established, which more realistically reflects the system's performance in long-term interactions and allows for rigorous comparison with traditional baseline models.

[0030] Based on the above steps, a complete personalized diet recommendation process is constructed: 1. Data Input: Basic user information, health goals, dietary preferences, and real-time status; 2. State Modeling: Quantize multi-dimensional state vectors and construct differentiated constraint feature maps. ; 3. Action Filtering: Through A greedy algorithm selects candidate recipes, and after constraint verification, outputs recommended results. 4. Feedback Collection: Obtain user ratings, nutritional intake data, and diversity feedback to calculate rewards. ; 5. Model Iteration: Updating Linear Models Function weights Adjust the exploration-utilization strategy; 6. Iterative optimization: Repeat steps 2-5 to improve the accuracy of recommendations in each round.

[0031] This process is simplified using differential constraint functions. By reducing computational complexity and combining the feedback and iterative characteristics of reinforcement learning, the specific path from state input to result output is clearly defined, and the personalized and adaptive recommendations are ensured through a dynamic adjustment mechanism, thus solving the problem of insufficient executability caused by the stacking of logic in traditional algorithms.

[0032] To further illustrate the effectiveness of the personalized dietary recommendation method based on differentiated constraints and improved reinforcement Q-learning mechanism, a systematic experiment was conducted under specific hardware and software conditions. The experiment was run on a workstation equipped with an Intel Core i5-12400F CPU and 32GB RAM, running Windows 11 64-bit. All algorithms were implemented using Python 3.9, with key libraries including: pandas 1.5.3 for data processing and analysis; numpy 1.23.5 for matrix operations and numerical computation; sqlite3 for local persistent storage of user information and reinforcement learning parameters; puDC 2.7.0 for linear programming modeling and nutritional goal solving; matplotlib 3.7.1 and seaborn 0.12.2 for visualizing recommendation results and evaluation metrics; and swifter for accelerating the mapping and computation of large-scale data. All experiments were completed without GPU acceleration, and the system's efficiency still met the requirements for large-scale simulation and evaluation.

[0033] Meanwhile, this system adopts the Q-Learning algorithm as the core strategy optimization method, and the relevant hyperparameter settings are as follows: Table 1. Hyperparameters of the Q-Learning Algorithm

[0034] The Q table uses user status (such as "activity level + recent rejection status") as the state space and recommendation strategies (such as favor_high_rating, favor_user_preference, etc.) as the action space. The system dynamically updates the Q value based on user feedback after each recommendation.

[0035] 1. User simulation scheme To systematically evaluate the recommendation capability and learning performance of the model, this application designed and implemented a simulated user mechanism. This mechanism simulates user behavior feedback in real-world usage scenarios by randomly generating a set of basic user attributes and preferences. The specific configuration is as follows: 50 independent users (episodes) are generated, each user is simulated for 30 days (daysper episode), and recommendations are made for each user based on three meals (breakfast, lunch, and dinner) per day, with a maximum of 5 recommendations per meal. The probability of a user accepting a recommendation is controlled by the simulation parameter `compliance_bias`, which is set to 0.7. The recommendation strategy set (i.e., the action space) includes the following five strategies: `standard`, `favor_high_rating` (favoring highly rated dishes), `favor_new_dishes` (favoring dishes not yet tried), and `favor_user_preference` (favoring dishes with high user preference scores). In comparative experiments, a baseline model without reinforcement learning is also introduced, i.e., the `standard` strategy is always used for recommendations, to evaluate the performance improvement effect brought by Q-Learning-based strategy learning.

[0036] 2. Model Evaluation Indicators Nutritional Deviation Index ( (CD): To measure how well the recommended dishes meet the user's nutritional needs, this application uses the absolute deviation between the actual calorie intake of the recommended dishes and the theoretical lower limit as a representative indicator. The smaller this indicator, the closer the system's nutritional recommendations are to the user's goals, and the higher the recommendation quality. The definition is as follows: .

[0037] Recommendation acceptance rate ( Recommendation Acceptance Rate (AR): This metric measures user satisfaction with the system's recommendations. It is defined as the proportion of recommendations that are accepted. This indicator can be used to compare the differences in user responses under reinforcement learning strategies and baseline strategies, and is one of the important metrics for measuring the practicality of a system. .

[0038] Recommendation rejection rate ( Recommendation rejection rate (RR): This measure of user dissatisfaction with the system's recommendation results is defined as the proportion of recommendations that are not accepted. This metric can be used to compare the differences in user responses under reinforcement learning strategies and baseline strategies, and is one of the important indicators for measuring the practicality of a system. .

[0039] Average number of rejections ( (ANR): This recommendation algorithm allows for multiple rounds of recommendations per meal until the user accepts or the maximum number of attempts is reached. This application records the average number of rejections per meal before it is accepted by the user, serving as a negative indicator reflecting the recommendation matching degree. The fewer the rejections, the more efficiently the system can hit the user's preferences within fewer attempts, resulting in higher recommendation efficiency. .

[0040] Recommended time ( ATPM): This application records the total time (in seconds) required for each meal recommendation to evaluate the model's efficiency under different strategies.

[0041] This metric helps verify the system's responsiveness in real-world deployments, especially whether it can maintain acceptable latency after enabling the reinforcement learning module.

[0042] Q-value convergence: To analyze the training effect of the reinforcement learning strategy, the system records the Q-value changes of each state-action pair in the Q-table. By plotting the evolution trend of the Q-value over time steps, it observes whether it gradually and steadily converges, reflecting whether the learning strategy is stable and effective.

[0043] 3. Results Analysis

[0044] After designing a personalized diet recommendation method that employs differentiated constraints to improve the Q-learning mechanism, this application uses a user simulation mechanism to analyze and compare the advantages of the DC-RL model designed in this application compared with the linear programming diet recommendation method DC model. The results are shown in Table 1.

[0045] Table 1. Experimental Results

[0046] Table 1 compares the performance of the DC model and the DC-RL model of this application on three core metrics: recommendation acceptance rate (AR), recommendation rejection rate (RR), and average time to recommendation (ATPM). Experimental results show that in simulated user interaction scenarios, the recommendation acceptance rate of the DC model is 0.70, while that of the DC-RL model increases to 0.73, a relative improvement of 3%; the corresponding recommendation rejection rate decreases from 0.30 in the DC model to 0.27 in the DC-RL model. This result indicates that the DC-RL model can more accurately capture and adapt to users' dynamic preferences, thereby effectively improving the user acceptance of the recommendation strategy.

[0047] In terms of recommendation efficiency, the average recommendation time of the DC model is 0.536s, while the average recommendation time of the DC-RL model is reduced to 0.472s, a reduction of 6.4%. In summary, the DC-RL recommendation system proposed in this application significantly improves operational efficiency while ensuring recommendation quality, and possesses good potential for practical deployment.

[0048] like Figure 4 As shown, Figure 4 The study demonstrates the evolution of the Q-values ​​of each user state-action pair over the daily meal time step. In the initial stage (first 50 steps), the Q-values ​​fluctuate drastically (e.g., the Q-value of the "high_no_rej-standard" combination jumps from 1 to 8), and the algorithm enters the exploration phase with high ε values. The system collects feedback through random actions and quickly adjusts the Q-values. After 200 steps, the Q-values ​​of each combination stabilize in the 7-9 range, the fluctuations converge significantly, and the algorithm enters the utilization phase, where the determinism of the recommendation strategy and user satisfaction tend to stabilize.

[0049] From the perspective of the Q-value differences of the state-action pairs, the average Q-value of the "favor_user_preference" and "standard" strategies in the "high_no_rej" state is 8.2, indicating that when users do not reject the strategy, the acceptance of using historical preferences or standard strategies is higher. In the "moderate_rej" state, the Q-value of the "favor_high_rating" strategy fluctuates more significantly, reflecting that the algorithm will specifically enhance the exploration of high-rated dishes. Moreover, the Q-value of this strategy does not have a significant trough in all states, making it a universal "safety net" strategy.

[0050] A rapid initial increase in Q-value corresponds to improved recommendation accuracy, while a later stabilization corresponds to the solidification of personalized strategies. The stratification of Q-values ​​enables scenario-based adaptation. This verifies the effectiveness of linear-reinforcement Q-learning, achieving both strategy self-optimization and overcoming the limitations of the traditional "one-size-fits-all" recommendation approach.

[0051] like Figure 5 As shown, Figure 5 The absolute deviation distribution between the recommended calorie count and the user's target calorie count for the DC-RL model and the DC model is shown. Using box plots to analyze features such as quartiles and dispersion, the differences in nutritional accuracy between the different models can be visually demonstrated. In terms of deviation levels, the median deviation of the DC-RL model is only about 200 calories, while the DC model's value is close to 600 calories, indicating a significantly higher nutritional match. This is because the DC-RL model incorporates nutritional constraints through a linear programming module, while the DC model relies solely on user feedback for optimization strategies, lacking a clear nutritional control mechanism.

[0052] like Figure 6 As shown, Figure 6By observing the frequency of strategy selection by the RL model under different user states, the dynamic adaptation logic of "state-strategy" is intuitively presented. The state dimension includes the degree of rejection and the frequency of rejection, and the strategies are divided into four categories: high-rating orientation, new dish exploration, user preference orientation, and standard strategy.

[0053] From the quantitative characteristics of state-policy matching, in the "high_no_rej" state (high activity with no rejection), user preference-oriented and new dish exploration strategies account for more than 60% of the total, and the model focuses on utilizing historical preferences and expanding dietary diversity; in the "moderate_no_rej" state (moderate activity with no rejection), high-rating-oriented strategies account for 45%, becoming the preferred strategy, thereby reducing the risk of recommendation rejection.

[0054] From the perspective of the impact of rejection states on policy frequency, the total number of policy choices for the "high_rej" (high rejection frequency) state is less than 200, and the policy distribution is scattered, showing the characteristics of "exploration contraction". In contrast, the policy selection frequency is significantly increased in the "no_rej" (no rejection) state. For example, the total number of choices for the "moderate_no_rej" state exceeds 1600, and the model can focus on selecting appropriate policies to improve recommendation efficiency.

[0055] This distribution validates the state-awareness capability of the RL model, whose policy selection is dynamically adjusted based on user rejection behavior and activity level, achieving precise "state-policy" adaptation for personalized recommendations.

Claims

1. The personalized dietary recommendation method that improves the reinforcement Q-learning mechanism using differentiated constraints is achieved through the following steps: S1. Data Acquisition and Preprocessing (1) Data cleaning and missing value handling We collected existing publicly available recipe data, removed duplicate recipes, and supplemented each recipe with nutritional indicator data. (2) Feature engineering and standardization Extract the ingredient quantity and cooking step length parameters from the recipe data in step (1); (3) User data and preference storage The SQLite database is used as the backend storage structure. The data in steps (1) and (2) are stored in the SQLite database, and a user basic information table, a recipe preference list Q table, and a reinforcement learning Q table are added to the database. S2. Reinforcement learning mechanism employing fusion linear programming algorithm (1) Basic Recommendations Based on the user's basic information, for new users without preference parameters, a linear programming algorithm is used to recommend three meals a day while meeting the user's nutritional needs. (2) Update the user preference list Q table After receiving algorithmic recommendations, users choose to accept or reject them. If they accept, the recommendations stop. The learning mechanism updates the user's preference list Q; if the user does not accept the suggestion, a linear algorithm is used to continue making recommendations. The learning mechanism updates the user's preference list, Q table; (3) Reward calculation and preference list Q table update Based on the preference list from step (2), The learning mechanism learns each user's preferences and forms a reinforcement learning Q-table. In the next recommendation, a reinforcement learning mechanism that integrates linear programming algorithms is used to recommend recipes.

2. The personalized dietary recommendation method using differentiated constraints to improve the reinforcement Q-learning mechanism as described in claim 1, characterized in that... The basic recommendation in step (1) of S2 includes: obtaining user information, including basic user information, health goals and real-time status, constructing a multi-dimensional state vector, and standardizing all indicators to the [0,1] interval; constructing dietary actions, which cover five categories of food: staple food, protein, vegetables, fruits and oils, to ensure that the actions match the nutritional needs of the user's status; and using linear function approximation. Initialize the weight vector Through feature mapping The cross-features of user state and eating behavior are transformed into a linearly computable form, initially... All values ​​are set to 0 to ensure that the algorithm learns adaptively from scratch.

3. The personalized dietary recommendation method using differentiated constraints to improve the reinforcement Q-learning mechanism as described in claim 1, characterized in that... In step (2) of S2, the following is adopted: Recommended steps for the learning mechanism: (1) Action selection strategy: The learning mechanism adopts Greedy algorithms select action strategies when... At that time, the optimal action with the highest current Q value is selected with a 90% probability, and a new action is randomly selected with a 10% probability to avoid the recommendation result becoming fixed; (2) Through linear The learning mechanism outputs the top 3 recipes ranked by Q-value as the initial recommendation set.

4. The personalized dietary recommendation method using differentiated constraints to improve the reinforcement Q-learning mechanism as described in claim 1, characterized in that... In step (3) of S2, The learning mechanism performs reward calculation and Q-value update steps, including: (1) Reward signal ( Design: Employs multi-dimensional, real-time feedback scoring to quantify user satisfaction with the recommended results and their nutritional suitability; reward value is... ; (2) Q-value update execution: Rewards based on user feedback and the next state ,pass Learning update rules adjust weight vector : (3) Adaptive adjustment logic: If the reward value of 3 consecutive recommendations is high... Automatically increase Up to 0.3; if Continue for 2 times, reduce The value was reduced to 0.05, enabling dynamic adaptation of the recommendation strategy.

Images