A dynamic programming multi-round search enhanced recommendation agent method

Abstract

This application discloses a dynamic programming-based multi-round retrieval-enhanced recommendation agent method, addressing the problems of inability to adapt to diverse user contexts and the tendency of open-loop retrieval processes to lead to incomplete knowledge completion, ultimately causing large-scale model inference illusions and limited recommendation performance. First, a hierarchical policy network is designed to deeply couple knowledge retrieval and recommendation tasks into a reinforcement learning policy network, dynamically generating structured retrieval plans that perceive the recommendation context. Second, a multi-round reflection mechanism is used to detect knowledge gaps and adjust the retrieval plan in reverse to achieve accurate knowledge completion, forming a closed-loop optimization process of perception-planning-execution-feedback. Finally, each round utilizes retrieval-enhanced knowledge to drive a large language model to complete the recommendation, and the policy network is optimized with a joint reward of accumulated retrieval cost and recommendation gain, achieving self-optimization of the retrieval process. This application has completed training, testing, and model optimization on multiple public datasets, validating the effectiveness of the proposed method.

Description

Technical Field

[0001] This application relates to the fields of recommender systems and large language models, specifically collaborative filtering technology based on large language models; this application relates to knowledge graph retrieval and retrieval enhancement generation technology; this application relates to agent decision-making technology based on reinforcement learning; this application also relates to deep learning representation learning and other related technologies. Background Technology

[0002] With the widespread adoption of the internet, a massive amount of information floods the network, growing exponentially and making it impossible for users to efficiently and quickly select the information they need, leading to a serious information overload problem. To alleviate information overload and improve information utilization, recommender systems have emerged. A recommender system is a system that uses users' historical behavioral data and personal preferences to recommend items or services that users may be interested in. Its goal is to provide personalized recommendations based on users' needs and interests, helping them find items or services that match their preferences more quickly, thus improving user experience and satisfaction. In recent years, large language models, with their powerful semantic understanding and reasoning capabilities, have reshaped the technological paradigm of recommender systems, breaking through the semantic representation limitations of traditional ID-based collaborative filtering methods. They can interpret users' complex interests and preferences through semantic understanding, providing a new solution to alleviate the data sparsity problem in recommender systems. However, the parameterized knowledge of large language models is static and general, often insufficient to capture signals related to users' personalized interests, frequently triggering reasoning illusions and resulting in poor recommendation performance.

[0003] To address these limitations, existing research introduces external knowledge graphs to supplement large language models with domain semantic structure signals, guiding user interest-based learning. Emerging retrieval-enhanced recommendation models further integrate contextual knowledge from knowledge graphs into the prompts of large language models, enriching the interest-based reasoning process and mitigating the illusion problem. However, current retrieval-enhanced recommendation paradigms suffer from three core limitations: First, most existing research treats knowledge retrieval and recommendation as independent stages, failing to perceive the needs of downstream recommendation tasks during the retrieval process, leading to a disconnect between retrieval content and recommendation goals. Second, these models typically execute static retrieval using fixed heuristic rules, failing to adapt to the differentiated retrieval needs of different user contexts and struggling to dynamically adjust the retrieval structure and depth. Third, due to the lack of feedback and correction mechanisms, the retrieval process is executed in an open loop, resulting in incomplete and imprecise retrieval content that cannot provide effective knowledge support for recommendation tasks.

[0004] This application takes this as its starting point and proposes a dynamic programming-based multi-round retrieval-enhanced recommendation agent method. Summary of the Invention

[0005] Current retrieval-enhanced recommendation systems suffer from three key limitations. First, most existing methods decouple knowledge retrieval from recommendation modeling, making the retrieval process unable to perceive downstream recommendation needs and only pursuing semantic relevance, which easily introduces redundant information unrelated to the recommendation task. Second, static retrieval mechanisms cannot adapt to diverse user contexts, making it difficult to dynamically adjust the retrieval structure and depth according to different user characteristics, resulting in significant performance degradation in data-sparse scenarios. Finally, the open-loop retrieval process lacks feedback and correction mechanisms, making it impossible to optimize the retrieval strategy based on recommendation performance, leading to incomplete knowledge completion, exacerbating the reasoning illusion of large language models, and impairing recommendation performance.

[0006] To address the aforementioned issues, this application discloses a dynamic programming-based multi-round retrieval enhancement method for recommendation agents, the overall framework of which is as follows: Figure 2 As shown in the figure, this application models the knowledge retrieval-enhanced recommendation task as a dynamic multi-round decision-making process, achieving end-to-end optimization through reinforcement learning. First, a hierarchical policy network is designed to dynamically generate a structured retrieval plan that perceives the recommendation context, enabling the retrieval process to perceive downstream recommendation needs and solving the problem of decoupling retrieval and recommendation. Second, a multi-round reflection mechanism is used to detect knowledge gaps and adjust the retrieval plan in reverse to achieve accurate knowledge completion, forming a closed-loop optimization process of perception-planning-execution-feedback. Finally, a joint reward optimization policy network based on accumulated retrieval cost and recommendation gain is used to achieve self-optimization of the retrieval process, improving recommendation performance while achieving precise control of retrieval costs.

[0007] The application details for each major module of this method are as follows:

[0008] 1. Search Action Generation Module

[0009] The system extracts user recommendation and retrieval needs, generates a unified state embedding, and transforms the unstructured recommendation context into a structured, executable retrieval plan through a hierarchical policy network. This hierarchical policy network decomposes the retrieval action into four progressive decision units: entity prioritization, relationship pruning, semantic path depth allocation, and historical association importance assessment. This significantly reduces the space complexity of single-step decisions, avoids ineffective exploration of redundant knowledge branches, and dynamically generates retrieval trajectories and budget allocation schemes adapted to the current user context.

[0010] 2. Reflection-driven knowledge completion module

[0011] Abstract retrieval actions are instantiated into specific queries on the knowledge graph. A multi-round reflection mechanism dynamically executes and adjusts the retrieval plan, iteratively filling knowledge gaps. First, the retrieval plan is broken down into multiple types of structured queries, and differentiated retrieval is performed. Then, deficiencies in the current knowledge subgraph are identified according to priority levels, generating targeted supplementary retrieval plans. Finally, through multiple iterations, gradual knowledge completion is achieved, and phased termination conditions are designed to ensure knowledge integrity while avoiding cost waste caused by excessive retrieval.

[0012] 3. Knowledge-enhanced recommendation module

[0013] This paper leverages structured knowledge obtained from multiple rounds of retrieval to enhance the recommendation instructions of a large language model, predicts the interaction scores between users and candidate items, and generates a personalized recommendation list in descending order of scores. Simultaneously, a hybrid reward function incorporating recommendation gain and retrieval cost is constructed to optimize the hierarchical policy network end-to-end using reinforcement learning. Furthermore, the knowledge encoding module is optimized based on the cross-entropy loss of the recommendation task, achieving deep coupling and joint optimization between the retrieval process and the recommendation task. Attached Figure Description

[0014] Figure 1 A schematic diagram illustrating the core limitations of existing search-enhancing recommendation paradigms;

[0015] Figure 2 This is a diagram illustrating the overall framework of the dynamic programming-enhanced multi-round retrieval recommendation agent method proposed in this application. Detailed Implementation

[0016] This application discloses a dynamic programming-enhanced multi-round retrieval recommendation agent method. The specific implementation steps of this application are as follows:

[0017] Step 1: Data preprocessing and partitioning of training and test sets

[0018] Acquire historical user-item interaction logs and corresponding domain knowledge graphs for the target recommendation scenario (such as various information recommendation platforms in different fields). Number users and items in the dataset sequentially, starting from 0. The external knowledge graph contains entities, relations, and triples associated with the items, and entities and relations are uniformly encoded. Record historical user-item interactions using implicit feedback: if a user has interacted with an item, the label is 1 (positive sample); if no interaction has occurred, the label is 0 (negative sample). If the dataset interactions are in rating form, items with ratings greater than or equal to a preset threshold are labeled 1, and those with ratings less than or equal to a preset threshold are labeled 0.

[0019] After obtaining the processed dataset, it is divided into a training set and a test set: for each user, their most recent interaction record is used as the true positive sample label in the test set, and the remaining interaction records are used as the user's historical behavior data. During the model evaluation phase, a predetermined number of negative samples are randomly sampled from the set of items that the user has not interacted with to form a candidate set for each test sample's positive samples. The model needs to sort the items in the candidate set to complete the recommendation task.

[0020] Step Two: Recommendation and Retrieval Needs Awareness and State Encoding

[0021] Historical user interaction data and candidate set information are extracted and encoded to obtain user recommendation and retrieval demand representations, respectively. These are then aggregated to generate a unified state embedding, which serves as the input to the downstream policy network. A pre-trained language model encoder is employed. The text description and attribute features of the items are encoded to obtain the semantic embedding of each item. Each aggregates the user's historical interaction set With candidate set The semantic embedding of items in the data yields a representation of the user's historical preferences and a representation of the candidate set of requirements.

[0022]

[0023] By calculating the pairwise similarity between historical interaction items and candidate items, a retrieval demand matrix is ​​constructed. This matrix is ​​then transformed using a two-layer fully connected network to obtain a representation of the user's retrieval demand. This representation reflects the potential semantic gaps that need to be filled through knowledge graph traversal. Finally, the recommendation requirement representation and the retrieval requirement representation are concatenated to obtain the final unified state embedding:

[0024]

[0025] in State embedding is used for embedding splicing operations. The task requirements of the current recommendation context are fully encoded to activate the downstream policy network.

[0026] Step 3: Hierarchical Strategy Network and Dynamic Retrieval Planning

[0027] A hierarchical strategy network with four levels of hierarchical decision-making is constructed. Using a unified state embedding as input, the retrieval action is decomposed into multi-level progressive decision-making units, dynamically generating structured retrieval plans and differentiated budget allocation schemes. The retrieval action is broken down into four progressive decision-making units, forming a complete decision structure. The first layer is entity priority sorting, which is the union of user history interactions and candidate sets. For all entities in the first layer, calculate the priority distribution and select high-priority entities as seed nodes for retrieval; the second layer is relation pruning, which involves the full set of relations in the knowledge graph. Assign importance scores and filter out subsets of relations whose importance exceeds a preset threshold. The first layer filters semantic noise and constrains the search direction. The second layer is semantic path depth allocation, which adaptively assigns differentiated maximum search depths to reasoning paths of different semantic types, deeply explores core knowledge paths, and performs lightweight retrieval on irrelevant paths. The third layer is historical association importance assessment, which predicts a scalar score reflecting the necessity of explicit path verification between candidate items and user historical interactions. Explicit path retrieval is triggered only when the score exceeds a threshold. Based on the above hierarchical structure, a system with trainable parameters is constructed. Policy network Through multi-level decision-making and dynamic planning of hierarchical retrieval trajectories, abstract recommendation and retrieval requirements are transformed into executable, structured action sequences. The policy network dynamically generates retrieval action sequences and budget allocation ratios, using the following formula:

[0028]

[0029] in For the first Hierarchical generated search actions To ensure the proportion of budget allocated to this level meets the following requirements: , For the total search budget, This involves embedding the state uniformly. The final result is a complete retrieval trajectory. And break it down into a structured search plan.

[0030] Step 4: Reflection-driven multi-round knowledge completion

[0031] Based on the generated search plan, differentiated searches are performed. Adaptive knowledge gap detection identifies deficiencies in the current search content, adjusts the search plan accordingly, and iteratively executes multiple rounds of searches to achieve gradual knowledge completion and optimization. The search trajectory is then tracked. Instantiated into four types of structured queries, each performing its corresponding retrieval operation, entity retrieval query set. Based on entity priority scores, core entities with high rankings are retrieved under budget constraints, and their textual descriptions, attribute values, and semantic features are extracted; relational query sets are expanded. Based on relation importance scores, selective relation expansion is performed on retrieved entities, expanding only those with importance exceeding a threshold. Relationship type; path traversal query set Based on the path depth allocation results, a dual-mode path traversal is performed. For candidate items, paths connecting to the user's historical interactions are searched first, while background knowledge entities undergo adaptive traversal according to the allocated depth; historical association query set. When the historical association importance score exceeds the threshold When this occurs, explicit candidate-historical path verification is triggered, extracting the complete path connecting candidate items and the user's historical interactions. After completing this round of retrieval, the knowledge subgraph of the l-th round is obtained. and the cumulative search cost for this round. Wherein, the threshold and These are pre-defined hyperparameters, all ranging from [0, 1], used to control the sparsity and precision of the retrieval. Five types of knowledge gaps are defined from highest to lowest priority: missing core entities, missing historical associations, missing key relationships, incomplete historical associations, and missing semantic paths. Based on this priority hierarchy, defects in the current knowledge subgraph are identified, and new supplementary retrieval plans are generated according to priority. Multiple rounds of iterative retrieval are performed. Each round detects gaps based on the knowledge subgraph from the previous round, performs supplementary retrieval, and updates the knowledge subgraph. Phased termination conditions are designed: during the training phase, the marginal value of the recommendation gain is calculated using real labels; during the inference phase, the termination timing is determined by the ranking stability of the recommendation lists over two consecutive rounds. The general termination condition is:

[0032]

[0033] in, The symbol for the empty set, This represents the logical OR operator. For the first The newly generated search plan, To recommend the marginal gain value, The minimum gain threshold is set to 1 x 10^(-4). This represents the maximum number of iteration rounds (a constant is set to 3 rounds). The final result is the complete retrieval knowledge accumulated over multiple rounds. In actual execution, the multi-round iterative retrieval process will immediately stop once one of the following three scenarios is triggered, indicating that the termination condition has been met:

[0034] 1. Knowledge Completeness Termination: When adaptive gap detection identifies the current knowledge subgraph and fails to detect any of the five preset knowledge gap types, a new supplementary retrieval plan is generated. When the set is empty, it means there are no gaps to fill, and the process terminates automatically.

[0035] 2. Termination of diminishing marginal returns: During the model training phase, between two consecutive iterations, the marginal increase in recommendation gain decreases due to knowledge saturation. Less than the set threshold This indicates that continuing the search only increases computational cost without providing a substantial improvement in recommendation performance, and the process terminates prematurely due to diminishing returns. During the model inference phase, this condition is adaptively replaced with a stable termination condition: calculating two consecutive rounds (the...) -1 round and the When the order of the Top-N recommendation lists generated in the two rounds remains unchanged (i.e., the composition and order of the items in the two rounds of recommendation lists are completely identical), the stable termination condition is determined to be met.

[0036] 3. Forced Boundary Termination: When the round index of multiple iterations... Reaching the set maximum number of iteration rounds To prevent infinite loops caused by map noise, a hard boundary is triggered to force termination. Finally, after satisfying any of the above termination conditions, the multi-round iteration ends, yielding the complete retrieval knowledge accumulated over multiple rounds. .

[0037] Step 5: Knowledge-enhanced Large Language Model Recommendation

[0038] The structured knowledge obtained from multiple rounds of retrieval is encoded and projected, and then input into a large language model along with user history interactions and candidate set information. This predicts the user's interaction score with candidate items, and generates a personalized recommendation list in descending order of score. A pre-trained language model encoder is used. ) for retrieval knowledge The text descriptions of entities, relations, and semantic paths are encoded and processed by a trainable two-layer MLP projector. The encoded knowledge features are mapped to the embedding space of a large language model to obtain knowledge enhancement tokens. Knowledge Enhancement Token User history interactions Candidate set Common Input Large Language Model ( Predict the probability of user interaction with each candidate item:

[0039]

[0040] Based on the predicted interaction probabilities, the items in the candidate set are sorted in descending order, and the top N items are selected to generate a personalized Top-N recommendation list for the user.

[0041] Step Six: Hybrid Reward Design and End-to-End Model Optimization

[0042] A hybrid reward function incorporating recommendation gain and retrieval cost is constructed to optimize the hierarchical policy network end-to-end using reinforcement learning. Simultaneously, the knowledge encoding module is optimized based on the cross-entropy loss of the recommendation task, achieving joint optimization of the retrieval process and the recommendation task. To balance the improvement in recommendation performance with the overhead of retrieval costs, a hybrid reward function is constructed:

[0043]

[0044] in, The recommendation gain brought by the retrieval of knowledge, that is, the improvement of the recommendation evaluation index after the introduction of external knowledge; The total cost accumulated over multiple rounds of retrieval; To balance the hyperparameters; This is a normalization operation within a batch.

[0045] Based on the policy gradient method of reinforcement learning, the policy network is optimized to maximize the expected reward, and the policy loss function is:

[0046]

[0047] To retrieve the number of actions in the trajectory, For the state embedding of the t-th decision level, For the hybrid reward The calculated dominance function value, The information entropy of the action probability distribution is output to the hierarchical policy network to measure the randomness of the policy and encourage the network to explore the action space. This is the entropy coefficient.

[0048] Adopting the standard autoregressive language modeling objective, the knowledge encoding and projection modules are optimized, and the cross-entropy loss function is:

[0049]

[0050] in For the set of token locations of the real label, The true token sequence output by the large language model. For the corresponding position The token For position Previous historical token sequences, The output probability distribution of the large language model; the model as a whole adopts a multi-task learning objective, and jointly optimizes the policy network and recommendation model end-to-end. The total loss function is:

[0051]

[0052] in, For policy network loss; The cross-entropy loss is used for the recommendation task. The total loss function is... The end-to-end training iteration and optimization termination conditions are specifically defined by setting a joint constraint mechanism that links the maximum number of training rounds with test set metrics to early stopping:

[0053] The entire recommendation agent system is fine-tuned and optimized, with a preset maximum global training epoch limit of 50 epochs. The model performs backpropagation under the Adam optimizer, decaying the total loss function epoch by epoch. The value of . At the end of each training step, forward recommendation inference is performed on an independent test set using the current intermediate model parameters, and the Recall@N and NDCG@N recommendation evaluation metrics for the candidate test sample set are calculated. The final training termination condition for the end-to-end optimization is defined as: if the total loss function If the model achieves complete convergence (i.e., the absolute difference of the total loss value fluctuation over three consecutive rounds is less than 1 x 10^(-5), or the target recommendation evaluation index (NDCG@N) on the test set does not show any monotonically increasing improvement over 10 consecutive rounds), then early stopping is immediately triggered, and the algorithm forcibly terminates the iterative training process of the current loss function. At this point, the system directly extracts and saves the model parameters of the round with the highest recommendation evaluation index on the test set throughout the entire history, as the optimal agent recommendation model in the final converged state, thus completing the entire training phase.

[0054] Step 7: Verify the effectiveness of the method

[0055] After model training, to verify the effectiveness of the proposed method, it was tested on multiple public datasets covering different fields such as movie recommendation, book recommendation, and music recommendation. After obtaining the interaction prediction scores of target users for recommended items, a Top-N personalized recommendation list was formed for each target user by selecting the top N items with the highest scores. The evaluation metrics used in the experiment were Recall (accuracy relative to all positive samples) and NDCG (NDCG focuses on the position of positive samples in the Top-N recommendation list; the higher the position, the larger the NDCG). Recall was calculated by dividing the number of correctly predicted positive samples in the Top-N recommendation list by the total number of actual positive samples for that user; NDCG was calculated by normalizing the cumulative gain value after logarithmic position decay of the positive sample positions in the recommendation list.

Claims

1. A dynamic programming-based multi-round retrieval-enhanced recommendation agent method, characterized in that, Includes the following steps: S1. Data Preprocessing and Dataset Construction: Obtain a recommendation dataset including user IDs and item IDs, along with a corresponding external knowledge graph. Unify the encoding of users, items, entities, and relationships in the knowledge graph, and divide the dataset into training and test sets according to a preset ratio. For each user, use their most recent interaction as the true label of the test set, and the remaining interactions as historical behavior data. Construct a candidate set containing positive and negative samples for each test sample. S2, Recommendation and Retrieval Needs Perception and State Encoding: Extract users' historical interaction data and candidate set information, encode them to obtain users' recommendation needs representation and retrieval needs representation respectively, and aggregate them to generate a unified state embedding, which serves as the input to the downstream policy network; S3. Hierarchical Strategy Network and Dynamic Retrieval Planning: Construct a hierarchical strategy network with four levels of hierarchical decision-making. With unified state embedding as input, decompose retrieval actions into multi-level progressive decision-making units, and dynamically generate structured retrieval plans and differentiated budget allocation schemes. S4. Reflection-driven multi-round knowledge completion: Based on the retrieval plan, differentiated retrieval is performed. The defects of the current retrieval content are identified through adaptive knowledge gap detection. The retrieval plan is adjusted in reverse and multiple rounds of retrieval are executed iteratively to complete the gradual completion and optimization of knowledge. S5. Knowledge-enhanced large language model recommendation: The structured knowledge obtained from multiple rounds of retrieval is encoded and projected, and input into the large language model together with the user's historical interaction and candidate set information to predict the user's interaction score with the candidate items, and generate a personalized recommendation list in descending order of score; S6. Hybrid Reward Design and End-to-End Model Optimization: Construct a hybrid reward function that includes recommendation gain and retrieval cost, and optimize the hierarchical policy network end-to-end using reinforcement learning. At the same time, optimize the knowledge encoding module based on the cross-entropy loss of the recommendation task to achieve joint optimization of the retrieval process and the recommendation task.

2. The dynamic programming multi-round retrieval enhancement recommendation agent method according to claim 1, characterized in that, The recommendation dataset includes user-item interaction behavior logs and item attribute metadata in the target recommendation business scenario. The supporting external knowledge graph contains entity, relation and triple data associated with the item. The implicit feedback processing of user-item interaction data is as follows: if a user has interacted with an item, the corresponding label is recorded as 1 as a positive sample. If no interaction occurs, the corresponding label is recorded as 0, which is a negative sample. The step of dividing the training set and the test set according to a preset ratio specifically includes: for each user, the most recent interaction on the timeline is used as a positive sample in the test set, and the remaining interactions are used as historical behavior data in the training set and the test set; when generating the candidate set, for each positive sample, a preset number of negative samples are randomly sampled from the set of items that the user has not interacted with, and the positive samples and the sampled negative samples together constitute the candidate set.

3. The dynamic programming multi-round retrieval enhancement recommendation agent method according to claim 1, characterized in that, In step S2, a pre-trained language model encoder is used to encode the text description and attribute features of the item to obtain the semantic embedding of the item. Each aggregates the user's historical interaction set With candidate set The semantic embedding of items in the data yields a representation of the recommendation requirements: ； A retrieval matrix is ​​constructed by calculating pairwise similarities between historical interactions and candidate items, and then transformed through a two-layer fully connected network to obtain a representation of the user's retrieval needs. , where i represents an item in the historical interaction set or candidate set; finally, the recommendation demand representation and the retrieval demand representation are concatenated to obtain a unified state embedding: ； in For embedded splicing operations, Represents the set of user's historical interactions The total number of items in the game. Indicates candidate set Total number of items in the list, absolute value symbol A unified representation of the number of elements within the corresponding set.

4. The dynamic programming multi-round retrieval enhancement recommendation agent method according to claim 1, characterized in that, During the retrieval phase, the hierarchical strategy network employs a four-layer hierarchical decision-making unit, namely entity priority ranking, relation pruning, semantic path depth allocation, and historical association importance assessment; the entity priority ranking is based on the union of user historical interactions and the candidate set. Calculate the priority distribution and select high-priority entities as seed nodes for retrieval; the relation pruning involves the full set of relations in the knowledge graph. Assign importance scores and filter task-related subsets of relationships. The semantic direction of the retrieval is constrained; the semantic path depth allocation adaptively assigns differentiated search depths to different types of inference paths, enabling in-depth exploration of key paths and lightweight retrieval of irrelevant paths; the historical association importance assessment predicts the necessity of explicit path verification between candidate items and user historical interactions, guiding the retrieval to complete the inference loop; based on a four-layer hierarchical decision-making unit, the policy network dynamically generates retrieval action sequences and budget allocation ratios, using the following formula: ； in For the first Hierarchical generated search actions To ensure the proportion of budget allocated to this level meets the following requirements: , For the total search budget, To achieve unified state embedding; ultimately, a complete retrieval trajectory is generated. The system generates a structured retrieval plan, wherein the total number of decision layers T is set to 4, corresponding to the four hierarchical decision units, and the time step t ranges from {1, 2, 3, 4}; the complete relation set... It is the set of all known entity relation types predefined in the knowledge graph.

5. The dynamic programming multi-round retrieval enhancement recommendation agent method according to claim 1, characterized in that, In step S4, the differentiated retrieval execution breaks down the retrieval plan into entity retrieval query sets. Relationship extended query set Path traversal query set Historical Relationship Query Set The following steps are performed: priority-based entity expansion, selective relation pruning, bimodal path traversal, and threshold-triggered historical association verification, respectively, to obtain the knowledge subgraph of the l-th round. and the cumulative search cost for this round. ; The adaptive knowledge gap detection process involves defining five types of knowledge gaps in descending order of priority: missing core entities, missing historical connections, missing key relationships, incomplete historical connections, and missing semantic paths. Based on these priorities, defects in the current knowledge subgraph are identified, and a new retrieval plan is generated. Multiple rounds of retrieval and knowledge gap detection are iteratively executed until a termination condition is met, ultimately yielding the accumulated complete retrieval knowledge from these rounds. .

6. The dynamic programming multi-round retrieval enhancement recommendation agent method according to claim 1, characterized in that, In step S5, a pre-trained language model encoder is used to process the retrieved knowledge. The text is encoded and mapped to the embedding space of a large language model through a trainable two-layer MLP projector to obtain a knowledge-enhancing token. The knowledge enhancement token User history interactions Textual representation, candidate set Textual representations are common inputs to large language models ( Predict the probability of user interaction with candidate items: ； Candidate items are sorted in descending order based on the predicted interaction probabilities, and the top N items are selected to generate a personalized recommendation list.

7. The dynamic programming multi-round retrieval enhancement recommendation agent method according to claim 1, characterized in that, The formula for the hybrid reward function is: ； in, The recommendation gain brought by the retrieval of knowledge, that is, the improvement of the recommendation evaluation index after the introduction of external knowledge; The total cost accumulated over multiple rounds of retrieval; To balance the hyperparameters; This is a normalization operation within a batch; Based on mixed rewards, the optimization objective of the policy network is to maximize the expected reward, and the policy loss function is: ； in, To retrieve the number of actions in the trajectory, It is the entropy coefficient; The recommendation task is optimized using the cross-entropy loss function: ； in For the set of token locations of the real labels, The true token sequence output by the large language model. For the corresponding position The token For position Previous historical token sequences, The output probability distribution of the large language model; the overall model adopts joint optimization with multi-task learning objectives, and the total loss function is: ； in, For policy network loss; Cross-entropy loss is used for recommendation tasks.

Images