Recommendation method and system based on reinforcement learning driven large model adaptive prompt

Abstract

The application discloses a recommendation method and system based on reinforcement learning driving large model adaptive prompt, the method comprises the following steps: obtaining user portrait text and performing semantic similarity matching to determine the corresponding prompt word type of the user; through a deep Q network (DQN) and a confidence upper bound algorithm (UCB), a preset multiple prompt optimization strategy is selected to recommend a prompt optimization strategy and obtain the prompt optimization strategy S; based on the prompt optimization strategy S and the corresponding prompt word type of the user, a large model is used to generate an optimized prompt word, a recommendation system is used for verification and performance index evaluation; continue iteration until the preset end condition is met to obtain the optimized prompt word for application in a target recommendation system. The application aims to introduce fine-grained user portraits and reinforcement learning driving large models to optimize personalized prompts for different user groups, solve the problem of rigid soft prompt templates and insufficient personalization in existing recommendation technologies, and improve recommendation accuracy.

Description

Technical Field

[0001] This invention relates to artificial intelligence recommendation systems, specifically to a recommendation method and system based on reinforcement learning-driven large model adaptive prompts. Background Technology

[0002] The core objective of data-driven recommendation is to provide users with highly personalized product data, such as news, content, and goods. Traditional recommendation methods primarily rely on analyzing users' historical interaction data, such as reading history, click behavior, and search records, to predict users' potential interests through recommendation algorithms and match these interests with the data to be recommended, thereby predicting data that users might be interested in. In recent years, with the development of Large Models (LLM), generative recommendation systems have gradually become a research hotspot. Compared to traditional methods, LLM can directly generate personalized recommendation content tailored to user needs through powerful semantic understanding and generative capabilities. However, when generating recommendation results for all users, LLM recommendation systems often rely on generic soft prompt templates. These soft prompt templates typically lack sufficient personalization and cannot deeply explore users' true needs and potential interest shifts. Although some systems have attempted to improve the intelligence of recommendations by introducing LLM to generate soft prompt templates, the generation and tuning process of soft prompt templates remains relatively rigid due to the lack of dynamic optimization mechanisms, resulting in recommendation performance that is difficult to achieve ideal results. How to solve the above challenges of generative LLM recommendation systems has become an urgent technical problem to be solved. Summary of the Invention

[0003] The technical problem to be solved by this invention is to provide a recommendation method and system based on reinforcement learning-driven large model adaptive prompts, addressing the aforementioned problems in existing recommendation technologies. This invention aims to improve recommendation accuracy by introducing fine-grained user profiles and a mechanism for optimizing personalized prompts for different user groups using a reinforcement learning-driven large model.

[0004] To solve the above-mentioned technical problems, the technical solution adopted by the present invention is as follows:

[0005] A recommendation method based on reinforcement learning-driven large model adaptive prompts includes the following steps:

[0006] S101, Obtain user profile text representing user preferences, and perform semantic similarity matching between the user profile text and multiple preset prompt word type descriptions to determine the prompt word type corresponding to the user;

[0007] S102, the encoded user profile text is used as the state, and the cue optimization strategy is used as the action. A deep Q-network (DQN) is used to combine the state, action, and the instant reward signal from the previous round. Choose the recommended prompt optimization strategy from a variety of prompt optimization strategies; based on the instant reward signal from the previous round. Update historical reward signals The confidence upper limit algorithm UCB is used based on historical reward signals. Select the recommended prompt optimization strategy from a variety of preset prompt optimization strategies; add the two recommended prompt optimization strategies to the strategy candidate pool and execute the strategy decision to obtain the final prompt optimization strategy S;

[0008] S103: Based on the final prompt optimization strategy S and the corresponding prompt word type for the user, the optimized prompt words are generated using a large model, and validated using a recommendation system. The performance metrics of the recommendation system are then evaluated, and the immediate reward signal for this round is updated and calculated. ;

[0009] S104, determine whether the preset termination condition is met. If the preset termination condition is met, output the final optimized prompt words for use in the target recommendation system to obtain recommendation results; otherwise, jump to step S102 to continue iterating.

[0010] Optionally, in step S102, a deep Q-network (DQN) is used to combine the state, action, and the immediate reward signal from the previous round. Among various cue optimization strategies, the recommended cue optimization strategy includes: firstly, combining the state, action, and the immediate reward signal from the previous round using a Deep Q-Network (DQN). Calculate the expected cumulative reward for each of the various suggestion optimization strategies; then utilize... - Greedy strategy uses a preset base exploration rate The probability of randomly selecting the recommended suggestion optimization strategy from multiple suggestion optimization strategies is to... The probability selection is based on the action that maximizes the expected cumulative reward after the update, and the recommended prompt optimization strategy is the one corresponding to that action.

[0011] Optionally, in step S102, based on the immediate reward signal from the previous round... Update historical reward signals The function expression is:

[0012] ;

[0013] in, For learning rate, For the first Historical reward signals for each round.

[0014] Optionally, in step S102, the confidence upper limit algorithm (UCB) is used based on historical reward signals. Among the preset multiple hint optimization strategies, the recommended hint optimization strategy includes: firstly, using the confidence upper limit algorithm (UCB) based on historical reward signals. Calculate the UCB score for each of the various prompt optimization strategies:

[0015] ;

[0016] in, For action The corresponding UCB score for the optimization strategy is suggested. For action The number of times the corresponding optimization strategy is tried. To obtain the maximum value, To explore the coefficient for dynamic attenuation, For iteration rounds, The total number of attempts is then used; the suggestion optimization strategy with the highest UCB score is then selected as the recommended suggestion optimization strategy.

[0017] Optionally, step S102, which involves adding the recommended suggestion optimization strategy to the strategy candidate pool and performing a strategy decision to obtain the final suggestion optimization strategy S, includes: assigning an initial weight vector to the suggestion optimization strategies in the strategy candidate pool. :

[0018] ;

[0019] in, The weights of the suggestion optimization strategy recommended for Deep Q-Network (DQN). The weights of the suggestion optimization strategy recommended by the Confidence Upper Limit (UCB) algorithm are used; the softmax function is applied to the weight vector. Normalization is performed:

[0020] ;

[0021] in, For weight Normalized weights, weights and The first The weight and the first weight Each weight, The final suggestion optimization strategy is determined by random sampling based on the normalized probability distribution. :

[0022] ;

[0023] in, Indicates a correspondence. For classification distribution, and They are respectively and Normalized weights.

[0024] Optionally, step S103, which generates optimized prompts based on the final prompt optimization strategy S and the prompt word type corresponding to the user using a large model, includes: generating the initial soft prompt optimization instruction corresponding to the final prompt optimization strategy S and the prompt word type corresponding to the user, and the instant reward signal from the previous round. The historical summary and suggested recommendation cases are combined to form a final suggestion optimization template to guide the large model to optimize the suggestion. The final suggestion optimization template is input into the large model to generate optimized suggestion words. The historical summary includes the final suggestion optimization strategies of the most recent rounds, the immediate reward signals and performance indicators.

[0025] Optionally, in step S103, the instant reward signal for the current round is updated. The function expression is:

[0026] ;

[0027] in, The number of performance indicators, For the first The weight of each performance metric, For the first The values ​​of each performance indicator in this round, For the first The values ​​of each performance metric in the previous round To prevent the removal of the zero constant.

[0028] Furthermore, the present invention also provides a recommendation system based on reinforcement learning-driven large model adaptive prompts, including an interconnected microprocessor and a memory, wherein the microprocessor is programmed or configured to execute the recommendation method based on reinforcement learning-driven large model adaptive prompts.

[0029] Furthermore, the present invention provides a computer-readable storage medium storing a computer program or instructions that are programmed or configured to execute the recommendation method based on reinforcement learning-driven large model adaptive prompts via a processor.

[0030] Furthermore, the present invention also provides a computer program product, including a computer program or instructions that are programmed or configured to execute the recommendation method based on reinforcement learning-driven large model adaptive prompts via a processor.

[0031] Compared with existing technologies, this invention mainly achieves the following beneficial effects: 1. By combining reinforcement learning and large language models, this invention realizes dynamic optimization and personalized generation of prompt words, significantly improving the recommendation accuracy and user satisfaction of news recommendation systems. 2. This invention introduces a collaborative strategy selection mechanism based on Deep Q-Network (DQN) and UCB algorithm, which can dynamically adjust the prompt optimization strategy according to user profiles, thereby achieving a balance between exploration and utilization, and improving the diversity and relevance of recommended content. 3. This invention provides more comprehensive data support for accurate recommendations by constructing user profile text representations to indicate user preferences. 4. This invention proposes a historical reward signal processing method based on sliding window and exponential moving average to capture long-term trends and dynamically adjust the direction of reinforcement learning strategy selection, further enhancing the adaptability and stability of the recommendation system. Attached Figure Description

[0032] Figure 1 This is a schematic diagram of the basic process of the method in an embodiment of the present invention.

[0033] Figure 2 This is a schematic diagram of the semantic similarity matching process in step S101 of an embodiment of the present invention.

[0034] Figure 3 This is a flowchart illustrating the process of obtaining the final prompt optimization strategy S in step S102 of an embodiment of the present invention.

[0035] Figure 4 This is a schematic diagram illustrating the iterative principle of an embodiment of the present invention. Detailed Implementation

[0036] To enable those skilled in the art to better understand the technical solution of this invention, the technical solution of this invention will be further described in detail below, taking news recommendation as an example and in conjunction with the accompanying drawings of the embodiments of this invention. It should be noted that the method of this invention is not limited to news recommendation, but can also be used for recommending various items such as goods, videos, audio, and papers.

[0037] like Figure 1 As shown, the recommendation method based on reinforcement learning-driven large model adaptive prompts in this embodiment includes the following steps:

[0038] S101, Obtain user profile text representing user preferences, and perform semantic similarity matching between the user profile text and multiple preset prompt word type descriptions to determine the prompt word type corresponding to the user;

[0039] S102, the encoded user profile text is used as the state, and the cue optimization strategy is used as the action. A deep Q-network (DQN) is used to combine the state, action, and the instant reward signal from the previous round. Choose the recommended prompt optimization strategy from a variety of prompt optimization strategies; based on the instant reward signal from the previous round. Update historical reward signals The confidence upper limit algorithm UCB is used based on historical reward signals. Select the recommended prompt optimization strategy from a variety of preset prompt optimization strategies; add the two recommended prompt optimization strategies to the strategy candidate pool and execute the strategy decision to obtain the final prompt optimization strategy S;

[0040] S103: Based on the final prompt optimization strategy S and the corresponding prompt word type for the user, the optimized prompt words are generated using a large model, and validated using a recommendation system. The performance metrics of the recommendation system are then evaluated, and the immediate reward signal for this round is updated and calculated. ;

[0041] S104, determine whether the preset termination condition is met. If the preset termination condition is met, output the final optimized prompt words for use in the target recommendation system to obtain recommendation results; otherwise, jump to step S102 to continue iterating.

[0042] Step S101, obtaining the user profile text representing user preferences, can employ the desired user profiling method. For example, as an optional implementation, this embodiment uses a large language model to analyze the user's fine-grained features based on the user's historical news reading data, including explicit interests, potential interest migration, reading style, regional preferences, and sentiment status, to generate personalized user profile text. Specifically, this includes: S201, Data Extraction: Extracting the user's historical reading data from the input dataset, including news titles, summaries, categories, regional tags, and sentiment analysis results. S202, Feature Analysis: Analyzing the user's fine-grained features by calling the API interface of the large language model, including explicit interest areas (such as technology, finance), potential interest migration trends (such as the emergence or decline of new areas), reading style classification (such as short / medium / long articles, professional / popular), sentiment tendency (such as the ratio of positive to negative sentiment), and regional preferences (such as "China: 60%)". S203, Standardized Output: Generating structured user profile text, containing the following fields: Primary Interests: Entity list (such as "technology, finance"). Reading style: Categorized by length (short / medium / long) and subject matter (e.g., "medium-length subject matter"). Recent changes in interests: Percentage of new areas or areas of declining interest (e.g., "AI healthcare accounts for 15% of new areas"). Sentiment: Ratio of positive to negative sentiment (e.g., "Negative: 40%"). Geographical preference: Regional distribution of attention (e.g., "China: 60%, Global: 40%").

[0043] like Figure 2As shown, step S101, which involves semantically matching the user profile text with a set of preset prompt word type descriptions to determine the prompt word type corresponding to the user, specifically includes: S301, reading the preset prompt word type descriptions. These preset prompt word type descriptions can be set according to actual needs. For example, as an optional implementation, in this embodiment, the preset prompt word types and their descriptions include: Structured Analysis Type: Detailed analytical content reflecting user preferences in finance, technology, and other fields. General Interest Type: Users have broad interests and focus on the interest, popularity, and readability of news. Exploratory Type: Users' recent interests have changed significantly, requiring a balance between recommendation diversity and precise matching. Cultural Background Type: Users have multicultural backgrounds and focus on the cultural relevance of news (such as international events and global trends). S302, encoding the user profile text and the preset prompt word type descriptions using the all-MiniLM-L6-v2 encoder, and then calculating the cosine similarity between the encoding results of the user profile text and each prompt word type description; S303, selecting the prompt word type with the highest similarity to the user profile of all users based on the similarity calculation results.

[0044] like Figure 3 and Figure 4 As shown, in step S102, the encoded user profile text is used as the state, and the prompt optimization strategy is used as the action. The state, action, and the instant reward signal from the previous round are combined through a deep Q-network (DQN). Choose the recommended prompt optimization strategy from a variety of prompt optimization strategies; based on the instant reward signal from the previous round. Update historical reward signals The confidence upper limit algorithm UCB is used based on historical reward signals. The system selects a recommended prompt optimization strategy from a set of preset prompt optimization strategies; adds the two recommended prompt optimization strategies to the strategy candidate pool and executes a strategy decision to obtain the final prompt optimization strategy S. In this embodiment, when the encoded user profile text is used as the state, the encoding of the user profile text includes: sentiment tendency: the quantitative value of the user's emotional state (e.g., negative percentage); regional preference weight: the attention of the dominant region (e.g., "China: 60%)); one-hot encoding of reading style: the one-hot representation of the user's reading style (e.g., Short / Medium / Long); and interest change indicator: the binary indicator of the user's interest migration trend. In addition, the preset prompt optimization strategies can be specified according to actual needs. For example, the preset prompt optimization strategies in this embodiment include:

[0045] Short Strategy: We recommend concise and refined news content, avoiding lengthy analysis or in-depth reports.

[0046] Medium Strategy: Recommends medium-length news articles, providing a balanced mix of background information and content.

[0047] Long Article Strategy: Recommend detailed and comprehensive news content, including in-depth analysis, historical background, and expert opinions.

[0048] Sentiment analysis: Optimize prompts based on the user's emotional state.

[0049] Region-based strategy: Focuses on region-based content recommendation.

[0050] Exploration strategy: Explore users' new interests and preferences, and prioritize recommending trending news (such as breaking news, popular events, and important events that users may be interested in).

[0051] Interest-focused strategy: Recommendations are made based on the user's primary interests.

[0052] In step S102 of this embodiment, a Deep Q-Network (DQN) is used to combine the state, action, and the instantaneous reward signal from the previous round. Among various cue optimization strategies, the recommended cue optimization strategy includes: firstly, combining the state, action, and the immediate reward signal from the previous round using a Deep Q-Network (DQN). Calculate the expected cumulative reward for each of the various suggestion optimization strategies; then utilize... - Greedy strategy uses a preset base exploration rate The probability of randomly selecting the recommended suggestion optimization strategy from multiple suggestion optimization strategies is to... The probability selection ensures that the cue optimization strategy corresponding to the action with the highest expected cumulative reward after the update is used as the recommended cue optimization strategy. In this embodiment, the Deep Q-Network (DQN) combines the state, action, and the instant reward signal from the previous round. When calculating the expected cumulative reward for each of the various cue optimization strategies, the Deep Q-Network (DQN) uses a Deep Neural Network (DNN) to select actions based on the state, which can be represented as:

[0053] ;

[0054] in, For state-based Select Action The expected cumulative reward (Q value). For deep Q-networks (DQN), the deep neural network (DNN) is used. These are the parameters of a deep neural network (DNN). This represents selecting actions from the Q-value vector output by a deep neural network (DNN). The corresponding expected cumulative reward (Q value), Q value in state The following prompts will be used to optimize the action. Then, the expected value of the cumulative rewards that may be obtained in the future. The new iteration rounds utilize immediate reward signals. and the Q value of the next state Calculate the target Q value To implement subsequent Q-value updates, the function expression is as follows:

[0055] ;

[0056] In the above formula, Discount factor ( The importance of controlling future rewards. Indicates a new state Choose the optimal action The Q value. The target Q value. After the calculation is complete, the Q-value is updated by minimizing the mean squared error (MSE) between the predicted Q-value and the target value using gradient descent, thereby realizing the new policy selection. The function expression is as follows:

[0057] ;

[0058] In the above formula Indicates dynamic updates. Represents the state in round t. Select action The expected cumulative reward (Q value). Indicates the target Q value. This represents the learning rate, which is 0.001 in this embodiment. Finally, using... - The greedy strategy is based on the Q-value to select the current strategy. In this embodiment, the base exploration rate is... Set to 0.2, strategy selection function The function expression is:

[0059] ;

[0060] In the above formula The choice indicates that in a given state Next, select the action value function. Maximize the action This corresponds to the optimization strategy.

[0061] In step S102 of this embodiment, based on the instant reward signal from the previous round... Update historical reward signals The function expression is:

[0062] ;

[0063] in, The learning rate (which can be set to a value as needed, for example, 0.1 in this implementation case). For the first Historical reward signals for each round. Real-time reward signals corresponding to the optimization strategy based on the prompts in each iteration round. Receive historical reward signal Combined with historical reward signals Calculate the corresponding UCB score, and the suggestion optimization strategy with the highest score is the suggestion optimization strategy selected by the Upper Confidence Bound (UCB) algorithm. The functional expression for calculating the UCB score in the traditional Upper Confidence Bound algorithm is:

[0064] ;

[0065] In the above formula, For the exploit item, it represents the strategy. The global historical average value For exploration items, it indicates the exploration of actions. The uncertainty of exploration The total number of attempts. For strategy The number of attempts, where t represents the number of training iterations. The exploration coefficient is a fixed value. This embodiment uses historical reward signals. Replacing the traditional UCB algorithm As an exploit, a dynamic decay exploration coefficient was used, and a truncation protection mechanism was employed. Specifically, in step S102, the confidence upper limit algorithm (UCB) was used based on historical reward signals. Among the preset multiple hint optimization strategies, the recommended hint optimization strategy includes: firstly, using the confidence upper limit algorithm (UCB) based on historical reward signals. Calculate the UCB score for each of the various prompt optimization strategies:

[0066] ;

[0067] in, For action The corresponding UCB score for the optimization strategy is suggested. For action The number of times the corresponding optimization strategy is tried. To obtain the maximum value, To explore the coefficient for dynamic attenuation, For iteration rounds, The total number of attempts is given; then, the suggestion optimization strategy with the highest UCB score is selected as the recommended suggestion optimization strategy. In the above formula, For the utilization item, among which This represents the action selected in the t-th iteration. The exponential moving average of historical reward signals. For action Number of attempts avoid When =0, the term is meaningless. The exploration item represents the action. The uncertainty of exploration Indicates the number of training iterations. To explore the coefficient for dynamic attenuation, The total number of attempts. The denominator of the exploration term is truncated to optimize the exploration stability with a low number of attempts.

[0068] In this embodiment, step S102, which involves adding the recommended suggestion optimization strategy to the strategy candidate pool and performing strategy decision-making to obtain the final suggestion optimization strategy S, includes: assigning initial weight vectors to the suggestion optimization strategies in the strategy candidate pool. :

[0069] ;

[0070] in, The weights of the suggestion optimization strategy recommended for Deep Q-Network (DQN). The weights for the hint optimization strategy recommended by the Confidence Upper Limit (UCB) algorithm can be selected according to actual needs, such as in this embodiment. It is 0.3. The value is 0.1; the softmax function is used to adjust the weight vector. Normalization is performed:

[0071] ;

[0072] in, For weight Normalized weights, weights and The first The weight and the first weight Each weight, The final suggestion optimization strategy is determined by random sampling based on the normalized probability distribution. :

[0073] ;

[0074] in, Indicates a correspondence. For classification distribution, and They are respectively and Normalized weights.

[0075] In step S103 of this embodiment, for users with different prompt word types, the optimal prompt optimization strategy is selected through the collaboration of Deep Q-Network (DQN) and Unified Confidence Limit (UCB) algorithm. This strategy combines the user's state and the reward signal generated during the recommendation process. The recommendation score, along with the real-time reward signal generated during the recommendation process, forms supplementary content for the prompt optimization instruction. In step S103 of this embodiment, when generating optimized prompt words using a large model based on the final prompt optimization strategy S and the user's corresponding prompt word type, the large model can be GLM-4-Flash or DeepSeek-V3, etc. Then, when validating and evaluating the performance metrics of the recommendation system using the recommendation system, the recommendation system can be replaced by a large model such as GLM-4-Flash or DeepSeek-V3. Step S103 in this embodiment includes: S401, embedding the initial soft suggestion optimization instruction corresponding to the user's corresponding suggestion word type into the suggestion optimization strategy, the previous round reward signal, historical summary, and the initial suggestion case composed of the recommendation results of the previous iteration output by the recommender (such as replacing it with a large model such as GLM-4-Flash or DeepSeek-V3) and the initial suggestion words, to guide the suggestion optimizer (such as replacing it with a large model such as GLM-4-Flash or DeepSeek-V3) to optimize and generate new recommendation suggestion words; Suggestion optimization strategy: the strategy description corresponding to the current strategy selected from DQN and UCB, see step S102 for details. Used to guide the large language model (LLM) to generate personalized suggestions. Previous round reward signal: reflects the change in the recommendation system performance in the previous iteration, used to dynamically adjust the direction of suggestion generation. Historical summary: the strategies and real-time reward signals of the most recent rounds recorded through a sliding window (window size = 5). And performance metrics, providing long-term trend information for LLM. S402, verify prompt word format, if historical interactive news data is missing: ${ } or candidate news data:$ Placeholders trigger a retry mechanism. S403: The optimized prompt words are used in the large-scale recommendation system for recommendations, and the recommendation results and performance are recorded. Simultaneously, it is determined whether to update the best prompt words. Specifically, except for the first recommendation which uses a predefined initial prompt template, the optimized prompt template is used thereafter. S404: Performance evaluation is performed and the reward signal is updated. Historical reward signals In addition to historical summaries, a new round of prompting optimization is performed by combining the RL policy network (reinforcement learning policy network, DQN+UCB) and the large model in this embodiment.

[0076] In step S103 of this embodiment, the instant reward signal for the current round is updated and calculated. The function expression is:

[0077] ;

[0078] in, The number of performance indicators, For the first The weight of each performance metric, For the first The values ​​of each performance indicator in this round, For the first The values ​​of each performance metric in the previous round To prevent division by zero constant (which can take the value of) In this embodiment, the performance metrics are four indicators: Group AUC, Mean Reciprocal Rank (MRR), Top 5 Normalized Cumulative Loss Gain (NDCG@5), and Top 10 Normalized Cumulative Loss Gain (NDCG@10), with corresponding weights of 0.4, 0.3, 0.2, and 0.1, respectively.

[0079] In step S104, the preset termination conditions are met when the performance index evaluation result meets the early termination condition, the performance index evaluation result is greater than the preset threshold, or the number of iterations is equal to the preset maximum number of iterations (e.g., 15 times). The early termination condition means that the change in the performance index evaluation result is lower than the preset threshold for a specified number of consecutive rounds (e.g., 10 consecutive rounds).

[0080] To verify the recommendation method based on reinforcement learning-driven large model adaptive prompts in this embodiment, the MIND (Microsoft News Dataset) news dataset was used in the experiments. Group AUC, Mean Reciprocal Rank (MRR), Normalized Diminished Cumulative Gain (NDCG@5) for the top 5, and Normalized Diminished Cumulative Gain (NDCG@10) for the top 10 were used. The large model GLM-4-Flash was used as the prompt optimizer, and DeepSeek-V3 as the recommender. Existing generative large model news recommendation models, including RecPrompt, LSTUR (a neural news recommendation model with long-term and short-term user representations), DKN (a deep knowledge-aware news recommendation model), NAML (a neural news recommendation model based on attention-based multi-view learning), NPA (a neural news recommendation model with personalized attention), and NRMS (a neural network news recommendation model based on multi-head attention), were compared with the method in this embodiment (UPGO). The experimental results of the comparison method are cited from the RecPrompt paper. To ensure the fairness and objectivity of the comparison, we tested the results on the unified generative large language model platform DeepSeek-V3. The method proposed in this embodiment (UPGO) and the prompt template proposed by RecPrompt were run on the above, thereby eliminating the impact of differences in the capabilities of the underlying language model on the recommendation performance, making the evaluation results more convincing and reproducible. The final comparison results are shown in Table 1.

[0081] Table 1. Experimental results comparing the performance of recommendation methods on the MIND dataset.

[0082]

[0083] As shown in Table 1, the method in this embodiment (UPGO) significantly outperforms existing classic recommendation models (such as LSTUR, DKN, NAML, NPA, NRMS) and the generative news recommendation model RecPrompt based on a large language model on the MIND news recommendation dataset. In particular, it achieves an improvement of approximately 2 percentage points in Group AUC and NDCG@5. Using the same underlying model (DeepSeek-V3) as RecPrompt, UPGO also shows significant advantages in user perception-related metrics such as Mean MRR and NDCG@10. The method in this embodiment (UPGO) combines the capabilities of reinforcement learning and a large language model to dynamically optimize prompts to adapt to different user groups and scenario needs. In each round of optimization, the system collaboratively selects the optimal strategy through a Deep Q-Network (DQN) and the UCB algorithm, and dynamically adjusts the recommendation direction by combining a reward signal mechanism. This design not only improves the performance of the recommendation system but also enhances its robustness and adaptability. Furthermore, the method in this embodiment (UPGO) further mines user feature information by introducing a fine-grained user profile generation method based on a large model. The system utilizes a large language model to analyze users' historical reading data, extracting multi-dimensional features such as explicit interests, latent interest transfer, reading style, regional preferences, and emotional state to generate standardized user profile text. These fine-grained features provide a solid foundation for subsequent suggestion word optimization, making the recommendation results more closely aligned with users' actual needs. It demonstrates excellent performance in news recommendation tasks, and its innovative design and efficient implementation provide a new solution for the field of personalized recommendations. The method can also be further extended to other recommendation scenarios, such as e-commerce product recommendations and video content recommendations, providing users with more accurate and diversified services.

[0084] Furthermore, this embodiment also provides a recommendation system based on reinforcement learning-driven large model adaptive prompts, including an interconnected microprocessor and a memory, wherein the microprocessor is programmed or configured to execute the recommendation method based on reinforcement learning-driven large model adaptive prompts.

[0085] Furthermore, this embodiment also provides a computer-readable storage medium storing a computer program or instructions that are programmed or configured to execute the recommendation method based on reinforcement learning-driven large model adaptive prompts via a processor.

[0086] Furthermore, this embodiment also provides a computer program product, including a computer program or instructions that are programmed or configured to execute the recommendation method based on reinforcement learning-driven large model adaptive prompts via a processor.

[0087] Those skilled in the art will understand that the technical solutions provided by this invention may take the form of a method, system, or computer program product. Therefore, this invention may take the form of a completely hardware embodiment, a completely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, this invention may take the form of a computer program product embodied on one or more computer-readable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code. This invention is described with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of the invention. It will be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, special-purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, produce an implementation of the flowchart... Figure 1 One or more processes and / or boxes Figure 1 The computer program instructions may also be stored in a computer-readable storage medium that can direct a computer or other programmable data processing device to operate in a particular manner, such that the instructions stored in the computer-readable storage medium produce an article of manufacture including instruction means, which are implemented in a process Figure 1 One or more processes and / or boxes Figure 1 The functions specified in one or more boxes. These computer program instructions may also be loaded onto a computer or other programmable data processing apparatus to cause a series of operational steps to be performed on the computer or other programmable apparatus to produce a computer-implemented process, thereby providing instructions that execute on the computer or other programmable apparatus for implementing the process. Figure 1 One or more processes and / or boxes Figure 1 The steps of the function specified in one or more boxes.

[0088] The above description is merely a preferred embodiment of the present invention. The scope of protection of the present invention is not limited to the above embodiments. All technical solutions falling within the scope of the present invention's concept are within the scope of protection of the present invention. It should be noted that for those skilled in the art, any improvements and modifications made without departing from the principles of the present invention should also be considered within the scope of protection of the present invention.

Claims

1. A recommendation method based on reinforcement learning-driven large model adaptive prompts, characterized in that, Includes the following steps: S101, Obtain user profile text representing user preferences, and perform semantic similarity matching between the user profile text and multiple preset prompt word type descriptions to determine the prompt word type corresponding to the user; S102, the encoded user profile text is used as the state, and the cue optimization strategy is used as the action. A deep Q-network (DQN) is used to combine the state, action, and the instant reward signal from the previous round. Choose the recommended suggestion optimization strategy from a variety of suggestion optimization strategies; Based on the previous round of instant reward signals Update historical reward signals The confidence upper limit algorithm UCB is used based on historical reward signals. Choose the recommended suggestion optimization strategy from the preset multiple suggestion optimization strategies; The two recommended suggestion optimization strategies are added to the strategy candidate pool and a strategy decision is made to obtain the final suggestion optimization strategy S; S103: Based on the final prompt optimization strategy S and the corresponding prompt word type for the user, the optimized prompt words are generated using a large model, and validated using a recommendation system. The performance metrics of the recommendation system are then evaluated, and the immediate reward signal for this round is updated and calculated. ; S104, determine whether the preset termination condition is met. If the preset termination condition is met, output the final optimized prompt words for use in the target recommendation system to obtain recommendation results. Otherwise, proceed to step S102 and continue iterating.

2. The recommendation method based on reinforcement learning-driven large model adaptive prompts according to claim 1, characterized in that, In step S102, a deep Q-network (DQN) is used to combine the state, action, and the immediate reward signal from the previous round. Among various cue optimization strategies, the recommended cue optimization strategy includes: firstly, combining the state, action, and the immediate reward signal from the previous round using a Deep Q-Network (DQN). Calculate the expected cumulative reward for each of the various suggestion optimization strategies; then utilize... - Greedy strategy uses a preset base exploration rate The probability of randomly selecting the recommended suggestion optimization strategy from multiple suggestion optimization strategies is to... The probability selection is based on the action that maximizes the expected cumulative reward after the update, and the recommended prompt optimization strategy is the one corresponding to that action.

3. The recommendation method based on reinforcement learning-driven large model adaptive prompts according to claim 1, characterized in that, In step S102, based on the instant reward signal from the previous round... Update historical reward signals The function expression is: ； in, For learning rate, For the first Historical reward signals for each round.

4. The recommendation method based on reinforcement learning-driven large model adaptive prompts according to claim 1, characterized in that, In step S102, the confidence upper limit algorithm UCB is used based on historical reward signals. Among the preset multiple hint optimization strategies, the recommended hint optimization strategy includes: firstly, using the confidence upper limit algorithm (UCB) based on historical reward signals. Calculate the UCB score for each of the various prompt optimization strategies: ； in, For action The corresponding UCB score for the optimization strategy is suggested. For action The number of times the corresponding optimization strategy is tried. To obtain the maximum value, To explore the coefficient for dynamic attenuation, For iteration rounds, The total number of attempts is then used; the suggestion optimization strategy with the highest UCB score is then selected as the recommended suggestion optimization strategy.

5. The recommendation method based on reinforcement learning-driven large model adaptive prompts according to claim 1, characterized in that, Step S102, which involves adding the recommended suggestion optimization strategy to the strategy candidate pool and performing strategy decision-making to obtain the final suggestion optimization strategy S, includes: assigning initial weight vectors to the suggestion optimization strategies in the strategy candidate pool. : ； in, The weights of the suggestion optimization strategy recommended for Deep Q-Network (DQN). The weights of the suggestion optimization strategy recommended by the Confidence Upper Limit (UCB) algorithm are used; the softmax function is applied to the weight vector. Normalization is performed: ； in, For weight Normalized weights, weights and The first The weight and the first weight Each weight, The final suggestion optimization strategy is determined by random sampling based on the normalized probability distribution. : ； in, Indicates a correspondence. For classification distribution, and They are respectively and Normalized weights.

6. The recommendation method based on reinforcement learning-driven large model adaptive prompts according to claim 1, characterized in that, Step S103, which generates optimized prompts based on the final prompt optimization strategy S and the prompt word type corresponding to the user using a large model, includes: generating the initial soft prompt optimization instruction corresponding to the final prompt optimization strategy S and the prompt word type corresponding to the user, and the instant reward signal from the previous round. The historical summary and suggested recommendation cases are combined to form a final suggestion optimization template to guide the large model to optimize the suggestion. The final suggestion optimization template is input into the large model to generate optimized suggestion words. The historical summary includes the final suggestion optimization strategies of the most recent rounds, the immediate reward signals and performance indicators.

7. The recommendation method based on reinforcement learning-driven large model adaptive prompts according to claim 1, characterized in that, In step S103, the instant reward signal for this round is updated and calculated. The function expression is: ； in, The number of performance indicators, For the first The weight of each performance metric, For the first The values ​​of each performance indicator in this round, For the first The values ​​of each performance metric in the previous round To prevent the removal of the zero constant.

8. A recommendation system based on reinforcement learning-driven large model adaptive prompts, comprising interconnected microprocessors and memory, characterized in that, The microprocessor is programmed or configured to execute the recommendation method based on reinforcement learning-driven large model adaptive prompts as described in any one of claims 1 to 7.

9. A computer-readable storage medium storing a computer program or instructions, characterized in that, The computer program or instructions are programmed or configured to execute, via a processor, the recommendation method based on reinforcement learning-driven large model adaptive prompts as described in any one of claims 1 to 7.

10. A computer program product, comprising a computer program or instructions, characterized in that, The computer program or instructions are programmed or configured to execute, via a processor, the recommendation method based on reinforcement learning-driven large model adaptive prompts as described in any one of claims 1 to 7.

Images