Resource recommendation method and apparatus

Abstract

The disclosure provides a resource recommendation method and device, relates to the technical field of artificial intelligence, in particular to the technical field of multi-modal feature fusion, large language model reasoning, reinforcement learning decision, intelligent recommendation and intelligent agent, and can be applied to various content pushing scenes. A specific embodiment of the method comprises: constructing a current state space based on multi-modal data of a user; performing feature extraction and fusion in the current state space to generate multi-modal fusion features; performing reasoning based on the multi-modal fusion features to generate a reasoning result; generating a multi-dimensional composite action based on the reasoning result; and executing the multi-dimensional composite action to recommend resources to the user.

Description

Technical Field

[0001] This disclosure relates to the field of artificial intelligence technology, particularly to the fields of multimodal feature fusion, large language model reasoning, reinforcement learning decision-making, intelligent recommendation, and intelligent agent technology, and can be applied to various content push scenarios. Background Technology

[0002] With the rapid development of information technology and multimedia content, multimodal information, including heterogeneous data such as images, text, and video, has been widely used in various recommendation scenarios. This necessitates that recommendation systems efficiently process and organically integrate multi-source heterogeneous information to improve the accuracy of recommended content and user experience. Ranking models, as the core module of recommendation systems, can construct optimization objectives based on user interaction feedback signals, providing a basis for recommendation decisions and driving the development of recommendation systems towards intelligence.

[0003] In current multimodal recommendation technologies, features are typically extracted from different modal data first, and then multimodal information is fused by feature concatenation. Models are then built based on DNN (Deep Neural Network) or traditional attention mechanisms to jointly learn and model user multimodal behavior, thereby forming corresponding recommendation strategies. Summary of the Invention

[0004] This disclosure presents a resource recommendation method and apparatus.

[0005] In a first aspect, embodiments of this disclosure propose a resource recommendation method, comprising: constructing a current state space based on user multimodal data; performing feature extraction and fusion in the current state space to generate multimodal fusion features; performing reasoning based on the multimodal fusion features to generate reasoning results; generating multidimensional composite actions based on the reasoning results; and executing the multidimensional composite actions to recommend resources to the user.

[0006] Secondly, embodiments of this disclosure propose a resource recommendation device, comprising: a construction module configured to construct a current state space based on user multimodal data; an extraction module configured to extract and fuse features in the current state space to generate multimodal fusion features; an inference module configured to perform inference based on the multimodal fusion features to generate inference results; a generation module configured to generate multidimensional composite actions based on the inference results; and a push module configured to execute the multidimensional composite actions to recommend resources to the user.

[0007] Thirdly, embodiments of this disclosure provide an electronic device, including: at least one processor; and a memory communicatively connected to the at least one processor; wherein the memory stores instructions executable by the at least one processor, the instructions being executed by the at least one processor to enable the at least one processor to perform the method as described in the first aspect.

[0008] Fourthly, embodiments of this disclosure provide a non-transitory computer-readable storage medium storing computer instructions for causing a computer to perform the methods described in the first aspect.

[0009] Fifthly, embodiments of this disclosure provide a computer program product including a computer program that, when executed by a processor, implements the method described in the first aspect.

[0010] The key or essential features of the embodiments disclosed herein are not intended to limit the scope of this disclosure. Other features of this disclosure will become readily apparent from the following description. Attached Figure Description

[0011] Other features, objects, and advantages of this disclosure will become more apparent from the following detailed description of non-limiting embodiments with reference to the accompanying drawings. The drawings are provided for a better understanding of the invention and are not intended to limit the scope of this disclosure. Wherein: Figure 1 This is a flowchart of an embodiment of the resource recommendation method according to the present disclosure; Figure 2 This is a flowchart of yet another embodiment of the resource recommendation method according to the present disclosure; Figure 3 This is the overall architecture diagram of a multimodal large language model resource recommendation system based on reinforcement learning; Figure 4 This is a schematic diagram of the structure of the multimodal sensing coding module; Figure 5 This is a schematic diagram illustrating the process of reinforcement learning decision-making and optimization. Figure 6 This is an overall flowchart of a multimodal large language model resource recommendation method based on reinforcement learning; Figure 7 This is a schematic diagram of the structure of one embodiment of the resource recommendation device according to the present disclosure; Figure 8 This is a block diagram of an electronic device used to implement the resource recommendation method of the embodiments of this disclosure. Detailed Implementation

[0012] The exemplary embodiments of this disclosure are described below with reference to the accompanying drawings, including various details of the embodiments to aid understanding, and should be considered merely exemplary. Therefore, those skilled in the art will recognize that various changes and modifications can be made to the embodiments described herein without departing from the scope and spirit of this disclosure. Similarly, for clarity and brevity, descriptions of well-known functions and structures are omitted in the following description.

[0013] It should be noted that, unless otherwise specified, the embodiments and features described in this disclosure can be combined with each other. This disclosure will now be described in detail with reference to the accompanying drawings and embodiments.

[0014] Figure 1 A flow 100 of an embodiment of a resource recommendation method according to the present disclosure is shown. The resource recommendation method includes the following steps: Step 101: Construct the current state space based on user multimodal data.

[0015] In this embodiment, the entity executing the resource recommendation method can collect user multimodal behavior data, integrate the user multimodal data, and construct the current state space.

[0016] User multimodal data refers to user-related data from different data sources and in different representation forms, capable of depicting user characteristics and contextual information from multiple dimensions. User multimodal data may include, but is not limited to, at least two of the following: user profiles, historical interaction sequences, contextual information, candidate resource pools, and real-time user feedback information. User profiles refer to a user's long-term interest vectors, demographic attributes, and historical statistical characteristics. Historical interaction sequences refer to the sequence of resource identifiers recently clicked, viewed, purchased, or favorited by the user, with each resource associated with its original multimodal data (such as images, text, video frames, audio, etc.). Contextual information refers to current time, geographical location, device type, and conversation scenario. A candidate resource pool refers to a set of multiple candidate resources to be recommended, each resource containing complete multimodal features. Real-time user feedback information refers to the user's explicit feedback (such as ratings, reviews, etc.) and implicit feedback (such as clicks, dwell time, swipe behavior, conversions, etc.) after the previous round of recommendations.

[0017] The current state space refers to the set of environmental states formed by uniformly modeling all available information in the recommendation scenario. It is used to represent all relevant information about the user, resources, and environment at the current moment and is the basis for the reinforcement learning agent to perform perception and decision-making.

[0018] Step 102: Perform feature extraction and fusion in the current state space to generate multimodal fusion features.

[0019] In this embodiment, the aforementioned execution entity can perform deep feature extraction and fusion on multimodal data in the current state space to generate multimodal fusion features.

[0020] Feature extraction and fusion refers to the process of extracting feature information from different modalities such as images and text from the state space, and achieving cross-modal semantic alignment through interactive computation to form a unified representation. For example, a visual encoder extracts visual feature vectors, a text encoder extracts text feature vectors, cross-modal attention fusion generates resource representation vectors, and a temporal encoder generates dynamic user interest representations. Multimodal fusion features are comprehensive feature vectors that integrate visual, textual, and temporal information, and can comprehensively reflect the relationship between resource content and user preferences.

[0021] Step 103: Perform inference based on multimodal fusion features to generate inference results.

[0022] In this embodiment, the aforementioned execution entity can convert numerical features into structured natural language prompts that can be understood by a large language model, perform advanced cognitive reasoning, and generate reasoning results.

[0023] Cognitive reasoning refers to the process of semantic understanding, logical analysis, and intent deduction based on multimodal fusion features and a large language model. The reasoning results can include structured reasoning chains and action intentions, which can guide subsequent decision generation.

[0024] Step 104: Based on the reasoning results, generate multi-dimensional composite actions.

[0025] In this embodiment, the aforementioned execution entity can generate multi-dimensional composite actions based on the reasoning results and numerical features of the large language model.

[0026] Multidimensional composite actions refer to a combination of actions that include multiple dimensions such as recommendation, explanation generation, exploration, and interaction. Unlike single recommendation actions, they can realize diversified recommendation services that are explainable, interactive, and explorable.

[0027] Step 105: Perform multi-dimensional composite actions to recommend resources to the user.

[0028] In this embodiment, the aforementioned execution entity can perform multi-dimensional composite actions to recommend resources to the user. These multi-dimensional composite actions may include outputting a list of recommended resources, generating natural language explanations for the recommendations, initiating exploratory recommendations, and engaging in interactive questioning with the user, thus achieving a complete output of the recommendation service.

[0029] The resource recommendation method provided in this disclosure achieves a unified representation of the environment by constructing a current state space, realizes cross-modal semantic understanding through multimodal feature fusion, achieves advanced cognitive decision-making through large language model reasoning, and generates multi-dimensional compound actions through reinforcement learning. Ultimately, it achieves accurate, interpretable, and interactive resource recommendation, effectively improving recommendation accuracy and user experience, while solving the cold start and information cocoon problems.

[0030] Figure 2 A flow 200 of yet another embodiment of the resource recommendation method according to this disclosure is shown. The resource recommendation method includes the following steps: Step 201: Construct the current state space based on user multimodal data.

[0031] In this embodiment, the specific operation of step 201 can be referred to Figure 1 The relevant descriptions of step 101 in the corresponding embodiments will not be repeated here.

[0032] Step 202: Extract visual feature vectors and text feature vectors from the current state space.

[0033] In this embodiment, the aforementioned execution entity can extract visual feature vectors through a visual encoder and extract text feature vectors through a text encoder.

[0034] Visual feature vectors are high-dimensional semantic vectors obtained by encoding resource images and video frames, used to represent visual content features. For example, a pre-trained visual encoder can be used to encode the current state space. The resource images are encoded, and a multi-head self-attention mechanism is used to capture global and local features within the images, outputting a visual feature vector. The pre-trained visual encoder can be a model such as the Vision Transformer, and the multi-head self-attention mechanism can simultaneously capture the global structure and local detail features of the image.

[0035] Text feature vectors are high-dimensional semantic vectors obtained by encoding text content such as resource titles and descriptions, used to represent text semantic information. For example, a pre-trained text encoder can be used to encode the current state space. The resource text is encoded, and the semantics of the text are captured through a multi-layer transformer to generate text feature vectors. The pre-trained text encoder can be, for example, BERT (Bidirectional Encoder Representations from Transformers) or RoBERTa (A Robustly Optimized BERT Pretraining Approach). Multi-layer Transformers can extract deep semantics from text layer by layer, achieving accurate semantic understanding.

[0036] Step 203: Perform feature fusion on the visual feature vector and the text feature vector to generate a resource representation vector.

[0037] In this embodiment, the aforementioned execution entity can perform cross-modal attention fusion on visual feature vectors and text feature vectors to generate resource representation vectors.

[0038] Resource representation vectors are unified resource feature vectors that fuse visual and textual modal information. They can completely represent all semantic information of a resource, avoiding comprehension biases caused by the lack of information from a single modality. For example, using a cross-attention fusion network, with visual feature vectors as queries and textual feature vectors as keys and values, the interaction weights between the visual and textual modalities are calculated to generate resource representation vectors. Among them, the cross-attention fusion network can achieve deep interaction between visual and textual modalities, dynamically allocate weights to highlight key modal information, and capture deeper semantic relationships across modalities better than simple splicing.

[0039] Resource representation vector The formula for calculating it is as follows: .

[0040] in, It is a visual feature vector. It is a text feature vector. It is a cross-modal attention fusion function, representing the... For Query, Using the key and value, the interaction weights between modalities are calculated through a cross-attention mechanism to achieve deep semantic fusion of visual and textual features, enabling the model to dynamically focus on more critical modal information in the current scene.

[0041] Step 204: Based on the resource representation vectors in the historical interaction sequence, perform dynamic user interest modeling to generate dynamic user interest representations.

[0042] In this embodiment, the aforementioned execution entity can use a time-series encoder to model user dynamic interests based on resource representation vectors in historical interaction sequences, thereby generating user dynamic interest representations.

[0043] User dynamic interest representation is a feature vector used to represent a user's current real-time interests and short-term preference changes. It can be dynamically updated following user behavior and context, solving the problem of user interest drift. For example, it can be a resource representation vector from a historical interaction sequence. The input is fed into a temporal encoder, which models sequence dependencies through a causal self-attention mechanism and outputs a dynamic representation of user interests. The temporal encoder can be used to process temporal interactive sequence data. It adopts a Transformer Decoder structure and a causal self-attention mechanism to ensure that only historical information is used to model sequence dependencies, avoid information leakage, and accurately capture the changing trends of user interests.

[0044] Step 205: Use a large language model to perform cognitive reasoning on the multimodal fusion features and output the reasoning chain and action intention.

[0045] In this embodiment, the aforementioned execution entity can utilize a large language model to perform cognitive reasoning on multimodal fusion features and output reasoning chains and action intentions.

[0046] Large language models possess powerful natural language understanding and logical reasoning capabilities. They can convert numerical features into natural language logical deductions, outputting interpretable reasoning chains and action intentions that guide decision-making.

[0047] In some embodiments, a large language model can perform cognitive reasoning through the following steps: First, structured text construction: using a pre-defined mapping function, the user's dynamic interests are represented. Candidate resource representation vector And contextual information is converted into natural language descriptions.

[0048] The pre-defined mapping function is used to map numerical feature vectors into readable natural language statements, such as: "The user recently viewed 'retro backpacks' (image features: brown leather, retro design; text features: top-grain cowhide, large capacity), and his / her historical preferences are 'outdoor products'."

[0049] Then, the prompt template is assembled: based on the natural language description and prompt word template, the target prompt word is assembled.

[0050] The prompt word template includes system instructions, user history, candidate resources, task objectives, etc., and is used to standardize the output of large language models.

[0051] The prompt template structure can be as follows: [System Command]: You are an intelligent recommendation assistant. Please perform inference and analysis based on the user's historical behavior.

[0052] [User History]: {User Interest Description}

[0053] [Candidate Resources]: {Candidate Resource 1 Description}, {Candidate Resource 2 Description}, ...

[0054] [Task Objective]: Analyze the user's current possible interests and preferences, and provide recommendation strategies.

[0055] Finally, the target cue words are input into the large language model for cognitive reasoning, and the reasoning chain and action intention are output.

[0056] Large language models can be, for example, LLaMA (Large Language Model Meta Artificial Intelligence) or ChatGLM (Chat General Language Model). The assembled target prompts are input into an open-source large language model or called via an API (Application Programming Interface). The model output consists of two parts: the inference chain and the action intent.

[0057] The reasoning chain is a structured deductive process. For example: "The user currently has a clear interest in retro style and leather materials, and has a long-term preference for outdoor products. Combining these two points, we can consider recommending resources that combine retro style and outdoor functionality."

[0058] Action intent is a structured result that includes modal emphasis, exploratory tendency, and category tendency. For example: {"Modal Emphasis": "Visual", "Exploration Tendency": "Yes", "Category Tendency": "Outdoor", "Interaction Needs": "No"}.

[0059] Step 206: Input the multimodal fusion features and inference results into the reinforcement learning policy network and output the probability distribution of each action dimension.

[0060] In this embodiment, the aforementioned execution entity can input multimodal fusion features and inference results into the reinforcement learning policy network and output the probability distribution of each action dimension.

[0061] The reinforcement learning policy network is used to output action decisions based on the state and reasoning results. A lightweight multilayer perceptron is used to construct the reinforcement learning policy network. The input is the vectorized representation of the reasoning results of the large language model and the numerical features generated in step two. The output is the probability distribution of each action dimension, which is used to select the optimal composite action.

[0062] Step 207: Select multidimensional composite actions from the multidimensional action space based on the probability distribution.

[0063] In this embodiment, the aforementioned execution entity can select the multidimensional composite action that maximizes the benefit from the multidimensional action space based on the probability distribution.

[0064] A multidimensional action space A is defined as a decision space containing actions across multiple dimensions. For example, a multidimensional action space A may include, but is not limited to, at least two of the following: recommendation actions, explanation generation actions, exploration actions, and interaction actions. The recommendation action refers to selecting candidate resources for recommendation, choosing K resources from a pool where K is variable (e.g., K=1, 3, 5). The explanation generation action refers to generating natural language recommendation reasons, generating natural language recommendation explanation text based on the inference chain output by a large language model. The exploration action refers to tentatively recommending new or low-exposure resources, used to decide whether to make exploratory recommendations from low-exposure resources, new resources, or categories that the user has not interacted with. The interaction action refers to asking the user questions to obtain preference information, used to decide whether to ask the user questions (e.g., "Do you prefer the appearance or the functionality?") to obtain more preference information.

[0065] The reinforcement learning policy network selects the optimal composite action based on the current state. The system executes the recommended action, displaying the selected K resources and their recommended explanations to the user. If interactive actions are included, interactive components are also displayed.

[0066] Step 208: Perform multi-dimensional composite actions to recommend resources to the user.

[0067] In this embodiment, the specific operation of step 208 can be referred to Figure 1 The relevant descriptions of step 105 in the corresponding embodiments will not be repeated here.

[0068] Step 209: Calculate the total reward based on real-time user feedback.

[0069] In this embodiment, the aforementioned execution entity can collect real-time user feedback information and calculate the total reward based on the user's real-time feedback information.

[0070] The total reward serves as a supervisory signal to guide model optimization. It is derived by weighting multi-dimensional feedback signals and is used to measure the quality of recommendation actions, aiming to maximize long-term returns. For example, the total reward can be generated by weighting and combining real-time user feedback information across multiple objectives. The real-time user feedback information may include, but is not limited to, at least one of the following: clicks, conversions, normalized dwell time, user satisfaction ratings, and exploration rewards. The multi-objective weighted combination involves assigning weights to different feedback metrics and summing them. Exploration rewards are used to encourage the model to explore new resources, avoid getting trapped in local optima, and balance short-term gains with long-term user retention.

[0071] Total Rewards The formula for calculating can be as follows: .

[0072] in, It's the weight of clicks. It is the weight of the transformation. It is a weight that is normalized based on the dwell time. It is the weight of the user satisfaction score. It is the weight of the exploration reward.

[0073] Step 210: Optimize the reinforcement learning policy network and the large language model based on the total reward.

[0074] In this embodiment, the aforementioned execution entity can optimize the reinforcement learning policy network and the large language model based on the total reward.

[0075] An online reinforcement learning framework is employed to continuously optimize the reinforcement learning policy network through real user interactions. Model optimization is achieved by jointly updating the reinforcement learning policy network and the large language model through reward signal backpropagation, forming a closed loop of perception-reasoning-decision-optimization, and continuously improving recommendation performance.

[0076] In some embodiments, the model optimization steps may be as follows: First, experience replay: the current spatial state Multidimensional compound movements Total Rewards and the next spatial state Constructing a quadruple sample Store the sample in the experience pool and extract the target quadruple sample from the experience pool.

[0077] Quadruple samples It serves as the foundational data for reinforcement learning training. The experience pool stores quadruple samples, and priority experience replay allows for the resampling of high-value samples, improving training efficiency.

[0078] Then, the algorithm selection is as follows: based on the total reward in the target quadruple sample, the proximal policy optimization algorithm is used to calculate the policy gradient, and the reinforcement learning policy network is updated according to the policy gradient.

[0079] PPO (Proximal Policy Optimization) is used to train the reinforcement learning policy network. PPO ensures training stability through importance sampling and pruning mechanisms.

[0080] Finally, joint fine-tuning: During the reinforcement learning policy network update process, the total reward in the target quadruple sample is backpropagated to the large language model through the policy gradient, and the large language model is updated using a low-rank adaptation method.

[0081] Policy gradients are used to update network parameters. Lightweight fine-tuning of the large language model is achieved through LoRA (Low-Rank Adaptation), making the inference chain generated by the large language model more conducive to maximizing long-term cumulative rewards, and realizing a closed-loop iteration of "perception-reasoning-decision-optimization".

[0082] In addition, the system supports offline pre-training and online fine-tuning: before the system goes live, the reinforcement learning policy network is trained offline using historical log data; after going live, the model parameters are continuously fine-tuned through online interaction to achieve rapid adaptation to new scenarios and new interests.

[0083] The resource recommendation method provided in this disclosure deeply integrates multimodal feature fusion, large language model cognitive reasoning, and reinforcement learning decision optimization to construct a closed-loop recommendation system. This system achieves cross-modal semantic understanding, dynamic interest adaptation, explainable recommendation, and continuous strategy optimization, significantly improving recommendation accuracy, user satisfaction, and system generalization ability.

[0084] The resource recommendation method provided in this disclosure, based on a deep fusion of reinforcement learning and a multimodal large language model, successfully solves the semantic understanding and dynamic decision-making problems in multimodal recommendation scenarios. By constructing a multimodal intelligent agent, leveraging the powerful reasoning capabilities of the large language model to deeply understand user interests, and combining reinforcement learning to continuously optimize online strategies, the method effectively improves the recommendation accuracy and adaptability of the system in complex interactive environments. Simultaneously, it can generate explainable recommendation reasons and supports interactive feedback from users through natural language, significantly enhancing the user experience and trustworthiness of the system.

[0085] Figure 3The overall architecture of a multimodal large language model-based resource recommendation system based on reinforcement learning is shown. The system models the recommendation process as a Markov decision process, comprising three core modules: a multimodal perception encoding module 301, a large language model cognitive reasoning module 302, and a reinforcement learning decision execution module 303. The multimodal perception encoding module 301 enables deep semantic understanding of heterogeneous information; the large language model cognitive reasoning module 302 endows the system with complex logical reasoning capabilities; and the reinforcement learning decision execution module 303 enables dynamic adaptive optimization of the recommendation strategy. These three modules work collaboratively to form a complete end-to-end resource recommendation system.

[0086] State generated by user interaction with the environment After processing through three main modules, the system ultimately outputs recommended actions and explanations, with user feedback forming a closed-loop optimization path. The multimodal perception coding module (301 pairs of states) The image and text data are used for feature extraction and fusion to output multimodal fusion features; the large language model cognitive reasoning module 302 generates reasoning chains and action intentions based on the multimodal fusion features; the reinforcement learning decision execution module 303 outputs multidimensional composite actions through a policy network. And perform recommendations; the total reward for user interaction that generates feedback information. The reinforcement learning training module is based on quadruplet samples. Update the policy network and the large language model, and generate the next state. This forms a complete closed-loop system.

[0087] Figure 4 A schematic diagram of the multimodal perception coding module is shown. The multimodal perception coding module includes a visual encoder 401, a text encoder 402, a cross-modal attention fusion network 403, and a temporal encoder 404.

[0088] The visual encoder 401 encodes the image or video input to generate a visual feature vector; the text encoder 402 encodes the text input to generate a text feature vector; the cross-modal attention fusion network 403 calculates the intermodal interaction weights of the visual feature vector and the text feature vector through a cross-attention mechanism to generate a unified resource representation vector; the temporal encoder 404 encodes the resource representation vectors in the user's historical interaction sequence in a temporal sequence, models sequence dependencies through a causal self-attention mechanism, generates a dynamic user interest representation, and completes the entire process of multimodal perception and interest modeling.

[0089] Figure 5 A schematic diagram illustrating the decision-making and optimization process in reinforcement learning is provided. The workflow of the reinforcement learning decision-making module is illustrated using a Markov decision process framework. The states are presented in a cyclical structure within the diagram. Multidimensional compound movements Total Rewards Next state The iterative relationship between them.

[0090] The agent comprises a large language model 501 and a policy network 502. State The large language model 501 and policy network 502 output multi-dimensional compound actions. The total reward is obtained by weighting and combining environmental feedback. The next state is finally generated by updating the policy network 502 based on PPO and fine-tuning the large language model 501 using LoRA. This completes the iterative optimization of decision-making.

[0091] Among them, state It includes user profiles, historical interaction sequences, contextual information, candidate resource pools, and real-time feedback information. Multi-dimensional composite actions. It includes recommended actions, explained and generated actions, exploration actions, and interactive actions. Total reward It is a weighted combination based on clicks, conversions, normalized dwell time, user satisfaction ratings, and exploration rewards. Next state It is the updated environment state after the user performs an action.

[0092] It's important to note that an intelligent agent refers to a system or entity capable of perceiving its environment and autonomously making decisions and executing tasks. In some scenarios, it can also be referred to as an "intelligent agent." Besides being understood as an entity handling user needs, an "intelligent agent" can also be understood as a service that intelligently invokes a single intelligent agent or combines multiple intelligent agents to meet user-expressed needs—in other words, a service delivery method.

[0093] Figure 6 The overall flowchart of a multimodal large language model resource recommendation method based on reinforcement learning is shown. The overall process of the resource recommendation method includes: state construction stage 601, multimodal encoding stage 602, large language model inference stage 603, reinforcement learning decision stage 604, and policy optimization stage 605.

[0094] The state construction phase (601) collects multimodal user data, integrates user profiles, historical interaction sequences, contextual information, candidate resource pools, and real-time user feedback to construct a state space. The multimodal encoding phase (602) extracts image features using a visual encoder, extracts text features using a text encoder, generates resource representations through cross-modal attention fusion, and generates dynamic user interest representations using a temporal encoder. The large language model inference phase (603) converts numerical features into structured natural language prompts, inputs them into the large language model, and outputs inference chains and action intentions. The reinforcement learning decision-making phase (604) receives the inference results and numerical features from the large language model through a policy network, outputs multidimensional composite actions, and executes recommendations and explanations. The policy optimization phase (605) collects user feedback, calculates the total reward, updates the policy network through PPO, and fine-tunes the large language model through LoRA, achieving a closed-loop iterative optimization throughout the entire process.

[0095] The resource recommendation method provided in this disclosure can be widely applied to various recommendation scenarios that require multimodal understanding and dynamic interaction. The following uses three typical application areas as examples to illustrate the specific application process in detail.

[0096] Application Area 1: E-commerce Platforms – Personalized Product Recommendations Across Product Categories. The application process on e-commerce platforms is as follows: User behavior collection: After a user logs into the e-commerce application, the system collects multimodal behavior data of the user in the current session in real time, including the main image of the product viewed, the product title and detailed description, the user's search terms, click and add-to-cart behavior sequences, etc.

[0097] Agent state construction: The multimodal perception coding module integrates the user's current browsing of "retro backpack" (image features include leather texture and retro design; text features include keywords such as "top-grain cowhide", "large capacity", and "commuting") with the user's historical behavior sequence (such as purchasing outdoor hiking shoes three days ago) to generate a comprehensive state representation that includes short-term interests and long-term preferences.

[0098] Reasoning and Decision-Making: The large language model receives state information and performs cognitive reasoning: "The user is currently interested in 'retro style' and 'leather material,' and has a long-term preference for 'outdoor products.' The current recommendation should be to explore cross-category combinations." Based on this, the reinforcement learning strategy network outputs an action: select a "retro leather outdoor water bottle bag" from the candidate pool (this item is a low-exposure new product) and generate explanatory text: "This water bottle bag continues your favorite retro leather style and also matches your outdoor hobbies."

[0099] Results Presentation and Feedback: The system displays recommended products and explanatory text on the user interface. When a user clicks to view details, the system receives a positive click reward. The reinforcement learning strategy network updates its parameters based on this feedback, reinforcing the strategy of "exploring across categories when users exhibit clear style preferences."

[0100] Application Area Two: Short Video or Streaming Media Platforms – Multimodal Content Recommendation and Interactive Guidance. The application process on short video platforms is as follows: User behavior collection: The system collects the video content watched by users in real time, including video cover images, video titles and tags, audio features in the video, and user behavior (such as completing the video, liking, commenting, and skipping).

[0101] Intelligent agent state construction: The multimodal perception and encoding module extracts features from the "food exploration" video currently being watched by the user, and combines them with the user's recent viewing sequence (such as three consecutive "camping" videos) to generate a dynamic interest representation of the user: "Recent preference for outdoor themes, current interest in food content".

[0102] Reasoning and Decision-Making: The large language model reasoned: "User interests are shifting from purely outdoor activities to food content related to the outdoors. We can consider recommending videos on 'outdoor camping food preparation' and actively guide users to clarify their preferences." The reinforcement learning strategy output action: Recommend an "outdoor camping cooking" video and simultaneously output the interactive action "Are you interested in outdoor-themed food content?", asking the user in the form of a pop-up window.

[0103] Results and Feedback: When a user watches a recommended video and provides a positive response to a question, the system receives a completion reward and an interaction reward. Based on this feedback, the agent increases the recommendation weight of outdoor food content in subsequent recommendations.

[0104] Application Area 3: Content Communities and Social Platforms – Image and Long Article Recommendations. The application process in content communities is as follows: User behavior collection: The system collects the text and image content that users read, including article cover images, article titles, body text, user comments and likes, and the user's following list in the community.

[0105] Intelligent agent state construction: The multimodal perception and encoding module extracts features from the "photography tips" type of text and images that the user is currently reading, and at the same time combines the user's long-term interests (following multiple "digital review" bloggers) to generate a comprehensive state representation.

[0106] Reasoning and Decision-Making: The large language model reasones: "The user has immediate interest in 'photography' and long-term interest in 'digital reviews,' and there is overlap between the two (mobile phone photography reviews). Content that combines these two themes can be recommended." The reinforcement learning strategy outputs the following action: Select a long article from the candidate pool titled "In-depth Review of the Latest Flagship Mobile Phone Photography Features" for recommendation, and generate the explanation: "This article combines photography techniques and digital reviews that you are interested in, and is recommended for reading."

[0107] Results and Feedback: When users clicked to read and stayed for longer than the average time, the system received positive feedback. The agent updated its strategy, learning the effectiveness of "recommending at the intersection of user interests."

[0108] In addition to recommendation scenarios in e-commerce, short videos, and content communities, the resource recommendation method provided in this disclosure can also be applied to: Intelligent Marketing and Advertising System: Based on user interaction with advertising materials (images, text, and videos), dynamically generate personalized advertising content and optimize delivery strategies.

[0109] Online education platforms recommend personalized course content and learning paths based on students' learning behaviors (watching videos, reading texts, answering questions and providing feedback).

[0110] Smart Home and IoT (Internet of Things): Based on user interactions with smart devices (voice commands, device usage records) and environmental information, personalized device linkage scenarios are recommended.

[0111] Digital Human Shopping Guide and Customer Service: Driven by a digital human intelligent agent, it provides users with immersive shopping guide and consultation services through multimodal perception and natural dialogue.

[0112] The resource recommendation method provided in this disclosure deeply integrates reinforcement learning with a multimodal large language model, enabling the recommendation system to possess cross-modal semantic understanding, dynamic policy optimization, natural language interaction, and interpretation generation capabilities. This can effectively improve user satisfaction and long-term system benefits in various recommendation scenarios.

[0113] The resource recommendation method provided in this disclosure achieves significant breakthroughs compared to existing recommendation systems in three dimensions: multimodal understanding capability, dynamic adaptability, interpretability, and interactive experience, and can bring the following core beneficial effects: First, it significantly improves recommendation accuracy and user satisfaction: Existing recommendation systems often use simple modal feature concatenation, making it difficult to capture the deep semantic relationships between multimodal information such as images, text, and videos, leading to discrepancies between recommendation results and the user's true intent. This disclosure utilizes a cross-modal attention fusion network, enabling the model to dynamically focus on more critical modal information in the current scenario, achieving deep semantic alignment of multimodal information. Simultaneously, a reinforcement learning strategy network continuously optimizes the recommendation strategy based on real-time user feedback, allowing the system to quickly adapt to short-term shifts in user interests. Compared to existing technologies, this disclosure improves core metrics such as click-through rate, conversion rate, and user dwell time.

[0114] Secondly, this invention achieves dynamic adaptation and long-term optimization of the recommendation strategy: Existing recommendation models are mostly static models trained offline, which cannot be adjusted in real time according to user feedback, and it is also difficult to balance the short-term immediate satisfaction of users with long-term retention goals. This embodiment models the recommendation process as a Markov decision process. Through the exploration mechanism of reinforcement learning, the system can proactively try to recommend new categories or items to users with less historical interaction, effectively avoiding the information cocoon effect. At the same time, the reward function design takes into account both short-term indicators such as click-through rate and conversion rate and long-term goals such as user retention rate and satisfaction, enabling the system to continuously improve user stickiness to the product while optimizing immediate benefits. Compared with traditional recommendation systems, this embodiment not only improves the monthly user retention rate, but also improves the recommendation accuracy during the cold start phase for new users.

[0115] Third, it effectively alleviates the cold start problem and improves the system's generalization ability: When new users or items are added, existing systems face a cold start dilemma due to a lack of historical interaction data, resulting in a significant drop in recommendation effectiveness. This embodiment of the disclosure fully utilizes the original features of new items, such as images and text, through a multimodal coding module, combined with the prior knowledge and reasoning ability of a large language model. Even without user interaction data, it can make reasonable recommendations based on content features. Simultaneously, the reinforcement learning exploration mechanism enables the system to proactively ask questions of new users or recommend diverse exploratory content, quickly establishing user interest profiles. Compared to traditional collaborative filtering methods, this embodiment of the disclosure improves the exposure effectiveness of new items in the first week after launch.

[0116] Further reference Figure 7 As an implementation of the methods shown in the above figures, this disclosure provides an embodiment of a resource recommendation device, which is similar to... Figure 3 Corresponding to the method embodiments shown, this device can be specifically applied to various electronic devices.

[0117] like Figure 7 As shown, the resource recommendation device 700 of this embodiment may include: a construction module 701, an extraction module 702, an inference module 703, a generation module 704, and a push module 705. The construction module 701 is configured to construct a current state space based on user multimodal data; the extraction module 702 is configured to extract and fuse features in the current state space to generate multimodal fusion features; the inference module 703 is configured to perform inference based on the multimodal fusion features to generate inference results; the generation module 704 is configured to generate multidimensional composite actions based on the inference results; and the push module 705 is configured to execute the multidimensional composite actions to recommend resources to the user.

[0118] In this embodiment, the specific processing of the resource recommendation device 700, including the construction module 701, extraction module 702, reasoning module 703, generation module 704, and push module 705, and the resulting technical effects, can be found by referring to [reference needed]. Figure 1 The relevant descriptions of steps 101-105 in the corresponding embodiments will not be repeated here.

[0119] In some optional implementations of this embodiment, the extraction module 702 is further configured to: extract visual feature vectors and text feature vectors from the current state space; perform feature fusion on the visual feature vectors and text feature vectors to generate resource representation vectors; and perform user dynamic interest modeling based on the resource representation vectors in the historical interaction sequence to generate user dynamic interest representations.

[0120] In some optional implementations of this embodiment, the extraction module 702 is further configured to: encode the resource image in the current state space using a pre-trained visual encoder, capture the global and local features inside the image through a multi-head self-attention mechanism, and output a visual feature vector; and encode the resource text in the current state space using a pre-trained text encoder, capture the text semantics through a multi-layer transformer, and generate a text feature vector.

[0121] In some optional implementations of this embodiment, the extraction module 702 is further configured to: use a cross-attention fusion network, with visual feature vectors as queries and text feature vectors as keys and values, calculate the interaction weights between the visual modality and the text modality, and generate a resource representation vector.

[0122] In some optional implementations of this embodiment, the extraction module 702 is further configured to: input the resource representation vector in the historical interaction sequence into the temporal encoder, model the sequence dependency relationship through the causal self-attention mechanism, and output the user dynamic interest representation.

[0123] In some optional implementations of this embodiment, the reasoning module 703 is further configured to: use a large language model to perform cognitive reasoning on multimodal fusion features and output reasoning chains and action intentions.

[0124] In some optional implementations of this embodiment, the reasoning module 703 is further configured to: convert user dynamic interest representation, candidate resource representation vector and context information into natural language description through a preset mapping function; assemble target prompt words based on natural language description and prompt word template; input the target prompt words into a large language model for cognitive reasoning, and output the reasoning chain and action intent.

[0125] In some optional implementations of this embodiment, the generation module 704 is further configured to: input multimodal fusion features and inference results into a reinforcement learning policy network, output the probability distribution of each action dimension, and select multidimensional composite actions from the multidimensional action space according to the probability distribution; In some optional implementations of this embodiment, the resource recommendation device 700 further includes an optimization module configured to calculate the total reward based on real-time user feedback information; and to optimize the reinforcement learning policy network and the large language model based on the total reward.

[0126] In some optional implementations of this embodiment, the optimization module is further configured to: perform multi-objective weighted combination of real-time user feedback information to generate a total reward, wherein the real-time user feedback information includes at least one of the following: clicks, conversions, normalized dwell time, user satisfaction score, and exploration reward.

[0127] In some optional implementations of this embodiment, the optimization module is further configured to: construct a quadruple sample from the current spatial state, multi-dimensional compound action, total reward, and next spatial state, and store it in the experience pool; extract a target quadruple sample from the experience pool; calculate the policy gradient using a proximal policy optimization algorithm based on the total reward in the target quadruple sample, and update the reinforcement learning policy network according to the policy gradient; during the reinforcement learning policy network update process, backpropagate the total reward in the target quadruple sample to the large language model through the policy gradient, and update the large language model using a low-rank adaptation method.

[0128] In some optional implementations of this embodiment, the user multimodal data includes at least two of the following: user profile, historical interaction sequence, context information, candidate resource pool, and real-time user feedback information.

[0129] In some optional implementations of this embodiment, the multidimensional action space includes at least two of the following: recommendation actions, explanation generation actions, exploration actions, and interaction actions.

[0130] The collection, storage, use, processing, transmission, provision, and disclosure of any type of information, such as user personal information, in this technical solution comply with relevant laws and regulations and do not violate public order and good morals.

[0131] According to embodiments of this disclosure, this disclosure also provides an electronic device, a readable storage medium, and a computer program product.

[0132] Figure 8A schematic block diagram of an example electronic device 800 that can be used to implement embodiments of the present disclosure is shown. The electronic device is intended to represent various forms of digital computers, such as laptop computers, desktop computers, workstations, personal digital assistants, servers, blade servers, mainframe computers, and other suitable computers. The electronic device may also represent various forms of mobile devices, such as personal digital processors, cellular phones, smartphones, wearable devices, and other similar computing devices. The components shown herein, their connections and relationships, and their functions are merely illustrative and are not intended to limit the implementation of the present disclosure described and / or claimed herein.

[0133] like Figure 8 As shown, device 800 includes a computing unit 801, which can perform various appropriate actions and processes based on a computer program stored in read-only memory (ROM) 802 or a computer program loaded from storage unit 808 into random access memory (RAM) 803. RAM 803 may also store various programs and data required for the operation of device 800. The computing unit 801, ROM 802, and RAM 803 are interconnected via bus 804. Input / output (I / O) interface 805 is also connected to bus 804.

[0134] Multiple components in device 800 are connected to I / O interface 805, including: input unit 806, such as keyboard, mouse, etc.; output unit 807, such as various types of monitors, speakers, etc.; storage unit 808, such as disk, optical disk, etc.; and communication unit 809, such as network card, modem, wireless transceiver, etc. Communication unit 809 allows device 800 to exchange information / data with other devices through computer networks such as the Internet and / or various telecommunications networks.

[0135] The computing unit 801 can be a variety of general-purpose and / or special-purpose processing components with processing and computing capabilities. Some examples of the computing unit 801 include, but are not limited to, a central processing unit (CPU), a graphics processing unit (GPU), various special-purpose artificial intelligence (AI) computing chips, various computing units running machine learning model algorithms, a digital signal processor (DSP), and any suitable processor, controller, microcontroller, etc. The computing unit 801 performs the various methods and processes described above, such as the resource recommendation method. For example, in some embodiments, the resource recommendation method may be implemented as a computer software program tangibly contained in a machine-readable medium, such as storage unit 808. In some embodiments, part or all of the computer program may be loaded and / or installed on device 800 via ROM 802 and / or communication unit 809. When the computer program is loaded into RAM 803 and executed by the computing unit 801, one or more steps of the resource recommendation method described above may be performed. Alternatively, in other embodiments, the computing unit 801 may be configured to perform the resource recommendation method by any other suitable means (e.g., by means of firmware).

[0136] Various embodiments of the systems and techniques described above herein can be implemented in digital electronic circuit systems, integrated circuit systems, field-programmable gate arrays (FPGAs), application-specific integrated circuits (ASICs), application-specific standard products (ASSPs), systems-on-a-chip (SoCs), payload-programmable logic devices (CPLDs), computer hardware, firmware, software, and / or combinations thereof. These various embodiments may include implementations in one or more computer programs that can be executed and / or interpreted on a programmable system including at least one programmable processor, which may be a dedicated or general-purpose programmable processor, capable of receiving data and instructions from a storage system, at least one input device, and at least one output device, and transmitting data and instructions to the storage system, the at least one input device, and the at least one output device.

[0137] The program code used to implement the methods of this disclosure may be written in any combination of one or more programming languages. This program code may be provided to a processor or controller of a general-purpose computer, special-purpose computer, or other programmable data processing apparatus, such that when executed by the processor or controller, the program code causes the functions / operations specified in the flowcharts and / or block diagrams to be implemented. The program code may be executed entirely on a machine, partially on a machine, as a standalone software package partially on a machine and partially on a remote machine, or entirely on a remote machine or server.

[0138] In the context of this disclosure, a machine-readable medium can be a tangible medium that may contain or store a program for use by or in conjunction with an instruction execution system, apparatus, or device. A machine-readable medium can be a machine-readable signal medium or a machine-readable storage medium. A machine-readable medium can be, but is not limited to, electronic, magnetic, optical, electromagnetic, infrared, or semiconductor systems, apparatus, or devices, or any suitable combination of the foregoing. More specific examples of machine-readable storage media include electrical connections based on one or more wires, portable computer disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fiber, portable compact disk read-only memory (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination of the foregoing.

[0139] To provide interaction with a user, the systems and techniques described herein can be implemented on a computer having: a display device for displaying information to the user (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor); and a keyboard and pointing device (e.g., a mouse or trackball) through which the user provides input to the computer. Other types of devices can also be used to provide interaction with the user; for example, feedback provided to the user can be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user can be received in any form (including sound input, voice input, or tactile input).

[0140] The systems and technologies described herein can be implemented in computing systems that include backend components (e.g., as a data server), or computing systems that include middleware components (e.g., an application server), or computing systems that include frontend components (e.g., a user computer with a graphical user interface or web browser through which a user can interact with implementations of the systems and technologies described herein), or any combination of such backend, middleware, or frontend components. The components of the system can be interconnected via digital data communication of any form or medium (e.g., a communication network). Examples of communication networks include local area networks (LANs), wide area networks (WANs), and the Internet.

[0141] Computer systems can include clients and servers. Clients and servers are generally located far apart and typically interact via communication networks. Client-server relationships are created by computer programs running on the respective computers and having a client-server relationship with each other. Servers can be cloud servers, distributed system servers, or servers incorporating blockchain technology.

[0142] It should be understood that the various forms of processes shown above can be used to rearrange, add, or delete steps. For example, the steps described in this disclosure can be executed in parallel, sequentially, or in different orders, as long as the desired result of the technical solution provided in this disclosure can be achieved, and this is not limited herein.

[0143] The specific embodiments described above do not constitute a limitation on the scope of protection of this disclosure. Those skilled in the art should understand that various modifications, combinations, sub-combinations, and substitutions can be made according to design requirements and other factors. Any modifications, equivalent substitutions, and improvements made within the spirit and principles of this disclosure should be included within the scope of protection of this disclosure.

Claims

1. A resource recommendation method, comprising: Construct the current state space based on user multimodal data; Feature extraction and fusion are performed in the current state space to generate multimodal fusion features; Reasoning is performed based on the multimodal fusion features to generate reasoning results; Based on the reasoning results, a multi-dimensional composite action is generated; Perform the aforementioned multi-dimensional composite actions to recommend resources to the user.

2. The method according to claim 1, wherein, The step of extracting and fusing features in the current state space to generate multimodal fused features includes: Extract visual feature vectors and text feature vectors from the current state space; The visual feature vector and the text feature vector are fused to generate a resource representation vector; User dynamic interest modeling is performed based on resource representation vectors in historical interaction sequences to generate user dynamic interest representations.

3. The method according to claim 2, wherein, The step of extracting visual feature vectors and text feature vectors from the current state space includes: A pre-trained visual encoder is used to encode the resource image in the current state space, and a multi-head self-attention mechanism is used to capture the global and local features inside the image, and output the visual feature vector. A pre-trained text encoder is used to encode the resource text in the current state space, and a multi-layer transformer is used to capture the text semantics to generate the text feature vector.

4. The method according to claim 2, wherein, The step of fusing the visual feature vector and the text feature vector to generate a resource representation vector includes: Using a cross-attention fusion network, with the visual feature vector as the query and the text feature vector as the key and value, the interaction weight between the visual modality and the text modality is calculated to generate the resource representation vector.

5. The method according to claim 2, wherein, The process of modeling user dynamic interests based on resource representation vectors in historical interaction sequences to generate user dynamic interest representations includes: The resource representation vectors in the historical interaction sequence are input into the temporal encoder, and the sequence dependency relationship is modeled through the causal self-attention mechanism to output the user dynamic interest representation.

6. The method according to claim 1, wherein, The reasoning based on the multimodal fusion features, generating the reasoning result, includes: The multimodal fusion features are used to perform cognitive reasoning using a large language model, and the reasoning chain and action intention are output.

7. The method according to claim 6, wherein, The process of using a large language model to perform cognitive reasoning on the multimodal fusion features, and outputting the reasoning chain and action intent, includes: By using a pre-defined mapping function, user dynamic interest representations, candidate resource representation vectors, and contextual information are converted into natural language descriptions. Based on the natural language description and prompt word template, assemble the target prompt words; The target prompt is input into the large language model for cognitive reasoning, and the reasoning chain and the action intention are output.

8. The method according to claim 6, wherein, The generation of multi-dimensional composite actions based on the reasoning results includes: The multimodal fusion features and the inference results are input into the reinforcement learning policy network, which outputs the probability distribution of each action dimension. Based on the probability distribution, the multidimensional composite action is selected from the multidimensional action space.

9. The method according to claim 8, wherein, The method further includes: The total reward is calculated based on real-time user feedback. Based on the total reward, the reinforcement learning policy network and the large language model are optimized.

10. The method according to claim 9, wherein, The optimization of the reinforcement learning policy network and the large language model based on the total reward includes: The total reward is generated by performing a multi-objective weighted combination of the real-time user feedback information, which includes at least one of the following: clicks, conversions, normalized dwell time, user satisfaction rating, and exploration rewards.

11. The method according to claim 9, wherein, The optimization of the reinforcement learning policy network and the large language model based on the total reward includes: The current spatial state, the multidimensional composite action, the total reward, and the next spatial state are constructed into a quadruple sample and stored in the experience pool. Draw target quadruplets from the experience pool; Based on the total reward in the target quadruple sample, the policy gradient is calculated using the near-end policy optimization algorithm, and the reinforcement learning policy network is updated according to the policy gradient. During the reinforcement learning policy network update process, the total reward in the target quadruple sample is backpropagated to the large language model through the policy gradient, and the large language model is updated using a low-rank adaptation method.

12. The method according to any one of claims 1-11, wherein, The user multimodal data includes at least two of the following: user profile, historical interaction sequence, contextual information, candidate resource pool, and real-time user feedback information.

13. The method according to any one of claims 7-11, wherein, The multidimensional action space includes at least two of the following: recommended actions, interpretation-generated actions, exploration actions, and interactive actions.

14. A resource recommendation device, comprising: The building module is configured to construct the current state space based on user multimodal data; The extraction module is configured to extract and fuse features in the current state space to generate multimodal fusion features; The inference module is configured to perform inference based on the multimodal fusion features and generate inference results; The generation module is configured to generate multi-dimensional composite actions based on the reasoning results; The push module is configured to perform the multi-dimensional composite action to recommend resources to the user.

15. An electronic device comprising: At least one processor; as well as A memory communicatively connected to the at least one processor; wherein, The memory stores instructions that can be executed by the at least one processor to enable the at least one processor to perform the method of any one of claims 1-13.

16. A non-transitory computer-readable storage medium storing computer instructions for causing the computer to perform the method of any one of claims 1-13.

17. A computer program product comprising a computer program that, when executed by a processor, implements the method according to any one of claims 1-13.

Images