Large model recommendation method and system based on comparative learning enhancement and model fine tuning

Abstract

The invention discloses a large model recommendation method and system based on comparative learning enhancement and model fine tuning, and relates to the technical field of personalized recommendation, and the method comprises the steps: obtaining historical interaction behaviors of a user, and constructing text modal representation and structured modal representation of the user and an article; performing representation enhancement through cross-modal contrast learning, and generating preference representation information of the user and the article through adaptive multi-modal fusion; generating a dynamic prompt according to the real-time behavior of the user and the preference representation information of the user and the article, fusing the preference representation information of the user and the article with the dynamic prompt through cross attention, and inputting the fused information into a preset large language model for model fine tuning; and when a user initiates a request, calling the fine-tuned large language model to generate a recommendation result, and providing an interpretable recommendation reason in combination with the inference ability of the comparative learning representation and the large language model. According to the method, technical paths of comparative learning and model fine tuning are fused, and the performance and personalization of article recommendation are improved.

Description

Technical Field

[0001] The present invention relates to the field of personalized recommendation technology, and more specifically, to a large model recommendation method and system based on contrastive learning enhancement and model fine-tuning. Background Art

[0002] In recent years, with the rapid development of large language models (LLMs), their application in personalized recommendation systems has attracted widespread attention. LLMs, with their powerful natural language understanding capabilities and rich semantic knowledge, can effectively mine deep information from historical user behavior data (such as comments, click records, search queries, etc.), thereby improving the performance of recommendation systems. However, most existing LLM-based recommendation methods rely solely on text features, while ignoring the important structured information contained in user-item interactions (such as social relationships, item associations, collaborative filtering signals, etc.). This structural information is crucial for accurately characterizing user preferences, especially in data sparse or cold-start scenarios.

[0003] While traditional recommendation systems (such as matrix factorization and graph neural networks) can effectively model the structured relationships between users and items, they are limited by their representational capabilities and struggle to fully exploit the semantic information in text data. Conversely, LLM-based recommendation methods, while leveraging the high-level semantic understanding capabilities of pre-trained language models, often overlook the topological structure of user-item interactions, resulting in a lack of personalized adaptability in recommendation results. Therefore, combining the semantic understanding advantages of LLMs with structured relationship modeling has become a key challenge in improving recommendation performance.

[0004] Contrastive learning, as a self-supervised learning paradigm, can enhance the model's discriminative ability by constructing pairs of positive and negative samples to learn implicit feature representations of users and items from different perspectives. In recommender systems, contrastive learning has been shown to effectively alleviate data sparsity and improve the robustness of representations. For example, by comparing different behavior sequences of the same user or different descriptions of the same item, the model can better capture the invariant characteristics of user preferences and item characteristics. However, how to organically combine contrastive learning with LLMs to simultaneously utilize textual semantics and structured information is a technical challenge that needs to be addressed. Summary of the Invention

[0005] In order to solve the above technical problems, the present invention proposes a large-model recommendation method and system based on contrastive learning enhancement and model fine-tuning. By integrating the multimodal representation enhancement of contrastive learning with the semantic reasoning ability of a large language model, the performance and personalization of item recommendations are improved.

[0006] A first aspect of the present invention provides a large model recommendation method based on contrastive learning enhancement and model fine-tuning, comprising the following steps:

[0007] Obtaining the user's historical interaction behavior, preprocessing the historical interaction behavior, and constructing text modal representations and structured modal representations of the user and items;

[0008] Representation enhancement is performed through cross-modal contrastive learning, and preference representation information of users and items is generated through adaptive multimodal fusion;

[0009] Generate dynamic prompts based on real-time user behavior and user and item preference representation information. Use cross-attention to fuse user and item preference representation information with dynamic prompts and input them into a preset large language model for model fine-tuning.

[0010] When a user initiates a request, prompts are dynamically generated, and the fine-tuned large language model is called to generate recommendation results. By combining contrastive learning representations and the reasoning capabilities of the large language model, explainable reasons for the recommendation are provided.

[0011] In this solution, the user's historical interaction behavior is obtained, preprocessed, and the textual modal representation and structured modal representation of the user and the item are constructed. Specifically:

[0012] Based on recommendation-related data sources, we obtain user-item interaction data, text data, and structured relationship data. We organize user interaction records in chronological order, perform weighted processing on discrete behaviors, and normalize continuous behaviors to generate a time-series behavior sequence.

[0013] The item-side text is spliced ​​with the item title and key attributes in the text data, a pre-trained language model is used to obtain sentence-level representation, and an item semantic embedding representation is generated. The user-side text is aggregated with user historical comments and spliced ​​with real-time search terms. A long text encoder is used to generate a user profile representation. The item semantic embedding representation and the user profile representation are used as text modality representations;

[0014] We construct an interaction graph by treating users and items as two types of nodes and interaction behaviors as edges. We use text modality representations to initialize the representations of user nodes and item nodes in the interaction graph. We use a relational graph convolutional network to process heterogeneous edge types, propagate information on edges of the same type, and then aggregate and update node representations across different types.

[0015] The graph attention mechanism is used to identify important neighbor nodes, which are weighted during the aggregation process. Graph pooling is used to generate global topological representations of users and items as structured modal representations.

[0016] In this solution, representation enhancement is performed through cross-modal contrastive learning, specifically:

[0017] A three-level contrastive learning framework is used to gradually enhance representations from fine-grained to coarse-grained. In intra-modal contrastive learning, different behavior sequences of the same user are used to construct positive samples, while similar behavior sequences of different users are used to construct negative samples. Through user-side contrastive learning, the temporal behavior representations of the same user are clustered in the vector space.

[0018] In item-side comparison, different descriptions of the same item are used to construct positive samples, while descriptions of different items in the same category are used to construct negative samples. This allows for learning to obtain the core features of items through item-side comparison.

[0019] In the user dimension comparison of cross-modal contrastive learning, positive samples are constructed using user text comment features and interaction graph features, and negative samples are constructed by randomly combining user modal representations. In the item dimension comparison, positive samples are constructed using item description text, and negative samples are constructed using a cross-modal combination of popular and unpopular items. Cross-modal contrastive learning eliminates semantic differences between modalities.

[0020] In interaction-level contrastive learning, positive samples are constructed by using items that users actually interact with and similar items generated through graph diffusion, while negative samples are constructed by using items of the same category but without interaction. This method captures the collaborative features between users and items.

[0021] Perform progressive optimization of intra-modal contrastive learning, cross-modal contrastive learning, and interaction-level contrastive learning to generate enhanced modal representations of users and items.

[0022] In this solution, user and item preference representation information is generated through adaptive multimodal fusion, specifically:

[0023] A gating mechanism is used to dynamically generate fusion weights. For the enhanced text modal representations of users and items, the text weights are calculated by calculating the frequency of user comments and the richness of item descriptions. Sigmoid function normalization is used, and complementary allocation is used to obtain the structural weights of the enhanced structured modal representations of users and items. Prior bias is introduced in cold start scenarios.

[0024] Using the text weight and structured weight to perform primary fusion using element-by-element weighting, and using the text modal representation after user and item enhancement as the query and the structured modal representation after user and item enhancement as the key-value pair, generating cross attention between modalities, and using the cross attention to perform advanced fusion;

[0025] A user interest memory is constructed based on the user's historical interaction behavior. The average of the text modal representation and structured modal representation of the user and item is stored using a sliding window method. The enhanced text modal representation and structured modal representation of the user and item are used to calculate the similarity in the user interest memory, and the high-level fusion weight is adjusted according to the similarity.

[0026] Through advanced fusion, cross-modal feature cross-fertilization is achieved, a multimodal interaction matrix is ​​established for low-rank decomposition, and preference representation information of users and items is generated.

[0027] In this solution, dynamic prompts are generated based on real-time user behavior and user and item preference information. Specifically:

[0028] Capture the user's latest interaction events in real time as the user's real-time behavior, enhance the user's real-time behavior with spatiotemporal context based on the spatiotemporal dimensions of the interaction events, generate a dense representation of the user's real-time behavior, and extract the current user and item preference representation information from the user interest memory as the basic anchor point of the user's interest;

[0029] A bidirectional LSTM network is used to perform temporal modeling on the dense representation of users' real-time behavior. The forward propagation method captures the users' explicit behavior patterns, while the backpropagation method captures the changes in their implicit needs. The hidden states of the bidirectional outputs are concatenated to generate a comprehensive representation vector containing the complete temporal context.

[0030] The similarity between the comprehensive representation vector and the basic anchor points of user interest is calculated and normalized to item-level weights through Softmax. The item description text of the user's real-time behavioral interaction is generated through BERT to generate item representations. Secondary feature-level weights are applied to the item representations to highlight key attributes.

[0031] The item-level weights and feature-level weights are used to weight the final item representations, the final item representations are clustered, and a natural language description is generated as a user interest summary. A scenario description is generated by combining the temporal scenario and the spatial scenario, and an executable recommendation guide is generated as a task instruction based on quantity restrictions and feature requirements.

[0032] Based on the user interest summary, scenario description and task instructions, structured dynamic prompts are generated, an update mechanism for short-term interest tracking and long-term preference calibration is established, behavioral mutation detection is performed through a sliding window, interest anchor points are updated in real time, offline clustering analysis is performed based on preset time intervals, stable preference patterns are identified, and user portrait reconstruction is triggered when the distance between the new and old cluster centers exceeds a threshold.

[0033] In this solution, the preference representation information of users and items is fused with dynamic prompts through cross-attention and input into the preset large language model for model fine-tuning. Specifically:

[0034] Align the user and item preference representation information with the dynamic prompts, use the text token as the query and the user preference vector as the key-value pair to calculate the preference-prompt cross attention and obtain the fused prompt vector;

[0035] Importing the hint vector into a preset large language model to obtain a hidden state, using the hidden state as a query and the item representation as a key-value pair to calculate item-context cross attention, and using the item-context cross attention to control the intensity of item information penetration;

[0036] Perform a residual connection between the representation of the original preset large language model and the output of the cross attention, and import the fused representation of the residual connection into the preset large language model for fine-tuning, maintaining the general language understanding ability of the first 6 layers of the preset large language model unchanged;

[0037] A low-rank adaptation module with a bottleneck structure is added between the Transformer layers for cross-modal adaptation. The LoRA technology is used to apply low-rank incremental updates to the attention layer to perform intermediate layer adaptation adjustments.

[0038] A new dual-branch task head for item recommendation sorting and list generation is added to the top-level task-specific layer, and fine-tuning training of the preset large language model is achieved through staged training.

[0039] In this solution, when a user initiates a request, prompts are dynamically generated and a fine-tuned large language model is used to generate recommendations. By combining contrastive learning representations with the reasoning capabilities of the large language model, explainable reasons for the recommendations are provided. Specifically:

[0040] When a user initiates a request, the preference representation information enhanced by contrastive learning is fused with real-time context features. The fused user vector is used to perform an approximate nearest neighbor search in the item library. The fused user vector is then concatenated with the candidate item features to construct a dynamic prompt.

[0041] The dynamic prompt is input into the fine-tuned large language model for multi-dimensional evaluation to generate item recommendation categories and explanations. The reason keywords of the explanations are reversely mapped to the attention peak area of ​​the contrastive learning representation to verify whether there is data support, and feature tracing is performed based on the verification results.

[0042] The graph path distance between the recommended items and the user's most recent n interaction items is checked for behavioral association, and interpretability is enhanced through the feature tracing and behavioral association. The final item recommendation results and recommendation reasons are output through the fine-tuned large language model.

[0043] A second aspect of the present invention provides a large model recommendation system based on contrastive learning enhancement and model fine-tuning, the system comprising a data representation module, a contrastive learning enhancement module, a dynamic prompt generation module, a large language model fine-tuning module, and an online reasoning and optimization module;

[0044] The data representation module is responsible for acquiring multimodal data, preprocessing and feature extraction of the multimodal data, and constructing text modal representations and structured modal representations of users and items;

[0045] The contrastive learning enhancement module performs representation enhancement through cross-modal contrastive learning and generates user and item preference representation information through adaptive multimodal fusion, thereby enhancing the personalized preference modeling capability.

[0046] The dynamic prompt generation module generates personalized prompts based on the user's real-time behavior and the user and item preference representation information, guiding the large language model to focus on the user's current interests;

[0047] The large language model fine-tuning module integrates the preference representation information of users and items with dynamic cross-attention, and inputs it into the preset large language model for recommendation task adaptation and fine-tuning;

[0048] When a user initiates a request, the online reasoning and optimization module dynamically generates prompts and calls a fine-tuned large language model to generate recommendation results. Combining contrastive learning representations with the reasoning capabilities of the large language model, it provides explainable reasons for recommendations, collects real-time user feedback, and adjusts the recommendation strategies of the contrastive learning module and the large language model.

[0049] Compared with the prior art, the present invention has the following beneficial effects:

[0050] The present invention achieves recommendation performance improvement and interpretability breakthroughs by integrating the multimodal representation enhancement of contrastive learning with the semantic reasoning ability of large language models. The user-item representation enhanced by contrastive learning significantly alleviates the data sparsity problem. Through dynamic prompt generation and multimodal fusion, while maintaining core preferences, the category coverage of the recommendation list is expanded, effectively solving the information cocoon problem. The hierarchical fine-tuning LLM reasoning supports thousands of concurrent personalized recommendation requests per second. The hierarchical fine-tuning strategy (bottom-level freezing + top-level adaptation) improves the accuracy of following recommendation instructions while retaining general language capabilities, and greatly improves the semantic consistency between the generated reasons and the real user preferences. By fusing contrastive learning representations and dynamic prompts through cross-attention, the accuracy of LLM's understanding of user preferences is improved. In the task of extracting product attributes, the fine-tuned model enhances the recognition ability of implicit features. The technical path of integrating contrastive learning and model fine-tuning provides a solution for personalized recommendation systems that combines high performance and interpretability. BRIEF DESCRIPTION OF THE DRAWINGS

[0051] In order to more clearly illustrate the technical solutions in the embodiments or exemplary embodiments of the present invention, the following briefly introduces the drawings required for use in the embodiments or exemplary descriptions. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained according to these drawings without paying any creative work.

[0052] Figure 1A flowchart of a large model recommendation method based on contrastive learning enhancement and model fine-tuning is shown;

[0053] Figure 2 A flowchart of representation enhancement via cross-modal contrastive learning is shown;

[0054] Figure 3 A flowchart of fine-tuning a preset large language model is shown;

[0055] Figure 4 Shown is a block diagram of a large model recommendation system based on contrastive learning enhancement and model fine-tuning. DETAILED DESCRIPTION

[0056] In order to more clearly understand the above-mentioned objects, features and advantages of the present invention, the present invention is further described in detail below in conjunction with the accompanying drawings and specific embodiments. It should be noted that, in the absence of conflict, the embodiments of the present application and the features therein can be combined with each other.

[0057] In the following description, many specific details are set forth to facilitate a full understanding of the present invention. However, the present invention may also be implemented in other ways different from those described herein. Therefore, the scope of protection of the present invention is not limited to the specific embodiments disclosed below.

[0058] like Figure 1 As shown, this embodiment provides a large model recommendation method based on contrastive learning enhancement and model fine-tuning, including:

[0059] S102, obtaining historical interaction behaviors of the user, preprocessing the historical interaction behaviors, and constructing text modal representations and structured modal representations of the user and the item;

[0060] S104, representation enhancement through cross-modal contrastive learning, and generating user and item preference representation information through adaptive multimodal fusion;

[0061] S106: Generate dynamic prompts based on the user's real-time behavior and the user and item preference representation information, fuse the user and item preference representation information with the dynamic prompts through cross-attention, and input them into a preset large language model for model fine-tuning;

[0062] S108, when the user initiates a request, dynamically generates prompts and calls the fine-tuned large language model to generate recommendation results, combining the contrastive learning representation and the reasoning ability of the large language model to provide explainable recommendation reasons.

[0063] It should be noted that user-item interaction data, text data, and structured relational data are obtained from recommendation-related data sources. User-item interaction data includes explicit user feedback (ratings, favorites) and implicit feedback (clicks, dwell time, purchase history) to construct a user-item interaction matrix. Text data includes user-generated content (comments, search queries) and item descriptions (titles, detail page copy), etc. Structured relational data includes social network relationships (user follow lists), item knowledge graphs (category-brand hierarchy), and spatiotemporal interaction records (recent browsing time). User interaction records are organized chronologically, discrete behaviors are weighted, and continuous behaviors are normalized to generate time-series behavior sequences. Sliding windows are used to segment very long sequences, and sparse sequences are filled with behaviors.

[0064] The item-side text is spliced ​​with the item title and key attributes from the text data, and the BERT pre-trained language model is used to obtain sentence-level representations to generate item semantic embedding representations. User historical comments are aggregated for user-tested text and spliced ​​with real-time search terms. A long text encoder is used to generate user portrait representations, and the item semantic embedding representations and user portrait representations are used as text modal representations. Users and items are regarded as two types of nodes, and interaction behaviors are used as edges to construct an interaction graph. Intermediate nodes such as categories and brands are preferably introduced as auxiliary nodes to enrich the relationship. The text modal representation is used to initialize the representations of user nodes and item nodes in the interaction graph, and a relational graph convolutional network is used to process heterogeneous edge types such as clicks, purchases, and follows. Information is propagated on the same type of edges in user-item interactions, and then node representations are updated across types of aggregation, such as user-social friends. The graph attention mechanism is used to identify important neighbor nodes, such as frequently interacting items, and weighting is performed during the aggregation process. Global topological representations of users and items are generated as structured modal representations through graph pooling.

[0065] Figure 2 A flowchart showing representation enhancement via cross-modal contrastive learning.

[0066] According to an embodiment of the present invention, representation enhancement is performed through cross-modal contrastive learning, specifically:

[0067] S202 uses a three-level contrastive learning framework to gradually enhance representations from fine-grained to coarse-grained. In intra-modal contrastive learning, different behavior sequences of the same user are used to construct positive samples, and similar behavior sequences of different users are used to construct negative samples. User-side contrastive learning aggregates the temporal behavior representations of the same user in the vector space. In item-side contrastive learning, different descriptions of the same item are used to construct positive samples, and descriptions of different items in the same category are used to construct negative samples. Item-side contrastive learning is used to obtain the core features of the items.

[0068] S204: In the user dimension comparison of cross-modal contrastive learning, positive samples are constructed using user text comment features and interaction graph features, and negative samples are constructed by randomly combining user modal representations. In the item dimension comparison, positive samples are constructed using item description texts, and negative samples are constructed using cross-modal combinations of popular and unpopular items. Cross-modal contrastive learning eliminates semantic differences between modalities.

[0069] S206, in interaction-level contrastive learning, positive samples are constructed using items that users actually interact with and similar items generated through graph diffusion, and negative samples are constructed using items of the same category but not interacted with. This captures user-item collaborative features through interaction-level contrastive learning;

[0070] S208 , performing progressive optimization of intra-modality contrastive learning, cross-modality contrastive learning, and interactive hierarchical contrastive learning to generate enhanced modal representations of users and items.

[0071] It should be noted that a three-level contrastive learning architecture is employed to gradually enhance representations from fine-grained to coarse-grained. The goal of intra-modal contrastive learning is to strengthen the discriminative features within a single modality. User-side contrastive learning constructs positive samples using different behavioral sequences of the same user, such as by segmenting click histories into time windows. It also uses similar behavioral sequences from different users, such as through Jaccard similarity sampling, to aggregate the temporal behavioral representations of the same user in vector space as contrastive targets. Item-side contrastive learning constructs positive samples using different descriptions of the same item, such as aggregated titles, detail pages, and review summaries. Textual descriptions of different items in the same category are used as negative samples. The contrastive goal is to improve the ability to extract core features from items. Intra-modal contrastive learning enhances the discriminative nature of single-modal features, improving the accuracy of temporal pattern recognition of user behavior sequences and the precision of extracting key features from item textual descriptions.

[0072] Cross-modal contrastive learning aims to bridge the semantic gap between modalities. In the user dimension, positive samples are constructed using user textual commentary features combined with inter-graph features. Negative samples are constructed using a random combination of user modal representations. Silent users (without textual data) are replaced with behavioral sequences. In the item dimension, positive samples are constructed using item descriptions, while negative samples are constructed using a cross-modal combination of popular and unpopular items. A gradient reversal layer is added to prevent adversarial training from being dominated by a particular modality. Cross-modal contrast effectively bridges the semantic gap, improves the spatial alignment of textual and graph representations, and addresses the problem of feature fragmentation.

[0073] Interaction-level contrastive learning aims to capture user-item collaboration information. Positive samples are constructed using real user-interacted items and similar items generated through graph diffusion. Hard negative samples are constructed by filtering items of the same category that have not been interacted with using a category classifier. Interaction-level contrastive learning improves coverage of long-tail items, particularly in cold-start scenarios, by mining hard negative samples.

[0074] It should be noted that a gating mechanism is used to dynamically generate fusion weights. For the enhanced text modal representations of users and items, the text weights are obtained by calculating the user comment frequency (such as comment frequency) and the richness of the item description text (description length, keyword density), normalized by the Sigmoid function, and complementary distribution are used to obtain the structured weights of the enhanced structured modal representations of users and items. Prior bias is introduced in cold start scenarios, such as increasing the text weights of new users by default. Fusion is performed at two levels to retain multi-granularity information. The text weights and structured weights are used to perform primary fusion using element-by-element weighting to ensure the stability of basic features. The enhanced text modal representations of users and items are used as queries, and the enhanced structured modal representations of users and items are used as key-value pairs to generate cross-attention between modalities. Advanced fusion is performed using the cross-attention, which can automatically identify complementary features and suppress redundant information. Based on the user's historical interaction behavior, a user interest memory is constructed. The sliding window method is used to store the mean of the text modal representation and structured modal representation of the user and item. The enhanced text modal representation and structured modal representation of the user and item are then similarly calculated in the user interest memory. The advanced fusion weights are adjusted based on the similarity. If the current behavior is highly consistent with long-term interests (such as continued purchase of the same brand), the weight of the dominant modality is increased. If a significant deviation is detected (such as suddenly browsing a new category), the fusion strategy is dynamically adjusted (such as enhancing the graph modality to capture behavioral changes), effectively solving the problem of interest drift while ensuring the continuity of recommendations. Through advanced fusion, cross-modal feature cross-pollination is achieved, and a multimodal interaction matrix is ​​established for low-rank decomposition to generate preference representation information for users and items.

[0075] It should be noted that the latest user interaction events are captured in real time as real-time user behavior, and the real-time user behavior is enhanced with spatiotemporal context according to the spatiotemporal dimensions of the interaction events, such as access time, device type, and geographic location, to generate a dense representation of the user's real-time behavior. The preference representation information of the current user and item is extracted from the user interest memory as the basic anchor point of the user's interest.

[0076] A bidirectional LSTM network is used to perform temporal modeling on the dense representation of users' real-time behavior. Forward propagation is used to obtain users' explicit behavior patterns, such as continuous purchases, and backpropagation is used to obtain changes in users' implicit needs. For example, by looking back at headphones, it is discovered that users are pursuing upgraded sound quality. The hidden states of the bidirectional outputs are spliced ​​to generate a comprehensive representation vector that contains the complete temporal context. The similarity between the comprehensive representation vector and the basic anchor points of user interests is calculated and normalized to item-level weights through Softmax. For example, a smart speaker that has been frequently interacted with recently may receive a high weight of 0.7, while an ordinary headphone that was viewed earlier may receive a weight of 0.1. BERT generates item representations from item descriptions generated by real-time user interactions. Secondary feature-level weighting is applied to these item representations to highlight key attributes, such as Dolby sound quality over red appearance. These item-level and feature-level weights are then weighted to generate a final item representation. This final item representation is clustered to generate a natural language description that serves as a user interest summary, including recent focus, preference evolution, and cross-modal associations. A scenario description is generated by combining temporal and spatial contexts, and actionable recommendation guidance is generated as task instructions based on quantity constraints and feature requirements. Structured dynamic prompts are generated based on these user interest summaries, scenario descriptions, and task instructions. An update mechanism is established for short-term interest tracking and long-term preference calibration. A sliding window is used to detect behavioral mutations and update interest anchors in real time. When a behavioral mutation is detected (e.g., clicking on three gaming laptops in a row), the weight of the relevant category is immediately increased. The forget gate mechanism of the LSTM is used to reduce the influence of old interests. Offline cluster analysis is performed at preset time intervals to identify stable preference patterns. User profile reconstruction is triggered when the distance between the new and old cluster centers exceeds a threshold.

[0077] Figure 3 A flowchart for fine-tuning a preset large language model is shown.

[0078] According to an embodiment of the present invention, the preference representation information of users and items is fused with dynamic prompts through cross-attention and input into a preset large language model for model fine-tuning, specifically:

[0079] S302: Align the user and item preference representation information with the dynamic prompts, use the text token as the query, and the user preference vector as the key-value pair to calculate the preference-prompt cross attention to obtain the fused prompt vector.

[0080] S304: Importing the hint vector into a preset large language model to obtain a hidden state, using the hidden state as a query and the item representation as a key-value pair to calculate item-context cross attention, and using the item-context cross attention to control the intensity of item information penetration;

[0081] S306: Perform a residual connection on the representation of the original preset large language model and the output of the cross attention, and import the fused representation after the residual connection into the preset large language model for fine-tuning, so as to maintain the general language understanding ability of the first 6 layers of the preset large language model unchanged;

[0082] S308: Add a low-rank adaptation module with a bottleneck structure between the Transformer layers for cross-modal adaptation, and use LoRA technology to apply low-rank incremental updates to the attention layer to perform intermediate layer adaptation adjustments;

[0083] S310, a new dual-branch task head for item recommendation sorting and list generation is added to the top-level task-specific layer, and fine-tuning training of the preset large language model is achieved through staged training.

[0084] It should be noted that the underlying parameters are frozen, keeping the parameters of the first six layers of base models such as BERT fixed. Only the affine transformation parameters of the LayerNorm layer are updated to adapt to changes in feature distribution. During intermediate layer adaptation, an adaptation module with a bottleneck dimension of 64 is added after each Transformer layer's FFN, and gradient clipping is used to prevent cross-modal training instability. A rank-8 factorization update is applied to the Q / K matrices, with an initial learning rate set to three times that of the backbone network. LoRa incremental updates are performed, reducing the number of new parameters through weight sharing. The newly added multimodal prediction head consists of a ranking branch constructed with a three-layer MLP and a generation branch constructed with a lightweight Transformer decoder. During fine-tuning training of a pre-built large language model, only the cross-attention layer and adapter are trained during the modality alignment phase, aiming to minimize the cosine distance between the large language text space and the multimodal space. During the instruction adaptation phase, the intermediate layer parameters are unfrozen to focus on optimizing prompt responsiveness. During the recommendation fine-tuning phase, full-task joint training is performed, with dynamic adjustment of loss weights, initially focusing on representation alignment and later on focusing on recommendation accuracy.

[0085] When a user initiates a request, prompts are dynamically generated, and the fine-tuned large language model is called to generate recommendation results. The contrastive learning representation and the reasoning ability of the large language model are combined to provide explainable recommendation reasons. Specifically, when a user initiates a request, the preference representation information enhanced by contrastive learning is fused with real-time context features, and the fused user vector is used to perform approximate nearest neighbor search in the item library. The user fusion vector is spliced ​​with the candidate item features to construct a dynamic prompt; the dynamic prompt is input into the fine-tuned large language model for multi-dimensional evaluation to generate item recommendation categories and explanations. The recommendation list is formatted as a JSON structure, including product ID, matching score, and key feature labels. The explanation adopts a two-layer structure of feature association and behavioral evidence. For example, the Bose QC45 headphones are recommended for the following reasons: they meet the noise reduction needs of the past three purchases (behavioral evidence); they weigh 250g and are suitable for commuting (scenario adaptation); and they are the same brand as the Bose speakers in the collection (brand preference). The reasoning keywords are reverse-mapped to the attention peak region of the contrastive learning representation to verify whether there is data support. Based on the verification results, feature traceability is performed. The graph path distance between the recommended item and the user's most recent n interactive items is checked for behavioral association. This feature traceability and behavioral association enhance interpretability, and the final item recommendation results and recommendation reasons are output through a fine-tuned large language model. When a user rejects a recommendation, the rejection keywords are recorded, and the ranking weight of subsequent recommendations is adjusted in real time. Negative feedback data is aggregated weekly, and the negative sample distribution in contrastive learning is recalculated to strengthen the modeling of difficult scenarios (such as price-sensitive users).

[0086] Figure 4 Shown is a block diagram of a large model recommendation system based on contrastive learning enhancement and model fine-tuning.

[0087] A second aspect of the present invention provides a large model recommendation system 4 based on contrastive learning enhancement and model fine-tuning, the system comprising a data representation module 401, a contrastive learning enhancement module 402, a dynamic prompt generation module 403, a large language model fine-tuning module 404, and an online reasoning and optimization module 405;

[0088] The data representation module is responsible for acquiring multimodal data, preprocessing and feature extraction of the multimodal data, and constructing text modal representations and structured modal representations of users and items;

[0089] The contrastive learning enhancement module performs representation enhancement through cross-modal contrastive learning and generates user and item preference representation information through adaptive multimodal fusion, thereby enhancing the personalized preference modeling capability.

[0090] The dynamic prompt generation module generates personalized prompts based on the user's real-time behavior and the user and item preference representation information, guiding the large language model to focus on the user's current interests;

[0091] The large language model fine-tuning module integrates the preference representation information of users and items with dynamic cross-attention, and inputs it into the preset large language model for recommendation task adaptation and fine-tuning;

[0092] When a user initiates a request, the online reasoning and optimization module dynamically generates prompts and calls a fine-tuned large language model to generate recommendation results. Combining contrastive learning representations with the reasoning capabilities of the large language model, it provides explainable reasons for recommendations, collects real-time user feedback, and adjusts the recommendation strategies of the contrastive learning module and the large language model.

[0093] The third aspect of the present invention provides a computer-readable storage medium, which includes a large model recommendation method program based on contrastive learning enhancement and model fine-tuning. When the large model recommendation method program based on contrastive learning enhancement and model fine-tuning is executed by a processor, it implements the steps of the large model recommendation method based on contrastive learning enhancement and model fine-tuning.

[0094] In the several embodiments provided in this application, it should be understood that the disclosed methods and systems can be implemented in other ways. The system embodiments described above are merely illustrative. For example, the division of the units is merely a logical function division. In actual implementation, there may be other division methods, such as: multiple units or components can be combined, or can be integrated into another system, or some features can be ignored, or not executed. In addition, the coupling, direct coupling, or communication connection between the components shown or discussed can be through some interfaces, indirect coupling or communication connection of devices or units, which can be electrical, mechanical or other forms. In addition, the functional units in the various embodiments of the present invention can all be integrated into one processing unit, or each unit can be a separate unit, or two or more units can be integrated into one unit; the above-mentioned integrated units can be implemented in the form of hardware or in the form of hardware plus software functional units.

[0095] Those skilled in the art will understand that all or part of the steps of implementing the above-mentioned method embodiment can be completed by hardware related to program instructions, and the aforementioned program can be stored in a computer-readable storage medium. When the program is executed, it executes the steps of the above-mentioned method embodiment; and the aforementioned storage medium includes: mobile storage devices, read-only memory (ROM, Read-Only Memory), random access memory (RAM, Random Access Memory), disks or optical disks, and other media that can store program codes.

[0096] The above description is only a specific embodiment of the present invention, but the scope of protection of the present invention is not limited thereto. Any technician familiar with this technical field can easily think of changes or replacements within the technical scope disclosed by the present invention, which should be covered by the scope of protection of the present invention.

Claims

1. A large model recommendation method based on contrastive learning enhancement and model fine-tuning, characterized by: The following steps are involved: Obtaining the user's historical interaction behavior, preprocessing the historical interaction behavior, and constructing text modal representations and structured modal representations of the user and items; Representation enhancement is performed through cross-modal contrastive learning, and preference representation information of users and items is generated through adaptive multimodal fusion; Generate dynamic prompts based on real-time user behavior and user and item preference representation information. Use cross-attention to fuse user and item preference representation information with dynamic prompts and input them into a preset large language model for model fine-tuning. When a user initiates a request, prompts are dynamically generated, and the fine-tuned large language model is called to generate recommendation results. By combining contrastive learning representations and the reasoning capabilities of the large language model, explainable reasons for the recommendation are provided.

2. The large model recommendation method based on contrastive learning enhancement and model fine-tuning according to claim 1 is characterized in that: Obtain the user's historical interaction behavior, preprocess the historical interaction behavior, and construct text modal representations and structured modal representations of users and items. Specifically: Based on recommendation-related data sources, we obtain user-item interaction data, text data, and structured relationship data. We organize user interaction records in chronological order, perform weighted processing on discrete behaviors, and normalize continuous behaviors to generate a time-series behavior sequence. The item-side text is spliced ​​with the item title and key attributes in the text data, a pre-trained language model is used to obtain sentence-level representation, and an item semantic embedding representation is generated. The user-side text is aggregated with user historical comments and spliced ​​with real-time search terms. A long text encoder is used to generate a user profile representation. The item semantic embedding representation and the user profile representation are used as text modality representations; We construct an interaction graph by treating users and items as two types of nodes and interaction behaviors as edges. We use text modality representations to initialize the representations of user nodes and item nodes in the interaction graph. We use a relational graph convolutional network to process heterogeneous edge types, propagate information on edges of the same type, and then aggregate and update node representations across different types. The graph attention mechanism is used to identify important neighbor nodes, which are weighted during the aggregation process. Graph pooling is used to generate global topological representations of users and items as structured modal representations.

3. The large model recommendation method based on contrastive learning enhancement and model fine-tuning according to claim 1 is characterized in that: Representation enhancement is performed through cross-modal contrastive learning, specifically: A three-level contrastive learning framework is used to gradually enhance representations from fine-grained to coarse-grained. In intra-modal contrastive learning, different behavior sequences of the same user are used to construct positive samples, while similar behavior sequences of different users are used to construct negative samples. Through user-side contrastive learning, the temporal behavior representations of the same user are clustered in the vector space. In item-side comparison, different descriptions of the same item are used to construct positive samples, while descriptions of different items in the same category are used to construct negative samples. This allows for learning to obtain the core features of items through item-side comparison. In the user dimension comparison of cross-modal contrastive learning, positive samples are constructed using user text comment features and interaction graph features, and negative samples are constructed by randomly combining user modal representations. In the item dimension comparison, positive samples are constructed using item description text, and negative samples are constructed using a cross-modal combination of popular and unpopular items. Cross-modal contrastive learning eliminates semantic differences between modalities. In interaction-level contrastive learning, positive samples are constructed by using items that users actually interact with and similar items generated through graph diffusion, while negative samples are constructed by using items of the same category but without interaction. This method captures the collaborative features between users and items. Perform progressive optimization of intra-modal contrastive learning, cross-modal contrastive learning, and interaction-level contrastive learning to generate enhanced modal representations of users and items.

4. The large model recommendation method based on contrastive learning enhancement and model fine-tuning according to claim 1 is characterized in that: Generate user and item preference representation information through adaptive multimodal fusion, specifically: A gating mechanism is used to dynamically generate fusion weights. For the enhanced text modal representations of users and items, the text weights are calculated by calculating the frequency of user comments and the richness of item descriptions. Sigmoid function normalization is used, and complementary allocation is used to obtain the structural weights of the enhanced structured modal representations of users and items. Prior bias is introduced in cold start scenarios. Using the text weight and structured weight to perform primary fusion using element-by-element weighting, and using the text modal representation after user and item enhancement as the query and the structured modal representation after user and item enhancement as the key-value pair, generating cross attention between modalities, and using the cross attention to perform advanced fusion; A user interest memory is constructed based on the user's historical interaction behavior. The average of the text modal representation and structured modal representation of the user and item is stored using a sliding window method. The enhanced text modal representation and structured modal representation of the user and item are used to calculate the similarity in the user interest memory, and the high-level fusion weight is adjusted according to the similarity. Through advanced fusion, cross-modal feature cross-fertilization is achieved, a multimodal interaction matrix is ​​established for low-rank decomposition, and preference representation information of users and items is generated.

5. The large model recommendation method based on contrastive learning enhancement and model fine-tuning according to claim 1 is characterized in that: Generate dynamic prompts based on real-time user behavior and user and item preference representation information, specifically: Capture the user's latest interaction events in real time as the user's real-time behavior, enhance the user's real-time behavior with spatiotemporal context based on the spatiotemporal dimensions of the interaction events, generate a dense representation of the user's real-time behavior, and extract the current user and item preference representation information from the user interest memory as the basic anchor point of the user's interest; A bidirectional LSTM network is used to perform temporal modeling on the dense representation of users' real-time behavior. The forward propagation method captures the users' explicit behavior patterns, while the backpropagation method captures the changes in their implicit needs. The hidden states of the bidirectional outputs are concatenated to generate a comprehensive representation vector containing the complete temporal context. The similarity between the comprehensive representation vector and the basic anchor points of user interest is calculated and normalized to item-level weights through Softmax. The item description text of the user's real-time behavioral interaction is generated through BERT to generate item representations. Secondary feature-level weights are applied to the item representations to highlight key attributes. The item-level weights and feature-level weights are used to weight the final item representations, the final item representations are clustered, and a natural language description is generated as a user interest summary. A scenario description is generated by combining the temporal scenario and the spatial scenario, and an executable recommendation guide is generated as a task instruction based on quantity restrictions and feature requirements. Based on the user interest summary, scenario description and task instructions, structured dynamic prompts are generated, an update mechanism for short-term interest tracking and long-term preference calibration is established, behavioral mutation detection is performed through a sliding window, interest anchor points are updated in real time, offline clustering analysis is performed based on preset time intervals, stable preference patterns are identified, and user portrait reconstruction is triggered when the distance between the new and old cluster centers exceeds a threshold.

6. The large model recommendation method based on contrastive learning enhancement and model fine-tuning according to claim 1 is characterized in that: The user and item preference representation information is fused with dynamic prompts through cross-attention and input into the preset large language model for model fine-tuning. Specifically: Align the user and item preference representation information with the dynamic prompts, use the text token as the query and the user preference vector as the key-value pair to calculate the preference-prompt cross attention and obtain the fused prompt vector; Importing the hint vector into a preset large language model to obtain a hidden state, using the hidden state as a query and the item representation as a key-value pair to calculate item-context cross attention, and using the item-context cross attention to control the intensity of item information penetration; Perform a residual connection between the representation of the original preset large language model and the output of the cross attention, and import the fused representation of the residual connection into the preset large language model for fine-tuning, maintaining the general language understanding ability of the first 6 layers of the preset large language model unchanged; A low-rank adaptation module with a bottleneck structure is added between the Transformer layers for cross-modal adaptation. The LoRA technology is used to apply low-rank incremental updates to the attention layer to perform intermediate layer adaptation adjustments. A new dual-branch task head for item recommendation sorting and list generation is added to the top-level task-specific layer, and fine-tuning training of the preset large language model is achieved through staged training.

7. The large model recommendation method based on contrastive learning enhancement and model fine-tuning according to claim 1 is characterized in that: When a user initiates a request, a prompt is dynamically generated, and a fine-tuned large language model is called to generate recommendation results. By combining contrastive learning representations and the reasoning capabilities of the large language model, explainable reasons for the recommendation are provided. Specifically: When a user initiates a request, the preference representation information enhanced by contrastive learning is fused with real-time context features. The fused user vector is used to perform an approximate nearest neighbor search in the item library. The fused user vector is then concatenated with the candidate item features to construct a dynamic prompt. The dynamic prompt is input into the fine-tuned large language model for multi-dimensional evaluation to generate item recommendation categories and explanations. The reason keywords of the explanations are reversely mapped to the attention peak area of ​​the contrastive learning representation to verify whether there is data support, and feature tracing is performed based on the verification results. The graph path distance between the recommended items and the user's most recent n interaction items is checked for behavioral association, and interpretability is enhanced through the feature tracing and behavioral association. The final item recommendation results and recommendation reasons are output through the fine-tuned large language model.

8. A large model recommendation system based on contrastive learning enhancement and model fine-tuning, characterized by: The large model recommendation method based on contrastive learning enhancement and model fine-tuning as described in any one of claims 1 to 7 is applied, the system comprising a data representation module, a contrastive learning enhancement module, a dynamic prompt generation module, a large language model fine-tuning module, and an online reasoning and optimization module; The data representation module is responsible for acquiring multimodal data, preprocessing and feature extraction of the multimodal data, and constructing text modal representations and structured modal representations of users and items; The contrastive learning enhancement module performs representation enhancement through cross-modal contrastive learning and generates user and item preference representation information through adaptive multimodal fusion, thereby enhancing the personalized preference modeling capability. The dynamic prompt generation module generates personalized prompts based on the user's real-time behavior and the user and item preference representation information, guiding the large language model to focus on the user's current interests; The large language model fine-tuning module integrates the preference representation information of users and items with dynamic cross-attention, and inputs it into the preset large language model for recommendation task adaptation and fine-tuning; When a user initiates a request, the online reasoning and optimization module dynamically generates prompts and calls a fine-tuned large language model to generate recommendation results. Combining contrastive learning representations with the reasoning capabilities of the large language model, it provides explainable reasons for recommendations, collects real-time user feedback, and adjusts the recommendation strategies of the contrastive learning module and the large language model.

Images