Personalized content generation method based on multi-modal large model and user behavior hidden vector
By constructing latent vectors of user behavior and a multimodal large model, the problem of one-sided user interest profiling in existing technologies is solved, enabling cross-platform and cross-modal personalized content generation. It can deeply understand user interests and generate highly relevant personalized content, and has continuous optimization capabilities.
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- SUZHOU AJIK INTELLIGENT TECHNOLOGY CO LTD
- Filing Date
- 2026-03-18
- Publication Date
- 2026-06-12
AI Technical Summary
Existing personalization systems cannot effectively integrate fragmented user behaviors across multiple platforms, nor can they deeply integrate preferences for different modalities such as text, images, and videos. This results in a one-sided user interest profile, and the generated recommendations or content only reflect local interests, failing to achieve in-depth understanding and satisfaction across domains and modalities.
By constructing latent vectors of user behavior and using a multimodal large model for content generation, including latent vector construction of user behavior, multimodal content understanding and representation, personalized preference alignment and fusion, and conditional content generation, an improved Transformer architecture and cross-variable attention mechanism are used to process ultra-long behavior sequences. Combined with a pre-trained multimodal large model for unified encoding and feature extraction, content that conforms to users' personalized preferences is generated.
It enables cross-platform and cross-modal deep personalized content generation, comprehensively and deeply characterizes user interests and preferences, generates content that closely matches user needs, and has the ability to continuously evolve and optimize to adapt to the dynamic changes in user interests.
Smart Images
Figure CN122198140A_ABST
Abstract
Description
Technical Field
[0001] This invention belongs to the field of artificial intelligence and machine learning technology, specifically a personalized content generation method based on multimodal large models and latent vectors of user behavior. Background Technology
[0002] By deeply mining users' cross-modal historical behavior across multiple platforms, a dynamic latent vector of user interests is constructed, which is then used to precisely control the generation process of a multimodal large model, enabling the automatic creation of personalized content.
[0003] Existing personalization systems are often limited to a single platform or a single modality, failing to effectively integrate fragmented user behaviors across multiple platforms or deeply integrate preferences reflected in different modalities such as text, images, and videos. This results in a one-sided and fragmented user interest profile, with generated recommendations or content only reflecting local interests, failing to achieve in-depth understanding and satisfaction across domains and modalities.
[0004] Based on this, the present invention designs a personalized content generation method based on a multimodal large model and latent vectors of user behavior to solve the above problems. Summary of the Invention
[0005] To overcome the shortcomings of existing technologies, this invention proposes a personalized content generation method based on a multimodal large model and latent vectors of user behavior. This invention primarily addresses the problem that existing personalized systems are often limited to a single platform or a single modality, failing to effectively integrate fragmented user behavior across multiple platforms or deeply integrate preferences reflected in different modalities such as text, images, and videos. This results in fragmented and incomplete user interest profiles, with generated recommendations or content only reflecting localized interests, failing to achieve deep understanding and satisfaction across domains and modalities.
[0006] The technical solution adopted by this invention to solve its technical problem is: a personalized content generation method based on a multimodal large model and latent vectors of user behavior, comprising:
[0007] S1. User behavior latent vector construction steps: Collect the target user's historical interaction behavior sequence on multiple content platforms. The historical interaction behavior includes clicks, browsing duration, comments, sharing and collection. After cleaning and standardizing the historical interaction behavior sequence, it is input into a behavior encoding network for training to generate a fixed-dimensional user behavior latent vector that represents the user's long-term interests and preferences.
[0008] S2. Multimodal content understanding and representation steps: Obtain original content data associated with the user's historical interaction behavior. The original content data includes at least one of text, image, video, and audio. Using a pre-trained multimodal large model, uniformly encode and extract features from the original content data to generate a content feature vector in the same semantic space as the user behavior latent vector.
[0009] S3. Personalized Preference Alignment and Fusion Step: The user behavior latent vector and the content feature vector generated in step S2 are fused and calculated to generate a personalized condition vector. The fusion calculation includes vector concatenation, attention-weighted summation, or nonlinear mapping through a lightweight adaptation network.
[0010] S4. Conditional content generation step: The personalized condition vector is used as the generation condition and input into a generative multimodal large model. The generative multimodal large model generates target content that conforms to the personalized preferences of the target user based on the input text prompts and the personalized condition vector.
[0011] Preferably, in step S1, the behavior encoding network is constructed using a sequence model based on an improved Transformer architecture, which handles extremely long historical interaction behavior sequences in the following way:
[0012] a. Sequence Segmentation Module: Divides extremely long historical interaction sequences into multiple continuous subsequences of fixed length according to time order;
[0013] b. Two-layer attention encoding module: including a local attention layer and a global attention layer;
[0014] Local attention layer: Within each subsequence, a key sparse attention mechanism is used to calculate the dependencies between historical interaction behaviors within the subsequence. The key sparse attention mechanism filters noise and redundant information and focuses on key behavioral features by retaining the top k columns with the highest weight values in the attention weight matrix.
[0015] Global attention layer: Above the local attention layer, a cross-variable attention mechanism is adopted, one-dimensional convolution operation is introduced to enhance the modeling of continuous features between time steps, and the attention calculation method is reconstructed to capture long-distance dependencies between different subsequences;
[0016] c. Improved position encoding module: In the input of the local attention layer and the global attention layer, a position encoding vector based on trigonometric functions is added to each historical interaction behavior to ensure that the model can clearly distinguish the absolute order between behaviors.
[0017] Preferably, the sequence segmentation module uses a sliding window method for segmentation, with overlapping portions between adjacent subsequences to preserve contextual continuity.
[0018] Preferably, the global attention layer employs a cross-variable attention mechanism, with a computational complexity of O(n log n). ,in For sequence length, The feature dimension is significantly lower than that of traditional self-attention mechanisms. ;
[0019] The global attention layer employs a cross-variable attention mechanism, and its core computation process is as follows:
[0020] Input representation:
[0021] Let the input sequence be ,in For sequence length, For feature dimensions.
[0022] Preferably, a self-attention mechanism is used to compute the query matrix. AND key matrix The dot product has a complexity of O(n log n). ;
[0023] The cross-variable attention mechanism reconstructs the computation method, transposing the query matrix to... The key matrix remains unchanged. Calculate the dot product of the two:
[0024]
[0025] in, This is a cross-variable attention weight matrix, representing the correlation between different variables;
[0026] Complexity analysis:
[0027] The computational complexity of the above is ,in For fixed feature dimensions, The sequence length;
[0028] Because in actual multivariate time series Therefore, This significantly reduces the computational burden of processing long sequences;
[0029] Output weighting:
[0030] Through attention weight matrix Log-value matrix After weighting, the enhanced feature representation is obtained:
[0031]
[0032] The output preserves the correlation information between variables and supports subsequent sequence modeling.
[0033] Preferably, the behavior encoding network is pre-trained through a self-supervised learning task, which includes next behavior prediction or behavior sequence mask reconstruction.
[0034] Preferably, in step S2, the pre-trained multimodal large model is a vision-language large model, which is trained through contrastive learning or cross-modal mask modeling tasks to achieve alignment of text and visual features in a unified vector space.
[0035] Preferably, in step S3, the fusion calculation specifically employs an attention mechanism network to calculate the attention weights of the user behavior latent vectors to each vector in the historical content feature vector sequence, and performs a weighted summation based on the attention weights to generate the personalized condition vector.
[0036] Preferably, the generative multimodal large model in step S4 is a diffusion model or an autoregressive large language model. When the generative multimodal large model is a diffusion model, the personalized conditional vector is injected into multiple sampling steps of the denoising process through a cross-attention mechanism.
[0037] Preferably, after step S4, the method further includes: S5. Feedback optimization step: pushing the generated target content to the target user and collecting new user interaction behaviors with the target content; based on the new interaction behaviors, updating the latent vector of user behavior of the target user and / or the parameters of the fusion computing network through online learning or incremental learning.
[0038] The beneficial effects of this invention are as follows:
[0039] 1. In this invention, deep personalization across platforms and modalities is achieved. By integrating the diverse behaviors of users on different content platforms and combining multimodal content understanding such as text, images, and videos, it is possible to comprehensively and deeply depict the user's interests and preferences, thereby generating content that truly meets the user's personalized needs.
[0040] 2. In this invention, extremely long user behavior sequences are efficiently processed using an improved Transformer architecture, particularly a cross-variable attention mechanism, which reduces computational complexity from traditional methods. Significantly reduced to Due to feature dimension Much smaller than the sequence length This method can efficiently and feasiblely model ultra-long historical behavior sequences, capture long-term interests, and effectively preserve the contextual continuity and key information of behavior sequences while filtering out noise through sliding window overlapping segmentation and key sparse attention, thereby reducing computational burden.
[0041] 3. In this invention, the precise alignment of interest representation and content semantics is achieved. By using a pre-trained multimodal large model, the original content of different modalities is mapped to a unified semantic vector space, so that the latent vector of user behavior extracted from user behavior and the content feature vector extracted from content are in the same space, laying a solid foundation for subsequent precise preference alignment and fusion.
[0042] 4. In this invention, the generated content is highly relevant and controllable: Through fusion methods such as attention mechanisms, the generated personalized condition vector can dynamically and selectively combine the user's long-term interests with specific content features. This vector is then injected as a condition into the generation model such as the diffusion model or the large language model, so that the final generated text, images and other content not only closely match the user's historical preferences, but also accurately respond to real-time text prompts, achieving a balance between personalization and controllable generation.
[0043] 5. In this invention, the system has the ability to continuously evolve and optimize. It is designed with feedback optimization steps. Through online learning or incremental learning, it uses the user's new interactive behavior with the generated content to update the latent vector of user behavior and model parameters in real time or periodically. This enables the system to adapt to the dynamic changes in user interests, realize the self-iteration of the model and the continuous improvement of personalized effects, and form a closed-loop optimization system. Attached Figure Description
[0044] The invention will now be further described with reference to the accompanying drawings.
[0045] Figure 1 This is a technical flowchart of the present invention;
[0046] Figure 2 This is a flowchart illustrating the present invention;
[0047] Figure 3 This is a flowchart illustrating the historical interaction sequence in this invention. Detailed Implementation
[0048] To make the technical means, creative features, objectives and effects of this invention easier to understand, the invention will be further described below in conjunction with specific embodiments.
[0049] Example 1
[0050] Example 1:
[0051] Multimodal graphic content generation based on social media browsing history;
[0052] In this embodiment, the target user is a food enthusiast active on multiple social media platforms, and the system performs the following steps:
[0053] User behavior latent vector construction: The system collects the user's historical interaction behavior sequence on food-related posts over the past year, including clicks, long-term browsing, collection, and sharing. After cleaning and standardization, the sequence is input into a behavior encoding network with an improved Transformer architecture (as described in claim 2, using sequence segmentation and a two-layer attention encoding module) for training. The network finally outputs a 512-dimensional vector as a latent vector representing the user's long-term interests in "high-value desserts" and "local specialty snacks".
[0054] Multimodal content understanding and representation: The system acquires the original content data contained in the posts that the user has interacted with in the past, including food pictures (images) and captions (text). Using a pre-trained visual-language large model (such as CLIP or ViLT), these images and texts are uniformly encoded to generate a series of content feature vectors in the same semantic space as the latent vectors of user behavior.
[0055] Personalized Preference Alignment and Fusion: The system inputs the user behavior latent vector obtained in step 1 and a series of historical content feature vectors obtained in step 2 into an attention mechanism network (as described in claim 8) for fusion. The network calculates the attention weight of the user latent vector to each historical content vector and finds that the user pays more attention to images with "bright colors" and "containing strawberry elements". Based on this, the historical content features are weighted and summed to generate a personalized condition vector.
[0056] Conditional content generation: When a user enters a short text prompt such as "afternoon tea recommendation", the system inputs the prompt along with the personalized condition vector into a generative multimodal large model (such as the text diffusion model StableDiffusion). Based on the guidance of the personalized condition vector, the model generates a beautiful "strawberry mousse cake" image and detailed description that matches the user's preferences.
[0057] Feedback optimization (optional): When a user "favorites" the generated content, the system uses this new interaction behavior to fine-tune some parameters of the behavior encoding network through incremental learning, and updates the user's latent vector of user behavior to make its preference representation of "mousse" type desserts stronger.
[0058] Example 2
[0059] Example 2:
[0060] Personalized short video ad generation based on e-commerce platform behavior sequences;
[0061] In this embodiment, the target user is a consumer with a large number of shopping and browsing records on an e-commerce platform. The system performs the following steps:
[0062] User behavior latent vector construction: The system integrates multiple historical interaction behavior sequences of users on different e-commerce apps, including browsing, adding to cart, purchasing, and watching product detail videos. Due to the excessive length of the sequences, the behavior encoding network first segments them into overlapping subsequences using a sliding window (as described in claim 3). Then, the network processes the sequences through a two-layer attention encoding module (local key sparse attention focuses on key products, and global cross-variable attention captures the long-term interest evolution across subsequences), ultimately generating a user behavior latent vector representing the user's preferences for multiple categories such as "outdoor sports equipment" and "high-end photography equipment".
[0063] Multimodal content understanding and representation: The system acquires product detail page data that users have interacted with, including product copy (text), promotional images (images), and introductory videos (videos). Using a pre-trained multimodal large model, the system extracts features from these multimodal contents and encodes them uniformly to generate a series of content feature vectors.
[0064] Personalized Preference Alignment and Fusion: The system fuses the latent vector of user behavior with the feature vector of historical content through a lightweight adaptation network (the nonlinear mapping method described in claim 3). This network learns to map multiple abstract preferences of users for "lightweight" and "waterproof performance" to specific content features and outputs a personalized condition vector.
[0065] Conditional content generation: The advertising system needs to generate a 15-second short video ad for the user. The system inputs the ad theme text prompt (such as "Conquer the mountains and wilderness") and personalized conditional vectors into a video generation diffusion model (such as Sora or a similar model). Under the control of personalized conditions, the model generates a short video of the main character using a lightweight waterproof tent and a high-end action camera for outdoor adventure. The narration and subtitles also reflect the emphasis on the user's preferred keywords.
[0066] Feedback optimization (optional): If a user watches a generated video ad but does not click on it, the system uses this negative feedback to quickly adjust the parameters of the fusion computing network through online learning. In the next generation, it reduces the weight of features related to "high-end photography equipment" and tries to strengthen other related points of interest.
[0067] Example 3
[0068] Example 3:
[0069] Personalized music clip generation based on cross-platform entertainment consumption behavior
[0070] In this embodiment, the target user frequently listens to music on music streaming platforms and watches video clips on video platforms. The system performs the following steps:
[0071] User behavior latent vector construction: The system collects users' listening history on music apps (including complete listening, favorites, and playlist creation) and their historical interaction sequences on video platforms (including playback, likes, and comments) of movie clips and concert excerpts. The behavior encoding network adopts a sequence model based on an improved Transformer (as described in claim 2), utilizing its global attention layer's cross-variable attention mechanism (as described in claims 4 and 5, with a computational complexity of O(n log n)). To efficiently model the cross-modal long-term association between music style preferences (such as "movie soundtrack" and "post-rock") and visual scene preferences (such as "grand aerial photography" and "quiet solitude") in long sequences, and finally output the latent vector of user behavior;
[0072] Multimodal content understanding and representation: The system acquires the audio of songs that users have listened to, the video images they have watched, and related tags. It uses a pre-trained multimodal large model (such as an audio-visual-language model trained through contrastive learning) to uniformly encode the audio, video frames, and text tags to generate content feature vectors.
[0073] Personalized preference alignment and fusion: The fusion is performed by concatenating vectors and then connecting them to a fully connected layer (as described in claim 3). The latent vector of user behavior is concatenated with the feature vector of recent interaction content and compressed to generate a personalized conditional vector. This vector incorporates the user's cross-modal emotional preferences (such as "epic feeling" and "loneliness").
[0074] Conditional content generation: The user inputs the prompt "background sound suitable for reading", and the system inputs the prompt and personalized condition vector into an autoregressive large language model (such as MusicLM or AudioLDM audio generation model). Based on the conditions, the model generates a pure music clip that incorporates the user's preferences, with a slight movie soundtrack style and ambient sound effects.
[0075] Feedback optimization (optional): The system periodically collects user interaction data on newly generated music clips and incrementally pre-trains the behavior encoding network through a self-supervised learning task (mask reconstruction as described in claim 6), enabling the latent vectors to dynamically capture subtle changes in user interests.
[0076] Example 4
[0077] Personalized image and text content generation for social media;
[0078] Application scenario: Generating personalized image captions for Weibo users;
[0079] Implementation steps:
[0080] User behavior latent vector construction;
[0081] Collect a user's interaction behavior sequence on platforms such as Weibo, Zhihu, and Xiaohongshu, including behaviors such as clicking on articles, reading time, commenting and liking, and sharing and forwarding;
[0082] Clean the behavior sequence: remove abnormal clicks (duration < 1 second), standardize timestamps, and normalize behavior weights;
[0083] The processed sequence input is used to improve the behavior encoding network of the Transformer architecture:
[0084] Sequence segmentation: The behavioral records of the past 6 months are divided into subsequences of 200 behaviors each using a sliding window, with an overlap of 50 behaviors;
[0085] Local attention: Use key sparse attention within each subsequence to retain the top 20% of key behaviors;
[0086] Global attention: Employs a cross-variable attention mechanism (complexity) ), capturing the evolution of interests across time periods;
[0087] Positional encoding: Add absolute positional encoding to each action;
[0088] After self-supervised pre-training (masked behavior prediction task), a 768-dimensional latent vector of user behavior is output.
[0089] Multimodal content understanding and representation;
[0090] Get 500 pieces of text and image content related to the user's historical interactions;
[0091] Unified encoding is performed using the CLIP-ViT-L / 14 visual language large model:
[0092] Text section: Extract text features from the title and body text;
[0093] Image section: Extract visual features from the accompanying images;
[0094] The multimodal features are projected onto the same 768-dimensional semantic space to obtain a sequence of content feature vectors;
[0095] Personalized preference alignment and integration;
[0096] Fusion is achieved using an attention mechanism network:
[0097] Use the implicit vector of user behavior as the query vector;
[0098] Calculate the attention weights for 500 historical content feature vectors;
[0099] We found that users place a high value on content related to "technology and digital products" and "minimalist design".
[0100] Weighted summation generates personalized conditional vectors;
[0101] Conditional content generation;
[0102] User input suggestion: "Recommend a keyboard suitable for programmers";
[0103] Input the prompt words and personalized conditional vectors into the StableDiffusionXL diffusion model;
[0104] Personalized conditions are injected into the denoising process through a cross-attention mechanism;
[0105] Result: A minimalist image of a mechanical keyboard with captions emphasizing features that appeal to users, such as "silent switches" and "programmable shortcut keys."
[0106] Feedback optimization;
[0107] The user clicks "favorite" on the generated content;
[0108] Based on this positive feedback, the behavior encoding network is fine-tuned through online learning;
[0109] Update the latent vector of user behavior and strengthen the weight of interest related to "peripheral equipment".
[0110] Example 5
[0111] Personalized video ad generation for e-commerce platforms;
[0112] Application scenario: Generating personalized product recommendation videos for Taobao users;
[0113] Implementation steps:
[0114] User behavior latent vector construction;
[0115] Collect user behavior sequences on Taobao, JD.com, and Douyin e-commerce platforms:
[0116] Browsing the product details page (duration > 30 seconds is considered valid);
[0117] Adding to cart, placing an order, and making a payment;
[0118] Completion rate of watching product demonstration videos;
[0119] The behavior encoding network employs a two-layer attention architecture:
[0120] Local layer: Identifies the core decision-making behaviors in a single shopping session;
[0121] Global layer: Use cross-variable attention analysis to analyze the consistency of shopping preferences across platforms;
[0122] Output a 512-dimensional latent vector of user behavior, representing multiple long-term preferences for "maternal and infant products" and "organic food";
[0123] Multimodal content understanding and representation;
[0124] Get the 300 product pages that the user has viewed in the past;
[0125] Feature extraction using the Florence-2 multimodal large model:
[0126] Textual features of product titles and descriptions;
[0127] Visual characteristics of the main product image and product detail images;
[0128] Keyframe features of product demonstration videos;
[0129] Align all features to a 512-dimensional shared space;
[0130] Personalized preference alignment and integration;
[0131] Nonlinear mapping using lightweight adaptor networks:
[0132] A 3-layer MLP network with hidden layer dimensions [512, 256, 512].
[0133] The activation function uses GELU;
[0134] Fuse user latent vectors with features of popular products in the current season;
[0135] Output personalized condition vectors to highlight the needs of "springtime infant skincare";
[0136] Conditional content generation;
[0137] Input prompt: "Essential skincare for babies in spring";
[0138] Use VideoCrafter to generate a diffusion model;
[0139] Personalized conditional vectors are injected through a spatiotemporal attention mechanism;
[0140] Generate a 15-second short video:
[0141] First 5 seconds: Showing common skin problems in babies during spring;
[0142] 5 seconds: We recommend a gentle organic moisturizer;
[0143] Last 5 seconds: Demonstration of correct usage;
[0144] The narration and subtitles emphasize several user concerns, including "no additives" and "suitable for sensitive skin."
[0145] Feedback optimization;
[0146] The user completes the purchase after watching the video;
[0147] Update the fusion network parameters through incremental learning:
[0148] 95% of historical parameters are retained;
[0149] Fine-tune the parameters by 5% using the new data;
[0150] Improve the accuracy of similar product recommendations.
[0151] Example 6
[0152] Personalized news summary generation;
[0153] Application scenario: Generating personalized news summaries for news app users;
[0154] Implementation steps:
[0155] User behavior latent vector construction;
[0156] Cross-platform data collection behaviors: browsing Toutiao, reading WeChat official accounts, and forwarding Weibo news;
[0157] Innovations of Behavioral Coding Networks:
[0158] Time-aware location coding: distinguishing differences in reading preferences between morning and evening;
[0159] Global attention uses one-dimensional convolution: enhancing the modeling of continuous reading sequences;
[0160] Complexity Processing several months of behavior records;
[0161] Output a 384-dimensional latent vector, representing multiple features such as "preference for financial news" and "attention to technology developments";
[0162] Multimodal content understanding and representation;
[0163] Get the news content that the user has read:
[0164] News article and headline;
[0165] Illustrations and infographics;
[0166] Related video reports;
[0167] Using OneVision multimodal encoder:
[0168] Supports unified understanding of text, images, and videos;
[0169] Output 384-dimensional content features;
[0170] Personalized preference alignment and integration;
[0171] Attention-weighted summation method:
[0172] Calculate the distribution of user attention across various news categories;
[0173] The highest level of attention was found to be paid to "Stock Market Opening Briefs";
[0174] Real-time integration of the latest financial data features;
[0175] Generate dynamic, personalized conditional vectors;
[0176] Conditional content generation;
[0177] Enter the text of the real-time news source;
[0178] Personalized conditional fine-tuning using GPT-4+:
[0179] Abstract length and focus are controlled based on conditional vectors;
[0180] Generate three formats:
[0181] Key points in one sentence (for quick browsing);
[0182] Three-part detailed summary (suitable for in-depth reading);
[0183] Key data list (for decision-making reference);
[0184] Automatically generate infographics for important information;
[0185] Feedback optimization;
[0186] Real-time collection of click, read completion rate, and sharing data;
[0187] Online learning update mechanism:
[0188] Attention weights are updated hourly.
[0189] Daily updates to converged network parameters;
[0190] Weekly retraining of behavior encoding networks;
[0191] Enables dynamic interest tracking.
[0192] The embodiments of the present invention have been described above with reference to the accompanying drawings. However, the present invention is not limited to the specific embodiments described above. The specific embodiments described above are merely illustrative and not restrictive. Those skilled in the art can make many other forms under the guidance of the present invention without departing from the spirit and scope of the claims. All of these forms are within the protection scope of the present invention.
Claims
1. A personalized content generation method based on a multimodal large model and latent vectors of user behavior, characterized in that, include: S1. User behavior latent vector construction steps: Collect the target user's historical interaction behavior sequence on multiple content platforms. The historical interaction behavior includes clicks, browsing duration, comments, sharing and collection. After cleaning and standardizing the historical interaction behavior sequence, it is input into a behavior encoding network for training to generate a fixed-dimensional user behavior latent vector that represents the user's long-term interests and preferences. S2. Multimodal content understanding and representation steps: Obtain original content data associated with the user's historical interaction behavior. The original content data includes at least one of text, image, video, and audio. Using a pre-trained multimodal large model, uniformly encode and extract features from the original content data to generate a content feature vector in the same semantic space as the user behavior latent vector. S3. Personalized Preference Alignment and Fusion Step: The user behavior latent vector and the content feature vector generated in step S2 are fused and calculated to generate a personalized condition vector. The fusion calculation includes vector concatenation, attention-weighted summation, or nonlinear mapping through a lightweight adaptation network. S4. Conditional content generation step: The personalized condition vector is used as the generation condition and input into a generative multimodal large model. The generative multimodal large model generates target content that conforms to the personalized preferences of the target user based on the input text prompts and the personalized condition vector.
2. The personalized content generation method based on a multimodal large model and latent vectors of user behavior according to claim 1, characterized in that: In step S1, the behavior encoding network is constructed using a sequence model based on an improved Transformer architecture, which handles extremely long historical interaction behavior sequences in the following way: a. Sequence Segmentation Module: Divides extremely long historical interaction sequences into multiple continuous subsequences of fixed length according to time order; b. Two-layer attention encoding module: including a local attention layer and a global attention layer; Local attention layer: Within each subsequence, a key sparse attention mechanism is used to calculate the dependencies between historical interaction behaviors within the subsequence. The key sparse attention mechanism filters noise and redundant information and focuses on key behavioral features by retaining the top k columns with the highest weight values in the attention weight matrix. Global attention layer: Above the local attention layer, a cross-variable attention mechanism is adopted, one-dimensional convolution operation is introduced to enhance the modeling of continuous features between time steps, and the attention calculation method is reconstructed to capture long-distance dependencies between different subsequences; c. Improved position encoding module: In the input of the local attention layer and the global attention layer, a position encoding vector based on trigonometric functions is added to each historical interaction behavior to ensure that the model can clearly distinguish the absolute order between behaviors.
3. The personalized content generation method based on a multimodal large model and latent vectors of user behavior according to claim 2, characterized in that: The sequence segmentation module uses a sliding window method for segmentation, with overlapping portions between adjacent subsequences to preserve contextual continuity.
4. The personalized content generation method based on a multimodal large model and latent vectors of user behavior according to claim 3, characterized in that: The global attention layer employs a cross-variable attention mechanism, with a computational complexity of O(n). ,in For sequence length, The feature dimension is significantly lower than that of traditional self-attention mechanisms. ; The global attention layer employs a cross-variable attention mechanism, and its core computation process is as follows: Input representation: Let the input sequence be ,in For sequence length, For feature dimensions.
5. The personalized content generation method based on a multimodal large model and latent vectors of user behavior according to claim 4, characterized in that: Self-attention mechanism calculates the query matrix. AND key matrix The dot product has a complexity of O(n log n). ; The cross-variable attention mechanism reconstructs the computation method, transposing the query matrix to... The key matrix remains unchanged. Calculate the dot product of the two: in, This is a cross-variable attention weight matrix, representing the correlation between different variables; Complexity analysis: The computational complexity of the above is ,in For fixed feature dimensions, The sequence length; Because in actual multivariate time series Therefore, This significantly reduces the computational burden of processing long sequences; Output weighting: Through attention weight matrix Log-value matrix We perform weighting to obtain the enhanced feature representation: The output preserves the correlation information between variables and supports subsequent sequence modeling.
6. The personalized content generation method based on a multimodal large model and latent vectors of user behavior according to claim 5, characterized in that: The behavior encoding network is pre-trained through a self-supervised learning task, which includes next behavior prediction or behavior sequence mask reconstruction.
7. The personalized content generation method based on a multimodal large model and latent vectors of user behavior according to claim 6, characterized in that: In step S2, the pre-trained multimodal large model is a vision-language large model, which is trained through contrastive learning or cross-modal masking modeling tasks to achieve alignment of text and visual features in a unified vector space.
8. The personalized content generation method based on a multimodal large model and latent vectors of user behavior according to claim 7, characterized in that: In step S3, the fusion calculation specifically employs an attention mechanism network to calculate the attention weights of the user behavior latent vectors on each vector in the historical content feature vector sequence, and performs a weighted sum based on the attention weights to generate the personalized condition vector.
9. The personalized content generation method based on a multimodal large model and latent vectors of user behavior according to claim 8, characterized in that: The generative multimodal large model in step S4 is either a diffusion model or an autoregressive large language model. When the generative multimodal large model is a diffusion model, the personalized conditional vector is injected into multiple sampling steps of the denoising process through a cross-attention mechanism.
10. The personalized content generation method based on a multimodal large model and latent vectors of user behavior according to claim 9, characterized in that: Following step S4, the method further includes: S5. Feedback optimization step: pushing the generated target content to the target user and collecting new user interaction behaviors with the target content; based on the new interaction behaviors, updating the latent vector of user behavior of the target user and / or the parameters of the fusion computing network through online learning or incremental learning.