Cross-platform intelligent recommendation method and device, electronic equipment and computer storage medium
By constructing a multimodal representation encoder and a cross-platform sharing network, integrating data from multiple platforms, and generating user/item representations with unified dimensions, the recommendation problem of traditional recommendation systems in cross-platform scenarios is solved, achieving personalized and real-time recommendation results.
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- GUANGDONG ZHUOSHANG TECH GRP CO LTD
- Filing Date
- 2026-03-14
- Publication Date
- 2026-06-12
AI Technical Summary
Traditional recommendation systems cannot achieve accurate, real-time, and personalized recommendations in cross-platform and multimodal scenarios, resulting in inconsistent characterization of user interests, serious duplication and bias, and difficulty in adapting to cross-platform scenario switching and interest migration.
By constructing a multimodal representation encoder and a cross-platform shared network, multimodal data from multiple platforms are integrated to generate user/item representations with a unified dimension. A recommendation list is generated through joint optimization of the model, and user interest representations are dynamically updated by combining offline pre-training and online fine-tuning.
It achieves unified processing of cross-platform data and comprehensive characterization of user interests, reduces recommendation duplication and bias, improves the real-time performance and accuracy of recommendations, adapts to user interest shifts, and provides personalized recommendation results.
Smart Images
Figure CN122196269A_ABST
Abstract
Description
Technical Field
[0001] This application relates to the field of artificial intelligence technology, and in particular to a cross-platform intelligent recommendation method, apparatus, electronic device, and computer storage medium. Background Technology
[0002] In consumption scenarios that integrate e-commerce, short videos, local services, and social media platforms, user behavior exhibits cross-platform, multimodal, and fragmented characteristics. Data is scattered across different modalities and platforms, forming data silos. Traditional recommendation systems rely on single-platform, single-modal data modeling, which cannot uniformly depict users' global interests, resulting in duplicate, biased, and lagging recommendations. They are also difficult to adapt to cross-platform scenario switching and interest migration. Therefore, in order to achieve accurate, real-time, and personalized recommendations across platforms and multiple modalities, it is urgent to build a unified representation and collaborative modeling system.
[0003] Traditional intelligent recommendation mainly adopts a single-platform, single-modal, offline modeling approach. For cross-platform and multimodal issues, existing common solutions achieve cross-platform data concatenation at the data level through log aggregation and ID mapping, and perform feature engineering on multimodal data separately before simple concatenation or weighted fusion. At the model level, collaborative filtering, matrix factorization, LR / MLP, and deep recall + fine ranking are the main methods. Cross-platform scenarios often adopt "separate platform modeling + result fusion / re-ranking", while multimodal scenarios adopt "separate encoding of single modalities + later fusion". At the engineering level, offline training + timed updates are used, and response speed is improved through caching and feature pre-computation. Cross-platform migration relies on rules or manual configuration of migration strategies. Summary of the Invention
[0004] To overcome the shortcomings of existing technologies, this invention provides a cross-platform intelligent recommendation method, device, electronic device, and computer storage medium, which effectively integrates multi-platform and multi-modal data, breaks down data silos, comprehensively portrays user interests, and achieves accurate, real-time, and personalized recommendations.
[0005] A first aspect of this application provides a cross-platform intelligent recommendation method, the method comprising: When multimodal data from multiple platforms is received, the multimodal data is processed by a pre-built multimodal representation encoder to output an initial user / item representation with a unified dimension; the initial user / item representation includes an initial user representation and an initial item representation. User behavior sequences are extracted from the log systems of the aforementioned multi-platform systems to construct cross-platform user profiles; The initial user / item representation and the cross-platform user profile are input into a preset cross-platform sharing network to output a target user / item representation that integrates cross-platform information through the cross-platform sharing network; the target user / item representation includes a target user representation and a target item representation. The target user / item representation is input into a preset joint optimization model to output a final recommendation list.
[0006] In an optional implementation, the method further includes: During the offline phase, all historical data is acquired to pre-train a preset base model; the base model includes the multimodal representation encoder and the cross-platform shared network. During the online phase, real-time user behavior data is acquired to fine-tune the pre-trained base model using the real-time user behavior data.
[0007] In an optional implementation, the method further includes: When a sudden user behavior is detected based on the real-time user behavior data, the weight coefficient of the category corresponding to the sudden user behavior is temporarily increased, and real-time scene features are injected; the sudden user behavior pattern is defined as the frequency of the same type of behavior exceeding a preset threshold within a unit of time; the real-time scene features include at least one of time, location, device type, and current task type; The dynamic user representation is adjusted based on the weight coefficients and real-time scene features to generate a scene-adaptive user representation; the dynamic user representation is obtained by combining offline pre-training with online fine-tuning.
[0008] In an optional implementation, the multimodal representation encoder includes a modality-specific sub-encoder and a cross-modal attention layer; the modality-specific sub-encoder is used to extract features from multimodal data and output a multimodal feature sequence; the cross-modal attention layer is used to achieve bidirectional semantic alignment between each modality feature sequence in the multimodal feature sequence through multiple rounds of cross-attention iteration, and output the initial user / item representation.
[0009] In an optional implementation, the cross-platform shared network includes a shared underlying network, a platform adaptive branch network, and a cross-platform distributed constraint network. The shared underlying network is used to extract global features from the initial user / item representation based on the cross-platform user profile, learn the global interest patterns of user cross-platform behavior and the cross-platform general features of items, and output user interest features and item general features. The platform adaptive branch network is used to perform platform-specific adaptation on the user interest features and the item general features, and output platform-adapted user interest representation and platform-adapted item features. The cross-platform distributed constraint network is used to constrain the platform-adapted user interest representation and the platform-adapted item features, and output the constrained target user / item representation.
[0010] In an optional implementation, the cross-platform distributed constraint network uses contrast loss and migration loss as joint constraint objectives.
[0011] In an optional implementation, the joint optimization model includes a recall layer, a fine ranking layer, and a re-ranking layer; the recall layer is used to perform approximate nearest neighbor retrieval based on the target user / item representation to generate an initial candidate set; the fine ranking layer is used to optimize the initial candidate set using a multi-task deep model to output a target candidate set; the re-ranking layer is used to optimize the target candidate set using a balanced optimization rule to generate the final recommendation list; wherein the recall layer and the fine ranking layer are jointly optimized by sharing a representation space.
[0012] A second aspect of this application provides a cross-platform intelligent recommendation device, the device comprising: The multimodal data processing module is used to process multimodal data input from multiple platforms through a pre-built multimodal representation encoder when the multimodal data is received, and output an initial user / item representation with a unified dimension; the initial user / item representation includes an initial user representation and an initial item representation. A cross-platform user profile building module is used to extract user behavior sequences from the log systems of the multiple platforms to build cross-platform user profiles. The cross-platform information fusion module is used to input the initial user / item representation and the cross-platform user profile into a preset cross-platform sharing network, so as to output the target user / item representation that integrates cross-platform information through the cross-platform sharing network; the target user / item representation includes the target user representation and the target item representation; The recommendation list generation module is used to input the target user / item representation into a preset joint optimization model, so as to output the final recommendation list through the joint optimization model.
[0013] A third aspect of this application provides an electronic device, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the computer program to implement the steps of the cross-platform intelligent recommendation method.
[0014] A fourth aspect of this application provides a computer-readable storage medium having a computer program stored thereon, wherein the computer program, when executed by a processor, implements the steps of the cross-platform intelligent recommendation method described above.
[0015] In summary, the cross-platform intelligent recommendation method, apparatus, electronic device, and computer storage medium provided in this application have at least one of the following beneficial effects: 1. When receiving multimodal data from multiple platforms, the system processes the data using a pre-built multimodal representation encoder, outputting an initial user / item representation with a unified dimension. This multimodal representation encoder breaks the limitations of traditional recommendation systems that rely on single-platform, single-modal data modeling. It unifies the processing of data from different modalities and platforms, outputting a unified dimension representation, thus laying the foundation for a unified characterization of user global interests. 2. By extracting user behavior sequences from multi-platform log systems to construct cross-platform user profiles, user behavior information from different platforms can be integrated to more comprehensively depict user interests and behavior patterns, thereby solving the problem of not being able to uniformly depict users' global interests. 3. Input the initial user / item representation and cross-platform user profile into a pre-defined cross-platform sharing network. This network then outputs a target user / item representation that integrates cross-platform information. The cross-platform sharing network comprehensively considers information from multiple platforms, optimizing the initial representation and ensuring that the target user / item representation incorporates rich cross-platform information, reducing the possibility of recommendation duplication and bias. 4. Input the initial user / item representation and cross-platform user profile into a pre-defined cross-platform sharing network. This network outputs a target user / item representation that integrates cross-platform information. A joint optimization model can further optimize the target representation that integrates cross-platform information. Combining real-time data and algorithms, a recommendation list that matches the user's current interests and needs can be quickly generated, improving the real-time performance and accuracy of recommendations and solving the problem of recommendation lag.
[0016] 5. The multimodal representation encoder processes multi-platform data, constructs cross-platform user profiles, and integrates cross-platform information through cross-platform network sharing. These steps enable the system to comprehensively utilize information from multiple platforms. When users switch between different platforms, the system can quickly adjust its recommendation strategy based on this comprehensive information, better adapting to cross-platform scenario switching. 6. By receiving multimodal data from multiple platforms in real time and extracting user behavior sequences to construct cross-platform user profiles, and continuously inputting new information into the cross-platform shared network and joint optimization model, the initial user / item representation and the target user / item representation can be dynamically updated. This dynamic update mechanism can promptly capture changes in user interests, thereby adapting to user interest migration and providing users with recommendations that better match their current interests. Attached Figure Description
[0017] Figure 1 This is a flowchart illustrating a cross-platform intelligent recommendation method according to an embodiment of this application; Figure 2 This is a schematic diagram of the data processing flow of a joint optimization model shown in an embodiment of this application; Figure 3This is a functional block diagram of a cross-platform intelligent recommendation device shown in an embodiment of this application; Figure 4 This is a schematic diagram of the structure of an electronic device shown in an embodiment of this application. Detailed Implementation
[0018] The present invention will be further described below with reference to the accompanying drawings and embodiments.
[0019] The following will clearly and completely describe the concept, specific structure, and technical effects of the present invention in conjunction with embodiments and accompanying drawings, so as to fully understand the purpose, features, and effects of the present invention. Obviously, the described embodiments are only a part of the embodiments of the present invention, not all of them. Other embodiments obtained by those skilled in the art based on the embodiments of the present invention without creative effort are all within the scope of protection of the present invention. Furthermore, all connections / linkages involved in the patent do not simply refer to direct contact between components, but rather to the ability to form a better connection structure by adding or reducing connecting accessories according to specific implementation conditions. The various technical features in this invention can be combined interactively without contradicting each other.
[0020] Reference Figure 1 The diagram shown is a flowchart illustrating a cross-platform intelligent recommendation method according to an embodiment of this application. The cross-platform intelligent recommendation method includes the following steps.
[0021] S11, when receiving multimodal data from multiple platforms, the multimodal data is processed by a pre-built multimodal representation encoder to output an initial user / item representation with a unified dimension.
[0022] The initial user / item representation includes an initial user representation and an initial item representation.
[0023] In some embodiments, the electronic device first constructs a multimodal representation encoder to replace "separate encoding of each single modality + subsequent concatenation". Specifically, the multimodal representation encoder includes a modality-specific sub-encoder and a cross-modal attention layer. The modality-specific sub-encoder is used to extract features from the multimodal data and output a multimodal feature sequence. The cross-modal attention layer is used to achieve bidirectional semantic alignment between each modality feature sequence in the multimodal feature sequence through multiple rounds of cross-attention iteration, and output the initial user / item representation.
[0024] Modality-specific sub-encoders are designed based on the input multimodal data. Electronic devices first define the input layer of the modality-specific sub-encoder, enabling it to receive data from multiple modalities such as text, images, video, audio, and action sequences. For example, a modality-specific sub-encoder can include, but is not limited to, text sub-encoders, image sub-encoders, video sub-encoders, audio sub-encoders, and action sequence sub-encoders. The text sub-encoder can use word vector models (such as Word2Vec and GloVe) to convert the segmented text into fixed-dimensional word vectors, or it can use pre-trained language models (such as BERT and GPT) to directly obtain the semantic representation of the text. If a pre-trained language model is used, it needs to be fine-tuned according to the task requirements. The text sub-encoder cleans the input text data, removing noise, special characters, etc., and performs word segmentation (for Chinese, tools such as Jieba can be used; for English, space segmentation can be used). Convolutional neural networks (CNNs) can be used as image sub-encoders, such as ResNet and VGG. The appropriate network structure can be selected based on the data characteristics and computing resources to extract feature vectors from the image. The image sub-encoder performs operations such as resizing and normalization on images to meet subsequent input requirements. The video sub-encoder can use a 3D-CNN or a combination of CNN and Recurrent Neural Network (RNN) architecture to process video data. 3D-CNN can capture both spatial and temporal information simultaneously, while the CNN+RNN structure first uses CNN to extract features from each frame and then uses RNN to model the frame sequence. The video sub-encoder splits the video into frames and performs the same preprocessing operations as the image sub-encoder on each frame. The audio sub-encoder uses a one-dimensional CNN or RNN to encode audio spectral features to obtain a semantic representation of the audio. The audio sub-encoder performs sampling rate conversion, noise reduction, and other processing on the audio to extract spectral features (such as Mel spectrum). The behavior sequence sub-encoder uses RNN or its variants (such as LSTM, GRU) to model behavior sequences, capturing the temporal features of user behavior. The behavior sequence sub-encoder encodes behavior sequences, mapping different types of behavior (such as clicks, purchases, favorites) to different numerical values or vectors.
[0025] Cross-modal attention layers employ appropriate attention mechanisms, such as self-attention or cross-attention. Self-attention captures relationships between elements within the same modality, while cross-attention achieves semantic alignment and fusion between different modalities. Feature vectors output from different modal sub-encoders are input into the cross-modal attention layer. By calculating attention weights between features from different modalities, the features are weighted and fused to obtain a unified-dimensional user / item representation (referred to as the initial user / item representation for ease of distinction). For example, for text and image modalities, the attention weights of text features on image features and vice versa are calculated, and then the two modal features are fused according to their weights.
[0026] S12, extract user behavior sequences from the log system of the multi-platform system to construct a cross-platform user profile.
[0027] In some embodiments, electronic devices can extract user behavior sequences from multi-platform logs and construct cross-platform user profiles through a unified ID system (such as a unified user ID / device ID mapping). Specifically, electronic devices can extract user behavior data from log systems of different platforms (such as e-commerce, short video, and local services), including user browsing history, purchase history, likes history, and comment history. Simultaneously, a unified user ID or device ID mapping table is constructed to unify user identifiers across different platforms. For example, if a user's ID is A on an e-commerce platform and B on a short video platform, a mapping relationship between A and B is established through device information or other related information to ensure accurate identification of the same user's behavior on different platforms. The unified user behavior data is then used to construct cross-platform user profiles from multiple dimensions (such as interests, spending power, and active time). For example, by analyzing the categories of goods purchased by users on e-commerce platforms and the types of videos liked on short video platforms, user interests can be determined.
[0028] S13, input the initial user / item representation and the cross-platform user profile into a preset cross-platform sharing network, so as to output the target user / item representation that integrates cross-platform information through the cross-platform sharing network.
[0029] The target user / item representation includes target user representation and target item representation.
[0030] In some embodiments, electronic devices may pre-construct a cross-platform shared network. This cross-platform shared network includes a shared underlying network, a platform-adaptive branch network, and a cross-platform distributed constraint network. The shared underlying network is used to extract global features from the initial user / item representation based on the cross-platform user profile, learn the global interest patterns of user cross-platform behavior and the cross-platform general features of items, and output user interest features and item general features (which can also be referred to as shared features). The platform-adaptive branch network is used to perform platform-specific adaptation on the user interest features and the item general features, outputting platform-adapted user interest representations and platform-adapted item features. The cross-platform distributed constraint network is used to constrain the platform-adapted user interest representations and platform-adapted item features, outputting the constrained target user / item representation.
[0031] Specifically, a shared underlying network is first designed to learn user global interests and general item features. Electronic devices can choose structures such as Multilayer Perceptron (MLP) or Graph Neural Network (GNN) to construct the shared underlying network. If the data has graph structure features (such as user-item interaction graphs), GNN may be more suitable; if the data is in tabular form or simple feature vectors, MLP may be a better choice. The initial user / item representation with unified dimensions output from step S11 and the cross-platform user profile constructed in step S12 are input into the cross-platform shared network. Through forward propagation of the cross-platform shared network, the global interest representation of users and the general feature representation of items across different platforms are learned. For example, by analyzing user purchasing behavior on multiple platforms, the overall preferences of users for different categories of goods can be learned.
[0032] Next, an adaptive branch network for each platform is constructed. Each platform branch employs adaptive layers (such as gating mechanisms, attention mechanisms, and lightweight MLPs) to learn platform-specific distributions and business objectives. Gating mechanisms can dynamically adjust feature weights based on the characteristics of different platforms; attention mechanisms can focus on features relevant to the current platform's business objectives; and lightweight MLPs can perform further non-linear transformations on shared features to adapt to the needs of different platforms. First, the business objectives of each platform are clarified. For example, the business objective of an e-commerce platform might be to increase product conversion rates, a short video platform's business objective might be to increase video completion rates, and a local services platform's business objective might be to encourage users to visit stores. Based on the business objectives of each platform, the shared features output by the shared underlying network across platforms are adjusted through adaptive layers to learn platform-specific feature distributions. For example, in the e-commerce platform branch, a gating mechanism is used to increase the weights of features related to product prices and promotional activities to improve the accuracy of product conversion rate prediction.
[0033] When performing platform-specific adaptation, a cross-platform distribution constraint network is constructed using cross-platform contrastive loss / transfer loss constraints. This network employs contrastive loss and transfer loss as joint constraint objectives to reduce distribution differences between platforms. Cross-platform contrastive loss compares the behavioral representations of the same user on different platforms, making them closer in the feature space. Transfer loss minimizes the feature distribution differences between the source and target platforms, enabling knowledge transfer. For example, in cold-start scenarios, knowledge learned from the rich data of the source platform (e.g., e-commerce platform) is used, and transfer loss constraints help the target platform (e.g., local services platform) quickly adapt to new users or items. Specifically, through joint optimization of cross-platform contrastive and transfer losses, the latent space distribution of user / item representations across multiple platforms is aligned, reducing distribution shifts caused by platform data heterogeneity (e.g., differences in user behavior patterns, inconsistent item feature dimensions), and outputting constrained target user / item representations (i.e., platform-independent shared embedding vectors).
[0034] (1) Contrastive loss: Contrastive learning brings the representation distance of similar entities across platforms (such as the IDs of the same user on different platforms) closer and widens the distance of dissimilar entities.
[0035] The specific mathematical form of cross-platform comparison loss is as follows: ; in, and For the same user, user representation on platforms A and B. To represent an item; This is a temperature coefficient used to control the "sharpness" of the probability distribution (the smoothness of the softmax).
[0036] (2) Migration Loss: The statistical differences (such as mean and covariance) in the representation distributions between platforms are minimized through domain adaptation techniques (such as Maximum Mean Discrepancy (MMD) and CORAL). In this embodiment, MMD is used, and its specific mathematical form is as follows: ; in, The kernel function maps the representation to the reproducing kernel Hilbert space (RKHS). and These represent the number of samples on platform A and platform B, respectively. is the norm in RKHS, used to measure the difference between two mean embeddings.
[0037] Furthermore, the contrast loss and migration loss can be weighted and summed to obtain the total loss function (i.e., the joint constraint objective): ; in, and It is a hyperparameter used to balance the strength of the two types of constraints.
[0038] S14, input the target user / item representation into a preset joint optimization model, so as to output the final recommendation list through the joint optimization model.
[0039] In some embodiments, the target user / item representation processed by a cross-platform distributed constraint network is used as input and passed to a preset joint optimization model. The joint optimization model further mines the potential associations between users and items based on these intermediate representations, which have eliminated platform-specific noise and possess cross-platform portability, to generate an accurate recommendation list that meets user needs. The joint optimization model includes a recall layer, a fine ranking layer, and a re-ranking layer. The recall layer performs approximate nearest neighbor retrieval based on the target user / item representation to generate an initial candidate set. The fine ranking layer optimizes the initial candidate set using a multi-task deep model to output a target candidate set. The re-ranking layer optimizes the target candidate set using a balanced optimization rule to generate the final recommendation list. The recall layer and the fine ranking layer jointly optimize by sharing a representation space. The working process of the recall layer, fine ranking layer, and re-ranking layer in the joint optimization model will be described in detail below: (1) Recall layer: Recall stage.
[0040] The recall layer receives target user representations and intermediate object representations (vector form) from the cross-platform distributed constraint network. These representations have eliminated platform-specific noise and are cross-platform transferable. Next, a suitable approximate nearest neighbor retrieval algorithm (such as Locality Sensitive Hashing (LSH), tree-based algorithms (such as KD-trees), graph-based algorithms (such as HNSW)) is selected to quickly retrieve item vectors similar to the target user's interests from a massive user / item vector pool, forming a candidate item set. Specifically, based on different recall strategies (such as user interest-based recall, popular item-based recall, collaborative filtering-based recall, etc.), for example, the cosine similarity between user vectors and item vectors is calculated to generate a similarity matrix; Top-K items (e.g., K=1000) are selected based on the similarity score, generating multiple candidate sets (referred to as the initial candidate set for easy differentiation). This candidate set contains items that best match the user's interests.
[0041] In addition, electronic devices can also perform preliminary filtering of the initial candidate set by combining business rules (such as timeliness weighting). For example, similarity of news items can be reduced by publication time to remove duplicate or highly similar items and ensure the diversity of the candidate set.
[0042] (2) Fine arrangement layer: fine arrangement stage.
[0043] In the fine-ranking stage, the candidate set is refined and ranked. Multi-task learning considers multiple business objectives, such as click-through rate (CTR) and conversion rate (CVR), to ensure the accuracy and comprehensiveness of the ranking results. Specifically, the fine-ranking layer receives the candidate set from the recall layer and integrates multi-platform behavioral data, multi-modal features, and business objectives (such as CTR and CVR) through a multi-task deep model (such as MMoE) to output a refined ranking score. This score reflects the item's attractiveness to users. The multi-task deep model can include multiple sub-tasks, such as CTR prediction and conversion rate prediction, improving the model's generalization ability by sharing underlying feature representations. Through forward propagation of the multi-task deep model, a ranking score is output for each candidate item. The ranking score can comprehensively consider multiple factors, such as user interest matching, item quality, and business objectives. Then, the initial candidate set is sorted from highest to lowest based on the ranking score, outputting the target candidate set.
[0044] (3) Rearrangement layer: Rearrangement stage.
[0045] The re-ranking stage further optimizes the refined ranking results. By introducing business rules and user scenario information, it ensures that the recommended list aligns with both user interests and practical application needs. Specifically, the target candidate set output from the refined ranking layer is input into the re-ranking layer. In the re-ranking layer, electronic devices can formulate balanced optimization rules for diversity, deduplication, timeliness, and scenario adaptation. Diversity rules ensure that the recommended results cover different types and categories of items, avoiding overly simplistic recommendations; deduplication rules remove duplicate items, improving the quality of the recommendations; timeliness rules prioritize the recommendation of newly released or highly popular items; and scenario adaptation rules adjust the recommended results according to different scenarios (such as different time periods or locations).
[0046] Based on the balanced optimization rules, the candidate item ranking list output in the fine-ranking stage is rearranged. Specifically, optimization algorithms such as greedy algorithms and genetic algorithms can be used to achieve this rearrangement. While satisfying the rules, this maximizes the accuracy of the recommendation results and the user experience, generating a final recommendation list that balances accuracy and experience. This list contains recommended items determined comprehensively based on user characteristics, item characteristics, business rules, and user scenarios.
[0047] Finally, electronic devices can also use test sets to evaluate the overall recommendation performance of the unified recall-refinement-replacement system. Based on evaluation metrics (such as accuracy, recall, F1 score, mean precision (MAP), etc.), the parameters and strategies of the recall, refinement, and replacement stages can be adjusted to improve the overall recommendation performance.
[0048] It should be noted that the recall layer and the ranking layer are jointly optimized through a shared representation space. Electronic devices can reduce "recall bias" by adjusting the similarity measurement method in the recall stage or the feature weights in the ranking stage, ensuring that items recalled in the recall stage achieve reasonable ranking scores in the ranking stage. By jointly optimizing the representation spaces of recall and ranking, the candidate set generated in the recall stage is made more consistent with the ranking target in the ranking stage.
[0049] In an optional implementation, the method further includes: During the offline phase, all historical data is acquired to pre-train a preset base model; the base model includes the multimodal representation encoder and the cross-platform shared network. During the online phase, real-time user behavior data is acquired to fine-tune the pre-trained base model using the real-time user behavior data.
[0050] In some embodiments, during the offline phase, the electronic device can pre-train a base model using historical full-data, enabling the base model to output stable user / item representations. The base model refers to the multimodal representation encoder and cross-platform shared network constructed in steps S11 and S12. During the offline phase, historical full-data is collected, including multi-source heterogeneous data such as user behavior data and item attribute data, and cleaned, preprocessed, and feature-engineered to ensure data integrity and accuracy, removing noisy data and outliers. During model training, the offline historical full-data can be used to pre-train the base model constructed in steps S11 and S12, so that stable user / item representations can be output when applied in the subsequent online phase. That is, the initial user / item representation output by the multimodal representation encoder in step S11 and the target user / item representation output by the cross-platform shared network in step S12 are both stable. Additionally, optimization algorithms such as batch gradient descent (BGD), stochastic gradient descent (SGD), or their variants (such as Adam) can be used to update the model parameters. The pre-trained model is evaluated using a validation set, and the model structure and hyperparameters are adjusted based on evaluation metrics (such as accuracy, recall, F1 score, etc.) to improve model performance.
[0051] During the online phase, electronic devices can incrementally update and stabilize user representations using real-time user behavior data (clicks, adding to cart, favorites, cross-platform navigation). Specifically, a real-time data stream processing pipeline is established to continuously feed real-time user behavior data generated during the online phase (such as clicks, adding to cart, favorites, cross-platform navigation, etc.) into the recommendation system. Simultaneously, message queues (such as Kafka) can be used to achieve real-time data transmission and buffering. Next, based on a pre-trained base model, the model is fine-tuned according to the real-time behavior data stream. For example, a sliding window mechanism can be used, considering only user behavior data within the most recent period, dynamically adjusting user interests based on new behaviors; or online learning algorithms, such as online gradient descent (OGD), can be used to update model parameters in real time. During fine-tuning, the focus is on updating the user interest representation. By continuously inputting new user behavior data, the model can capture dynamic changes in user interests. For example, when a user frequently clicks on a certain type of item, the model will adjust the user interest representation to be more aligned with the feature direction of that type of item.
[0052] Through the aforementioned optional implementation methods, a basic model containing multimodal coding and a cross-platform shared network is pre-trained using historical full-data in the offline phase to output stable user / item representations. In the online phase, the model is fine-tuned using real-time behavioral data to capture dynamic changes in user interests. This approach combines historical and real-time data, ensuring both representation stability and timely reflection of interest changes, effectively improving the accuracy of the recommendation system's grasp of user interests and enhancing recommendation performance.
[0053] In an optional implementation, the method further includes: When a sudden user behavior is detected based on the real-time user behavior data, the weight coefficient of the category corresponding to the sudden user behavior is temporarily increased, and real-time scene features are injected; the sudden user behavior pattern is defined as the frequency of the same type of behavior exceeding a preset threshold within a unit of time; the real-time scene features include at least one of time, location, device type, and current task type; The dynamic user representation is adjusted based on the weight coefficients and real-time scene features to generate a scene-adaptive user representation; the dynamic user representation is obtained by combining offline pre-training with online fine-tuning.
[0054] In some embodiments, when a sudden user behavior pattern (the frequency of the same behavior exceeding a preset threshold within a unit of time) is detected in real time, the system will take further action based on the existing dynamically changing user interest representation (i.e., the user representation continuously adjusted during the previous online update process). First, the weight coefficient of the category corresponding to the sudden behavior is temporarily increased. Taking a sudden user interest in sneakers as an example, the system will increase the weight coefficient of the sneaker-related category in the user interest model, so that sneakers will receive more attention in subsequent calculations and recommendations. At the same time, the system will inject real-time scene features. Real-time scene features cover multiple dimensions such as time, location, device type, and current task type. For example, if the user's sudden behavior is detected at 8 pm, the location is their home address, the device used is a mobile phone, and the current task type may be leisure shopping, the system will combine these scene features with the user's sudden behavior information. Furthermore, based on the temporarily increased weight coefficient and the injected real-time scene features, the values of the dimensions related to the sudden behavior category and scene features in the dynamic user representation are adjusted. For example, in response to sudden behavior related to sneakers, the system will add values related to dimensions such as sports interest and fashion preferences to the dynamic user representation; considering the time context of 8 pm, it may further adjust values related to dimensions such as nighttime shopping habits and leisure sports needs. Through such adjustments, a scenario-adapted user representation is generated.
[0055] The dynamic user representation is a multi-dimensional vector, with each dimension corresponding to a different aspect of the user's interests, obtained based on the preceding offline pre-training and online incremental updates (i.e., online fine-tuning). In the offline pre-training phase, the base model is trained using historical full-scale data (covering rich user behavior data and item attribute data from multiple platforms). The base model learns the complex relationship patterns between users and items from this massive dataset, constructing an initial semantic framework of user interests and generating an initial user representation vector. This vector contains multiple dimensions of the user's long-term stable interests, such as the user's preference for different categories of items and the persistence of those interests. In the online incremental update phase, the system receives real-time user behavior data streams from various platforms, such as clicks, adding to cart, favorites, and cross-platform navigation. Based on the pre-trained base model, this real-time data is used to fine-tune the model. During fine-tuning, the model dynamically adjusts the values of each dimension of the user representation vector according to the user's latest behavior. For example, when a user frequently browses a certain type of new products recently, the model will increase the value of the interest dimension corresponding to that type of item, and may also adjust the values of other related dimensions, such as the user's acceptance of new things. After online incremental updates, the user representation vector can reflect the changing trends of user interests in real time, thus forming a dynamic user representation. It is a multi-dimensional vector, with each dimension corresponding to a different aspect of the user's interests.
[0056] Context-adapted user representations can more accurately reflect a user's interest state under specific sudden behaviors and scenarios. In the subsequent recommendation process, relevant item vectors are retrieved more accurately based on these context-adapted user representations to further optimize the recommendation list, thereby providing users with recommendation results that are more in line with their current needs and scenarios.
[0057] Through the aforementioned optional implementation methods, when sudden user behavior is detected, the corresponding category weight coefficient is temporarily increased and real-time scene features are injected. Based on the dynamic user representation obtained through offline pre-training and online fine-tuning, the relevant dimension values are adjusted to generate a scene-adaptive user representation. This representation accurately reflects the user's interests in a specific state. Subsequent recommendation processes use this representation to retrieve and optimize the item vector list, providing users with recommendations that better match their current needs and scenarios, thereby improving recommendation accuracy and user satisfaction.
[0058] In some embodiments, electronic devices can also design online A / B testing schemes, randomly dividing users into different groups and using different recommendation models or strategies for each group. By comparing user behavior metrics (such as click-through rate, conversion rate, dwell time, etc.) across different groups, the performance of different models or strategies is evaluated. Based on the results of the online A / B test, the model is iterated on in a lightweight manner. Lightweight iteration allows for adjustments to only some parameters or structures of the model, avoiding large-scale model retraining and improving iteration efficiency. For example, if it is found that the parameters of a certain adaptive layer have a significant impact on the recommendation performance, only the parameters of that layer can be optimized and adjusted.
[0059] Reference Figure 3 The diagram shown is a functional block diagram of a cross-platform intelligent recommendation device according to an embodiment of this application.
[0060] In some embodiments, the cross-platform intelligent recommendation device 30 may include multiple functional modules composed of computer program segments. The computer programs of each program segment of the cross-platform intelligent recommendation device 30 may be stored in the memory of an electronic device and executed by at least one processor to perform (see details). Figure 1 (Description) This describes the cross-platform intelligent recommendation function. Based on its functions, it can be divided into multiple functional modules. These modules may include: a multimodal data processing module 301, a cross-platform user profile construction module 302, a cross-platform information fusion module 303, and a recommendation list generation module 304. The term "module" in this application refers to a series of computer program segments that can be executed by at least one processor and perform a fixed function, stored in memory. In this embodiment, the functions of each module will be detailed in subsequent embodiments.
[0061] The multimodal data processing module 301 is used to process the multimodal data received from multiple platforms through a pre-built multimodal representation encoder and output an initial user / item representation with a unified dimension; the initial user / item representation includes an initial user representation and an initial item representation.
[0062] The cross-platform user profile building module 302 is used to extract user behavior sequences from the log systems of the multiple platforms to build cross-platform user profiles.
[0063] The cross-platform information fusion module 303 is used to input the initial user / item representation and the cross-platform user profile into a preset cross-platform sharing network, so as to output the target user / item representation that integrates cross-platform information through the cross-platform sharing network; the target user / item representation includes the target user representation and the target item representation.
[0064] The recommendation list generation module 304 is used to input the target user / item representation into a preset joint optimization model, so as to output the final recommendation list through the joint optimization model.
[0065] It should be understood that the various variations and specific embodiments of the cross-platform intelligent recommendation method provided in the above embodiments are also applicable to the cross-platform intelligent recommendation device of this embodiment. Through the foregoing detailed description of the cross-platform intelligent recommendation method, those skilled in the art can clearly understand the implementation method of the cross-platform intelligent recommendation device in this embodiment. For the sake of brevity, it will not be described in detail here.
[0066] See Figure 4 The diagram shown is a schematic representation of the structure of an electronic device according to an embodiment of this application. In a preferred embodiment of this application, the electronic device 4 includes a memory 41, at least one processor 42, and at least one communication bus 43.
[0067] Those skilled in the art should understand that Figure 4 The structure of the electronic device shown does not constitute a limitation of the embodiments of this application. It can be a bus structure or a star structure. The electronic device 4 may also include more or fewer other hardware or software than shown, or different component arrangements.
[0068] In some embodiments, the electronic device 4 is a device capable of automatically performing numerical calculations and / or information processing according to pre-set or stored instructions. Its hardware includes, but is not limited to, microprocessors, application-specific integrated circuits (ASICs), programmable gate arrays (FPGAs), digital processors, and embedded devices. The electronic device 4 may also include user equipment, which includes, but is not limited to, any electronic product capable of human-computer interaction with a user via a keyboard, mouse, remote control, touchpad, or voice control device, such as a personal computer, tablet computer, smartphone, or digital camera.
[0069] In the embodiments provided in this application, it should be understood that the disclosed methods, apparatuses, computer-readable storage media, and electronic devices can be implemented in other ways. For example, the apparatus embodiments described above are merely illustrative. For instance, the division of modules is only a logical functional division, and in actual implementation, there may be other division methods. For example, multiple components or modules may be combined or integrated into another device, or some features may be ignored or not executed. Furthermore, the coupling or direct coupling or communication connection shown or discussed may be through some interfaces; the indirect coupling or communication connection between devices, components, or modules may be electrical, mechanical, or other forms.
[0070] The components described as separate parts may or may not be physically separate. The components shown as components may or may not be physical modules; that is, they may be located in one place or distributed across multiple network modules. Some or all of the components can be selected to achieve the purpose of this embodiment according to actual needs.
[0071] Furthermore, the functional modules in the various embodiments of the present invention can be integrated into one processing module, or each component can exist physically separately, or two or more modules can be integrated into one module. The integrated modules described above can be implemented in hardware or as software functional modules.
[0072] If the integrated module is implemented as a software functional module and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention, in essence, or the part that contributes to the prior art, or all or part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of the present invention. The aforementioned storage medium includes: USB flash drive, portable hard drive, read-only memory (ROM). Various media that can store program code, such as only memory, random access memory (RAM), magnetic disks or optical disks.
[0073] It should be noted that, for the sake of simplicity, the foregoing method embodiments are all described as a series of actions. However, those skilled in the art should understand that the present invention is not limited to the described order of actions, because according to the present invention, some steps can be performed in other orders or simultaneously. Furthermore, those skilled in the art should also understand that the embodiments described in the specification are preferred embodiments, and the actions and modules involved are not necessarily essential to the present invention.
[0074] In the above embodiments, the descriptions of each embodiment have different focuses. For parts not described in detail in a certain embodiment, please refer to the relevant descriptions of other embodiments.
[0075] The above is a detailed description of the preferred embodiments of the present invention. However, the present invention is not limited to the embodiments described. Those skilled in the art can make various equivalent modifications or substitutions without departing from the spirit of the present invention. All such equivalent modifications or substitutions are included within the scope defined by the claims of this application.
Claims
1. A cross-platform intelligent recommendation method, characterized in that, The method includes: When multimodal data from multiple platforms is received, the multimodal data is processed by a pre-built multimodal representation encoder to output an initial user / item representation with a unified dimension; the initial user / item representation includes an initial user representation and an initial item representation. User behavior sequences are extracted from the log systems of the aforementioned multi-platform systems to construct cross-platform user profiles; The initial user / item representation and the cross-platform user profile are input into a preset cross-platform sharing network to output a target user / item representation that integrates cross-platform information through the cross-platform sharing network; the target user / item representation includes a target user representation and a target item representation. The target user / item representation is input into a preset joint optimization model to output a final recommendation list.
2. The cross-platform intelligent recommendation method according to claim 1, characterized in that, The method further includes: During the offline phase, all historical data is acquired to pre-train a preset base model; the base model includes the multimodal representation encoder and the cross-platform shared network. During the online phase, real-time user behavior data is acquired to fine-tune the pre-trained base model using the real-time user behavior data.
3. The cross-platform intelligent recommendation method according to claim 2, characterized in that, The method further includes: When a sudden user behavior is detected based on the real-time user behavior data, the weight coefficient of the category corresponding to the sudden user behavior is temporarily increased, and real-time scene features are injected; the sudden user behavior pattern is defined as the frequency of the same type of behavior exceeding a preset threshold within a unit of time; the real-time scene features include at least one of time, location, device type, and current task type; The dynamic user representation is adjusted based on the weight coefficients and real-time scene features to generate a scene-adaptive user representation; the dynamic user representation is obtained by combining offline pre-training with online fine-tuning.
4. The cross-platform intelligent recommendation method according to claim 1, characterized in that, The multimodal representation encoder includes a modality-specific sub-encoder and a cross-modal attention layer. The modality-specific sub-encoder is used to extract features from multimodal data and output a multimodal feature sequence. The cross-modal attention layer is used to achieve bidirectional semantic alignment between each modality feature sequence in the multimodal feature sequence through multiple rounds of cross-attention iteration, and output the initial user / item representation.
5. The cross-platform intelligent recommendation method according to claim 1, characterized in that, The cross-platform shared network includes a shared underlying network, a platform adaptive branch network, and a cross-platform distributed constraint network. The shared underlying network is used to extract global features from the initial user / item representation based on the cross-platform user profile, learn the global interest pattern of user cross-platform behavior and the cross-platform general features of items, and output user interest features and item general features. The platform adaptive branch network is used to perform platform-specific adaptation on the user interest features and the general features of the items, and outputs the platform-adapted user interest representation and the platform-adapted item features; the cross-platform distributed constraint network is used to constrain the platform-adapted user interest representation and the platform-adapted item features, and outputs the constrained target user / item representation.
6. The cross-platform intelligent recommendation method according to claim 5, characterized in that, The cross-platform distributed constraint network uses contrast loss and migration loss as joint constraint objectives.
7. The cross-platform intelligent recommendation method according to claim 1, characterized in that, The joint optimization model includes a recall layer, a fine ranking layer, and a re-ranking layer. The recall layer is used to perform approximate nearest neighbor retrieval based on the target user / item representation to generate an initial candidate set. The fine ranking layer is used to optimize the initial candidate set through a multi-task deep model to output a target candidate set. The re-ranking layer is used to optimize the target candidate set through a balanced optimization rule to generate the final recommendation list. The recall layer and the fine ranking layer are jointly optimized by sharing a representation space.
8. A cross-platform intelligent recommendation device, characterized in that, The device includes: The multimodal data processing module is used to process multimodal data input from multiple platforms through a pre-built multimodal representation encoder when the multimodal data is received, and output an initial user / item representation with a unified dimension; the initial user / item representation includes an initial user representation and an initial item representation. A cross-platform user profile building module is used to extract user behavior sequences from the log systems of the multiple platforms to build cross-platform user profiles. The cross-platform information fusion module is used to input the initial user / item representation and the cross-platform user profile into a preset cross-platform sharing network, so as to output the target user / item representation that integrates cross-platform information through the cross-platform sharing network; the target user / item representation includes the target user representation and the target item representation; The recommendation list generation module is used to input the target user / item representation into a preset joint optimization model, so as to output the final recommendation list through the joint optimization model.
9. An electronic device, characterized in that, It includes a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the computer program to implement the steps of the cross-platform intelligent recommendation method according to any one of claims 1 to 7.
10. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by the processor, it implements the steps of the cross-platform intelligent recommendation method according to any one of claims 1 to 7.