Micro-video popularity prediction system and method

Abstract

The invention relates to a micro-video popularity prediction system and method. The method comprises the following steps: constructing a micro-video data set; respectively training a micro-video retrieval module, a multi-mode processing module, a GRU4Rec module and a prediction module by utilizing the micro-video data set; a to-be-queried micro-video is obtained, on one hand, the to-be-queried micro-video is input to a micro-video retrieval module and a multi-modal processing module to be processed, and multi-modal features are obtained; on the other hand, the information is input to a GRU4Rec module to be processed, and recommendation features are obtained; and carrying out fusion processing on the obtained multi-modal features and the recommendation features to obtain a micro-video popularity prediction result corresponding to the micro-video to be queried. Compared with the prior art, the social network information and the recommendation system information except the video content are integrated, so that the factors influencing the popularity of the micro-video are more comprehensively captured, the influence of the user behavior on the popularity of the video is simulated, the time dynamic state of the event sequence can be captured, the understanding of the user behavior and the video propagation is enhanced, and the user experience is improved. And the accuracy of micro-video popularity prediction is greatly improved.

Description

Technical Field

[0001] The present invention relates to the technical field of big data processing, and in particular to a micro video popularity prediction system and method. Background Art

[0002] With the widespread use of smart mobile terminals and the rapid development of information technology, micro-video platforms have gradually become a media method that penetrates all scenarios. Users can easily share their lives and express their opinions on these platforms, generating multi-modal content such as text, pictures, audio, and video. In order to improve the accuracy of recommendations and tap into commercial value, micro-video popularity prediction has become a key research area.

[0003] There are many methods that can be applied to the research of micro video popularity prediction, and feature engineering is one of the important algorithms. The steps of feature engineering mainly include data collection and cleaning, feature extraction and construction, feature processing and optimization, feature selection and evaluation, which can maximize the extraction of features from the original data to achieve better prediction results.

[0004] Feature engineering determines the upper limit of machine learning and has good interpretability and adjustability. However, feature engineering is highly dependent on expert knowledge and the quality of feature selection, and obtaining high-quality data is still a big challenge, which may limit the generalization ability of the model.

[0005] With the improvement and development of machine learning methods, deep learning algorithms have become a powerful tool for modeling multimodal data. Models such as HMMVED and MTFM use neural networks such as ResNet and ViT to process visual data, and neural networks such as BERT and AnglE to process text data. They have the advantage of processing multimodal data and can capture cross-modal correlations. They have good results in the field of micro-video popularity prediction. Despite this, the use of deep learning models still has the following defects: the factors that affect the popularity of micro-videos include not only the text and video of micro-videos, but also user behavior, recommendation algorithms and other factors. Many existing studies focus on cross-modal relationships within micro-videos, but do not fully utilize the full content of the data. They often only focus on limited aspects such as video and text, ignoring the in-depth analysis of user behavior and the impact of event propagation on social networks. Summary of the invention

[0006] The purpose of the present invention is to overcome the defects of the above-mentioned prior art and to provide a micro video popularity prediction system and method, which can effectively improve the prediction accuracy of micro video popularity.

[0007] The objective of the present invention can be achieved by the following technical scheme: a micro video popularity prediction system, comprising a multimodal processing module, a micro video retrieval module, a GRU4Rec (Gated Recurrent Units for Recommendations) module and a prediction module, wherein the multimodal processing module is used to extract multimodal features from micro video data, and interact with the micro video retrieval module to output a multimodal feature vector;

[0008] The micro video retrieval module performs retrieval based on multimodal features to obtain similar videos;

[0009] The GRU4Rec module processes user session sequences based on multimodal features and combines the neural Hawkes process to simulate the temporal dynamics of user behavior and outputs recommendation features;

[0010] The prediction module is used to fuse multimodal features and recommendation features, and output a micro video popularity prediction result.

[0011] A micro video popularity prediction method comprises the following steps:

[0012] S1, build a micro video dataset;

[0013] S2. Using the micro video data set, respectively train a micro video retrieval module, a multimodal processing module, a GRU4Rec module and a prediction module, wherein the micro video retrieval module is used to retrieve video data similar to the query video;

[0014] The multimodal processing module is used to extract multimodal features from the query video;

[0015] The GRU4Rec module is used to process the query video to obtain the recommended features;

[0016] The prediction module is used to integrate multimodal features and recommendation features and output micro-video popularity prediction results;

[0017] S3, obtaining the micro video to be queried, and inputting it into the micro video retrieval module and the multimodal processing module for processing to obtain multimodal features;

[0018] On the other hand, the input is sent to the GRU4Rec module for processing to obtain the recommended features;

[0019] S4. The multimodal features and recommendation features obtained in step S3 are fused and processed, and a micro video popularity prediction result corresponding to the micro video to be queried is outputted.

[0020] Furthermore, the micro video data set in step S1 includes video data watched by multiple different users on different social platforms, and is divided into a training set, a validation set and a test set according to a preset ratio.

[0021] Furthermore, the working process of the micro video retrieval module includes:

[0022] Generate retrieval vector: Convert the video into text, combine it with the original text to generate composite text, and then encode it to obtain the retrieval vector;

[0023] Retrieve similar videos: Based on the retrieval vector, similar videos are retrieved and filtered out from the memory library through similarity calculation and comparison.

[0024] Furthermore, the specific process of generating the retrieval vector is: for the micro video, the BLIP-2 model is used to analyze the video content and generate description subtitles for each frame;

[0025] Use the CLAP model to process audio and obtain audio description text annotations;

[0026] After the video subtitles, audio annotations and original text descriptions are spliced ​​together, they are processed by the UAE-Large architecture to generate a retrieval vector that can encapsulate the key attributes of the video.

[0027] Furthermore, the specific process of retrieving similar videos is: using the Bank Retriever mechanism, using the maximum inner product search algorithm to traverse the memory bank composed of frame-text pairs, and by calculating the similarity score between the target video and the instance stored in the memory bank, the top N most similar videos are screened out from the memory bank.

[0028] Furthermore, the working process of the multimodal processing module includes:

[0029] Multimodal feature extraction: extract visual, audio and text information from micro videos;

[0030] Cross-modal bipolar interaction: A cross-attention network with positive and negative attention modules is used. Positive attention captures consistent features between modalities, while negative attention highlights inconsistent information. After complex calculation and integration, a comprehensive modal representation is generated.

[0031] Retrieval interaction enhancement: The retrieval similarity scores are weighted and aggregated into instance representations, and the attention pooling strategy is used to obtain the final feature representation. At the same time, the label information is encoded to obtain the retrieval label embedding, and the feature interaction is integrated through AUG-MMRA.

[0032] Furthermore, the specific process of the multimodal feature extraction is as follows: in terms of vision, the key frames are selected and processed by a pre-trained visual model;

[0033] For audio, Audio Spectrogram Transformer is used to process audio clips to generate spectrum matrix for further processing;

[0034] In terms of text, it is processed by a pre-trained language model;

[0035] Finally, the audio-visual features are integrated and input tokens are generated to construct a multimodal feature representation.

[0036] Furthermore, the comprehensive modality representation includes the audio-visual guided text feature T i p and the positive audio-visual features of text guidance C i p :

[0037]

[0038] in, is the projection matrix.

[0039] Furthermore, the working process of the GRU4Rec module includes:

[0040] GRU4Rec part: uses GRU units to process user session sequences and capture user behavior patterns;

[0041] NHP (Neural Hawkes Process) is based on the GRU-enhanced model: the user interaction code is input into NHP, and the sequence data is processed by the GRU layer to simulate the temporal dynamics of user behavior;

[0042] The recommendation score is obtained from GRU4Rec, the recommendation likelihood probability is obtained from NHP, and the recommendation features are constructed.

[0043] Furthermore, the working process of the prediction module includes:

[0044] By integrating multimodal features and recommendation features, a multi-layer perceptron is used to predict video popularity parameters, and the mean square error is used as the loss function to optimize the parameters and output the micro video popularity prediction value.

[0045] Compared with the prior art, the present invention has the following advantages:

[0046] The present invention designs a multimodal processing module, a micro video retrieval module, a GRU4Rec module and a prediction module, wherein the micro video retrieval module provides the multimodal processing module with retrieval vectors and similar video information; the multimodal processing module transmits the extracted and processed multimodal features to the GRU4Rec module based on the neural Hawkes process; the recommendation features output by the GRU4Rec module and the multimodal features output by the multimodal processing module are input into the prediction module together, and the micro video popularity prediction result is output after calculation. The above functional modules work together to process the input micro video data in turn, and can combine the recommendation system and multimodal retrieval to fully and effectively integrate the social network and recommendation system information outside the video content, so as to more comprehensively capture the factors affecting the popularity of the micro video and significantly improve the prediction accuracy.

[0047] The present invention uses a micro-video retrieval module to generate a retrieval vector. For micro-videos, the BLIP-2 model is used to analyze the video content and generate description subtitles for each frame. The CLAP model is used to process the audio to obtain audio description text annotations. The video subtitles, audio annotations and original text descriptions are then spliced ​​and processed through the UAE-Large architecture to generate a retrieval vector that can encapsulate the key attributes of the video, thereby unifying the audio-visual text modality of the micro-video and greatly improving the accuracy of subsequent popularity prediction.

[0048] The present invention utilizes a multimodal processing module to comprehensively extract multimodal features including visual, audio and text information, and takes into account the inconsistency between the audio-visual and text modalities of micro-videos. It captures the consistent features between modalities, highlights the inconsistent information, and integrates them into a comprehensive modal representation through cross-modal bipolar interaction, which can deepen the understanding of the video content. In addition, it also effectively integrates the retrieval instance information through retrieval interaction enhancement, and can deeply mine the correlation between video features and popularity.

[0049] The present invention integrates the neural Hawkes process and the GRU4Rec model. On the one hand, GRU4Rec is used to accurately capture user behavior patterns. On the other hand, NHP is used to dynamically update event intensity and integrate complex time dependencies and nonlinear effects. The two are organically combined to comprehensively consider the impact of user behavior and event dynamics on video popularity. It can truly simulate the impact of user behavior on video popularity, enable the model to capture the temporal dynamics of event sequences, and enhance the model's understanding and prediction capabilities of user behavior and video propagation. BRIEF DESCRIPTION OF THE DRAWINGS

[0050] Figure 1 It is a schematic diagram of the method flow of the present invention;

[0051] Figure 2 A schematic diagram of the pre-training process of micro video prediction in an embodiment;

[0052] Figure 3Schematic diagram of the application process of micro video prediction in the embodiment. DETAILED DESCRIPTION

[0053] The present invention is described in detail below with reference to the accompanying drawings and specific embodiments.

[0054] Example

[0055] A micro video popularity prediction system includes a multimodal processing module, a micro video retrieval module, a GRU4Rec module and a prediction module, wherein the multimodal processing module is used to extract multimodal features from micro video data, and interact with the micro video retrieval module to output a multimodal feature vector;

[0056] The micro video retrieval module retrieves similar videos based on multimodal features;

[0057] The GRU4Rec module processes user session sequences based on multimodal features and combines the neural Hawkes process to simulate the temporal dynamics of user behavior and outputs recommendation features.

[0058] The prediction module is used to fuse multimodal features and recommendation features, and output the micro-video popularity prediction results.

[0059] Based on the above system, a micro video popularity prediction method is implemented, such as Figure 1 As shown, the following steps are included:

[0060] S1, build a micro video dataset;

[0061] S2. Using the micro video data set, respectively train a micro video retrieval module, a multimodal processing module, a GRU4Rec module and a prediction module, wherein the micro video retrieval module is used to retrieve video data similar to the query video;

[0062] The multimodal processing module is used to extract multimodal features from the query video;

[0063] The GRU4Rec module is used to process the query video to obtain the recommended features;

[0064] The prediction module is used to integrate multimodal features and recommendation features and output micro-video popularity prediction results;

[0065] S3, obtaining the micro video to be queried, and inputting it into the micro video retrieval module and the multimodal processing module for processing to obtain multimodal features;

[0066] On the other hand, the input is sent to the GRU4Rec module for processing to obtain the recommended features;

[0067] S4. The multimodal features and recommendation features obtained in step S3 are fused and processed, and a micro video popularity prediction result corresponding to the micro video to be queried is outputted.

[0068] This embodiment applies the above solution and executes the following in the GPU (Graphics Processing Unit): Figure 2 and Figure 3 The pre-training and application process shown in the figure mainly includes:

[0069] Step 1: Data preparation

[0070] The MicroLens-100k dataset is used, which contains 19,738 unique micro videos watched by 100,000 users from different online video platforms, divided into 15,779 training samples, 1,973 validation samples and 1,972 test samples.

[0071] Step 2: Construction and training of micro video retrieval module

[0072] Retrieval vector generation: For micro videos, the BLIP-2 model is used to analyze the video content and generate description subtitles for each frame; the CLAP model is used to process the audio to obtain audio description text annotations. After splicing the video subtitles, audio annotations and original text descriptions, they are processed by the UAE-Large architecture to generate a retrieval vector that can encapsulate the key attributes of the video, thereby unifying the audio-visual text modality of the micro video and improving the accuracy of popularity prediction.

[0073] Bank Retriever mechanism: This mechanism selects the top N most similar videos from the memory bank by evaluating the similarity scores between the target video and the stored instances. After receiving the query video, the maximum inner product search algorithm is used to traverse the memory bank consisting of frame-text pairs to accurately retrieve videos with high relevance.

[0074] Step 3: Multimodal processing module construction and training

[0075] Multimodal feature extraction: Extract visual, audio, and text information from micro videos. Visually, select key frames and process them using a pre-trained visual model. For audio, use Audio Spectrogram Transformer to process audio clips to generate a spectrum matrix for further processing. Text is processed by a pre-trained language model. Finally, integrate audio-visual features and generate input tokens to construct a multimodal feature representation.

[0076] Cross-modal bipolar interaction: Taking into account the inconsistency between the audio-visual and text modalities of micro-videos, a cross-attention network with positive and negative attention modules is designed. Positive attention captures consistent features between modalities, while negative attention highlights inconsistent information. After complex calculation and integration, a comprehensive modal representation is generated to deepen the understanding of video content.

[0077] Among them, the audio-visual guided text feature T i p and the positive audiovisual features of text guidance C i p The calculation is as follows:

[0078]

[0079] In the formula, is the projection matrix.

[0080] Retrieval interaction enhancement: Retrieval instances are integrated to optimize popularity prediction, and the retrieval similarity scores are weighted to aggregate instance representations, and the final feature representation is obtained with the help of the attention pooling strategy. At the same time, the label information is encoded to obtain the retrieval label embedding, and the feature interaction is integrated through AUG-MMRA to accurately capture the popularity trend.

[0081] Step 4: Construction and training of GRU4Rec module based on neural Hawkes process

[0082] GRU4Rec: In the popularity prediction model, GRU4Rec is crucial to capturing user session sequences and behavior patterns. It embeds user click events into feature vectors and then processes them through the GRU layer to capture long-term and short-term dependencies. It then passes through the fully connected layer to obtain recommendation scores, which are then converted into probability distributions for popularity prediction through the sigmoid function. The calculation core of GRU is:

[0083] NHP is based on the GRU enhanced model part: the neural Hawkes process processes the hidden state of the recurrent neural network according to the past event sequence to adjust the intensity of future events accordingly. When integrating NHP with GRU4Rec, the user interaction is encoded into NHP, and the time gating mechanism is used to learn the impact of time intervals on model state updates. The final prediction is obtained by processing the sequence data through the GRU layer.

[0084] Step 5: Prediction network construction and training

[0085] The concatenated vector of micro-video feature components is input into the AUG-MMRA output layer and processed by the fully connected layer. At the same time, the recommendation score of GRU4Rec and the recommendation possibility of NHP are combined to predict the popularity parameters of the target video through a multi-layer perceptron, and the mean square error is used as the loss function during training.

[0086] This scheme constructs a RecMMR model (i.e., micro-video popularity prediction model), in which the connection relationship between the modules is as follows: the micro-video retrieval module provides the retrieval vector and similar video information to the multi-modal processing module; the multi-modal processing module transmits the extracted and processed multi-modal features to the GRU4Rec module based on the neural Hawkes process; the output of the GRU4Rec module and the features of the multi-modal processing module are jointly input into the prediction network, and the micro-video popularity prediction result is output after calculation. Each module cooperates and processes the input data in turn to form a complete micro-video popularity prediction process.

[0087] The features of each module are:

[0088] Micro video retrieval module: Through advanced visual, audio model and language model processing, the multimodal information of micro videos is accurately unified as a retrieval vector. With the help of the Bank Retriever mechanism, similar videos are efficiently retrieved, providing key references for subsequent predictions and improving prediction accuracy.

[0089] Multimodal processing module: comprehensively extracts multimodal features of audio-visual texts, cleverly handles modal inconsistency issues through cross-modal bipolar interaction, effectively integrates retrieval instance information through retrieval interaction enhancement, and deeply mines the relationship between video features and popularity, laying a solid foundation for accurate prediction.

[0090] GRU4Rec module based on neural Hawkes process: GRU4Rec accurately captures user behavior patterns, and NHP dynamically updates event intensity, integrating complex time dependencies and nonlinear effects. The two are organically combined to comprehensively consider the impact of user behavior and event dynamics on video popularity, significantly enhancing the model prediction performance.

[0091] Prediction network module: scientifically integrate multi-module features and recommendation information, use multi-layer perceptron as the core to predict popularity parameters, use mean square error as the loss function to effectively optimize model parameters, and ensure stable and accurate output of popularity prediction values. The calculation formula of mean square error is:

[0092] The effectiveness of this solution has not been verified. This embodiment has been experimented on the MicroLens-100k dataset, comparing RecMMR with 9 baseline models (CBAN, CLSTM, HMMVED, MASSL, SVR, MFTM, Hyfea, TMALL, MMRA), and establishing ablation experiments on each module (Audio, GRU, NHP) to verify the advancement of the model and the effectiveness of each module. The experiments were trained and tested according to the experimental regulations of the corresponding datasets.

[0093] From the comparison in Table 1, it can be seen that the prediction accuracy (Pearson correlation coefficient PLCC, Spearman correlation coefficient SRC) of the RecMMR model on the test set is higher than that of the other baselines, and the test loss (normalized mean square error nMSE, mean absolute error MAE) is lower than the test error of the other baselines, which fully proves that the RecMMR model of this solution performs better in predicting the popularity of short videos. On MicroLens-100k, the test PLCC, test SRC, test nMSE, and test MAE of this solution are 0.5573, 0.5587, 0.7051, and 23.097, respectively. The experimental results are better than the currently published mainstream algorithms and have a high recognition rate.

[0094] Table 1 Comparison of Baseline experimental data

[0095] Model nMSE SRC MAE PLCC CBAN 0.8169 0.4659 25.203 0.4288 CLSTM 0.8416 0.4706 25.533 0.4213 HMMVED 0.8926 0.3526 27.402 0.3232 MASSL 0.9273 0.3804 25.163 0.3292 SVR 1.0193 0.4048 25.043 0.3212 MFTM 0.876 0.4137 26.417 0.3524 Hyfea 0.8400 0.4320 26.099 0.4082 TMALL 0.8564 0.4395 26.114 0.4013 MMRA 0.7854 0.4947 24.339 0.4804 RecMMR 0.7051 0.5587 23.097 0.5573

[0096] In addition, this embodiment also establishes an ablation experiment, respectively eliminating the Audio module (noAudio), the GRU module (noGRU) and the NHP module (noNHP) from the model, adding the complete model (RecMMR), and then training these four models separately. Afterwards, the top 50, 200 and 500 videos with the highest real popularity, and the bottom 50, 200 and 500 videos with the lowest scores were selected from the test samples, and the popularity was predicted using the above four models, and the average predicted popularity score was calculated. From the ablation experiment in Table 2, it can be seen that the order of the predicted popularity scores of the six groups is: Top50>Top200>Top500>Bottom500>Bottom200>Bottom50, which shows that the performance of the prediction model is in line with expectations. In addition, compared with the actual number of comments in each group, RecMMR achieved the best performance (indicated by bold and italics) in most cases, which is closest to the true value. In terms of evaluation indicators, RecMMR outperforms the other three variants in nMSE, SRC, and MAE, verifying the effectiveness of adding audio information, multimodal information for users, and feedback sequences for videos. It is worth noting that removing the neural Hawkes process will lead to a significant drop in performance, which emphasizes the effectiveness of aggregating user feedback sequences and mining video recommendation mechanisms for popularity prediction.

[0097] Table 2 Comparison of ablation model performance data

[0098] Prediction noAudio noGRU noNHP RecMMR Top50 75.000 87.908 71.688 88.030 Top200 57.690 64.322 55.616 64.266 Top500 43.111 44.420 41.604 44.443 Bottom500 27.714 26.017 26.339 25.866 Bottom200 22.638 22.544 22.818 22.447 Bottom50 23.409 22.414 19.536 22.183 nMSE 0.7413 0.7080 0.8002 0.7051 SRC 0.5286 0.5477 0.5064 0.5587 MAE 23.814 23.198 24.359 23.097

[0099] In summary, this scheme combines the recommendation mechanism with multimodal retrieval for the first time for micro-video popularity prediction, which can effectively integrate social network and recommendation system information outside the video content, so as to more comprehensively capture the factors affecting the popularity of micro-videos and significantly improve the prediction accuracy. This scheme also integrates the neural Hawkes process and the GRU4Rec model to simulate the impact of user behavior on video popularity, so that the model can capture the temporal dynamics of event sequences and enhance the model's understanding and prediction of user behavior and video propagation. After experimental comparison, the prediction accuracy of this scheme is better than the current mainstream algorithm on MicroLens-100k, and has achieved a 12.9% performance improvement on key evaluation indicators, surpassing the existing state-of-the-art methods.

Claims

1. A micro video popularity prediction system, characterized in that: It includes a multimodal processing module, a micro video retrieval module, a GRU4Rec module and a prediction module. The multimodal processing module is used to extract multimodal features from micro video data, and interact with the micro video retrieval module to output a multimodal feature vector. The micro video retrieval module performs retrieval based on multimodal features to obtain similar videos; The GRU4Rec module processes user session sequences based on multimodal features and combines the neural Hawkes process to simulate the temporal dynamics of user behavior and outputs recommendation features; The prediction module is used to fuse multimodal features and recommendation features, and output a micro video popularity prediction result.

2. A micro video popularity prediction method, characterized in that: The following steps are involved: S1, build a micro video dataset; S2. Using the micro video data set, respectively train a micro video retrieval module, a multimodal processing module, a GRU4Rec module and a prediction module, wherein the micro video retrieval module is used to retrieve video data similar to the query video; The multimodal processing module is used to extract multimodal features from the query video; The GRU4Rec module is used to process the query video to obtain the recommended features; The prediction module is used to integrate multimodal features and recommendation features and output micro-video popularity prediction results; S3, obtaining the micro video to be queried, and inputting it into the micro video retrieval module and the multimodal processing module for processing to obtain multimodal features; On the other hand, the input is sent to the GRU4Rec module for processing to obtain the recommended features; S4. The multimodal features and recommendation features obtained in step S3 are fused and processed, and a micro video popularity prediction result corresponding to the micro video to be queried is outputted.

3. A micro video popularity prediction method according to claim 2, characterized in that: In step S1, the micro video data set includes video data watched by multiple different users on different social platforms, and is divided into a training set, a validation set and a test set according to a preset ratio.

4. A micro video popularity prediction method according to claim 2, characterized in that: The working process of the micro video retrieval module includes: Generate retrieval vector: Convert the video into text, combine it with the original text to generate composite text, and then encode it to obtain the retrieval vector; Retrieve similar videos: Based on the retrieval vector, similar videos are retrieved and filtered out from the memory library through similarity calculation and comparison.

5. A micro video popularity prediction method according to claim 4, characterized in that: The specific process of generating the retrieval vector is as follows: for the micro video, the BLIP-2 model is used to analyze the video content and generate description subtitles for each frame; Use the CLAP model to process audio and obtain audio description text annotations; After the video subtitles, audio annotations and original text descriptions are spliced ​​together, they are processed by the UAE-Large architecture to generate a retrieval vector that can encapsulate the key attributes of the video; The specific process of retrieving similar videos is: using the Bank Retriever mechanism, using the maximum inner product search algorithm to traverse the memory bank composed of frame-text pairs, and by calculating the similarity score between the target video and the instance stored in the memory bank, the top N most similar videos are screened out from the memory bank.

6. A micro video popularity prediction method according to claim 2, characterized in that: The working process of the multimodal processing module includes: Multimodal feature extraction: extract visual, audio and text information from micro videos; Cross-modal bipolar interaction: A cross-attention network with positive and negative attention modules is used. Positive attention captures consistent features between modalities, while negative attention highlights inconsistent information. After complex calculation and integration, a comprehensive modal representation is generated. Retrieval interaction enhancement: The retrieval similarity scores are weighted and aggregated into instance representations, and the attention pooling strategy is used to obtain the final feature representation. At the same time, the label information is encoded to obtain the retrieval label embedding, and the feature interaction is integrated through AUG-MMRA.

7. A micro video popularity prediction method according to claim 6, characterized in that: The specific process of the multimodal feature extraction is as follows: in terms of vision, key frames are selected and processed by a pre-trained visual model; For audio, Audio Spectrogram Transformer is used to process audio clips to generate spectrum matrix for further processing; In terms of text, it is processed by a pre-trained language model; Finally, the audio-visual features are integrated and input tokens are generated to construct a multimodal feature representation.

8. A micro video popularity prediction method according to claim 6, characterized in that: The comprehensive modality representation includes audio-visual guided text features T i p and the positive audiovisual features of text guidance C i p : in, is the projection matrix.

9. A micro video popularity prediction method according to claim 2, characterized in that: The working process of the GRU4Rec module includes: GRU4Rec part: uses GRU units to process user session sequences and capture user behavior patterns; NHP is based on the GRU-enhanced model: the user interaction code is input into NHP, and the sequence data is processed by the GRU layer to simulate the temporal dynamics of user behavior; The recommendation score is obtained from GRU4Rec, the recommendation likelihood probability is obtained from NHP, and the recommendation features are constructed.

10. A micro video popularity prediction method according to claim 2, characterized in that: The working process of the prediction module includes: By integrating multimodal features and recommendation features, a multi-layer perceptron is used to predict video popularity parameters, and the mean square error is used as the loss function to optimize the parameters and output the micro video popularity prediction value.

Images