A short video popularity prediction method and device based on learnable graph enhancement

Abstract

This invention discloses a method and apparatus for predicting the popularity of short videos based on learnable graph augmentation, belonging to the field of artificial intelligence technology. The method includes: collecting keyframe sets, text data sets, and behavioral data sets from short videos; constructing a graph edge to obtain a heterogeneous social graph; inputting this graph into a heterogeneous attribute graph neural network; extracting content features from the target video; performing odd-hop propagation and even-hop propagation; employing feature fusion operations and learnable sparse masks; and inputting this into a trained popularity prediction model to obtain the predicted future popularity of the target video. This invention, by integrating learnable graph augmentation and social propagation relationship modeling, can effectively utilize unlabeled data even when labeled samples are insufficient, enhancing the model's robustness and generalization ability, thereby more accurately predicting the popularity of short videos.

Description

Technical Field

[0001] This invention relates to the field of artificial intelligence technology, and in particular to a method and apparatus for predicting the popularity of short videos based on learnable graph augmentation. Background Technology

[0002] With the rapid development of short video platforms, a massive amount of short video content is continuously generated. User interaction behaviors (such as likes, comments, and shares) have a significant impact on the speed and scope of video dissemination. Therefore, predicting the future popularity of short videos can provide important references for content distribution strategies, recommendation system optimization, and platform operation decisions. Current commonly used methods mostly rely on visual content and textual information to build prediction models, but they neglect user interaction behaviors and the social propagation relationships between videos, resulting in insufficient modeling of the impact during actual dissemination. On the other hand, short video data generally exhibits a scarcity of annotations; only a small number of videos have reliable popularity data, while a large number of newly uploaded videos lack annotations, making it difficult for prediction models to maintain stable performance in real-world platform scenarios. Existing methods still have limitations in enhancing the utilization of unlabeled samples, preserving graph structure information, and performing robust predictions in social relationship environments. Therefore, a short video popularity prediction method is needed that can fully utilize multimodal information and social propagation structures, and still have good generalization ability in scenarios with insufficient annotation. Summary of the Invention

[0003] To address the technical problems of existing technologies that neglect user interaction behavior and the social propagation relationships between videos, resulting in insufficient modeling of the impact during actual propagation, this invention provides a method and apparatus for predicting the popularity of short videos based on learnable graph augmentation. The technical solution is as follows: On the one hand, a short video popularity prediction method based on learnable graph augmentation is provided. This method is implemented by a short video popularity prediction device based on learnable graph augmentation, and includes: S1: Collect keyframe sets, text data sets, and behavior data sets of the original short video data, input them into a pre-trained visual coding model, perform behavior feature extraction processing, and then instantiate them into different types of nodes. Perform graph edge construction to obtain a heterogeneous node set and a heterogeneous edge set. After graph structure integration processing, a heterogeneous social graph is obtained. The heterogeneous node set includes video nodes, user nodes, and attribute nodes. S2: Using a feature masking or replacement strategy, the heterogeneous social graph is processed into a single modality and input into a heterogeneous attribute graph neural network for fusion processing to obtain an initial multimodal representation of the short video. The initial multimodal representation of the short video includes multimodal fusion node output and single modal node output. S3: Through the heterogeneous social graph, content features are extracted, odd-hop propagation and even-hop propagation are performed, and the sparse random projection matrix is ​​input. After fusion and aggregation by the fusion module, the video node propagation representation is obtained. The video node propagation representation has semantic enhancement of propagation structure and content features. S4: Select the target video, and based on the propagation representation of the video nodes, perform global semantic similarity calculation and similar sample screening, then use feature fusion operation, employ a learnable sparse mask, and obtain a preset number of enhanced views through multiple random forward propagations; S5: Using consistency constraints, the prediction results of the preset number of enhanced views are regularized and optimized, and then input into the trained popularity prediction model to output the future popularity prediction results of the target video.

[0004] Preferably, the keyframe set, text data set, and behavior data set of the raw short video data collected in S1 are input into a pre-trained visual coding model for behavior feature extraction processing. These are then instantiated into different types of nodes, and graph edge construction is performed to obtain a heterogeneous node set and a heterogeneous edge set. After graph structure integration processing, a heterogeneous social graph is obtained. The heterogeneous node set includes video nodes, user nodes, and attribute nodes, including: S11: Collect raw short video data through short video data sources; S12: Using a unified sampling method, keyframes are extracted from the visual frames of each short video in the original short video data to obtain a keyframe set; S13: For each short video in the original short video data, collect the video title and descriptive text generated from the visual content to obtain a text data set; S14: For each short video in the original short video data, obtain the user's likes, comments, and playback interaction data between the user and the video to obtain a set of behavioral data; S15: Input the set of keyframes into a pre-trained visual coding model to obtain the visual features corresponding to each short video; S16: Input the text data set into the text encoding model to extract semantic information, and obtain the title features, descriptive text features and video-attribute association relationship; S17: Based on the behavioral data set, after behavioral feature extraction processing, user interaction features and user-video interaction relationships are obtained; S18: Using a preset node type definition, user interaction features, visual features, title features, and descriptive text features are instantiated into different types of nodes to obtain a heterogeneous node set, which includes video nodes, user nodes, and attribute nodes. S19: Based on the user-video interaction relationship and the video-attribute association relationship, construct a graph edge to obtain a heterogeneous edge set, which includes user-video interaction edges and video-attribute association edges; S110: Based on the heterogeneous node set and the heterogeneous edge set, a heterogeneous social graph is obtained through graph structure integration processing.

[0005] Preferably, step S2 employs a feature masking or replacement strategy to perform unimodal processing on the heterogeneous social graph, inputs it into a heterogeneous attribute graph neural network, and performs fusion processing to obtain an initial multimodal representation of the short video. The initial multimodal representation of the short video includes multimodal fusion node outputs and unimodal node outputs, including: S21: Using a feature masking or replacement strategy, the heterogeneous social graph is processed into a single modality to obtain a single modality graph. The structure of the single modality graph retains only the attribute information corresponding to the preset modality. S22: Input the heterogeneous social graph into the heterogeneous attribute graph neural network to perform graph feature propagation and representation learning, and obtain the multimodal fusion node output, wherein the multimodal fusion node output includes the learned cross-modal semantics; S23: Input the single-modal graph into the heterogeneous attribute graph neural network to perform graph feature propagation and representation learning to obtain the single-modal node output, wherein the single-modal node output includes the enhanced semantic expression of the modality itself; S24: Input the output of the multimodal fusion node and the output of the single-modal node into the fusion module for feature fusion to obtain the initial multimodal representation of the short video. The initial multimodal representation has modal independence and cross-modal semantic correlation.

[0006] Preferably, in step S3, content feature extraction, odd-hop propagation, and even-hop propagation are performed through the heterogeneous social graph. A sparse random projection matrix is ​​input, and the matrix is ​​fused and aggregated by a fusion module to obtain a video node propagation representation. This video node propagation representation has semantic enhancements in terms of propagation structure and content features, including: S31: Based on the type of relation edges in the heterogeneous social graph, content features are extracted to obtain initial node features; S32: Using the heterogeneous social graph, a positive adjacency matrix of a specific relationship type is used to perform odd-hop propagation to obtain direct interaction propagation features, which are used to characterize the direct interaction relationships between nodes. S33: Using the heterogeneous social graph, even-numbered hop propagation is performed using a reverse adjacency matrix of a specific relationship type and the corresponding degree matrix to obtain indirect diffusion propagation features, which are used to characterize the indirect diffusion influence between nodes. S34: Input the direct interactive propagation features and indirect diffusion propagation features into the sparse random projection matrix to compress the dimension, and obtain the compressed multi-hop propagation information; S35: Based on the compressed multi-hop propagation information, the fusion module processes the information to obtain the fused propagation features of the fused content semantics and social propagation structure; S36: Aggregate the fused propagation features with the initial node features to obtain a video node propagation representation, which has semantic enhancement of propagation structure and content features.

[0007] Preferably, the selected target video in S4, based on the video node propagation representation, undergoes global semantic similarity calculation and similar sample screening, followed by feature fusion operation, and employs a learnable sparse mask. After multiple random forward propagations, a preset number of enhanced views are obtained, including: S41: Select the target video, and based on the propagation representation of the video nodes, calculate the similarity value between the target video node and other nodes through global semantic similarity calculation; S42: Using the similarity value, similar samples are filtered on the heterogeneous social graph to obtain a sample set that is semantically similar to the target video node. The sample set includes a preset number of video nodes with the highest similarity ranking. S43: Based on the sample set, supplementary feature information is obtained by averaging the relation channel features, wherein the relation channel features include feature vectors that characterize different types of semantic relationships between nodes; S44: A feature fusion operation is used to aggregate the supplementary feature information with the feature information of the target video node in the current propagation view to obtain a retrieval enhancement representation. The feature fusion operation includes concatenation, addition and weighted summation. S45: A learnable sparse mask is used to sparsify the retrieval enhancement representation in the feature dimension to obtain the masked feature representation. The sparse mask is used to adaptively select important features during training. S46: Based on the feature representation after the mask, a preset number of enhanced views are obtained through multiple random forward propagations, wherein the random forward propagation includes introducing random deactivation or noise into the network.

[0008] Preferably, step S5 employs consistency constraints to regularize and optimize the prediction results of the preset number of enhanced views, inputs them into the trained popularity prediction model, and outputs the future popularity prediction result of the target video, including: S51: Using consistency constraints, the prediction results of the preset number of augmented views are regularized to obtain stable prediction results. The consistency constraints aim to minimize the differences in model prediction results under different augmented views. S52: By employing diversity constraints, the stable prediction results are optimized to obtain a structure-preserving enhanced representation. The diversity constraints aim to prevent the feature representations of multiple enhanced views from collapsing into the same subspace. S53: Input the enhanced representation that preserves the structure into the trained popularity prediction model to obtain intermediate inference results; S54: Analyze and perform feature mapping on the intermediate inference results, and output the future popularity prediction results of the target video.

[0009] Preferably, the training method for the popularity prediction model includes: S61: Obtain video samples with real popularity labels to obtain a labeled sample set; S62: Obtain video samples without real popularity labels to get an unlabeled sample set; S63: Based on the labeled sample set, the supervised loss is calculated by the supervised regression module; S64: Using a data augmentation module, the propagation representation in the unlabeled sample set is randomly augmented to obtain a preset number of augmented views; S65: Construct a prediction network; S66: Input the preset number of enhanced views into the prediction network to obtain the corresponding popularity prediction results; S67: Based on the popularity prediction results, the consistency loss function is used to calculate the consistency regularization loss, which is used to measure the difference or similarity of prediction results under different enhanced views. S68: Statistically analyze the corresponding popularity prediction results to obtain a batch prediction distribution; S69: The diversity regularization module is used to calculate the batch prediction distribution to obtain the diversity regularization loss, which is used to prevent the model prediction results from collapsing. S610: Based on the supervision loss, consistency regularization loss, and diversity regularization loss, a joint loss function is obtained by weighted summation; S611: Using the backpropagation algorithm, the parameters of the prediction network are semi-supervised and trained based on the joint loss function to obtain the trained popularity prediction model. The semi-supervised optimization training includes updating the network parameters of the prediction network.

[0010] On the other hand, a short video popularity prediction device based on learnable graph augmentation is provided. This device is applied to a short video popularity prediction method based on learnable graph augmentation, and the device includes: Heterogeneous social graph module: used to collect keyframe sets, text data sets and behavior data sets of raw short video data, input them into a pre-trained visual coding model, perform behavior feature extraction processing, and then instantiate them into different types of nodes, perform graph edge construction, and obtain heterogeneous node sets and heterogeneous edge sets. After graph structure integration processing, a heterogeneous social graph is obtained. The heterogeneous node set includes video nodes, user nodes and attribute nodes. Multimodal representation module: used to perform unimodal processing on the heterogeneous social graph using feature masking or replacement strategies, input it into the heterogeneous attribute graph neural network, perform fusion processing, and obtain the initial multimodal representation of the short video. The initial multimodal representation of the short video includes multimodal fusion node output and unimodal node output. The semantic enhancement module is used to extract content features, perform odd-hop propagation and even-hop propagation through the heterogeneous social graph, input a sparse random projection matrix, and fuse and aggregate it through the fusion module to obtain the video node propagation representation. The video node propagation representation has semantic enhancement of propagation structure and content features. Enhanced View Module: This module selects a target video and, based on the propagation representation of the video nodes, performs global semantic similarity calculation and similar sample filtering, then employs feature fusion operations and a learnable sparse mask. After multiple random forward propagations, a preset number of enhanced views are obtained. Popularity prediction module: It is used to regularize and optimize the prediction results of the preset number of enhanced views by applying consistency constraints, input them into the trained popularity prediction model, and output the future popularity prediction results of the target video.

[0011] On the other hand, a short video popularity prediction device based on learnable graph augmentation is provided. The short video popularity prediction device based on learnable graph augmentation includes: a processor; a memory, wherein computer-readable instructions are stored in the memory, and when the computer-readable instructions are executed by the processor, the method described in any one of the above-described short video popularity prediction methods based on learnable graph augmentation is implemented.

[0012] On the other hand, a computer-readable storage medium is provided, characterized in that program code is stored in the computer-readable storage medium.

[0013] The beneficial effects of the technical solutions provided in the embodiments of the present invention include at least the following: Significantly improve model performance in small sample scenarios by integrating learnable graph augmentation technology with social propagation relationship modeling, which can deeply mine potential features and correlation information in unlabeled data without relying on a large amount of manually labeled data.

[0014] Graph augmentation mechanisms can adaptively learn the topological structure and complex relationships of data, while social propagation relationship modeling captures the dynamic patterns of user behavior in the real world, improving generalization performance.

[0015] It provides flexible and versatile technical solutions that are not only applicable to short video popularity prediction scenarios, but can also be migrated and applied to other fields involving graph structure data and social relationship modeling (such as information flow recommendation, user influence analysis, public opinion prediction, etc.). Attached Figure Description

[0016] To more clearly illustrate the technical solutions in the embodiments of the present invention, the accompanying drawings used in the description of the embodiments will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0017] Figure 1 This is a flowchart of a short video popularity prediction method based on learnable graph enhancement provided by an embodiment of the present invention; Figure 2 This is a schematic diagram of a perceptual heterogeneous graph provided in an embodiment of the present invention; Figure 3 This is a block diagram of a short video popularity prediction device based on learnable graph enhancement provided in an embodiment of the present invention; Figure 4 This is a schematic diagram of the structure of a short video popularity prediction device based on learnable graph enhancement provided in an embodiment of the present invention. Detailed Implementation

[0018] The technical solution of the present invention will now be described with reference to the accompanying drawings.

[0019] In embodiments of the present invention, words such as "exemplarily," "for example," etc., are used to indicate that something is an example, illustration, or description. Any embodiment or design described as "exemplary" in the present invention should not be construed as being more preferred or advantageous than other embodiments or designs. Specifically, the use of the word "exemplary" is intended to present the concept in a concrete manner. Furthermore, in embodiments of the present invention, the meaning expressed by "and / or" can be both, or either one.

[0020] In the embodiments of this invention, the terms "image" and "picture" may sometimes be used interchangeably. It should be noted that, without emphasizing the distinction between them, they convey the same meaning. Similarly, the terms "of," "corresponding (relevant)," and "corresponding" may sometimes be used interchangeably. It should be noted that, without emphasizing the distinction between them, they convey the same meaning.

[0021] In this embodiment of the invention, sometimes a subscript such as W1 may be written in a non-subscript form such as W1. When the difference is not emphasized, the meaning they express is the same.

[0022] To make the technical problems, technical solutions and advantages of the present invention clearer, a detailed description will be given below in conjunction with the accompanying drawings and specific embodiments.

[0023] This invention provides a method for predicting the popularity of short videos based on learnable graph augmentation. This method can be implemented by a short video popularity prediction device based on learnable graph augmentation, which can be a terminal or a server. Figure 1 The flowchart shown is for a short video popularity prediction method based on learnable graph augmentation. The processing flow of this method may include the following steps: Preferably, keyframe sets, text data sets, and behavior data sets of raw short video data are collected, input into a pre-trained visual coding model, and processed for behavior feature extraction. These are then instantiated into different types of nodes, and graph edge construction is performed to obtain a heterogeneous node set and a heterogeneous edge set. After graph structure integration processing, a heterogeneous social graph is obtained. The heterogeneous node set includes video nodes, user nodes, and attribute nodes, including: Collect raw short video data from short video data sources; Using a unified sampling method, keyframes are extracted from the visual frames of each short video in the original short video data to obtain a keyframe set; For each short video in the original short video data, the video title and descriptive text generated from the visual content are collected to obtain a text data set; For each short video in the original short video data, obtain the user's interaction data between the user and the video, including likes, comments, and playback behavior data, to obtain a set of behavioral data; The keyframe set is input into a pre-trained visual coding model to obtain the visual features corresponding to each short video. The text dataset is input into a text encoding model for semantic information extraction, yielding title features, descriptive text features, and video-attribute associations. Based on the aforementioned behavioral data set, user interaction features and user-video interaction relationships are obtained through behavioral feature extraction processing. Using a preset node type definition, user interaction features, visual features, title features, and descriptive text features are instantiated into different types of nodes to obtain a heterogeneous node set, which includes video nodes, user nodes, and attribute nodes. Based on the user-video interaction relationship and the video-attribute association relationship, a graph edge is constructed to obtain a heterogeneous edge set, which includes user-video interaction edges and video-attribute association edges. Based on the heterogeneous node set and the heterogeneous edge set, a heterogeneous social graph is obtained through graph structure integration processing.

[0024] In some embodiments, several keyframes are extracted from each short video using a uniform sampling method, and these keyframes are input into a pre-trained visual coding model to obtain the corresponding visual features, denoted as . Simultaneously, the video title and descriptive text generated from the visual content are input into a text encoding model to extract semantic information, resulting in title features and descriptive text features, denoted as follows: and .

[0025] Furthermore, based on user interactions with videos (including likes, comments, and plays), user interaction information is extracted to characterize the short video dissemination relationships. According to different data instance types, videos, users, and the aforementioned multimodal attributes are respectively set as different types of nodes, and corresponding graph edges are constructed based on user-video interaction relationships and video-attribute association relationships, thus forming a heterogeneous social graph. Its overall structure can be represented as follows: .

[0026] Among them, the node set Includes video nodes, user nodes, and attribute nodes, and an edge set. This includes user-video interaction relationships and video-attribute association relationships. The heterogeneous social graph constructed through this step simultaneously represents the multimodal content and social propagation relationships of short videos, providing structured input for subsequent graph representation learning and propagation modeling.

[0027] Preferably, a feature masking or replacement strategy is employed to perform unimodal processing on the heterogeneous social graph, which is then input into a heterogeneous attribute graph neural network for fusion processing to obtain an initial multimodal representation of the short video. This initial multimodal representation of the short video includes multimodal fusion node outputs and unimodal node outputs, including: The heterogeneous social graph is processed into a single modality by employing a feature masking or replacement strategy to obtain a single modality graph. The structure of the single modality graph retains only the attribute information corresponding to the preset modality. The heterogeneous social graph is input into a heterogeneous attribute graph neural network to perform graph feature propagation and representation learning, and a multimodal fusion node output is obtained, which includes the learned cross-modal semantics. The unimodal graph is input into the heterogeneous attribute graph neural network to perform graph feature propagation and representation learning, and the unimodal node output is obtained. The unimodal node output includes the enhanced semantic expression of the modality itself. The output of the multimodal fusion node and the output of the single-modal node are input into the fusion module for feature fusion to obtain the initial multimodal representation of the short video. The initial multimodal representation has modal independence and cross-modal semantic correlation.

[0028] In some embodiments, the multimodal fusion graph retains the association between video nodes and all their multimodal attributes for extracting cross-modal semantics; the single-modal graph retains only the attribute information corresponding to a specific modality, masking or replacing features irrelevant to the current modality to avoid interference between different modalities and enhance the semantic expression of that modality itself. By performing graph feature propagation and representation learning separately on the two types of graphs and fusing their node outputs, an initial video representation that combines modality independence and cross-modal semantic association is obtained, providing input for subsequent propagation modeling and graph augmentation.

[0029] Preferably, through the heterogeneous social graph, content feature extraction, odd-hop propagation, and even-hop propagation are performed. A sparse random projection matrix is ​​input, and the data is fused and aggregated by a fusion module to obtain a video node propagation representation. This video node propagation representation has semantic enhancement of propagation structure and content features, including: Based on the type of relation edges in the heterogeneous social graph, content features are extracted to obtain initial node features; Using the heterogeneous social graph, a positive adjacency matrix of a specific relationship type is used to perform odd-hop propagation to obtain direct interaction propagation features, which are used to characterize the direct interaction relationships between nodes. Using the heterogeneous social graph, even-hop propagation is performed using a reverse adjacency matrix of a specific relationship type and the corresponding degree matrix to obtain indirect diffusion propagation features, which are used to characterize the indirect diffusion influence between nodes. The direct interactive propagation features and indirect diffusion propagation features are input into a sparse random projection matrix for dimensionality compression to obtain compressed multi-hop propagation information. Based on the compressed multi-hop propagation information, the fusion module processes it to obtain the fused propagation features of the fused content semantics and social propagation structure; The fused propagation features are aggregated with the initial node features to obtain a video node propagation representation, which has semantic enhancements in propagation structure and content features.

[0030] In some embodiments, a heterogeneous graph neural network based on propagation perception performs multi-hop feature propagation and aggregation on video nodes, and integrates content semantics and social propagation structure through a multi-hop propagation mechanism that separates odd and even numbers to obtain a propagation representation.

[0031] Specifically, for any node feature in a heterogeneous graph For different types of relation edges, propagation operations of odd hops (representing direct interactive propagation) and even hops (representing indirect diffusion propagation) are performed respectively. Specifically, based on relation type... For example, its odd-hop propagation is represented as:

[0032] Even-numbered hop propagation is represented as:

[0033] in These are the node features after the k-th round of odd-numbered hop propagation under relation e. It is the inverse of the degree matrix corresponding to relation e. and Relationship The forward and reverse adjacency matrices, This is the corresponding degree matrix. These are the features of all nodes after the (k-1)th iteration. To avoid the increase in feature dimensionality caused by multi-hop aggregation, the propagated features are multiplied by a sparse random projection matrix. Compression is represented as:

[0034] in, It is the final feature of node v after the k-th iteration. It is L2 normalization. It is the sparse random projection matrix corresponding to the odd / even hops in the k-th round under relation e. It is the compressed propagation characteristic.

[0035] Through the aforementioned propagation and random projection compression mechanism, multi-hop propagation information can be aggregated without destroying the heterogeneous relational structure, enabling node representations to simultaneously possess both propagation structure and content features as semantic enhancements, thus providing propagation-aware representations for subsequent graph enhancement and prediction.

[0036] The target video is selected, and based on the propagation representation of the video nodes, after global semantic similarity calculation and similar sample screening, feature fusion operation is performed, and a learnable sparse mask is used. After multiple random forward propagations, a preset number of enhanced views are obtained. Preferably, a target video is selected, and based on the propagation representation of the video nodes, global semantic similarity calculation and similar sample screening are performed. Then, feature fusion operation is adopted, and a learnable sparse mask is used. After multiple random forward propagations, a preset number of enhanced views are obtained, including: Select a target video, and based on the propagation representation of the video nodes, calculate the similarity value between the target video node and other nodes through global semantic similarity calculation; Using the similarity value, similar samples are filtered on the heterogeneous social graph to obtain a sample set that is semantically similar to the target video node. The sample set includes a preset number of video nodes with the highest similarity ranking. Based on the sample set, supplementary feature information is obtained by averaging the relation channel features. The relation channel features include feature vectors that characterize different types of semantic relationships between nodes. A feature fusion operation is used to aggregate the supplementary feature information with the feature information of the target video node in the current propagation view to obtain a retrieval enhancement representation. The feature fusion operation includes concatenation, addition and weighted summation. A learnable sparse mask is used to sparsify the retrieval enhancement representation in the feature dimension to obtain the masked feature representation. The sparse mask is used to adaptively select important features during training. Based on the feature representation after the mask, a preset number of enhanced views are obtained through multiple random forward propagations, wherein the random forward propagation includes introducing random deactivation or noise into the network.

[0037] In some embodiments, a learnable graph data augmentation module generates structure-preserving augmented representations under consistency and diversity constraints to improve the ability to utilize propagation features under semi-supervised conditions.

[0038] Specifically, in terms of communication representation Based on this, a set of samples similar to the target video node is selected based on global semantic similarity, and their relational channel features are averaged to supplement the feature information of the node in the current propagation view, thus forming an enhanced retrieval representation. Subsequently, a learnable sparse mask is introduced into the feature dimension of the enhanced propagation representation, and several enhanced views are generated through multiple random forward propagations. Consistency constraints are used to keep the prediction results of different enhanced views stable. At the same time, diversity constraints are introduced to avoid feature collapse caused by multiple enhanced views, thereby generating differentiated enhanced representations without destroying the propagation structure.

[0039] For the An augmented view, whose prediction result is denoted as... Consistency constraints are used to make the prediction results of multiple views tend to be consistent, and can be expressed as:

[0040] in To increase the number of views, This represents the average of the prediction results for each augmented view. To avoid excessive similarity among augmented views, a diversity constraint is introduced. This constraint suppresses view collapse by using the similarity between different augmentation features. This loss can be expressed as:

[0041] in and These represent the propagation features after different enhanced views.

[0042] Through the combined effects of the above-mentioned retrieval enhancement, consistency constraints, and diversity constraints, an enhanced view with stability and distinctiveness can be generated without destroying the original propagation structure, enabling the propagation representation to still have robust semantic expression capabilities even with insufficient annotation.

[0043] Preferably, a consistency constraint is applied to regularize and optimize the prediction results of the preset number of enhanced views, which are then input into the trained popularity prediction model to output the future popularity prediction result of the target video, including: A consistency constraint is applied to regularize the prediction results of the preset number of augmented views to obtain stable prediction results. The consistency constraint aims to minimize the difference in model prediction results under different augmented views. By employing diversity constraints, the stable prediction results are optimized to obtain a structure-preserving enhanced representation. The diversity constraints aim to prevent the feature representations of multiple enhanced views from collapsing into the same subspace. The enhanced representation that preserves the structure is input into the trained popularity prediction model to obtain intermediate inference results; The intermediate inference results are parsed and feature-mapped to output the future popularity prediction results of the target video.

[0044] Preferably, the training method for the popularity prediction model includes: Obtain video samples with real popularity tags to obtain a labeled sample set; Obtain video samples without real popularity labels to obtain an unlabeled sample set; Based on the labeled sample set, the supervised loss is calculated by the supervised regression module. A data augmentation module is used to randomly augment the propagation representation in the unlabeled sample set to obtain a preset number of augmented views; Construct a prediction network; The preset number of enhanced views are input into the prediction network to obtain the corresponding popularity prediction results; Based on the popularity prediction results, the consistency regularization loss is calculated using the consistency loss function, which is used to measure the difference or similarity of prediction results under different enhanced views. The corresponding popularity prediction results are statistically analyzed to obtain a batch prediction distribution; A diversity regularization module is used to calculate the batch prediction distribution to obtain the diversity regularization loss, which is used to prevent the model prediction results from collapsing. Based on the aforementioned supervision loss, consistency regularization loss, and diversity regularization loss, a joint loss function is obtained by weighted summation; The backpropagation algorithm is used to perform semi-supervised optimization training on the parameters of the prediction network based on the joint loss function to obtain the trained popularity prediction model. The semi-supervised optimization training includes updating the network parameters of the prediction network.

[0045] The following experiment verifies the effectiveness of the above-mentioned propagation-aware semi-supervised prediction method for short video popularity based on learnable graph enhancement: To evaluate the performance of the short video popularity prediction model provided in this embodiment, the experiment used the publicly available MicroLens short video dataset, dividing it into two subsets: MicroLens-views and MicroLens-comments, to construct a heterogeneous graph suitable for short videos. Both subsets treat video frames and titles as attribute-type nodes and initialize multimodal features using the AngloE, ViT, and BLIP models, respectively. For interaction-type nodes, the MicroLens-views subset establishes interaction edges based on user comments, using the logarithm of video views as the popularity metric, while the MicroLens-comments subset uses video category as the interaction attribute and the logarithm of comment count as the popularity metric. Considering the challenges of few-shot learning scenarios, the experiment strictly divided the dataset into training, validation, and test sets according to an 8%:2%:90% ratio to verify the model's training performance in environments with extremely limited labeled data. In terms of evaluation metrics, this embodiment uses three metrics: normalized mean squared error (nMSE), mean absolute error (MAE), and Spearman rank correlation coefficient (SRC). These metrics are used to quantitatively evaluate the model performance from three dimensions: the degree of deviation of the predicted values, the magnitude of the average error, and the monotonic consistency of the predicted ranking.

[0046] To verify the effectiveness of the proposed method in short video popularity prediction, 10 representative benchmark methods were selected for comparative analysis. These methods covered feature engineering methods (SVR, HyFea), classic deep learning methods (CLSTM, TMALL, CBAN), and emerging graph neural network methods (RGCN, HetSANN, SeHGNN, RpHGNN). The experimental results show that the proposed method significantly outperforms all baselines on both the MicroLens-views and MicroLens-comments datasets, achieving the best performance in the three key metrics: nMSE, MAE, and SRC. Particularly under limited sample conditions, compared to existing end-to-end or pre-computation graph neural network methods, the method effectively integrates the multimodal features and complex social interaction information of short video content by introducing Socially Aware Graph Network (SAGN) and Graph Data Augmentation (GDA) strategies. This not only reduces prediction error but also significantly improves the accuracy of popularity ranking, demonstrating the technical advantages of this method in leveraging unlabeled data to enhance model robustness and generalization ability.

[0047] Table 1 Experimental results for the two datasets

[0048] The above is an introduction to the method embodiments. The following describes the solution described in this application through device embodiments.

[0049] Figure 3 This is a block diagram illustrating a short video popularity prediction apparatus based on learnable graph augmentation, according to an exemplary embodiment. The apparatus is used in a short video popularity prediction method based on learnable graph augmentation. (Refer to...) Figure 3 The device includes a heterogeneous social graph module, a multimodal representation module, a semantic enhancement module, an enhanced view module, and a popularity prediction module.

[0050] Heterogeneous social graph module: used to collect keyframe sets, text data sets and behavior data sets of raw short video data, input them into a pre-trained visual coding model, perform behavior feature extraction processing, and then instantiate them into different types of nodes, perform graph edge construction, and obtain heterogeneous node sets and heterogeneous edge sets. After graph structure integration processing, a heterogeneous social graph is obtained. The heterogeneous node set includes video nodes, user nodes and attribute nodes. Multimodal representation module: used to perform unimodal processing on the heterogeneous social graph using feature masking or replacement strategies, input it into the heterogeneous attribute graph neural network, perform fusion processing, and obtain the initial multimodal representation of the short video. The initial multimodal representation of the short video includes multimodal fusion node output and unimodal node output. The semantic enhancement module is used to extract content features, perform odd-hop propagation and even-hop propagation through the heterogeneous social graph, input a sparse random projection matrix, and fuse and aggregate it through the fusion module to obtain the video node propagation representation. The video node propagation representation has semantic enhancement of propagation structure and content features. Enhanced View Module: This module selects a target video and, based on the propagation representation of the video nodes, performs global semantic similarity calculation and similar sample filtering, then employs feature fusion operations and a learnable sparse mask. After multiple random forward propagations, a preset number of enhanced views are obtained. Popularity prediction module: It is used to regularize and optimize the prediction results of the preset number of enhanced views by applying consistency constraints, input them into the trained popularity prediction model, and output the future popularity prediction results of the target video.

[0051] A short video popularity prediction device based on learnable graph augmentation, the short video popularity prediction device based on learnable graph augmentation includes: a processor; a memory, the memory storing computer-readable instructions.

[0052] Figure 4 This is a schematic diagram of the structure of a short video popularity prediction device based on learnable graph enhancement provided in an embodiment of the present invention, as shown below. Figure 4As shown, a short video popularity prediction device based on learnable graph augmentation can include the above-mentioned... Figure 3 The illustrated device is a short video popularity prediction apparatus based on learnable graph enhancement. Optionally, the short video popularity prediction apparatus 410 based on learnable graph enhancement may include a first processor 2001.

[0053] Optionally, the short video popularity prediction device 410 based on learnable graph enhancement may also include a memory 2002 and a transceiver 2003.

[0054] The first processor 2001, memory 2002, and transceiver 2003 can be connected via a communication bus.

[0055] The following is combined Figure 4 The components of the short video popularity prediction device 410 based on learnable graph augmentation are described in detail below: The first processor 2001 is the control center of the short video popularity prediction device 410 based on learnable graph enhancement. It can be a single processor or a collective term for multiple processing elements. For example, the first processor 2001 can be one or more central processing units (CPUs), application-specific integrated circuits (ASICs), or one or more integrated circuits configured to implement embodiments of the present invention, such as one or more digital signal processors (DSPs), or one or more field-programmable gate arrays (FPGAs).

[0056] Optionally, the first processor 2001 can perform various functions of the short video popularity prediction device 410 based on learnable graph enhancement by running or executing software programs stored in the memory 2002 and calling data stored in the memory 2002.

[0057] In a specific implementation, as one example, the first processor 2001 may include one or more CPUs, for example... Figure 4 CPU0 and CPU1 are shown in the diagram.

[0058] In a specific implementation, as one example, the short video popularity prediction device 410 based on learnable graph enhancement may also include multiple processors, for example... Figure 4The first processor 2001 and the second processor 2004 are shown in the diagram. Each of these processors can be a single-core processor or a multi-core processor. Here, a processor can refer to one or more devices, circuits, and / or processing cores used to process data (such as computer program instructions).

[0059] The memory 2002 is used to store the software program that executes the present invention, and is controlled by the first processor 2001 to execute it. The specific implementation method can be referred to the above method embodiment, and will not be repeated here.

[0060] Optionally, the memory 2002 may be a read-only memory (ROM) or other type of static storage device capable of storing static information and instructions, random access memory (RAM) or other type of dynamic storage device capable of storing information and instructions, or electrically erasable programmable read-only memory (EEPROM), compact disc read-only memory (CD-ROM) or other optical disc storage, optical disc storage (including compressed optical discs, laser discs, optical discs, digital universal optical discs, Blu-ray discs, etc.), magnetic disk storage media or other magnetic storage devices, or any other medium capable of carrying or storing desired program code in the form of instructions or data structures and accessible by a computer, but not limited thereto. The memory 2002 may be integrated with the first processor 2001 or may exist independently, and may be connected via the interface circuit of the learnable graph-enhanced short video popularity prediction device 410. Figure 4 (Not shown in the image) is coupled to the first processor 2001, and this embodiment of the invention does not specifically limit this.

[0061] The transceiver 2003 is used to communicate with network devices or with terminal devices.

[0062] Alternatively, transceiver 2003 may include a receiver and a transmitter. Figure 4 (Not shown separately). The receiver is used to implement the receiving function, and the transmitter is used to implement the transmitting function.

[0063] Optionally, the transceiver 2003 can be integrated with the first processor 2001 or exist independently, and can be connected to the interface circuit of the short video popularity prediction device 410 based on learnable graph enhancement. Figure 4 (Not shown in the image) is coupled to the first processor 2001, and this embodiment of the invention does not specifically limit this.

[0064] It should be noted that, Figure 4 The structure of the short video popularity prediction device 410 based on learnable graph enhancement shown in the figure does not constitute a limitation on the router. Actual knowledge structure recognition devices may include more or fewer components than shown, or combine certain components, or have different component arrangements.

[0065] Furthermore, the technical effect of the short video popularity prediction device 410 based on learnable graph enhancement can be referred to the technical effect of the short video popularity prediction method based on learnable graph enhancement described in the above method embodiments, and will not be repeated here.

[0066] It should be understood that the first processor 2001 in the embodiments of the present invention may be a central processing unit (CPU), or it may be other general-purpose processors, digital signal processors (DSPs), application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. The general-purpose processor may be a microprocessor or any conventional processor, etc.

[0067] It should also be understood that the memory in the embodiments of the present invention can be volatile memory or non-volatile memory, or may include both volatile and non-volatile memory. The non-volatile memory can be read-only memory (ROM), programmable read-only memory (PROM), erasable programmable read-only memory (EPROM), electrically erasable programmable read-only memory (EEPROM), or flash memory. The volatile memory can be random access memory (RAM), which is used as an external cache. By way of example, but not limitation, many forms of random access memory (RAM) are available, such as static RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), double data rate synchronous DRAM (DDR SDRAM), enhanced synchronous DRAM (ESDRAM), synchronous linked DRAM (SLDRAM), and direct rambus RAM (DR RAM).

[0068] The above embodiments can be implemented, in whole or in part, by software, hardware (such as circuits), firmware, or any other combination thereof. When implemented using software, the above embodiments can be implemented, in whole or in part, as a computer program product. The computer program product includes one or more computer instructions or computer programs. When the computer instructions or computer programs are loaded or executed on a computer, all or part of the processes or functions described in the embodiments of the present invention are generated. The computer can be a general-purpose computer, a special-purpose computer, a computer network, or other programmable device. The computer instructions can be stored in a computer-readable storage medium or transmitted from one computer-readable storage medium to another. For example, the computer instructions can be transmitted from one website, computer, server, or data center to another website, computer, server, or data center via wired (e.g., infrared, wireless, microwave, etc.) means. The computer-readable storage medium can be any available medium that a computer can access or a data storage device such as a server or data center that includes one or more sets of available media. The available medium can be a magnetic medium (e.g., floppy disk, hard disk, magnetic tape), an optical medium (e.g., DVD), or a semiconductor medium. A semiconductor medium can be a solid-state drive.

[0069] It should be understood that the term "and / or" in this article is merely a description of the relationship between related objects, indicating that three relationships can exist. For example, A and / or B can represent: A existing alone, A and B existing simultaneously, or B existing alone. A and B can be singular or plural. Additionally, the character " / " in this article generally indicates an "or" relationship between the preceding and following related objects, but it can also represent an "and / or" relationship. Please refer to the context for a more accurate understanding.

[0070] In this invention, "at least one" means one or more, and "more than one" means two or more. "At least one of the following" or similar expressions refer to any combination of these items, including any combination of a single item or a plurality of items. For example, at least one of a, b, or c can represent: a, b, c, ab, ac, bc, or abc, where a, b, and c can be a single item or multiple items.

[0071] It should be understood that, in various embodiments of the present invention, the order of the above-mentioned process numbers does not imply the order of execution. The execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiments of the present invention.

[0072] Those skilled in the art will recognize that the units and algorithm steps of the various examples described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are implemented in hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementations should not be considered beyond the scope of this invention.

[0073] Those skilled in the art will clearly understand that, for the sake of convenience and brevity, the specific working processes of the devices, apparatuses, and units described above can be referred to the corresponding processes in the foregoing method embodiments, and will not be repeated here.

[0074] In the several embodiments provided by this invention, it should be understood that the disclosed devices, apparatuses, and methods can be implemented in other ways. For example, the apparatus embodiments described above are merely illustrative; for instance, the division of units is only a logical functional division, and in actual implementation, there may be other division methods. For example, multiple units or components may be combined or integrated into another device, or some features may be ignored or not executed. Furthermore, the coupling or direct coupling or communication connection shown or discussed may be through some interfaces; the indirect coupling or communication connection between devices or units may be electrical, mechanical, or other forms.

[0075] The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the units can be selected to achieve the purpose of this embodiment according to actual needs.

[0076] In addition, the functional units in the various embodiments of the present invention can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit.

[0077] If the aforementioned functions are implemented as software functional units and sold or used as independent products, they can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of this invention, or the part that contributes to the prior art, or a part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of this invention. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0078] The above description is merely a specific embodiment of the present invention, but the scope of protection of the present invention is not limited thereto. Any variations or substitutions that can be easily conceived by those skilled in the art within the technical scope disclosed in the present invention should be included within the scope of protection of the present invention. Therefore, the scope of protection of the present invention should be determined by the scope of the claims.

Claims

1. A method for predicting the popularity of short videos based on learnable graph augmentation, characterized in that, The method includes: S1: Collect keyframe sets, text data sets, and behavior data sets of the original short video data, input them into a pre-trained visual coding model, perform behavior feature extraction processing, and then instantiate them into different types of nodes. Perform graph edge construction to obtain a heterogeneous node set and a heterogeneous edge set. After graph structure integration processing, a heterogeneous social graph is obtained. The heterogeneous node set includes video nodes, user nodes, and attribute nodes. S2: Using a feature masking or replacement strategy, the heterogeneous social graph is processed into a single modality and input into a heterogeneous attribute graph neural network for fusion processing to obtain an initial multimodal representation of the short video. The initial multimodal representation of the short video includes multimodal fusion node output and single modal node output. S3: Through the heterogeneous social graph, content features are extracted, odd-hop propagation and even-hop propagation are performed, and the sparse random projection matrix is ​​input. After fusion and aggregation by the fusion module, the video node propagation representation is obtained. The video node propagation representation has semantic enhancement of propagation structure and content features. S4: Select the target video, and based on the propagation representation of the video nodes, perform global semantic similarity calculation and similar sample screening, then use feature fusion operation, employ a learnable sparse mask, and after multiple random forward propagations, obtain a preset number of enhanced views; S5: Using consistency constraints, the prediction results of the preset number of enhanced views are regularized and optimized, and then input into the trained popularity prediction model to output the future popularity prediction results of the target video.

2. The short video popularity prediction method based on learnable graph enhancement according to claim 1, characterized in that, The keyframe set, text data set, and behavior data set of the raw short video data collected in S1 are input into a pre-trained visual coding model for behavior feature extraction. These are then instantiated into different types of nodes, and graph edge construction is performed to obtain a heterogeneous node set and a heterogeneous edge set. After graph structure integration processing, a heterogeneous social graph is obtained. The heterogeneous node set includes video nodes, user nodes, and attribute nodes, including: S11: Collect raw short video data through short video data sources; S12: Using a unified sampling method, keyframes are extracted from the visual frames of each short video in the original short video data to obtain a keyframe set; S13: For each short video in the original short video data, collect the video title and descriptive text generated from the visual content to obtain a text data set; S14: For each short video in the original short video data, obtain the user's likes, comments, and playback interaction data between the user and the video to obtain a set of behavioral data; S15: Input the set of keyframes into a pre-trained visual coding model to obtain the visual features corresponding to each short video; S16: Input the text data set into the text encoding model to extract semantic information, and obtain the title features, descriptive text features and video-attribute association relationship; S17: Based on the behavioral data set, after behavioral feature extraction processing, user interaction features and user-video interaction relationships are obtained; S18: Using a preset node type definition, user interaction features, visual features, title features, and descriptive text features are instantiated into different types of nodes to obtain a heterogeneous node set, which includes video nodes, user nodes, and attribute nodes. S19: Based on the user-video interaction relationship and the video-attribute association relationship, construct a graph edge to obtain a heterogeneous edge set, which includes user-video interaction edges and video-attribute association edges; S110: Based on the heterogeneous node set and the heterogeneous edge set, a heterogeneous social graph is obtained through graph structure integration processing.

3. The short video popularity prediction method based on learnable graph enhancement according to claim 1, characterized in that, The S2 method employs a feature masking or replacement strategy to perform unimodal processing on the heterogeneous social graph, inputs it into a heterogeneous attribute graph neural network, and performs fusion processing to obtain an initial multimodal representation of the short video. This initial multimodal representation includes multimodal fusion node outputs and unimodal node outputs, including: S21: Using a feature masking or replacement strategy, the heterogeneous social graph is processed into a single modality to obtain a single modality graph. The structure of the single modality graph retains only the attribute information corresponding to the preset modality. S22: Input the heterogeneous social graph into the heterogeneous attribute graph neural network to perform graph feature propagation and representation learning, and obtain the multimodal fusion node output, wherein the multimodal fusion node output includes the learned cross-modal semantics; S23: Input the single-modal graph into the heterogeneous attribute graph neural network to perform graph feature propagation and representation learning to obtain the single-modal node output, wherein the single-modal node output includes the enhanced semantic expression of the modality itself; S24: Input the output of the multimodal fusion node and the output of the single-modal node into the fusion module for feature fusion to obtain the initial multimodal representation of the short video. The initial multimodal representation has modal independence and cross-modal semantic correlation.

4. The short video popularity prediction method based on learnable graph enhancement according to claim 1, characterized in that, S3 extracts content features, performs odd-hop propagation and even-hop propagation through the heterogeneous social graph, inputs a sparse random projection matrix, and fuses and aggregates the data through a fusion module to obtain a video node propagation representation. This video node propagation representation has semantic enhancements in terms of propagation structure and content features, including: S31: Based on the type of relation edges in the heterogeneous social graph, content features are extracted to obtain initial node features; S32: Using the heterogeneous social graph, a positive adjacency matrix of a specific relationship type is used to perform odd-hop propagation to obtain direct interaction propagation features, which are used to characterize the direct interaction relationships between nodes. S33: Using the heterogeneous social graph, even-numbered hop propagation is performed using a reverse adjacency matrix of a specific relationship type and the corresponding degree matrix to obtain indirect diffusion propagation features, which are used to characterize the indirect diffusion influence between nodes. S34: Input the direct interactive propagation features and indirect diffusion propagation features into the sparse random projection matrix to compress the dimension, and obtain the compressed multi-hop propagation information; S35: Based on the compressed multi-hop propagation information, the fusion module processes the information to obtain the fused propagation features of the fused content semantics and social propagation structure; S36: Aggregate the fused propagation features with the initial node features to obtain a video node propagation representation, which has semantic enhancement of propagation structure and content features.

5. The short video popularity prediction method based on learnable graph enhancement according to claim 1, characterized in that, The selected target video in S4, based on the propagation representation of the video nodes, undergoes global semantic similarity calculation and similar sample screening, followed by feature fusion operation and a learnable sparse mask. After multiple random forward propagations, a preset number of enhanced views are obtained, including: S41: Select the target video, and based on the propagation representation of the video nodes, calculate the similarity value between the target video node and other nodes through global semantic similarity calculation; S42: Using the similarity value, similar samples are filtered on the heterogeneous social graph to obtain a sample set that is semantically similar to the target video node. The sample set includes a preset number of video nodes with the highest similarity ranking. S43: Based on the sample set, supplementary feature information is obtained by averaging the relation channel features, wherein the relation channel features include feature vectors that characterize different types of semantic relationships between nodes; S44: A feature fusion operation is used to aggregate the supplementary feature information with the feature information of the target video node in the current propagation view to obtain a retrieval enhancement representation. The feature fusion operation includes concatenation, addition and weighted summation. S45: A learnable sparse mask is used to sparsify the retrieval enhancement representation in the feature dimension to obtain the masked feature representation. The sparse mask is used to adaptively select important features during training. S46: Based on the feature representation after the mask, a preset number of enhanced views are obtained through multiple random forward propagations, wherein the random forward propagation includes introducing random deactivation or noise into the network.

6. The short video popularity prediction method based on learnable graph enhancement according to claim 1, characterized in that, S5 employs consistency constraints to regularize and optimize the prediction results of the preset number of enhanced views, inputs them into the trained popularity prediction model, and outputs the future popularity prediction results of the target video, including: S51: Using consistency constraints, the prediction results of the preset number of augmented views are regularized to obtain stable prediction results. The consistency constraints aim to minimize the differences in model prediction results under different augmented views. S52: By employing diversity constraints, the stable prediction results are optimized to obtain a structure-preserving enhanced representation. The diversity constraints aim to prevent the feature representations of multiple enhanced views from collapsing into the same subspace. S53: Input the enhanced representation that preserves the structure into the trained popularity prediction model to obtain intermediate inference results; S54: Analyze and perform feature mapping on the intermediate inference results, and output the future popularity prediction results of the target video.

7. The short video popularity prediction method based on learnable graph enhancement according to claim 5, characterized in that, The training method for the popularity prediction model includes: S61: Obtain video samples with real popularity labels to obtain a labeled sample set; S62: Obtain video samples without real popularity labels to get an unlabeled sample set; S63: Based on the labeled sample set, the supervised loss is calculated by the supervised regression module; S64: Using a data augmentation module, the propagation representation in the unlabeled sample set is randomly augmented to obtain a preset number of augmented views; S65: Construct a prediction network; S66: Input the preset number of enhanced views into the prediction network to obtain the corresponding popularity prediction results; S67: Based on the popularity prediction results, the consistency loss function is used to calculate the consistency regularization loss, which is used to measure the difference or similarity of prediction results under different enhanced views. S68: Statistically analyze the corresponding popularity prediction results to obtain a batch prediction distribution; S69: The diversity regularization module is used to calculate the batch prediction distribution to obtain the diversity regularization loss, which is used to prevent the model prediction results from collapsing. S610: Based on the supervision loss, consistency regularization loss, and diversity regularization loss, a joint loss function is obtained by weighted summation; S611: Using the backpropagation algorithm, the parameters of the prediction network are semi-supervised and trained based on the joint loss function to obtain the trained popularity prediction model. The semi-supervised optimization training includes updating the network parameters of the prediction network.

8. A short video popularity prediction device based on learnable graph augmentation, wherein the short video popularity prediction device based on learnable graph augmentation is used to implement the short video popularity prediction method based on learnable graph augmentation as described in any one of claims 1-7, characterized in that, The device includes: Heterogeneous social graph module: used to collect keyframe sets, text data sets and behavior data sets of raw short video data, input them into a pre-trained visual coding model, perform behavior feature extraction processing, and then instantiate them into different types of nodes, perform graph edge construction, and obtain heterogeneous node sets and heterogeneous edge sets. After graph structure integration processing, a heterogeneous social graph is obtained. The heterogeneous node set includes video nodes, user nodes and attribute nodes. Multimodal representation module: used to perform unimodal processing on the heterogeneous social graph using feature masking or replacement strategies, input it into the heterogeneous attribute graph neural network, perform fusion processing, and obtain the initial multimodal representation of the short video. The initial multimodal representation of the short video includes multimodal fusion node output and unimodal node output. The semantic enhancement module is used to extract content features, perform odd-hop propagation and even-hop propagation through the heterogeneous social graph, input a sparse random projection matrix, and fuse and aggregate it through the fusion module to obtain the video node propagation representation. The video node propagation representation has semantic enhancement of propagation structure and content features. Enhanced View Module: This module selects a target video and, based on the propagation representation of the video nodes, performs global semantic similarity calculation and similar sample filtering, then employs feature fusion operations and a learnable sparse mask. After multiple random forward propagations, a preset number of enhanced views are obtained. Popularity prediction module: It is used to regularize and optimize the prediction results of the preset number of enhanced views by applying consistency constraints, input them into the trained popularity prediction model, and output the future popularity prediction results of the target video.

9. A short video popularity prediction device based on learnable graph augmentation, characterized in that, The short video popularity prediction processor based on learnable graph enhancement; a memory storing computer-readable instructions, which, when executed by the processor, implement the method as described in any one of claims 1 to 7.

10. A computer-readable storage medium, characterized in that, The computer-readable storage medium contains program code that can be invoked by a processor to execute the method as described in any one of claims 1 to 7.

Images