Multi-feature learning online video popularity prediction method based on time information perception

Abstract

The invention discloses a multi-feature learning online video popularity prediction method based on time information perception. The method comprises the following steps: acquiring multi-feature data and playing time sequence data related to a to-be-predicted video; inputting the multi-feature data into feature sub-encoders in a preset multi-feature encoder according to categories, and determining visual, acoustic, text, social and metadata features; inputting the playing time sequence data into a hierarchical popularity sequence feature extraction network in a preset multi-feature encoder to determine popularity sequence features of the to-be-predicted video; inputting the visual, acoustic, text, social, metadata and popularity sequence features of the to-be-predicted video into a preset feature fusion network for feature fusion; and inputting the fused to-be-predicted video features into a preset popularity prediction network to obtain a prediction result. Through the technical scheme of the invention, various features related to the video popularity can be comprehensively considered, and the popularity prediction performance is improved.

Description

Technical Field

[0001] The present invention relates to the fields of deep learning, data mining, and multimodal learning, and specifically to a multi-feature learning online video popularity prediction method based on time information perception, which belongs to the application field of online video platform service providers on the Internet to optimize content recommendations and improve user experience. Background Art

[0002] In recent years, the popularity of large online video platforms such as TikTok, YouTube and Bilibili has profoundly affected people's daily lives. Users can log in to these platforms at any time to consume various forms of content, such as videos, images and texts, or upload their life stories, share their opinions, etc. Unfortunately, only a small part of the massive amount of social media content can become the focus of the public. Specifically, some carefully produced videos have been slow to get a response, while some casually uploaded videos have unexpectedly become popular. There may be some potential factors that affect the popularity of content.

[0003] One of the goals of video popularity prediction is to estimate the popularity of a video at a certain point in the future. This research can not only provide important clues for content providers in terms of dynamic traffic allocation, advertising marketing strategy planning, and recommendation algorithm optimization, but also help understand current user preferences and market trends, thereby guiding content producers to create content with high popularity potential and improve user satisfaction and retention of the overall platform. In previous studies on video popularity prediction, some considered static content features in the video, such as text content, social attributes, emotional features, or a combination of visual and emotional features. The other part considered it from a time series perspective, focusing on using the early viewing patterns of the video, or considering a limited combination with content features.

[0004] In multimodal learning, some studies have extracted content features from the visual, textual and acoustic modalities of videos to predict popularity, or combined the user's social attributes as a modality with content features. However, there is still a lack of a method that can effectively overcome the following challenges: (1) Multimodal content features such as visual, acoustic, and textual features in online videos present videos from different angles, but they lack temporal context information and are difficult to capture the dynamics of popularity; (2) Due to the complex information propagation and interaction in social networks, such as changes in user behavior or the influence of platform recommendation algorithms, the popularity trends observed from the time dimension will have a certain degree of randomness, which increases the difficulty of modeling popularity sequences. (3) The contribution of each feature to popularity is different in different time periods, so it is necessary to find a method to adaptively learn the weight distribution of features from different sources to improve the robustness of fusion. Summary of the invention

[0005] The present invention aims to address the deficiencies of existing video popularity prediction technologies and proposes a multi-feature learning online video popularity prediction method based on time information perception. The method designs four key time contexts, namely Holiday Eve (HDE), Holiday (HLD), Published Day Count (PBD) and Time Zone (TMZ), introduces popularity sequence features and the key time contexts as a supplement to time information, and enhances the dynamic sensitivity of multimodal content features. Advanced pre-trained models are used to extract features of multimodal content such as vision, acoustics, and text. At the same time, feature categories of social and metadata are designed and feature extraction is performed separately. The playback data of a certain time period is input into a hierarchical popularity sequence feature extraction network for feature extraction to determine the popularity sequence features of the video to be predicted. The above features are input into a preset feature fusion network for adaptive feature fusion to determine the features of the video to be predicted, ensuring the maximum effectiveness and timeliness of multi-feature fusion and improving the performance of video popularity prediction.

[0006] The present invention proposes a method for predicting online video popularity based on multi-feature learning based on time information perception, comprising:

[0007] (1) Obtaining multi-feature data and playback timing data related to the video to be predicted;

[0008] (2) inputting the multi-feature data into a feature sub-encoder in a preset multi-feature encoder according to the category to determine visual, acoustic, text, social and metadata features;

[0009] (3) inputting the playback time sequence data into a hierarchical popularity sequence feature extraction network in a preset multi-feature encoder to determine the popularity sequence features of the video to be predicted;

[0010] (4) Inputting the visual, acoustic, text, social, metadata and popularity sequence features of the video to be predicted into a preset feature fusion network for feature fusion;

[0011] (5) The fused features of the video to be predicted are input into the preset popularity prediction network to obtain the prediction result.

[0012] The preset multi-feature encoder is based on multi-feature data and is input into different sub-encoders according to different categories: the visual, acoustic, text and other data of the video to be predicted are divided into multi-modal content information and input into the visual, acoustic, and text feature sub-encoders; the social information of the user who uploaded the video to be predicted and the metadata information of the video to be predicted are input into the social and metadata feature sub-encoders; the playback time series data is input into a hierarchical popularity sequence feature extraction network to determine the popularity sequence features of the video to be predicted. After the preliminary extraction of the visual, acoustic, text and other multi-modal content information, they are respectively input into the corresponding multi-layer perceptron MLP. v 、MLP a 、MLP t In the method, a feature vector based on element alignment is determined. The visual, acoustic, text, social, metadata and popularity sequence features processed and determined by the preset multi-feature encoder are necessary features for predicting video popularity in the method.

[0013] The social and metadata feature sub-encoder is a network structure that integrates multiple embedding layers and fully connected layers. In the social and metadata feature sub-encoder, the embedding layer is used to process discrete digital features, and the fully connected layer is used to process continuous digital features. The social information of the user who uploaded the video to be predicted and the metadata information of the video to be predicted processed by the social and metadata feature sub-encoder are calculated to determine the social features based on element alignment. and metadata features The calculation formula is as follows:

[0014]

[0015] In the above formulas (1) and (2), LeakyReLU represents the LeakyReLU activation function, z s and z m They represent the social and metadata features determined by concatenation after being processed by the embedding layer and the fully connected layer, respectively. s and w m represents the learnable weight, b s and b m represents a learnable bias.

[0016] The playback timing data of the video to be predicted is composed of the video popularity calculated based on the uniformly sampled playback index in a certain time period and the key time context features processed and combined by the embedding layer; the key time context includes Holiday Eve (HDE), Holiday (HLD), Published Day Count (PBD) and Time Zone (TMZ). Different key time contexts are combined into different key time context features: key time context feature temb1 It consists of HDE, HLD and TMZ, which represent the time context in hours; the key time context feature t emb2 It consists of HDE and HLD, which represent the time context in days; the key time context feature t emb3 It consists of PBD, HDE and HLD, which represent the time context of the video to be predicted on the target prediction date. The playback time sequence data of the video to be predicted is input into the hierarchical popularity sequence feature extraction network to determine the popularity sequence feature.

[0017] The popularity of the video is determined by a combination of seven indicators, including the number of video playbacks, the number of video likes, the number of video coins, the number of video collections, the number of video shares, the number of video comments, and the number of video bullet screens. The above indicators are obtained through uniform sampling in a certain period of time, and each sampling point can be regarded as a time point. The video popularity calculated according to the sampling frequency of the playback indicators can be divided into short-term popularity time series information and long-term popularity time series information. In order to avoid the adverse effects of huge numerical range differences in popularity scores on network training, the popularity of the i-th video at the k-th time point is The calculation formula is as follows:

[0018]

[0019] In the above formula (3), represents the number of views of the i-th video at the k-th time point, represents the number of likes for the i-th video at the k-th time point, represents the amount of coins inserted into the i-th video at the k-th time point, represents the number of collections of the i-th video at the k-th time point, represents the number of shares of the i-th video at the k-th time point, represents the number of comments on the i-th video at the k-th time point, It represents the amount of bullet comments of the ith video at the kth time point. Assuming there are n time points in total, the popularity of the ith video after uniform sampling in a certain time period can be expressed as:

[0020] The hierarchical popularity sequence feature extraction network includes two time series feature extraction modules, each of which is composed of two layers of LSTM (Long Short-Term Memory) long short-term memory networks. First, the playback time series data of the video to be predicted is input into the hierarchical popularity sequence feature extraction network to calculate the short-term popularity trend I S and long-term popularity trends I L , IS and I L Use I K (K∈{S,L}) uniformly represents, I K The calculation formula is as follows:

[0021] I K =LeakyReLU(w K z K +b K ),K∈{S,L} (4)

[0022] In the above formula (4), LeakyReLU represents the LeakyReLU activation function, z K represents short-term popularity time series information or long-term popularity time series information, w K represents the learnable weight, b K represents a learnable bias.

[0023] The short-term popularity trend I S and the key temporal context feature t emb1 Input to the first time series feature extraction module to determine the short-term popular hidden state sequence The calculation formula is as follows:

[0024]

[0025] In the above formula (5), Represents the first layer of LSTM network in the first time series feature extraction module, Represents the second layer LSTM network in the first time series feature extraction module, [I S ;t emb1 ] indicates I S and t emb1 Perform feature concatenation. Take H S Hidden state at the last time step As a short-term fluctuation feature. I L and the key temporal context feature t emb2 Input to the second time series feature extraction module for feature extraction to determine the long-term popular hidden state sequence The calculation formula is as follows:

[0026]

[0027] In the above formula (6), Represents the first layer of LSTM network in the second time series feature extraction module, Represents the second layer LSTM network in the second time series feature extraction module, Express I L and t emb2 Perform feature concatenation. Finally, L The attention mechanism is applied to all time steps of to obtain the output of the hierarchical popularity sequence feature extraction network, i.e., the popularity sequence feature The calculation formula is as follows:

[0028]

[0029] In the above formula (7), represents H after softmax normalization L The weight of the feature at the i-th time step in , Indicates H L The features of the i-th time step in . The calculation formula is as follows:

[0030]

[0031] In the above formula (8), Indicates H L The initial weight of the feature at the i-th time step in is calculated as follows:

[0032]

[0033] In the above formula (9), tanh represents the tanh activation function, represents the transpose of a learnable query vector, w p represents the learnable weight, b p represents a learnable bias.

[0034] The preset feature fusion network includes: a feature scoring stage and a feature fusion stage; in the feature scoring stage, the time context of the video to be predicted on the target prediction date and the attention mechanism are used to realize the adaptive fusion of multimodal content features such as vision, acoustics, and text, as well as social, metadata, and popularity sequence features. After z∈{v,a,t,s,m,p}, the feature scoring stage gives a scoring weight α to each feature z Calculate. z Afterwards, the feature fusion stage will fuse each feature to determine the features of the video to be predicted

[0035] The feature scoring stage is to determine the scoring weights corresponding to different features according to a certain preset condition, highlighting the contribution of different features to popularity prediction. emb3As a precondition, with each feature Do similarity calculation to get the corresponding initial score a z , and its calculation formula is as follows:

[0036]

[0037] In the above formula (10), tanh represents the tanh activation function, represents the transpose of a learnable query vector, w z and v z represents the learnable weight, b z Represents a learnable bias. After obtaining the initial score of each feature, the score weight α of each feature is determined by the following calculation. z , and its calculation formula is as follows:

[0038]

[0039] In the feature fusion stage, the score α of each feature is determined. z Afterwards, α z Apply to the corresponding In the process, the features of the video to be predicted are determined by fusion The calculation formula is as follows:

[0040]

[0041] The preset popularity prediction network takes the features of the video to be predicted Input into a fully connected layer for calculation to determine the popularity of the video to be predicted The calculation formula is as follows:

[0042]

[0043] In the above formula (13), represents the learnable weight vector, represents the learnable bias vector. BRIEF DESCRIPTION OF THE DRAWINGS

[0044] Figure 1 This is a structural diagram of a multi-feature learning network based on time information perception.

[0045] Figure 2 This is a diagram of a preset multi-feature encoder structure.

[0046] Figure 3 Extracting network structure graph for a hierarchical popularity sequence feature. DETAILED DESCRIPTION

[0047] The present invention is described below in conjunction with the accompanying drawings and embodiments. It should be understood that the present disclosure can be implemented in various forms and should not be interpreted as limiting the present disclosure. On the contrary, these embodiments are provided to provide a more thorough and complete understanding of the present disclosure.

[0048] It should be understood that the concepts such as “first” and “second” mentioned in the present invention are only used to distinguish different modules, devices or units, and are not used to limit the order of functions performed by these modules, devices or units or the dependencies.

[0049] Figure 1 The present invention relates to a multi-feature learning network structure diagram based on time information perception, which is composed of a preset multi-feature encoder, a preset feature fusion network and a preset popularity prediction network. The preset multi-feature encoder includes visual, acoustic, and text feature sub-encoders for determining the visual, acoustic, and textual multimodal content features of the video to be predicted, metadata and social feature sub-encoders for determining the social information of the user who uploaded the video to be predicted and the metadata information of the video to be predicted, and a hierarchical popularity sequence feature extraction network for determining the popularity sequence features. The preset feature fusion network includes a feature scoring stage and a feature fusion stage, introduces the time context and attention mechanism of the video to be predicted on the target prediction date, realizes the adaptive fusion of multimodal content features such as vision, acoustics, and text, and social, metadata, and popularity sequence features, and determines the features of the video to be predicted. The preset popularity prediction network determines the popularity of the video to be predicted through the features of the video to be predicted. The performance of the multimodal learning network based on time information perception changes with the introduction or removal of different features and key time contexts.

[0050] Specifically, in order to illustrate the implementation manner of the present invention, in an embodiment, first, it is possible but not limited to obtain target data of multiple videos to be predicted and playback data of a certain time period from a certain online video platform website. The target data of the video to be predicted includes: video frames, video cover, video audio, video title and introduction, video metadata information, and social information of the user who uploaded the video to be predicted. The playback data of the video to be predicted in a certain time period may refer to the dynamic data of one or more indicators of the video at different time points within a period of time, and may include but is not limited to: the number of video playbacks, the number of video likes, the number of video coins, the number of video collections, the number of video shares, the number of video comments, and the number of video barrages. For example, the number of video playbacks can be: the sequence of the number of playbacks of the video i to be predicted at the past n time points The sampling frequency of the playback data may be set to every two hours. The metadata information of the video to be predicted may include but is not limited to the contents listed in Table 1:

[0051] Table 1 Metadata information of the video to be predicted

[0052] variable explain Video category The partition number assigned to the video Video resolution The pixel dimensions of the video Video honor count The number of times a video is recommended by the platform Video tag count The number of tags attached to the video Video segment count The number of segments a video is divided into Video originality Whether the video is user-generated Video downloadable Whether the video can be downloaded from the platform to the local computer Video reprintable Is it forbidden to forward the video to other platforms? Video access mode Do I need to pay to watch the video content? Video collaboration Whether the video is a collaboration with other users High bitrate Does the video support high bitrate? Interactive video Does the video content allow audience interaction? Panoramic video Whether the video is panoramic PGC video Is the video produced by a professional team? Playlist video Whether the video is part of a series Scripted video Is the video content guided by a script? Task participation Whether the video is part of a platform activity or task

[0053] The social information of the user who uploaded the video to be predicted may include but is not limited to the content listed in Table 2:

[0054] Table 2 Social information of users who uploaded the video to be predicted

[0055]

[0056]

[0057] The proposed four key time contexts are shown in Table 3:

[0058] Table 3 Key time context

[0059] variable explain Holiday Eve(HDE) Is the current date the day before a holiday? Holiday(HLD) Is the current date a holiday? Published day count(PBD) The number of days the video is uploaded to the platform Time zone(TMZ) The current time period

[0060] Figure 2 It is a structural diagram of a preset multi-feature encoder of the present invention, and the encoder includes a visual feature sub-encoder, an acoustic feature sub-encoder, a text feature sub-encoder, a metadata feature sub-encoder, a social feature sub-encoder, and a hierarchical popularity sequence feature extraction network. Before the target data of the video to be predicted is input into the preset multi-feature encoder, a preprocessing operation is required. The visual feature sub-encoder may be, but is not limited to, composed of a pre-trained ResNet-152 model, a visual emotion classification model based on AlexNet, and a NIMA-VGG16 model, wherein the ResNet-152 model is used to extract deep visual features, the visual emotion classification model based on AlexNet is used to extract visual emotion features, and the NIMA-VGG16 model is used for visual quality scoring. The video frame may be, but is not limited to, intercepting the 0.5 second, 3 seconds, 6 seconds, 9 seconds, 12 seconds, and 15 seconds of the video, and the video frame and the video cover are input into the visual feature sub-encoder and the multi-layer perceptron MLP v In the example above, we determine the visual features based on element alignment. The acoustic feature sub-encoder can be, but is not limited to, composed of a pre-trained VGGish model and an emotion2vec model, wherein the VGGish model is used to extract deep audio features and the emotion2vec model is used to extract audio emotional features. The video audio can be, but is not limited to, the first 15 seconds of the video. The audio is input to the acoustic feature sub-encoder and the multi-layer perceptron MLP. a In the example, acoustic features based on element alignment are determined The text feature sub-encoder can be, but is not limited to, composed of a pre-trained BERT model and a sentiment analysis toolkit on PaddleNLP, where the BERT model is used to extract deep text features of the video title and description, and the sentiment analysis toolkit on PaddleNLP is used to determine the sentiment scores of the video title and description. The title and description of each video are input into the text feature sub-encoder and the multi-layer perceptron MLP. t In the text feature set, we determine the text features based on element alignment. The output dimension of the multilayer perceptron can be set to 128 but is not limited to it.

[0061] The metadata feature sub-encoder and social feature sub-encoder are composed of multiple embedding layers for processing discrete features and fully connected layers for processing continuous features. The metadata information of each video to be predicted and the social information of the user who uploaded the video to be predicted are respectively input into the metadata feature sub-encoder and social feature sub-encoder, and finally processed by the fully connected layer to determine the social features based on element alignment. and metadata features The calculation formula is as follows:

[0062]

[0063] In the above formulas (1) and (2), LeakyReLU represents the LeakyReLU activation function, z s and z m They represent the social and metadata features determined by concatenation after being processed by the embedding layer and the fully connected layer, respectively. s and w m represents the learnable weight, b s and b m Represents a learnable bias. The output dimension of the fully connected layer can be, but is not limited to, set to 128.

[0064] The playback timing data consists of the video popularity calculated based on the uniformly sampled playback indicators in a certain time period and different key time context features processed and combined by the embedding layer. The popularity of the video is determined by a combination of seven indicators: the number of video plays, the number of video likes, the number of video coins, the number of video collections, the number of video shares, the number of video comments, and the number of video barrages. The above indicators are all obtained through uniform sampling in a certain time period, and each sampling point can be regarded as a time point. The video popularity calculated according to the sampling frequency of the playback indicators can be divided into short-term popularity time series information and long-term popularity time series information. In order to avoid the adverse effects of large numerical range differences in popularity scores on network training, the popularity of the i-th video at the k-th time point is The calculation formula is as follows:

[0065]

[0066] In the above formula (3), represents the number of views of the i-th video at the k-th time point, represents the number of likes for the i-th video at the k-th time point, represents the amount of coins inserted into the i-th video at the k-th time point, represents the number of collections of the i-th video at the k-th time point, represents the number of shares of the i-th video at the k-th time point, represents the number of comments on the i-th video at the k-th time point, It represents the amount of bullet comments of the ith video at the kth time point. Assuming there are n time points in total, the popularity of the ith video after uniform sampling in a certain time period can be expressed as:

[0067] Key time contexts may include Holiday Eve (HDE), Holiday (HLD), Published daycount (PBD), and Time zone (TMZ). Key time context features emb1 It can be composed of HDE, HLD and TMZ, which represent the time context in hours; the key time context feature t emb2 It can be composed of HDE and HLD, which represent the time context in days; the key time context feature t emb3 It can be composed of PBD, HDE and HLD, which represents the time context of the video to be predicted on the target prediction date.

[0068] Figure 3 : is the hierarchical popularity sequence feature extraction network structure diagram of the present invention, which includes two time series feature extraction modules, each of which is composed of two layers of LSTM networks. First, the playback time series data of the video to be predicted is input into the hierarchical popularity sequence feature extraction network to calculate the short-term popularity trend I S and long-term popularity trends I L , I S and I L Use I K (K∈{S,L}) uniformly represents, I K The calculation formula is as follows:

[0069] I K =LeakyReLU(w K z K +b K ),K∈{S,L} (4)

[0070] In the above formula (4), LeakyReLU represents the LeakyReLU activation function, z K represents short-term popularity time series information or long-term popularity time series information, w K represents the learnable weight, b K represents a learnable bias.

[0071] The short-term popularity trend I S and the key temporal context feature t emb1 Input to the first time series feature extraction module to determine the short-term popular hidden state sequence The calculation formula is as follows:

[0072]

[0073] In the above formula (5), Represents the first layer of LSTM network in the first time series feature extraction module, Represents the second layer LSTM network in the first time series feature extraction module, [I S ;t emb1 ] indicates I S and t emb1 Perform feature concatenation. Take H S Hidden state at the last time step As a short-term fluctuation feature. I L and the key temporal context feature t emb2 Input to the second time series feature extraction module for feature extraction to determine the long-term popular hidden state sequence The calculation formula is as follows:

[0074]

[0075] In the above formula (6), Represents the first layer of LSTM network in the second time series feature extraction module, Represents the second layer LSTM network in the second time series feature extraction module, Express I L and t emb2 Perform feature concatenation. Finally, L The attention mechanism is applied to all time steps of to obtain the output of the hierarchical popularity sequence feature extraction network, i.e., the popularity sequence feature The calculation formula is as follows:

[0076]

[0077] In the above formula (7), represents H after softmax normalization L The weight of the feature at the i-th time step in , Indicates H L The features of the i-th time step in . The calculation formula is as follows:

[0078]

[0079] In the above formula (8), Indicates H L The initial weight of the feature at the i-th time step in is calculated as follows:

[0080]

[0081] In the above formula (9), tanh represents the tanh activation function, represents the transpose of a learnable query vector, w p represents the learnable weight, b p represents a learnable bias.

[0082] Getting all features After z∈{v,a,t,s,m,p}, the feature scoring stage gives a scoring weight α to each feature z Calculate the key time context feature t emb3 As a precondition, with each feature Do similarity calculation to get the corresponding initial score a z , and its calculation formula is as follows:

[0083]

[0084] In the above formula (10), tanh represents the tanh activation function, represents the transpose of a learnable query vector, w z and v z represents the learnable weight, b z Represents a learnable bias. After obtaining the initial score of each feature, the score weight α of each feature is determined by the following calculation. z , and its calculation formula is as follows:

[0085]

[0086] In the feature fusion stage, the score α of each feature is determined. z Afterwards, α z Apply to the corresponding In the process, the features of the video to be predicted are determined by fusion The calculation formula is as follows:

[0087]

[0088] The preset popularity prediction network takes the features of the video to be predicted Input into a fully connected layer for calculation to determine the popularity of the video to be predicted The calculation formula is as follows:

[0089]

[0090] In the above formula (12), represents the learnable weights, represents a learnable bias.

[0091] The prediction effect of the embodiment can be measured by four indicators: mean squared error (MSE), mean absolute error (MAE), Spearman's rank correlation coefficient (SRCC) and coefficient of determination (R2). The comparison prediction results of the present invention and other methods are shown in Table 4. MSE, MAE, SRCC and R 2 The calculation formulas are shown in (14), (15), (16) and (17), where represents the popularity prediction value of the i-th video, p i represents the true popularity of the i-th video on the target prediction date, represents the mean of video popularity, d i Represents the rank difference of each pair of data of two variables, and n represents the total number of videos:

[0092]

[0093] The smaller the MSE value is, the more accurate the prediction result of the embodiment for popularity is, and the same applies to MAE; the value range of SRCC is [-1,1], and the higher the absolute value is, the stronger the monotonic relationship between the two variables is, thus representing the more accurate and reliable the prediction result of the embodiment is; the value range of the determination coefficient (R2) is [0,1], and R2 measures the ability of the regression model to explain the observed data. The higher its value is, the more data variation can be explained by the embodiment, thus representing the more accurate and reliable the prediction result of the embodiment is.

[0094] Table 4 Comparison prediction results of the present invention and other methods

[0095] method MSE MAE SRCC R2 Support Vector Regression 4.5811 1.7814 0.5653 0.1645 Multiple Linear Regression 2.4147 1.2026 0.7820 0.5648 The present invention 0.7341 0.5603 0.9349 0.8680

[0096] In Table 4, the above prediction results are based on 50791 video data. When training the multivariate linear regression, L2 regularization with a penalty coefficient of 0.001 was used to suppress overfitting of the model; the kernel function of the support vector regression used the RBF (Radial Basis Function).

[0097] The above is only a preferred embodiment of the present invention. Those skilled in the art should understand that the disclosure scope designed by the present invention is not limited to the technical solution formed by the specific combination of the above technical features. All equivalent changes and modifications made according to the scope of the patent application of the present invention should fall within the scope of the present invention.

Claims

1. A method for predicting online video popularity based on multi-feature learning based on time information perception, characterized in that: include: (1) Obtaining multi-feature data and playback timing data related to the video to be predicted; (2) inputting the multi-feature data into a feature sub-encoder in a preset multi-feature encoder according to the category to determine visual, acoustic, text, social and metadata features; (3) inputting the playback time sequence data into a hierarchical popularity sequence feature extraction network in a preset multi-feature encoder to determine the popularity sequence features of the video to be predicted; (4) Inputting the visual, acoustic, text, social, metadata and popularity sequence features of the video to be predicted into a preset feature fusion network for feature fusion; (5) The fused features of the video to be predicted are input into the preset popularity prediction network to obtain the prediction result.

2. According to claim 1, the method for predicting online video popularity based on multi-feature learning based on time information perception is characterized in that: The preset multi-feature encoder in step (2) is based on multi-feature data and is input into different sub-encoders according to different categories: the visual, acoustic, and text data of the video to be predicted are divided into multi-modal content information and input into the visual, acoustic, and text feature sub-encoders; Inputting the social information of the user who uploaded the video to be predicted and the metadata information of the video to be predicted into the social and metadata feature sub-encoder; inputting the playback time sequence data into a hierarchical popularity sequence feature extraction network to determine the popularity sequence feature of the video to be predicted; After the initial extraction of visual, acoustic, and textual multimodal content information, they are input into the corresponding MLP multi-layer perceptrons (MLPs) v 、MLP a 、MLP t In the embodiment, a feature vector based on element alignment is determined, and visual, acoustic, text, social, metadata and popularity sequence features are processed and determined by the preset multi-feature encoder.

3. According to claim 2, the method for predicting online video popularity based on multi-feature learning based on time information perception is characterized in that: The social and metadata feature sub-encoder is a network structure that integrates multiple embedding layers and fully connected layers. In the social and metadata feature sub-encoder, the embedding layer is used to process discrete digital features, and the fully connected layer is used to process continuous digital features. The social information of the user who uploaded the video to be predicted and the metadata information of the video to be predicted processed by the social and metadata feature sub-encoder are calculated to determine the social features based on element alignment. and metadata features The calculation formula is as follows: In the above formulas (1) and (2), LeakyReLU represents the LeakyReLU activation function, z s and z m They represent the social and metadata features determined by concatenation after being processed by the embedding layer and the fully connected layer, respectively. s and w m represents the learnable weight, b s and b m represents a learnable bias.

4. According to claim 1, the method for predicting online video popularity based on multi-feature learning based on time information perception is characterized in that: In step (1), the playback timing data of the video to be predicted is composed of the video popularity calculated based on the uniformly sampled playback index in a certain time period and the key time context features processed and combined by the embedding layer; the key time context includes Holiday Eve (HDE), Holiday (HLD), Published Day Count (PBD) and Time Zone (TMZ); different key time contexts are combined into different key time context features: key time context feature t emb1 It consists of HDE, HLD and TMZ, which represent the time context in hours; the key time context feature t emb2 It consists of HDE and HLD, which represent the time context in days; Key temporal context features t emb3 It consists of PBD, HDE and HLD, which represent the time context of the video to be predicted on the target prediction date; the playback time sequence data of the video to be predicted is input into the hierarchical popularity sequence feature extraction network to determine the popularity sequence feature 5. According to claim 4, the method for predicting online video popularity based on multi-feature learning based on time information perception is characterized in that: The video popularity calculated based on the uniformly sampled playback index in the certain time period is determined comprehensively by seven indicators: the video playback volume, the video like volume, the video coin volume, the video collection volume, the video sharing volume, the video comment volume, and the video barrage volume. Each indicator is obtained through uniform sampling in a certain time period, and each sampling point can be regarded as a time point; the video popularity calculated according to the sampling frequency of the playback index is divided into short-term popularity time series information and long-term popularity time series information; the popularity of the i-th video at the k-th time point The calculation formula is as follows: In the above formula (3), represents the number of views of the i-th video at the k-th time point, represents the number of likes for the i-th video at the k-th time point, represents the amount of coins inserted into the i-th video at the k-th time point, represents the number of collections of the i-th video at the k-th time point, represents the number of shares of the i-th video at the k-th time point, represents the number of comments on the i-th video at the k-th time point, represents the amount of bullet comments for the ith video at the kth time point; assuming there are n time points in total, the popularity of the ith video after uniform sampling in a certain time period can be expressed as:

6. According to claim 4, the method for predicting online video popularity based on multi-feature learning based on time information perception is characterized in that: The hierarchical popularity sequence feature extraction network includes two time series feature extraction modules, each of which is composed of two layers of LSTM long short-term memory networks; the playback time series data of the video to be predicted is input into the hierarchical popularity sequence feature extraction network to calculate the short-term popularity trend I S and long-term popularity trends I L , I S and I L Use I K (K∈{S,L}) uniformly represents, I K The calculation formula is as follows: AND K =LeakyReLU(W K WITH K +b K ),K∈{S,L} (4) In the above formula (4), LeakyReLU represents the LeakyReLU activation function, z K represents short-term popularity time series information or long-term popularity time series information, w K represents the learnable weight, b K represents a learnable bias; The short-term popularity trend I S and the key temporal context feature t emb1 Input to the first time series feature extraction module to determine the short-term popular hidden state sequence The calculation formula is as follows: In the above formula (5), Represents the first layer of LSTM network in the first time series feature extraction module, Represents the second layer LSTM network in the first time series feature extraction module, [I S ;t emb1 ] indicates I S and t emb1 Perform feature splicing operation; take H S Hidden state at the last time step As a short-term volatility feature; Will I L The key time context feature temb2 is input into the second time series feature extraction module for feature extraction to determine the long-term popular hidden state sequence The calculation formula is as follows: In the above formula (6), Represents the first layer of LSTM network in the second time series feature extraction module, Represents the second layer LSTM network in the second time series feature extraction module, Express I L and t emb2 Perform feature concatenation operation; finally, L The attention mechanism is applied to all time steps of to obtain the output of the hierarchical popularity sequence feature extraction network, i.e., the popularity sequence feature The calculation formula is as follows: In the above formula (7), represents H after softmax normalization L The weight of the feature at the i-th time step in , Indicates H L The features of the i-th time step in ; The calculation formula is as follows: In the above formula (8), Indicates H L The initial weight of the feature at the i-th time step in is calculated as follows: In the above formula (9), tanh represents the tanh activation function, represents the transpose of a learnable query vector, w p represents the learnable weight, b p represents a learnable bias.

7. The method for predicting online video popularity based on multi-feature learning based on time information perception according to claim 1, characterized in that: The preset feature fusion network in step (4) includes: a feature scoring stage and a feature fusion stage. The feature scoring stage uses the time context of the video to be predicted on the target prediction date and the attention mechanism to achieve the adaptive fusion of visual, acoustic, and textual multimodal content features, as well as social, metadata, and popularity sequence features; obtain all features After z∈{v,a,t,s,m,p}, the feature scoring stage gives each feature a score weight α z Calculate α z After that, the feature fusion stage will fuse each feature to determine the features of the video to be predicted 8. The method for predicting online video popularity based on multi-feature learning based on time information perception according to claim 7 is characterized in that: In the feature scoring stage, the key time context feature t emb3 As a precondition, with each feature Do similarity calculation to get the corresponding initial score a z , and its calculation formula is as follows: In the above formula (10), tanh represents the tanh activation function, represents the transpose of a learnable query vector, w z and v z represents the learnable weight, b z represents a learnable bias; After obtaining the initial score of each feature, the score weight α of each feature is determined through calculation. z , and its calculation formula is as follows:

9. The method for predicting online video popularity based on multi-feature learning based on time information perception according to claim 7, characterized in that: In the feature fusion stage, the score α of each feature is determined. z Afterwards, α z Apply to the corresponding In the process, the features of the video to be predicted are determined by fusion The calculation formula is as follows: The preset popularity prediction network takes the features of the video to be predicted Input into a fully connected layer for calculation to determine the popularity of the video to be predicted The calculation formula is as follows: In the above formula (13), represents the learnable weight vector, represents the learnable bias vector.

Images