Video hotness prediction method, video recommendation method, system and storage medium

Abstract

The application discloses a video heat prediction method, a video recommendation method, a system and a storage medium, and belongs to the field of artificial intelligence.In the application, visual enhancement features are obtained by fusing attention features, global semantic features and dynamic features of each frame; the global semantic features are obtained by weighting and summing each frame feature in a video frame feature sequence based on corresponding importance weights; the dynamic features of each frame are obtained by inputting the video frame feature sequence into a state space model; the attention features are obtained by inputting the global semantic features and the video frame feature sequence into a sparse attention model after splicing; wherein, the sparse attention model constructs an attention window according to the importance weights of each video frame, the video frame with a higher importance weight corresponds to a larger attention window, and the video frame with a lower importance weight corresponds to a smaller attention window.Based on this, the application can realize accurate video heat prediction with lower calculation and storage overheads.

Description

Technical Field

[0001] This invention belongs to the field of artificial intelligence technology, and more specifically, relates to a video popularity prediction method, a video recommendation method, a system, and a storage medium. Background Technology

[0002] In recent years, with the development of short video platforms and social media, video has become an important medium for online content dissemination and user interaction. Video popularity prediction refers to predicting the number of views, platform popularity score, or overall popularity value a video may receive within a preset future time frame based on signals such as visual content, text descriptions, user information, and statistical metadata available before or early in the video's release. This technology can provide important data for content recommendation, traffic scheduling, and the governance of risky content.

[0003] Existing video popularity prediction methods typically employ a combination of multimodal feature modeling and historical sample retrieval enhancement. First, features such as video frames, text descriptions, user attributes, and statistical information are extracted, and then an attention mechanism, hierarchical fusion network, or regression model is used to form a representation of the current video. Subsequently, a historical sample database is used to retrieve historical videos related to the current video, and the retrieved historical sample features are incorporated into the prediction process as external contextual information to improve the instability of predictions when relying solely on the current video content.

[0004] However, in real-world video platforms, the temporal span, semantic density, and propagation patterns of video content vary significantly. Some key semantic segments influencing popularity may only appear in a few frames of a long video sequence, while a large number of ordinary frames contribute little to the final popularity score. Existing methods often employ fixed-interval sampling, fixed-frame-count sampling, or short-time-window modeling. While these methods reduce computational costs, they easily miss sparsely distributed highlight segments, resulting in incomplete video content representation and lower accuracy in popularity prediction. Conversely, directly performing dense modeling on all video frames would significantly increase computational and storage costs, hindering deployment in large-scale video scenarios. Summary of the Invention

[0005] In view of the above-mentioned defects or improvement needs of the existing technology, the present invention provides a video popularity prediction method, a video recommendation method, a system and a storage medium to solve the technical problem that the existing technology cannot achieve accurate video popularity prediction with low computational and storage overhead.

[0006] To achieve the above objectives, in a first aspect, the present invention provides a video popularity prediction method, comprising: performing a popularity prediction operation on the video to be predicted: Obtain the video frame sequence of the video to be predicted S The text description and owner attribute information are extracted and feature-encoded separately to obtain the corresponding video frame feature sequences. Text features and attribute features; video frame feature sequences A sequence of features composed of the individual frames in a video frame sequence; Based on video frame feature sequences Extracting its visual enhancement features: This involves extracting the feature sequence from the video frames. The feature differences between the i-th frame and the (i-1)-th frame are input into the frame scoring model to obtain the importance weight of the i-th frame; ; Video frame sequence S The number of frames; the importance weight of the first frame is obtained by inputting the preset initial feature differences into the frame scoring model; the video frame feature sequence The features of each frame in the video are weighted and summed based on their corresponding importance weights to obtain the global semantic features; the video frame feature sequence is then processed. The input is fed into the state-space model to obtain the dynamic features of each frame; the global semantic features are then combined with the video frame feature sequence. After concatenation, the data is input into a sparse attention model, and the features at the corresponding positions of the global semantic features are extracted from the output of the sparse attention model as attention features. The attention features, global semantic features, and dynamic features of each frame are fused to obtain the corresponding visual enhancement features. During computation, the sparse attention model uses a window attention mechanism to process the global semantic features and video feature sequences. Each frame feature in the video is assigned a corresponding attention window; the attention window corresponding to the global semantic feature covers both the global semantic feature and the frequency frame feature sequence; the video frame feature sequence The size of the attention window corresponding to each frame feature is determined based on its importance weight; the larger the importance weight of the frame feature, the larger the size of the attention window. Construct a feature set including visual enhancement features, text features, and attribute features of the video to be predicted; fuse the features in the feature set to obtain a comprehensive prediction feature; The comprehensive prediction features are input into the prediction head to obtain the predicted popularity value of the video to be predicted.

[0007] More preferably, the above state-space model is a selective state-space model.

[0008] More preferably, the step size calculation formula in the above selective state-space model is:

[0009] in, This refers to the softplus function; Indicates linear layer operations; These are the frame features input into the selective state-space model; Frame features The corresponding scan step size; This is the step size adjustment amount; The weight matrix is ​​a learnable weight matrix; This is the global semantic relevance adjustment coefficient; For the corresponding global semantic features; for and Similarity; This is the importance adjustment coefficient, and ; Frame features Importance weights.

[0010] More preferably, the above-mentioned operation of performing popularity prediction on the video to be predicted further includes: extracting retrieval enhancement features of the video to be predicted: The similarity between the multimodal features of the video to be predicted and the aggregated multimodal features corresponding to each cluster in the historical sample feature set is calculated separately, and used as the matching score between the video to be predicted and the corresponding cluster; the multimodal features of the video to be predicted are the concatenated features of the visual enhancement features, text features and attribute features of the video to be predicted. Obtain the K clusters with the highest matching scores and use them as candidate clusters for the video to be predicted; normalize the matching scores of the K candidate clusters for the video to be predicted to obtain the corresponding K matching weights. ; The aggregated multimodal features corresponding to each candidate cluster of the video to be predicted are weighted and summed based on the corresponding matching weights, and used as the retrieval enhancement features of the video to be predicted. At this point, the feature set also includes: retrieval enhancement features for the video to be predicted; The historical sample feature set includes: multimodal features of multiple historical videos carrying corresponding real popularity values; the multimodal features of historical videos are the concatenation features of visual enhancement features, text features and attribute features of historical videos; the visual enhancement features of historical videos are extracted based on their video frame feature sequences; the video frame feature sequences, text features and attribute features of historical videos are features after feature encoding of the video frame sequences, text descriptions and owner attribute information of historical videos respectively. The historical sample feature set is divided into multiple clusters, and the division method is as follows: The equal-frequency binning method is used to partition the true heat of the historical sample feature set, thereby dividing the historical sample feature set into multiple non-overlapping subsets. Cluster analysis is performed on each subset based on its multimodal features to obtain multiple clusters and the aggregated multimodal features corresponding to each cluster. The aggregated multimodal features are the cluster centers of the multimodal features in the corresponding clusters. Each subset has a corresponding label. Each cluster has a corresponding label.

[0011] More preferably, the above-mentioned fusion of features in the feature group includes: The visual enhancement features of the video to be predicted are used as the visual input, and the text features of the video to be predicted are used as the text input. Based on the cross-attention mechanism, the attention features of the text side on the visual side are calculated respectively. And visual attention features to text. ; Calculate the retrieval enhancement features and attention features of the video to be predicted. The Hadamard product yields the visual historical interaction features; Calculate the retrieval enhancement features and attention features of the video to be predicted. The Hadama product yields the textual history interaction features; Attention features Attention characteristics The visual historical interaction features, textual historical interaction features, and attribute features of the video to be predicted are concatenated to obtain comprehensive prediction features.

[0012] More preferably, the frame scoring model, state space model, sparse attention model, and prediction head described above are trained in the following manner: Obtain the training sample set; the training samples in the training sample set are historical videos, and the corresponding tags are the actual popularity values; Perform a popularity prediction operation on each training sample in the training sample set to obtain the corresponding predicted popularity value; Construct training objectives; whereby the training objectives include minimizing the difference loss between the predicted popularity value and the corresponding label for each training sample; Based on the training objective, the parameters in the frame scoring model, state space model, sparse attention model, and prediction head are updated simultaneously.

[0013] More preferably, when performing a popularity prediction operation on the video to be predicted further includes extracting retrieval enhancement features of the video to be predicted, the operation of performing popularity prediction on the training samples further includes: extracting retrieval enhancement features of the training samples; The training objectives also include: minimizing the difference loss between the first matching distribution and the first preset prior distribution, and minimizing the difference loss between the second matching distribution and the second preset prior distribution; wherein, the first preset prior distribution is a uniform distribution; the second preset prior distribution is a distribution composed of the labels of all training samples; the first matching distribution is obtained by recording the candidate clusters of the training samples obtained during the extraction of retrieval enhancement features; the labels of the candidate clusters of all training samples constitute the first matching distribution; the second matching distribution is obtained by: for each training sample, determining the true heat range of its historical sample feature set based on its label, and then establishing a mapping with the corresponding subset in the historical sample feature set; the labels of the subsets corresponding to all training samples constitute the second matching distribution; based on the training objectives, when updating the parameters in the frame scoring model, state space model, sparse attention model and prediction head simultaneously, the aggregated multimodal features corresponding to the candidate clusters of each training sample are also updated simultaneously.

[0014] In a second aspect, the present invention provides a video popularity prediction system, comprising: a memory and a processor, wherein the memory stores a computer program, and the processor executes the computer program to perform the video popularity prediction method provided in the first aspect of the present invention.

[0015] Thirdly, the present invention provides a video recommendation method, which uses the video popularity prediction method provided in the first aspect of the present invention to predict the popularity of each video on the platform and obtain the predicted popularity value of each video; the videos on the platform are sorted from high to low according to the predicted popularity value and a preview sequence is generated and pushed to the user.

[0016] Fourthly, the present invention also provides a computer-readable storage medium comprising a stored computer program, wherein, when the computer program is executed by a processor, it controls the device where the storage medium is located to execute the video popularity prediction method provided in the first aspect of the present invention or the video recommendation method provided in the third aspect of the present invention.

[0017] Fifthly, the present invention also provides a computer program product, including a computer program / instruction, which, when executed by a processor, implements the video popularity prediction method provided in the first aspect of the present invention or the video recommendation method provided in the third aspect of the present invention.

[0018] In summary, the above-described technical solutions conceived in this invention can achieve the following beneficial effects: 1. This invention provides a video popularity prediction method, which constructs a feature group including visual enhancement features, text features, and attribute features of the video to be predicted; fuses the features in the feature group to obtain a comprehensive prediction feature; inputs the comprehensive prediction feature into a prediction head to obtain the predicted popularity value of the video to be predicted; wherein, the visual enhancement feature is obtained by fusing attention features, global semantic features, and dynamic features of each frame; the global semantic feature is obtained by fusing the video frame feature sequence. The features of each frame are obtained by weighted summation based on their corresponding importance weights, focusing on the global features of the video to be predicted; the dynamic features of each frame are obtained by summing the video frame feature sequences. The input is obtained by the state-space model, focusing on the evolution of frames over time; attention features are obtained by combining global semantic features with the video frame feature sequence. The concatenated images are input into a sparse attention model, which focuses on the interaction between highlight segments across time locations. The sparse attention model constructs an attention window based on the importance weight of each video frame. Video frames with higher importance weights correspond to larger attention windows to capture the interaction between distant but semantically related highlight segments. Video frames with lower importance weights correspond to smaller attention windows to reduce invalid interactions between redundant frames. Based on this, the present invention does not require dense modeling of all video frames, and can enhance the correlation modeling between highlight segments while reducing computational load, thereby achieving accurate video heat prediction with lower computational and storage overhead.

[0019] 2. Furthermore, in the video popularity prediction method provided by this invention, the step size calculation formula in the selective state-space model is as follows: ;in, The value is negative. Based on this design, for video frames with high importance weights, a smaller scan step size is used for dense scanning to preserve key semantic changes; for video frames with low importance weights, a larger scan step size is used for sparse scanning, thereby further reducing the computational overhead caused by redundant video frames.

[0020] 3. Furthermore, in the video popularity prediction method provided by this invention, the popularity prediction operation for the video to be predicted further includes: extracting retrieval enhancement features of the video to be predicted; wherein, considering that existing flat memory banks usually store historical video samples in the same memory space, as the number of historical samples increases, the retrieval scale and noise interference increase accordingly, this invention constructs a fixed-size prototype grid according to popularity partitioning and topic clustering, firstly dividing the historical sample feature set into multiple non-overlapping subsets according to its true popularity value; then performing clustering analysis on each subset based on its multimodal features to obtain multiple clusters; this method can reduce the interference of irrelevant historical context and improve the relevance and scalability of structured memory retrieval.

[0021] 4. Further, in the video popularity prediction method provided by this invention, the training objective further includes: minimizing the difference loss between the first matching distribution and the first preset prior distribution, and minimizing the difference loss between the second matching distribution and the second preset prior distribution; wherein, the first preset prior distribution is a uniform distribution; the second preset prior distribution is a distribution composed of the labels of all training samples; the first matching distribution is obtained by recording the candidate clusters of the training samples obtained during the extraction of retrieval enhancement features; the labels of all candidate clusters of the training samples constitute the first matching distribution; the second matching distribution is obtained by determining the true popularity interval of the historical sample feature set to which each training sample belongs based on its label, and then establishing a mapping with the corresponding subset of the historical sample feature set; the labels of the subsets corresponding to all training samples constitute the second matching distribution; based on the above design, the matching weights of the training samples can be avoided from being overly concentrated in a few clusters, so that the aggregated multimodal features corresponding to each cluster can be more fully utilized, thereby further improving the effectiveness of retrieval enhancement features, and further improving the accuracy of video popularity prediction. Attached Figure Description

[0022] Figure 1 A flowchart of a video popularity prediction method provided in an embodiment of the present invention.

[0023] Figure 2 This is a schematic diagram illustrating the process of acquiring visual enhancement features according to an embodiment of the present invention. Detailed Implementation

[0024] To make the objectives, technical solutions, and advantages of this invention clearer, the invention will be further described in detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative and not intended to limit the invention. Furthermore, the technical features involved in the various embodiments of this invention described below can be combined with each other as long as they do not conflict with each other.

[0025] Example 1 A video popularity prediction method, such as Figure 1 As shown, this includes: performing a popularity prediction operation on the video to be predicted: S1. Obtain the video frame sequence of the video to be predicted. S The text description and owner attribute information are extracted and feature-encoded separately to obtain the corresponding video frame feature sequences. Text features and attribute features; video frame feature sequences A sequence of features composed of the individual frames in a video frame sequence; In this embodiment, the video frame sequence SThe prediction tool can be the video to be predicted, or it can be obtained by extracting video frames from the video to be predicted according to a preset number of frames or a preset time interval. The text description of the video to be predicted includes one or more of the following: title, tags, introduction, subtitles, publication time, and text information available on the platform. The owner attribute information of the video to be predicted includes some historical statistical information, such as the number of followers, historical likes, and video theme of the owner.

[0026] It should be noted that the above feature encoding can adopt any feature encoding method, and no limitation is made here.

[0027] Preferably, in one optional implementation, for a video frame sequence, a visual encoder (such as an image or video coding network capable of outputting frame-level features) can be used to extract features from each frame in the video frame sequence to obtain frame-level visual features, thus forming a video frame feature sequence. For text descriptions, a text encoder can be used to extract text semantic features from the text descriptions to obtain text features. For owner attribute information, embedding layers, normalization layers, or fully connected layers can be used to extract features from it to generate attribute features.

[0028] S2, Based on video frame feature sequences Extracting its visual enhancement features: This involves extracting the feature sequence from the video frames. The feature differences between the i-th frame and the (i-1)-th frame are input into the frame scoring model to obtain the importance weight of the i-th frame; ; Video frame sequence S The number of frames; the importance weight of the first frame is obtained by inputting the preset initial feature differences into the frame scoring model; the video frame feature sequence The features of each frame in the video are weighted and summed based on their corresponding importance weights to obtain the global semantic features; the video frame feature sequence is then processed. The input is fed into the state-space model to obtain the dynamic features of each frame; the global semantic features are then combined with the video frame feature sequence. After concatenation, the data is input into a sparse attention model, and the features at the corresponding positions of the global semantic features are extracted from the output of the sparse attention model as attention features. The attention features, global semantic features, and dynamic features of each frame are fused to obtain the corresponding visual enhancement features. During computation, the sparse attention model uses a window attention mechanism to process the global semantic features and video feature sequences. Each frame feature in the video is assigned a corresponding attention window; the attention window corresponding to the global semantic feature covers both the global semantic feature and the frequency frame feature sequence; the video frame feature sequence The size of the attention window corresponding to each frame feature is determined based on its importance weight; the larger the importance weight of a frame feature, the larger the size of the attention window.

[0029] like Figure 2 The diagram shown illustrates the process of acquiring visual enhancement features provided in this embodiment.

[0030] In the above process, an adaptive sparse window is constructed based on the importance weight of each video frame. Video frames with higher importance weights correspond to a larger interaction range, while those with lower importance weights correspond to a smaller interaction range. Attention weights between queries, keys, and values ​​are calculated within the corresponding window. For salient video frames, a wider window is set to capture the interaction relationships between distant but semantically related highlight segments; for ordinary video frames, a narrower attention window is set to reduce invalid interactions between redundant frames. By calculating attention weights only within the corresponding window using the sparse attention model, invalid interactions between redundant video frames can be reduced, and the interaction relationships between distant but semantically related video frames can be captured. Compared to full attention, this method calculates attention weights only within a content-aware sparse window, thereby enhancing the association modeling between highlight segments while reducing computational cost.

[0031] It should be noted that the frame scoring model evaluates the importance weights based on the feature changes between video frames, so that video frames with more significant changes tend to receive higher importance weights, while video frames with higher redundancy tend to receive lower importance weights. The above frame scoring model is a machine learning model, preferably a deep learning model, such as CNN, MLP, etc., but this is not a limitation.

[0032] It should be noted that there are various ways to fuse attention features, global semantic features, and dynamic features of each frame, such as gated fusion, concatenating attention features, global semantic features, and dynamic features of each frame, adjusting attention features, global semantic features, and dynamic features of each frame to the same size before weighted summation, or adjusting attention features, global semantic features, and dynamic features of each frame to the same size before calculating the geometric mean, etc. No limitation is made here. Preferably, gated fusion is used for attention features, global semantic features, and dynamic features of each frame.

[0033] In this embodiment, the aforementioned preset initial feature difference is a 0 vector with the same dimension as the input dimension of the frame scoring model.

[0034] It should be noted that the aforementioned state-space model can be a primitive state-space model, a structured state-space model, a selective state-space model, etc., and is not limited here. Preferably, in one optional implementation, the aforementioned state-space model is a selective state-space model, such as the Mamba model.

[0035] Preferably, in one optional implementation, the step size calculation formula in the above selective state-space model is:

[0036] in, This refers to the softplus function; Indicates linear layer operations; These are the frame features input into the selective state-space model; Frame features The corresponding scan step size; This is the step size adjustment amount; The weight matrix is ​​a learnable weight matrix; This is the global semantic relevance adjustment coefficient; For the corresponding global semantic features; for and Similarity; This is the importance adjustment coefficient, and This ensures that, all other things being equal, the higher the global semantic relevance and the greater the importance weight, the smaller the corresponding scan step size. Frame features Importance weights. In one optional implementation, The value is -0.1.

[0037] Through the above design, dense scanning is performed with a smaller scanning step size for video frames with high importance weights in order to preserve key semantic changes; sparse scanning is performed with a larger scanning step size for video frames with low importance weights in order to reduce the computational overhead caused by redundant video frames; and forward and backward temporal information are fused through bidirectional scanning.

[0038] The above method yields temporal enhancement features that include long-distance temporal dependencies and specular fragment interaction information.

[0039] S3. Construct a feature set including visual enhancement features, text features, and attribute features of the video to be predicted; fuse the features in the feature set to obtain a comprehensive prediction feature; It should be noted that there are various ways to fuse the features in a feature group, such as concatenating the features in the feature group or adjusting the features in the feature group to the same size before weighted summation, etc., and no limitation is made here. Preferably, in an optional implementation, when the feature group does not include the retrieval enhancement features of the video to be predicted, fusion is performed in the following way: The visual enhancement features of the video to be predicted are used as the visual input, and the text features of the video to be predicted are used as the text input. Based on the cross-attention mechanism, the attention features of the text side on the visual side are calculated respectively. And visual attention features to text. This yields cross-modal aligned visual and textual features; attention features. Corresponding to a new visual feature; attention feature This corresponds to a new text feature; Attention features Attention characteristics Visual enhancement features and attribute features are combined to obtain comprehensive predictive features.

[0040] S4. Input the comprehensive prediction features into the prediction head to obtain the predicted popularity value of the video to be predicted.

[0041] It should be noted that the prediction head is a regression model, such as a linear regression head or MLP, etc., which is not limited here. It outputs the popularity value of the video to be predicted within a preset future time range through regression.

[0042] In one optional implementation, the above-mentioned operation of performing popularity prediction on the video to be predicted further includes: extracting retrieval enhancement features of the video to be predicted. The similarity between the multimodal features of the video to be predicted and the aggregated multimodal features corresponding to each cluster in the historical sample feature set is calculated separately, and used as the matching score between the video to be predicted and the corresponding cluster; the multimodal features of the video to be predicted are the concatenated features of the visual enhancement features, text features and attribute features of the video to be predicted. Obtain the K clusters with the highest matching scores and use them as candidate clusters for the video to be predicted; normalize the matching scores of the K candidate clusters for the video to be predicted to obtain the corresponding K matching weights. ; The aggregated multimodal features corresponding to each candidate cluster of the video to be predicted are weighted and summed based on the corresponding matching weights to serve as the retrieval enhancement features of the video to be predicted. The retrieval enhancement features are used to represent the correlation information between the video to be predicted and historical videos, and to provide historical context for popularity prediction.

[0043] At this point, the feature set also includes: retrieval enhancement features for the video to be predicted; The historical sample feature set includes: multimodal features of multiple historical videos carrying corresponding real popularity values; the multimodal features of historical videos are the concatenation features of visual enhancement features, text features and attribute features of historical videos; the visual enhancement features of historical videos are extracted based on their video frame feature sequences; the video frame feature sequences, text features and attribute features of historical videos are features after feature encoding of the video frame sequences, text descriptions and owner attribute information of historical videos respectively. The historical sample feature set is divided into multiple clusters, and the division method is as follows: The equal-frequency binning method is used to partition the true popularity of the historical sample feature set, thereby dividing the historical sample feature set into multiple non-overlapping subsets. Cluster analysis is performed on each subset based on its multimodal features to achieve topic clustering, resulting in multiple clusters and the aggregated multimodal features corresponding to each cluster. The aggregated multimodal features are the cluster centers of the multimodal features in the corresponding clusters. Each cluster has a corresponding label.

[0044] In this embodiment, the historical sample feature set is divided into multiple non-overlapping subsets according to the quantiles of the true popularity values. There are various methods for clustering analysis, such as the K-Means algorithm and the K-Medoids algorithm, which are not limited here.

[0045] Existing flat memory repositories typically store historical video samples in the same memory space. As the number of historical samples increases, the retrieval scale and noise interference also increase accordingly. This embodiment constructs a fixed-size prototype grid based on popularity partitioning and topic clustering. This approach can reduce interference from irrelevant historical context and improve the relevance and scalability of structured memory retrieval.

[0046] When the multimodal features of the video to be predicted are inconsistent with the dimensions of the multimodal features in the historical sample feature set, nonlinear mapping and normalization can be performed on the multimodal features of the video to be predicted so that they are in a matching feature space with the multimodal features in the historical sample feature set.

[0047] Preferably, in an optional implementation, when the feature group further includes: retrieval enhancement features of the video to be predicted, the features in the feature group are fused, including: The visual enhancement features of the video to be predicted are used as the visual input, and the text features of the video to be predicted are used as the text input. Based on the cross-attention mechanism, the attention features of the text side on the visual side are calculated respectively. And visual attention features to text. ; Calculate the retrieval enhancement features and attention features of the video to be predicted. The Hadamard product yields the visual historical interaction features; Calculate the retrieval enhancement features and attention features of the video to be predicted. The Hadama product yields the textual history interaction features; Attention features Attention characteristics The visual historical interaction features, textual historical interaction features, and attribute features of the video to be predicted are concatenated to obtain comprehensive prediction features.

[0048] In one optional implementation, the frame scoring model, state space model, sparse attention model, and prediction head described above are trained in the following manner: Obtain the training sample set; the training samples in the training sample set are historical videos, and the corresponding tags are the actual popularity values; Perform a popularity prediction operation on each training sample in the training sample set to obtain the corresponding predicted popularity value; Construct training objectives; among which, the training objectives include: minimizing the prediction loss, that is, minimizing the difference loss between the predicted popularity value and the corresponding label of each training sample; Based on the training objective, the parameters in the frame scoring model, state space model, sparse attention model, and prediction head are updated simultaneously.

[0049] It should be noted that the aforementioned difference loss can be measured by Huber loss, MSE loss, MAE loss, etc., and no specific method is used here, but Huber loss is preferred.

[0050] In one optional implementation, when performing a popularity prediction operation on the video to be predicted further includes extracting retrieval enhancement features of the video to be predicted, the operation of performing popularity prediction on the training samples further includes: extracting retrieval enhancement features of the video to be predicted. The training objectives also include: minimizing the load balancing loss, i.e., minimizing the difference loss between the first matching distribution and the first preset prior distribution, and minimizing the difference loss between the second matching distribution and the second preset prior distribution (making the first matching distribution as close as possible to a uniform distribution, and the second matching distribution as close as possible to the second preset prior distribution); wherein, the first preset prior distribution is a uniform distribution; the second preset prior distribution is a distribution composed of the labels of all training samples, usually a long-tailed distribution; the first matching distribution is obtained by recording the candidate clusters of the training samples obtained during the extraction of retrieval enhancement features; the labels of all training samples are used to form the first matching distribution; the second matching distribution is obtained by determining the true popularity interval of the historical sample feature set to which each training sample belongs based on its label, and then establishing a mapping with the corresponding subset in the historical sample feature set; the labels of the subsets corresponding to all training samples are used to form the second matching distribution. Based on the training objective, when updating the parameters in the frame scoring model, state space model, sparse attention model and prediction head simultaneously, the aggregated multimodal features corresponding to the candidate clusters of each training sample are also updated at the same time.

[0051] Based on the above design, the matching weights of the training samples can be avoided from being overly concentrated in a few clusters, so that the aggregated multimodal features corresponding to each cluster can be more fully utilized, thereby further improving the effectiveness of retrieval enhancement features and further improving the accuracy of video popularity prediction.

[0052] By minimizing the difference loss between the matching distribution and the preset prior distribution, we can reduce uneven distribution of candidate clusters and route collapse.

[0053] The prediction loss and load balancing loss can be weighted and summed to obtain the total training loss. Then, the parameters of the frame scoring model, state space model, sparse attention model, prediction head, and aggregated multimodal features corresponding to the candidate clusters of each training sample can be updated through backpropagation.

[0054] In one alternative implementation, the number of subsets is 6, the number of clusters in each subset is 32, and K is 5. It should be understood that the above parameters are only used to illustrate one possible implementation, and the parameters can be adjusted according to data scale, computing resources, and business needs without changing the basic technical concept of the present invention.

[0055] It should be noted that the aforementioned videos can be short videos, micro-videos, etc., and are not limited here. The video popularity prediction method provided in this embodiment can be applied to scenarios such as content recommendation, traffic scheduling, traffic management, and risky content governance. Taking the traffic management scenario applied to a video platform as an example, by predicting the popularity of a video to be published, the platform can assess the video's dissemination potential based on the predicted video popularity value, and provide a reference for video recommendation, traffic allocation, and content operation decisions. For example, for videos with high predicted popularity, their recommendation exposure can be increased; for videos with low predicted popularity, their display strategy can be adjusted according to the platform's operational needs, thereby improving the platform's traffic utilization efficiency.

[0056] Example 2 This embodiment provides a video popularity prediction system, including: a memory and a processor. The memory stores a computer program, and the processor executes the computer program to perform the video popularity prediction method provided in Embodiment 1 of this invention.

[0057] The relevant technical solutions are the same as the video popularity prediction method provided in Example 1, and will not be repeated here.

[0058] Example 3 This embodiment provides a video recommendation method. It adopts the video popularity prediction method provided in Embodiment 1 of the present invention to predict the popularity of each video on the platform and obtain the predicted popularity value of each video. The videos on the platform are sorted from high to low according to the predicted popularity value and a preview sequence is generated and pushed to the user.

[0059] The relevant technical solutions are the same as the video popularity prediction method provided in Example 1, and will not be repeated here.

[0060] Example 4 This embodiment also provides a computer-readable storage medium, which includes a stored computer program, wherein when the computer program is run by a processor, it controls the device where the storage medium is located to execute the video popularity prediction method provided in Embodiment 1 or the video recommendation method provided in Embodiment 3 of the present invention.

[0061] The relevant technical solutions are the same as the video popularity prediction method provided in Example 1 and the video recommendation method provided in Example 3, and will not be repeated here.

[0062] Example 5 This embodiment also provides a computer program product, including a computer program / instruction, which, when executed by a processor, implements the video popularity prediction method provided in Embodiment 1 or the video recommendation method provided in Embodiment 3 of the present invention.

[0063] The relevant technical solutions are the same as the video popularity prediction method provided in Example 1 and the video recommendation method provided in Example 3, and will not be repeated here.

[0064] Those skilled in the art will readily understand that the above description is merely a preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions, and improvements made within the spirit and principles of the present invention should be included within the scope of protection of the present invention.

Claims

1. A method for predicting video popularity, characterized in that, include: Perform a popularity prediction operation on the video to be predicted: Obtain the video frame sequence of the video to be predicted S The text description and owner attribute information are extracted and feature-encoded separately to obtain the corresponding video frame feature sequences. Text features and attribute features; the video frame feature sequence A sequence of features composed of the individual frames in a video frame sequence; Based on video frame feature sequences Extracting its visual enhancement features: This involves extracting the feature sequence from the video frames. The feature differences between the i-th frame and the (i-1)-th frame are input into the frame scoring model to obtain the importance weight of the i-th frame; ; Video frame sequence S The number of frames; the importance weight of the first frame is obtained by inputting the preset initial feature difference into the frame scoring model; the video frame feature sequence The features of each frame in the video are weighted and summed based on their corresponding importance weights to obtain the global semantic features of the video to be predicted; the video frame feature sequence is then processed. The input is fed into the state-space model to obtain the dynamic features of each frame; the global semantic features are then combined with the video frame feature sequence. After concatenation, the data is input into a sparse attention model, and the features at the corresponding positions of the global semantic features are extracted from the output of the sparse attention model as attention features. The attention features, global semantic features, and dynamic features of each frame are fused to obtain the corresponding visual enhancement features. The sparse attention model is calculated based on a window attention mechanism for global semantic features and video feature sequences. Each frame feature in the video is assigned a corresponding attention window; the attention window corresponding to the global semantic feature covers both the global semantic feature and the frequency frame feature sequence; the video frame feature sequence The size of the attention window corresponding to each frame feature is determined based on its importance weight; the larger the importance weight of the frame feature, the larger the size of the attention window. Construct a feature set including visual enhancement features, text features, and attribute features of the video to be predicted; fuse the features in the feature set to obtain a comprehensive prediction feature; The comprehensive prediction features are input into the prediction head to obtain the predicted popularity value of the video to be predicted.

2. The video popularity prediction method according to claim 1, characterized in that, The state-space model is a selective state-space model.

3. The video popularity prediction method according to claim 2, characterized in that, The formula for calculating the step size in the selective state-space model is as follows: in, This refers to the softplus function; Indicates linear layer operations; These are the frame features input into the selective state-space model; Frame features The corresponding scan step size; This is the step size adjustment amount; The weight matrix is ​​a learnable weight matrix; This is the global semantic relevance adjustment coefficient; For the corresponding global semantic features; for and Similarity; This is the importance adjustment coefficient, and ; Frame features Importance weights.

4. The video popularity prediction method according to claim 1, characterized in that, The process of performing popularity prediction on the video to be predicted further includes: extracting retrieval enhancement features from the video to be predicted. The similarity between the multimodal features of the video to be predicted and the aggregated multimodal features corresponding to each cluster in the historical sample feature set is calculated separately, and used as the matching score between the video to be predicted and the corresponding cluster; the multimodal features of the video to be predicted are the concatenated features of the visual enhancement features, text features and attribute features of the video to be predicted. Obtain the K clusters with the highest matching scores and use them as candidate clusters for the video to be predicted; normalize the matching scores of the K candidate clusters for the video to be predicted to obtain the corresponding K matching weights. ; The aggregated multimodal features corresponding to each candidate cluster of the video to be predicted are weighted and summed based on the corresponding matching weights, and used as the retrieval enhancement features of the video to be predicted. At this time, the feature group also includes: retrieval enhancement features of the video to be predicted; The historical sample feature set includes: multimodal features of multiple historical videos carrying corresponding real popularity values; the multimodal features of historical videos are the concatenation features of visual enhancement features, text features and attribute features of historical videos; the visual enhancement features of historical videos are extracted based on their video frame feature sequences; the video frame feature sequences, text features and attribute features of historical videos are features obtained by feature encoding the video frame sequences, text descriptions and owner attribute information of historical videos respectively; The historical sample feature set is divided into multiple clusters, and the division method is as follows: The historical sample feature set is partitioned by the equal frequency binning method, and then the historical sample feature set is divided into multiple non-overlapping subsets. Cluster analysis is performed on each subset based on its multimodal features to obtain multiple clusters and the aggregated multimodal features corresponding to each cluster. The aggregated multimodal features are the cluster centers of the multimodal features in the corresponding clusters. Each cluster has a corresponding label.

5. The video popularity prediction method according to claim 4, characterized in that, The process of fusing features in a feature group includes: The visual enhancement features of the video to be predicted are used as the visual input, and the text features of the video to be predicted are used as the text input. Based on the cross-attention mechanism, the attention features of the text side on the visual side are calculated respectively. And visual attention features to text. ; Calculate the retrieval enhancement features of the video to be predicted and the attention features. The Hadamard product yields the visual historical interaction features; Calculate the retrieval enhancement features of the video to be predicted and the attention features. The Hadama product yields the textual history interaction features; Attention features Attention characteristics The visual historical interaction features, textual historical interaction features, and attribute features of the video to be predicted are concatenated to obtain comprehensive prediction features.

6. The video popularity prediction method according to any one of claims 1-5, characterized in that, The frame scoring model, the state space model, the sparse attention model, and the prediction head are trained in the following manner: Obtain a training sample set; the training samples in the training sample set are historical videos, and the corresponding tags are the actual popularity values; Perform a popularity prediction operation on each training sample in the training sample set to obtain the corresponding predicted popularity value; Construct a training objective; wherein the training objective includes: minimizing the difference loss between the predicted popularity value and the corresponding label of each training sample; Based on the training objective, the parameters in the frame scoring model, the state space model, the sparse attention model, and the prediction head are updated simultaneously.

7. The video popularity prediction method according to claim 6, characterized in that, When the operation of performing popularity prediction on the video to be predicted further includes extracting the retrieval enhancement features of the video to be predicted, the operation of performing popularity prediction on the training samples further includes: extracting the retrieval enhancement features of the training samples; The training objective further includes: minimizing the difference loss between the first matching distribution and the first preset prior distribution, and minimizing the difference loss between the second matching distribution and the second preset prior distribution; wherein, the first preset prior distribution is a uniform distribution; the second preset prior distribution is a distribution composed of the labels of all training samples; the first matching distribution is obtained by recording the candidate clusters of the training samples obtained during the extraction of retrieval enhancement features; the labels of all candidate clusters of the training samples are used to form the first matching distribution; the second matching distribution is obtained by: for each training sample, determining the true popularity interval of its historical sample feature set based on its label, and then establishing a mapping with the corresponding subset in the historical sample feature set; the labels of the subsets corresponding to all training samples are used to form the second matching distribution. When updating the parameters in the frame scoring model, the state space model, the sparse attention model, and the prediction head simultaneously based on the training objective, the aggregated multimodal features corresponding to the candidate clusters of each training sample are also updated at the same time.

8. A video popularity prediction system, characterized in that, include: A memory and a processor, wherein the memory stores a computer program, and the processor executes the computer program to perform the video popularity prediction method according to any one of claims 1-7.

9. A video recommendation method, characterized in that, Using the video popularity prediction method according to any one of claims 1-8, the popularity of each video on the platform is predicted to obtain the predicted popularity value of each video; the videos on the platform are sorted from high to low according to the predicted popularity value, and a preview sequence is generated and pushed to the user.

10. A computer-readable storage medium, characterized in that, The computer-readable storage medium includes a stored computer program, wherein when the computer program is run by a processor, it controls the device where the storage medium is located to perform the video popularity prediction method according to any one of claims 1-7 or the video recommendation method according to claim 9.

Images