Prediction method and device for playing duration of multimedia data

Abstract

The invention provides a prediction method and device for multimedia data playing duration. The prediction method comprises the following steps: inputting user data and multimedia data of a to-be-recommended multimedia item into a pre-trained prediction model as original data; performing feature extraction on the original data in the deep session interest network layer to obtain a multi-dimensional deep feature vector; performing feature extraction on the original data in the residual network layer to obtain a multi-dimensional shallow feature vector; performing compensation processing on the deep feature vector based on the shallow feature vector to generate a target feature vector; and predicting the target feature vector according to a multi-stage prediction tower to obtain a multi-stage playing duration probability when the user plays the multimedia data. According to the invention, the prediction result of the low-order playing duration is transmitted to the high-order playing duration to assist model learning, so that the accuracy of the model prediction result is improved.

Description

Technical Field

[0001] This disclosure relates to the field of data processing technology, specifically to a method and apparatus for predicting the playback duration of multimedia data. Background Technology

[0002] In the recommendation system of an application platform, the duration of user consumption is a core indicator for measuring user interest. For example, in a music or video recommendation system, the duration of a user's playback of a song / video is a core indicator for measuring user interest, and skipping after a certain playback time also indicates the degree of user disinterest.

[0003] However, when using playback duration for interest modeling, the related technologies have the drawback that the prediction targets at each stage of the prediction model are optimized independently, and the prediction results of playback duration at lower stages cannot assist in the prediction of playback duration at higher stages. On the other hand, when using deep neural networks to extract user features and multimedia features, the gradient of the network layer responsible for extracting basic features in the deep neural network will gradually weaken or even disappear during backpropagation, which may result in the loss of key features of multimedia data and reduce the accuracy of the prediction results. Summary of the Invention

[0004] This disclosure provides a method and apparatus for predicting the playback duration of multimedia data, in order to solve the problems that, when using playback duration for interest modeling, the prediction results of low-stage playback duration cannot assist in the prediction of high-stage playback duration, and that deep neural networks are prone to gradient vanishing when extracting complex features, which affects the learning effect of playback duration-related features.

[0005] Firstly, this disclosure provides a method for predicting the playback duration of multimedia data, the prediction method including: User data and multimedia data of the multimedia items to be recommended are used as raw data input into a pre-trained prediction model. The prediction model includes a deep conversational interest network layer, a residual network layer, and a multi-stage prediction tower. Feature extraction is performed on the original data in the deep conversational interest network layer to obtain multi-dimensional deep feature vectors; and, Feature extraction is performed on the original data in the residual network layer to obtain multi-dimensional shallow feature vectors; The deep feature vector is compensated based on the shallow feature vector to generate the target feature vector. The compensation process is used to make up for the feature loss caused by the deep conversational interest network layer during feature extraction. Based on the prediction of the target feature vector using a multi-stage prediction tower, the probability of multi-stage playback duration for users when playing multimedia data is obtained.

[0006] This disclosure provides a method for predicting the playback duration of multimedia data. User data and multimedia data of the recommended multimedia items are input as raw data into a pre-trained prediction model. Feature extraction is performed on the raw data in a deep conversational interest network layer to obtain multi-dimensional deep feature vectors. Feature extraction is also performed on the raw data in a residual network layer to obtain multi-dimensional shallow feature vectors. The deep feature vectors are compensated based on the shallow feature vectors to generate target feature vectors. The target feature vectors are then predicted using a multi-stage prediction tower to obtain the multi-stage playback duration probability of the user when playing multimedia data. This disclosure transfers the prediction results of low-order playback durations to high-order playback durations to assist model learning. Simultaneously, the residual network optimizes the feature extraction results, mitigating the gradient vanishing problem and improving the accuracy of the model's prediction results.

[0007] Secondly, this disclosure provides a multimedia data playback duration prediction device, the device comprising: The data acquisition module is used to input user data and multimedia data of the multimedia items to be recommended as raw data into the pre-trained prediction model. The prediction model includes a deep conversational interest network layer, a residual network layer, and a multi-stage prediction tower. The deep feature extraction module is used to extract features from the raw data in the deep conversational interest network layer to obtain multi-dimensional deep feature vectors; and, The shallow feature extraction module is used to extract features from the original data in the residual network layer to obtain multi-dimensional shallow feature vectors. The feature compensation module is used to compensate the deep feature vector based on the shallow feature vector to generate the target feature vector. The compensation process is used to make up for the feature loss caused by the deep conversational interest network layer during feature extraction. The probability prediction module is used to predict the target feature vector based on the multi-stage prediction tower, and obtain the probability of multi-stage playback duration when the user plays multimedia data.

[0008] Thirdly, this disclosure provides an electronic device, including: a memory and a processor, which are communicatively connected to each other. The memory stores computer instructions, and the processor executes the computer instructions to perform the multimedia data playback duration prediction method of the first aspect or any corresponding embodiment described above.

[0009] Fourthly, this disclosure provides a computer program product, including computer instructions for causing a computer to execute the multimedia data playback duration prediction method of the first aspect or any corresponding embodiment described above.

[0010] The multimedia data playback duration prediction method and apparatus provided in this disclosure include: inputting user data and multimedia data of the multimedia items to be recommended as raw data into a pre-trained prediction model; extracting features from the raw data in a deep conversational interest network layer to obtain multi-dimensional deep feature vectors; extracting features from the raw data in a residual network layer to obtain multi-dimensional shallow feature vectors; compensating the deep feature vectors based on the shallow feature vectors to generate target feature vectors; and predicting the target feature vectors based on a multi-stage prediction tower to obtain the multi-stage playback duration probability of the user when playing multimedia data. This disclosure transfers the prediction results of low-order playback durations to high-order playback durations to assist model learning, and simultaneously optimizes the feature extraction results through a residual network, mitigating the gradient vanishing problem and improving the accuracy of the model's prediction results. Attached Figure Description

[0011] To more clearly illustrate the technical solutions in the specific embodiments or related technologies of this disclosure, the accompanying drawings used in the description of the specific embodiments or related technologies will be briefly introduced below. Obviously, the accompanying drawings described below are some embodiments of this disclosure. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0012] Figure 1 This is a schematic diagram illustrating an application scenario according to an embodiment of this disclosure; Figure 2 This is a schematic flowchart of a first method for predicting the playback duration of multimedia data according to an embodiment of the present disclosure; Figure 3 This is a schematic diagram of the structure of the prediction model in a multimedia data playback duration prediction method according to an embodiment of the present disclosure; Figure 4 This is a schematic diagram of the structure of the deep conversational interest network layer in the prediction model according to an embodiment of the present disclosure; Figure 5 This is a schematic diagram of the structure of the residual network layer in the prediction model according to an embodiment of the present disclosure; Figure 6 This is a second flowchart illustrating a method for predicting the playback duration of multimedia data according to an embodiment of the present disclosure; Figure 7 This is a third flowchart illustrating a method for predicting the duration of multimedia data playback according to an embodiment of the present disclosure; Figure 8 This is a schematic diagram of the fourth process of a method for predicting the playback duration of multimedia data according to an embodiment of the present disclosure; Figure 9This is a structural block diagram of a multimedia data playback duration prediction device according to an embodiment of the present disclosure; Figure 10 This is a schematic diagram of the hardware structure of an electronic device according to an embodiment of the present disclosure. Detailed Implementation

[0013] To make the objectives, technical solutions, and advantages of the embodiments of this disclosure clearer, the technical solutions of the embodiments of this disclosure will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this disclosure, and not all embodiments. Based on the embodiments of this disclosure, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this disclosure.

[0014] It is understood that before using the technical solutions disclosed in the various embodiments of this disclosure, users should be informed of the types, scope of use, and usage scenarios of the personal information involved in this disclosure in an appropriate manner in accordance with relevant laws and regulations, and user authorization should be obtained.

[0015] The terms "first" and "second" are used for descriptive purposes only and should not be construed as indicating or implying relative importance or implicitly specifying the number of technical features indicated. Thus, a feature defined as "first" or "second" may explicitly or implicitly include one or more of that feature. In the description of this disclosure, "a plurality of" means two or more, unless otherwise expressly specified.

[0016] As one optional application scenario of this disclosure embodiment, such as Figure 1 As shown, the recommendation system of this application platform may include at least one terminal device and at least one server. Figure 1 The example shows that the recommendation system includes a computer 101, a mobile terminal 102, and a server 103, and the terminal devices such as the computer 101 and the mobile terminal 102 are connected to the server 103 through a network 110.

[0017] Specifically, the terminal device can be a smartphone, tablet, laptop, PDA, desktop computer, game console, smart TV, smart wearable device, in-vehicle terminal, VR (Virtual Reality) device, AR (Augmented Reality) device, etc. Server 103 can be a standalone physical server, a server cluster, a distributed system, or a cloud server providing cloud services. Network 110 can be a wired or wireless network, examples of which include, but are not limited to, the Internet, corporate intranet, local area network, wide area network, mobile communication network, and combinations thereof.

[0018] In the recommendation system of an application platform, the duration of user consumption is a core indicator for measuring user interest. For example, in a music or video recommendation system, the duration of a user's playback of a song / video is a core indicator for measuring user interest, and skipping after a certain playback time also indicates the degree of user disinterest.

[0019] When using playback duration to model interests, related technologies simply and crudely categorize the complex degree of user interest into "whether the playback is completed," losing much intermediate information. For example, there is a significant difference in the intensity of interest reflected by whether a user skips after 30 seconds versus skipping at the end of the playback. However, the single objective of "whether the playback is completed" cannot reflect the difference between the two, thus causing the prediction model to be unable to accurately distinguish the subtle differences in user interests and limiting the upper limit of its recommendation accuracy.

[0020] Other related technologies, while attempting to compensate for the shortcomings of a single objective through multiple objectives, have encountered the problem of a lack of collaborative mechanisms between objectives. Even if the prediction model simultaneously predicts multiple objectives such as "whether the playback duration is greater than 30 seconds" and "whether the playback is completed", these objectives are often treated as independent tasks within the prediction model. The learning results of lower-order objectives (such as whether the playback duration is greater than 30 seconds) cannot be effectively used to assist the optimization of higher-order objectives (such as whether the playback is completed), thus limiting the overall performance improvement of the multi-objective prediction model.

[0021] Other related technologies have introduced deep neural network structures to improve the accuracy of prediction models. However, when using complex deep neural networks to learn user interests, the gradients of the network layers responsible for extracting basic features in the neural network will gradually weaken or even disappear during backpropagation. This makes it impossible to effectively update the key features of multimedia data, ultimately making it difficult for the prediction model to learn complex targets related to the playback duration of multimedia data, such as "a certain type of user usually only listens to the first half of a certain style of music".

[0022] Based on this, embodiments of this disclosure provide a method for predicting the playback duration of multimedia data. The method involves inputting user data and multimedia data of the recommended multimedia items as raw data into a pre-trained prediction model; extracting features from the raw data in a deep conversational interest network layer to obtain multi-dimensional deep feature vectors; extracting features from the raw data in a residual network layer to obtain multi-dimensional shallow feature vectors; compensating the deep feature vectors based on the shallow feature vectors to generate target feature vectors; and predicting the target feature vectors using a multi-stage prediction tower to obtain the multi-stage playback duration probability of the user when playing multimedia data. This disclosure transfers the prediction results of low-order playback durations to high-order playback durations to assist model learning, while simultaneously optimizing the feature extraction results through a residual network, mitigating the gradient vanishing problem and improving the accuracy of the model's prediction results.

[0023] According to an embodiment of this disclosure, a method for predicting the duration of multimedia data playback is provided. It should be noted that the steps shown in the flowchart in the accompanying drawings can be executed in a computer system such as a set of computer-executable instructions. Furthermore, although a logical order is shown in the flowchart, in some cases, the steps shown or described may be executed in a different order than that shown here.

[0024] This embodiment provides a method for predicting the playback duration of multimedia data, which can be used in the recommendation system of the aforementioned application platform. Figure 2 This is a flowchart of a method for predicting the playback duration of multimedia data according to an embodiment of the present disclosure, such as... Figure 2 As shown, the process includes the following steps: Step S201: Input the user data and the multimedia data of the multimedia items to be recommended as raw data into the pre-trained prediction model. The prediction model includes a deep conversational interest network layer, a residual network layer, and a multi-stage prediction tower.

[0025] This disclosure first obtains user data of the target users and multimedia data of the projects to be recommended. The target users refer to the specific users on the application platform to whom multimedia projects need to be recommended. The user data refers to data describing the target users' own characteristics and behavioral history, including static and dynamic features. Static features may include age, gender, registered region, etc., while dynamic features may include historical clicks, playback, search, favorites, sharing records, listening frequency, historical completion rate, etc., as well as the time, device, and network environment in which these behaviors occurred.

[0026] Multimedia projects to be recommended refer to information carriers that are composed of a combination of various media forms and are created, processed, stored, transmitted and displayed through computers, such as audio, video and animation.

[0027] Multimedia data refers to information about multimedia projects to be recommended, including video and audio features. Video features may include title, tags, category, uploader, cover image, audio waveform, and video frame features, while music features may include song title, genre, duration, artist, album, genre, release year, audio spectrum, rhythm, and mood tags.

[0028] Figure 3 This is a schematic diagram of the prediction model in a multimedia data playback duration prediction method according to an embodiment of this disclosure. Figure 3As shown, the pre-trained prediction model in this disclosure includes an input layer, a deep conversational interest network layer, a residual network layer, a multi-stage prediction tower, and an output layer. User data of the target user and multimedia data of the multimedia items to be recommended are input into the prediction model as raw data. The input layer of the prediction model receives the raw data and performs preliminary feature extraction to obtain raw data features, which largely preserve the data characteristics of the original data. The deep conversational interest network layer is configured to perform deep extraction of the raw data features to obtain deeper levels of user interest preferences. The residual network layer consists of one or more cascaded residual blocks, which are configured to perform shallow extraction of the raw data features to obtain shallow features that are easily filtered out from the raw data features.

[0029] When using deep conversational interest networks (PINs) to learn user interests, on the one hand, the PPNs tend to extract deeper features from the original data, filtering out shallower features. These shallower features often contain rich and subtle details in the original data, which can affect the representation of user interests in the original data. On the other hand, the gradients of the PPNs gradually weaken or even disappear during backpropagation, making it impossible to effectively update the key features of the multimedia data. Furthermore, the shallow feature vectors extracted by the PPNs without residual networks compensate for the deep feature vectors extracted by the PPNs. After multiple nonlinear transformations, the shallower user interests in the original data are almost completely smoothed or forgotten. As a result, the prediction model cannot capture the subtle features in the original data, leading to a significant decrease in the generalization ability of the prediction model when faced with new users or new content that have not appeared in the training data.

[0030] In one embodiment, the multi-stage prediction tower consists of multiple prediction towers, each employing a multi-layered progression network (MLP) structure. Specifically, it includes an independent tower for predicting "heart" collection behavior, and four cascaded playback duration prediction towers for predicting playback duration. These cascaded towers utilize a low-order output-assisted high-order association mechanism. Specifically, the first playback duration prediction tower performs an initial prediction based on deep features extracted by the preceding network. Its output prediction probability is fused with the original deep features and used as input to the second duration tower. This prediction process is progressive; each subsequent tower always predicts the next playback duration based on the prediction results of the previous tower, significantly improving the accuracy of the final prediction result. Finally, the output layer maps the final outputs of each prediction tower in the multi-stage prediction tower to specific prediction probabilities, such as the probability of a heart collection and the probability of each stage's duration.

[0031] Step S202: Extract features from the original data in the deep conversational interest network layer to obtain multi-dimensional deep feature vectors.

[0032] Figure 4 This is a schematic diagram of the structure of the deep conversational interest network layer in the prediction model according to an embodiment of this disclosure. Figure 4 As shown, the deep conversational interest network layer includes a conversation segmentation layer, a conversational interest extraction layer, a conversational interest interaction layer, and a conversational interest activation layer.

[0033] Specifically, the original data features include user behavior features and multimedia data features. The session segmentation layer segments the user data features in the original data features into multiple independent sessions based on preset segmentation rules. For example, the segmentation rules can be based on fixed time intervals, so that each session contains a series of logically closely related user behaviors that occur in a short period of time. Different sessions may represent different interest topics.

[0034] Furthermore, for each independent session, the session interest extraction layer utilizes a sequence encoder such as a bidirectional Transformer or LSTM to capture the local contextual semantics of behaviors within the session. Since behaviors within a session typically revolve around a relatively focused core session interest—for example, continuously listening to multiple songs by the same artist—the embedding vector sequence of each behavior in the session can be encoded into a single vector representing the core session interest. Finally, the session interest extraction layer outputs a session interest sequence composed of single vectors representing all core session interests, where each single vector represents the user's core session interest within a specific time segment.

[0035] A Transformer self-attention mechanism is introduced into the conversation interest interaction layer and the conversation interest activation layer to model the conversation interest sequence and further learn the mutual weights between different conversation interests through self-attention calculation. For example, a user's conversation interest in "sports music" from last week and "commuter podcast" from this week have different reference values ​​for predicting the currently playing work background music. Therefore, the conversation interest interaction layer and the conversation interest activation layer calculate the importance of each conversation interest sequence in historical conversations relative to the current multimedia item to be predicted, and generate a weighted user interest representation based on the calculation results.

[0036] On the other hand, the deep conversational interest network layer also performs deep feature extraction on the multimedia data features in the original data features to obtain deep multimedia data features, and then fuses the deep multimedia data features with the weighted user interest representation as the deep feature vector output by the deep conversational interest network layer.

[0037] Step S203: Extract features from the original data in the residual network layer to obtain multi-dimensional shallow feature vectors.

[0038] Figure 5This is a schematic diagram of the residual network layer structure in the prediction model according to an embodiment of this disclosure. Figure 5 As shown, the residual network layer consists of one or more cascaded residual blocks. These residual blocks are used to perform nonlinear transformations and feature abstraction on the original data features. Specifically, each residual block, through a parallel path of "identity mapping" and "nonlinear transformation," allows the input original data features to be directly passed to the output of the residual network layer. The output is then added to the result after transformation by convolutional or fully connected layers and activation functions to obtain the final output. This ensures that subtle features at the lower levels of the original data can also propagate forward, mitigating the gradient vanishing problem. In one embodiment, the calculation formula for each residual block is:

[0039] in, To input the original data characteristics of the residual block, This represents the nonlinear transformation result of the convolutional (or fully connected) layers and activation functions of the residual network. This is the output for the residual block.

[0040] Step S204: Compensate the deep feature vector based on the shallow feature vector to generate the target feature vector. The compensation process is used to make up for the feature loss caused by the deep conversational interest network layer during feature extraction.

[0041] When the Deep Conversation Interest Network (DSIN) layer extracts deep feature vectors of users through conversation segmentation and attention mechanisms, it inevitably filters out or weakens some subtle shallow feature information in the original data. Therefore, this disclosure introduces a residual network layer that works in parallel with the DSIN layer. Due to the structural characteristics of the identity mapping in the residual network layer, shallow feature vectors from the input layer or lower network layers that have not undergone deep nonlinear transformation can be transmitted to higher layers with near-lossless transmission. Furthermore, the shallow feature vectors transmitted by the residual network can be fused with the deep feature vectors output by the DSIN layer through vector concatenation, element-wise addition, or weighted fusion, thereby compensating for the original data details that may have been lost during the DSIN feature extraction process in the final target feature vector. The target feature vector not only provides deep feature vectors containing deep user interests for the next level of the multi-stage prediction tower, but also retains shallow feature vectors with key details of the original data, thus laying the foundation for a significant improvement in the final prediction accuracy.

[0042] Step S205: Based on the multi-stage prediction tower, predict the target feature vector to obtain the multi-stage playback duration probability of the user when playing multimedia data.

[0043] The multi-stage prediction tower in this embodiment consists of N sequentially connected prediction towers, where N is a positive integer greater than 1. First, the target feature vector is initially predicted using the first-order prediction tower, thus outputting the first playback duration probability for the first stage. Then, the target feature vector and the first playback duration probability are input into the second-order prediction tower for probability prediction of the playback duration in the next stage. Specifically, for the nth-order prediction tower, the playback duration probability output by the (n-1)th-order prediction tower is fused with the target feature vector as input to predict a more accurate nth-stage playback duration probability. Finally, the last prediction tower outputs the multi-stage playback duration probability for the user when playing multimedia data, where n ∈ [2, N].

[0044] This embodiment provides a method for predicting the playback duration of multimedia data. On the one hand, it uses a residual network to retain shallow feature vectors containing key details of the original data based on deep feature vectors, thus alleviating the gradient vanishing problem. On the other hand, it improves the accuracy of the model prediction results by passing the prediction results of low-order playback duration to high-order playback duration to assist in model learning.

[0045] This embodiment provides a method for predicting the playback duration of multimedia data, which can be used in the recommendation system of the aforementioned application platform. Figure 6 This is a flowchart of a method for predicting the playback duration of multimedia data according to an embodiment of the present disclosure, such as... Figure 6 As shown, the process includes the following steps: Step S601 involves inputting user data and multimedia data of the multimedia items to be recommended as raw data into a pre-trained prediction model. The prediction model includes a deep conversational interest network layer, a residual network layer, and a multi-stage prediction tower. For details, please refer to [link to details]. Figure 2 Step S201 of the illustrated embodiment will not be described again here.

[0046] Step S602: Extract features from the original data in the deep conversational interest network layer to obtain multi-dimensional deep feature vectors.

[0047] Specifically, step S602 includes: Step S6021: In the deep conversational interest network layer, the original data is divided into multiple conversations according to a preset time interval, and each conversation is represented as a sequence of user behavior.

[0048] Based on preset segmentation rules, the user data features in the original data features are segmented into multiple independent sessions. Preferably, in a specific embodiment of this disclosure, the segmentation rule can be a preset time interval.

[0049] Based on a preset time interval as a segmentation criterion, for example, if the interval between two user actions exceeds 30 minutes, the user data features in the original data features are segmented into multiple sessions that are relatively independent in time but highly correlated in internal behavior. Each session is defined as a user behavior sequence, which represents a series of related interactions generated by the user within a certain continuous time period, such as several songs played continuously during a commute.

[0050] Step S6022: Obtain the deviation information between any two sessions and inject the deviation information into the embedded representation of the user behavior sequence.

[0051] In one embodiment of this disclosure, a linear transformation is performed on the user behavior representations of any two sessions to obtain a linear vector corresponding to each session. The dot product and cosine similarity of the transformed two linear vectors are then calculated to obtain the deviation score between any two sessions. The deviation information calculated in this way can dynamically reflect the differences in session content. For example, a session mainly focused on "rock music" and a session mainly focused on "classical music" will have a large negative deviation, while two similar "commuter podcast" sessions will have a small positive deviation. Alternatively, other methods for calculating deviation information can be selected, and this disclosure does not limit this approach.

[0052] Preferably, this deviation information is injected into the user behavior sequence of each session, thereby labeling the behavior data of each session with a context label. This allows the prediction of the playback duration of multimedia data to not only see "what multimedia data the user played", but also to perceive "in what different contexts the user played the multimedia data", thus improving the accuracy of the prediction results.

[0053] Step S6023: Extract the session interest feature vector from the user behavior sequence injected with deviation information, and use the session interest feature vector as the deep feature vector.

[0054] Deep feature learning is performed on user behavior sequences with injected contextual bias information. Preferably, a bidirectional long short-term memory network or a self-attention mechanism can be used to extract a fixed-length session interest feature vector from the behavior sequence of each independent session, which can highly summarize the core interests of that session. Finally, a deep feature vector containing context labels is generated based on the interest feature vectors of all sessions.

[0055] Step S603: Feature extraction is performed on the original data in the residual network layer to obtain multi-dimensional shallow feature vectors. For details, please refer to [link to relevant documentation]. Figure 2 Step S203 of the illustrated embodiment will not be described again here.

[0056] Step S604: Compensate the deep feature vectors based on the shallow feature vectors to generate the target feature vector. The compensation process is used to compensate for feature loss caused by the deep conversational interest network layer during feature extraction. For details, please refer to [link to details]. Figure 2 Step S204 of the illustrated embodiment will not be described again here.

[0057] Step S605: Predict the target feature vector based on the multi-stage prediction tower to obtain the probability of multi-stage playback duration when the user plays multimedia data. For details, please refer to [link to details]. Figure 2 Step S205 of the illustrated embodiment will not be described again here.

[0058] This embodiment provides a method for predicting the playback duration of multimedia data. By calculating and injecting contextual deviation information between sessions into the user behavior sequence, and then performing deep feature learning, a deep feature vector containing contextual labels is finally generated. This allows the extracted session interest feature vector to clearly reflect the dynamic semantic representation of how user interests evolve in specific time, scene, and other contextual environments, thereby enhancing the accuracy and generalization ability of the prediction model in predicting the final behavior.

[0059] This embodiment provides a method for predicting the playback duration of multimedia data, which can be used in the recommendation system of the aforementioned application platform. Figure 7 This is a flowchart of a method for predicting the playback duration of multimedia data according to an embodiment of the present disclosure, such as... Figure 7 As shown, the process includes the following steps: Step S701: Input the user data and the multimedia data of the multimedia items to be recommended as raw data into the pre-trained prediction model. The prediction model includes a deep conversational interest network layer, a residual network layer, and a multi-stage prediction tower. For details, please refer to... Figure 2 Step S201 of the illustrated embodiment will not be described again here.

[0060] Step S702: Feature extraction is performed on the original data in the deep conversational interest network layer to obtain multi-dimensional deep feature vectors. For details, please refer to [link to details]. Figure 2 Step S202 of the illustrated embodiment will not be described again here.

[0061] Step S703: Extract features from the original data in the residual network layer to obtain multi-dimensional shallow feature vectors.

[0062] Specifically, step S703 includes: Step S7031: Input the raw data into the feature extraction network.

[0063] Step S7032: Perform at least one nonlinear transformation on the original data through a feature extraction network to extract shallow feature vectors from the original data; wherein the dimension of the shallow features is related to the dimension of the original data; the feature extraction network is a residual network, and at least one nonlinear transformation is implemented through residual blocks in the residual network.

[0064] In this embodiment, the residual network layer consists of one or more cascaded residual blocks, which are used to perform nonlinear transformations and feature abstraction on the original data features. Specifically, each residual block directly transmits the input original data to the output of the residual network layer through a parallel path of "identity mapping" and "nonlinear transformation," and adds the original data to the result after transformation by convolutional or fully connected layers and activation functions to obtain the final output. This ensures that subtle features at the lower levels of the original data can also propagate forward, alleviating the gradient vanishing problem.

[0065] In some optional implementations, step S7032 above includes: a1. Obtaining the original data features of the raw data. When the raw data is input into the prediction model, the input layer of the prediction model receives the raw data and performs preliminary feature extraction to obtain the raw data features. The raw data features largely retain the data characteristics in the raw data.

[0066] a2. The original data features are input into the convolutional layer of the residual block for convolution processing to obtain convolutional features, wherein the residual block is an identity residual block. The original data features are input into one or more convolutional layers inside the residual block. The convolutional layer performs sliding window-style local perception and weighted summation on the input original data features through its internal convolutional kernel to obtain convolutional features. Preferably, the residual block in this disclosure is an identity residual block, that is, the identity mapping path does not require linear transformation of the original data features and can directly pass the input original data features.

[0067] a3. Input the convolutional features into the activation function for non-linear mapping to obtain the first feature vector. Input the convolutional features obtained from the convolutional layer into the activation function (e.g., ReLU, Sigmoid) of the residual network layer. The activation function performs non-linear mapping on the input convolutional features to obtain the first feature vector.

[0068] a4. Add the first feature vector to the original data features to obtain the shallow feature vector of the original data. Add the original data features to the first feature vector after transformation by convolution or fully connected layers and activation functions to obtain the final output shallow feature vector. This ensures that subtle features at the lower levels of the original data can also propagate forward, alleviating the gradient vanishing problem. The calculation formula for each residual block is:

[0069] in, To input the original data characteristics of the residual block, This represents the nonlinear transformation result of the convolutional (or fully connected) layers and activation functions of the residual network. This is the output for the residual block.

[0070] Step S704: Compensate the deep feature vectors based on the shallow feature vectors to generate the target feature vector. The compensation process is used to compensate for feature loss caused by the deep conversational interest network layer during feature extraction. For details, please refer to [link to details]. Figure 2 Step S204 of the illustrated embodiment will not be described again here.

[0071] Step S705: Predict the target feature vector based on the multi-stage prediction tower to obtain the probability of multi-stage playback duration when the user plays multimedia data. For details, please refer to [link to details]. Figure 2 Step S205 of the illustrated embodiment will not be described again here.

[0072] This embodiment provides a method for predicting the playback duration of multimedia data. It fuses the shallow feature vectors transmitted by the residual network with the deep feature vectors output by the DSIN layer, thereby compensating for any original data details that might be lost during the DSIN feature extraction process in the final target feature vector. This target feature vector not only provides the next level of the multi-stage prediction tower with deep feature vectors containing profound user interests, but also retains the shallow feature vectors that preserve key details of the original data, thus laying a solid foundation for a significant improvement in the final prediction accuracy.

[0073] This embodiment provides a method for predicting the playback duration of multimedia data, which can be used in the recommendation system of the aforementioned application platform. Figure 8 This is a flowchart of a method for predicting the playback duration of multimedia data according to an embodiment of the present disclosure, such as... Figure 8 As shown, the process includes the following steps: Step S801 involves inputting user data and multimedia data of the multimedia items to be recommended as raw data into a pre-trained prediction model. The prediction model includes a deep conversational interest network layer, a residual network layer, and a multi-stage prediction tower. For details, please refer to [link to relevant documentation]. Figure 2 Step S201 of the illustrated embodiment will not be described again here.

[0074] Step S802: Feature extraction is performed on the original data in the deep conversational interest network layer to obtain multi-dimensional deep feature vectors. For details, please refer to [link to details]. Figure 2 Step S202 of the illustrated embodiment will not be described again here.

[0075] Step S803: Feature extraction is performed on the original data in the residual network layer to obtain multi-dimensional shallow feature vectors. For details, please refer to [link to relevant documentation]. Figure 2 Step S203 of the illustrated embodiment will not be described again here.

[0076] Step S804: Compensate the deep feature vectors based on the shallow feature vectors to generate the target feature vector. The compensation process is used to compensate for feature loss caused by the deep conversational interest network layer during feature extraction. For details, please refer to [link to details]. Figure 2 Step S204 of the illustrated embodiment will not be described again here.

[0077] Step S805: Based on the multi-stage prediction tower, predict the target feature vector to obtain the multi-stage playback duration probability of the user when playing multimedia data.

[0078] Specifically, step S805 includes: Step S8051: Input the target feature vector into the multi-stage prediction tower to predict the playback behavior target of the user playing multimedia data with increasing duration; wherein, the playback behavior target is characterized as whether the playback duration of the multimedia data exceeds multiple increasing thresholds. The increasing thresholds refer to the playback duration thresholds corresponding to the playback behavior target set in each stage of the prediction tower in an increasing order. In one embodiment, this disclosure first sets the playback behavior target in each stage of the prediction tower in an increasing order. For example, the playback behavior target of the first stage prediction tower is set to "Is the playback duration of the multimedia item to be recommended greater than 30 seconds?", where 30 seconds is the playback duration threshold of the first stage prediction tower; the playback behavior target of the second stage prediction tower is set to "Is the playback duration of the multimedia item to be recommended greater than half of its total duration?", where half of the total duration is the playback duration threshold of the second stage prediction tower; the playback behavior target of the third stage prediction tower is set to "... Is the playback duration of the recommended multimedia project greater than three-quarters of its total duration? Here, three-quarters of the total duration is the playback duration threshold of the third-order prediction tower; the playback behavior target of the fourth-order prediction tower is set as "Is the playback duration of the recommended multimedia project equal to the total duration?", that is, whether the recommended multimedia project is completed, where the total duration is the playback duration threshold of the fourth-order prediction tower. Since this application uses a "lower-order assists higher-order" learning mechanism for the multi-stage prediction tower, the playback duration threshold is also set in an increasing order when setting the playback behavior target in each prediction tower.

[0079] In some optional implementations, step S8051 above includes: a1. The target feature vector is input into the first-order prediction tower of the multi-stage prediction tower to predict the first probability that the duration of the user's playback of multimedia data exceeds a first preset duration. The target feature vector, generated after compensating the deep feature vector based on the shallow feature vector, is input into the first-order prediction tower. The first-order prediction tower predicts the target feature vector based on its own internally set first playback behavior target and outputs a probability value, i.e., the first probability. For example, the first playback behavior target of the first-order prediction tower is set to "playback duration greater than 30 seconds". Preferably, a first benchmark value can be set for the first probability. When the first probability is greater than the first benchmark value, it indicates that the probability that the playback duration exceeds the first playback behavior target is relatively high. Conversely, it is considered that the probability that the user will immediately skip the multimedia data is relatively high. That is, the first probability can be used to filter out content that the user may immediately skip. The first playback behavior target and the first benchmark value can be adjusted according to the actual situation, and this disclosure does not impose any restrictions on them.

[0080] a2. Input the target feature vector and the first probability into the second-order prediction tower of the multi-stage prediction tower to predict the second probability that the duration of the user playing multimedia data is greater than the second preset duration.

[0081] After obtaining the first probability of the multimedia data based on the first-order prediction tower, the first probability is converted into a vector representation. This first probability is then fused with the target feature vector through vector concatenation and sent as input to the second-order prediction tower. The second-order prediction tower predicts the target feature vector based on its internally set second playback behavior target and outputs the second probability. For example, the second playback behavior target of the second-order prediction tower is set to "playback duration greater than half the duration of the multimedia item to be recommended." Preferably, a second benchmark value can be set for the second probability. When the second probability is greater than the second benchmark value, it indicates a high probability that the playback duration exceeds the second playback behavior target. Conversely, it suggests that although the user is willing to play briefly, there is a high probability that they will abandon further playback of the multimedia data. In other words, the second probability can be used to filter out content that the user may not be interested in after playing it for a short period. The second playback behavior target and the second benchmark value can be adjusted according to actual circumstances, and this disclosure does not impose any restrictions on this.

[0082] a3. Input the target feature vector and the second probability into the third prediction tower of the multi-stage prediction tower to predict the third probability that the duration of the user playing multimedia data is greater than the third preset duration.

[0083] After obtaining the second probability of multimedia data based on the second-order prediction tower, the second probability is converted into a vector representation. This second probability is then fused with the target feature vector through vector concatenation and sent as input to the third-order prediction tower. The third-order prediction tower predicts the target feature vector based on its internally set third playback behavior target and outputs the third probability. For example, the third playback behavior target of the third-order prediction tower is set to "playback duration greater than three-quarters of the duration of the multimedia item to be recommended." Preferably, a third benchmark value can be set for the third probability. When the third probability is greater than the third benchmark value, it indicates a high probability that the playback duration exceeds the third playback behavior target. Conversely, it suggests that although the probability of the user playing more than half the duration of the multimedia item to be recommended is high, their willingness to continue playing the latter half of the recommended item is low. In other words, the third probability can be used to filter out multimedia items to be recommended that are unlikely to attract users to play for extended periods. The third playback behavior target and the third benchmark value can be adjusted according to actual circumstances, and this disclosure does not impose any restrictions on them.

[0084] a4. Input the target feature vector and the third probability into the fourth prediction tower of the multi-stage prediction tower to predict the fourth probability that the duration of the user playing multimedia data is greater than the fourth preset duration.

[0085] In this disclosure, the playback duration represented by the first preset duration is less than the playback duration represented by the second preset duration, the playback duration represented by the second preset duration is less than the playback duration represented by the third preset duration, and the playback duration represented by the third preset duration is less than the playback duration represented by the fourth preset duration. Preferably, the playback duration represented by the first preset duration can be 30 seconds, the playback duration represented by the second preset duration can be half the duration of the item to be recommended, the playback duration represented by the third preset duration can be three-quarters of the duration of the item to be recommended, and the playback duration represented by the fourth preset duration can be the complete playback duration of the item to be recommended. In practical applications, the playback duration represented by each preset duration can be adjusted according to the size relationship. This disclosure does not limit the specific values.

[0086] After obtaining the third probability of the multimedia data based on the third-order prediction tower, the third probability is converted into a vector representation. This third probability is then fused with the target feature vector through vector concatenation and sent as input to the fourth-order prediction tower. The fourth-order prediction tower predicts the target feature vector based on its internally set fourth playback behavior target and outputs the fourth probability. For example, the fourth playback behavior target of the fourth-order prediction tower is set to "play the recommended multimedia item in its entirety." Preferably, a fourth benchmark value can be set for the fourth probability. When the fourth probability is greater than the fourth benchmark value, it indicates a high probability that the playback duration exceeds the fourth playback behavior target. Conversely, it suggests that although the user is willing to watch most of the recommended multimedia item (e.g., willing to play up to three-quarters of the recommended item's duration), their willingness to continue playing the recommended item in its entirety is low. In other words, the fourth probability can be used to filter out recommended multimedia items that are unlikely to attract users to play them in their entirety. The fourth playback behavior target and the fourth benchmark value can be adjusted according to actual circumstances, and this disclosure does not impose any restrictions on this.

[0087] This disclosure, through this hierarchical structure, builds upon the prediction of lower-level playback behavior targets to predict higher-level playback behavior targets. This enables the prediction model to accurately distinguish subtle differences in user interests and improves the accuracy of the model's prediction results for "complete playback of recommended multimedia items".

[0088] a5. The target feature vector is input into the fifth-order prediction tower of the multi-stage prediction tower to predict the fifth probability that the user will perform a "favorite" operation on the multimedia data. The fifth-order prediction tower in this disclosure is a heart prediction tower, used to predict the probability that the user will perform a "favorite" operation on the multimedia item to be recommended. In one embodiment, although the fifth-order prediction tower operates in parallel with other playback duration prediction towers, it independently predicts the fifth probability that the user will provide positive explicit feedback on the multimedia item to be recommended, such as clicking the heart to favorite it.

[0089] The fifth-order prediction tower can calculate the similarity between the target feature vector of the multimedia project to be recommended and the historical target feature vector of the multimedia projects already collected by the user, and assign corresponding weights to the similarity calculation results of each dimension. Finally, the fifth probability is obtained by weighted fusion of the similarity calculation results of each dimension. For example, the fifth-order prediction tower can obtain the music style feature of the multimedia project to be recommended from the target feature vector, and calculate the similarity between its music style feature and the music style feature of the multimedia projects already collected by the user to obtain the first similarity result. It can also obtain the singer feature of the multimedia project to be recommended from the target feature vector, and calculate the similarity between its singer feature and the singer feature of the multimedia projects already collected by the user to obtain the second similarity result. The first similarity calculation result and the second similarity result are assigned corresponding weights respectively, and the fifth probability is obtained by weighted fusion of the first similarity calculation result and the second similarity result. The selection of dimensions and the weight assigned to each dimension can be adjusted according to the actual situation, and this disclosure does not impose any restrictions on this.

[0090] In another embodiment, a fourth probability can be introduced to assist in the calculation of the fifth probability. For example, the fourth probability can be assigned a corresponding weight, and the fourth probability can be weighted and fused together with the similarity results of each dimension to obtain the fifth probability. That is, when deciding whether a user will perform a collection operation on the recommended multimedia items, the predicted probability of the recommended multimedia data being played in its entirety is introduced.

[0091] In another embodiment, this disclosure can optimize the prediction model by calculating the loss at each order of the prediction tower:

[0092] in, The cross-entropy loss for the collection target, The cross-entropy loss is for the goal of completing the broadcast. For the cross-entropy loss of the third-order prediction tower, For the cross-entropy loss of the second-order prediction tower, The cross-entropy loss of the first-order prediction tower; ~ For multi-stage target weights, specifically, , , , Each weight can be set based on the actual scenario. This disclosure only provides a preferred embodiment and does not imply a specific limitation on the weights.

[0093] a6. Obtain scene data when the user is playing multimedia data, and extract context features from the scene data. For example, whether the current time is a weekday or weekend, whether it is a holiday, whether the user is commuting or resting at home, etc.

[0094] a7. Assign corresponding weights to the fourth and fifth probabilities based on contextual features. Dynamically adjust the weight ratio of the fourth and fifth probabilities based on contextual features. For example, in a commuting scenario, users may be more inclined to listen to a complete song rather than add it to their favorites, so the fourth probability is assigned a higher weight; while in a leisure exploration scenario, users may be more willing to add their favorite songs to their favorites for later enjoyment, in which case the weight of the fifth probability will be increased accordingly. This dynamic weight allocation mechanism improves the recommendation system's ability to perceive and satisfy users' real-time intentions.

[0095] a8. Combine the weighted fourth and fifth probabilities to calculate the total predicted value of the multimedia data. For each multimedia item to be recommended, calculate its corresponding total predicted value by weighted summation. The total predicted value represents the user's expectation of completing and adding the recommended multimedia item to their favorites in the current scenario.

[0096] a9. Sort multiple multimedia data based on the total predicted value of each multimedia data, and recommend multimedia data with a total predicted value greater than a preset probability threshold to the user.

[0097] The recommendation system sorts all multimedia items to be recommended in descending order based on the total predicted value for each item, generating an optimal recommendation list. Then, the top M ranked items, or all items with total predicted values ​​exceeding a preset probability threshold, can be pushed to the user interface.

[0098] This disclosure ranks recommended items based on users' expectations of completing the viewing and adding them to their favorites. It can not only filter out the high-quality content most likely to attract users based on the total predicted value, but also flexibly control the quantity and quality of recommendations by setting the top M or probability thresholds, thereby significantly improving the personalization of recommendations, user click-through rate and satisfaction, while avoiding information overload.

[0099] This embodiment also provides a multimedia data playback duration prediction device, which is used to implement the above embodiments and preferred embodiments; details already described will not be repeated. As used below, the term "module" can be a combination of software and / or hardware that implements a predetermined function. Although the device described in the following embodiments is preferably implemented in software, hardware implementation, or a combination of software and hardware, is also possible and contemplated.

[0100] This embodiment provides a device for predicting the playback duration of multimedia data, such as... Figure 9 As shown, the device includes: The data acquisition module 901 is used to input user data and multimedia data of the multimedia items to be recommended as raw data into the pre-trained prediction model. The prediction model includes a deep conversational interest network layer, a residual network layer, and a multi-stage prediction tower.

[0101] The deep feature extraction module 902 is used to extract features from the raw data in the deep conversational interest network layer to obtain multi-dimensional deep feature vectors. And, The shallow feature extraction module 903 is used to extract features from the original data in the residual network layer to obtain multi-dimensional shallow feature vectors.

[0102] The feature compensation module 904 is used to compensate the deep feature vector based on the shallow feature vector to generate the target feature vector. The compensation process is used to make up for the feature loss caused by the deep conversational interest network layer during feature extraction.

[0103] The probability prediction module 905 is used to predict the target feature vector based on the multi-stage prediction tower to obtain the probability of multi-stage playback duration when the user plays multimedia data.

[0104] In some alternative implementations, the deep feature extraction module 902 includes: The data partitioning unit is used to divide the raw data into multiple sessions according to a preset time interval in the deep conversational interest network layer. Each session is represented as a sequence of user behavior.

[0105] The deviation calculation unit is used to obtain deviation information between any two sessions and inject the deviation information into the embedded representation of the user behavior sequence.

[0106] The deep feature extraction unit extracts the conversation interest feature vector from the user behavior sequence injected with deviation information, and uses the conversation interest feature vector as the deep feature vector.

[0107] In some alternative implementations, the shallow feature extraction module 903 includes: The feature input unit is used to input raw data into the feature extraction network.

[0108] The nonlinear transformation unit is used to perform at least one nonlinear transformation on the original data through the feature extraction network to extract shallow feature vectors of the original data; wherein the dimension of the shallow features is related to the dimension of the original data; the feature extraction network is a residual network, and at least one nonlinear transformation is implemented through residual blocks in the residual network.

[0109] In some alternative implementations, the nonlinear transformation unit includes: The original feature acquisition subunit is used to acquire the original data features of the original data.

[0110] The convolution processing subunit is used to input the original data features into the convolutional layer of the residual block for convolution processing to obtain convolutional features, wherein the residual block is an identity residual block.

[0111] The nonlinear mapping subunit is used to input the convolutional features into the activation function for nonlinear mapping to obtain the first feature vector.

[0112] The shallow feature extraction unit is used to add the first feature vector to the original data features to obtain the shallow feature vector of the original data.

[0113] In some alternative implementations, the probability prediction module 905 includes: The vector input unit is used to input the target feature vector into the multi-stage prediction tower to predict the playback behavior target of the user playing multimedia data with increasing duration; wherein, the playback behavior target is characterized as whether the playback duration of the multimedia data exceeds multiple increasing thresholds.

[0114] In some alternative implementations, the vector input unit includes: The first probability prediction unit is used to input the target feature vector into the first-order prediction tower of the multi-stage prediction tower to predict the first probability that the duration of the user playing multimedia data is greater than the first preset duration.

[0115] The second probability prediction unit is used to input the target feature vector and the first probability into the second-order prediction tower of the multi-stage prediction tower to predict the second probability that the duration of the user playing multimedia data is greater than the second preset duration.

[0116] The third probability prediction unit is used to input the target feature vector and the second probability into the third-order prediction tower of the multi-stage prediction tower to predict the third probability that the duration of the user playing multimedia data is greater than the third preset duration.

[0117] The fourth probability prediction unit is used to input the target feature vector and the third probability into the fourth-order prediction tower of the multi-stage prediction tower to predict the fourth probability that the duration of the user playing multimedia data is greater than the fourth preset duration.

[0118] The fifth probability prediction unit is used to input the target feature vector into the fifth-order prediction tower of the multi-stage prediction tower to predict the fifth probability that the user will perform a collection operation on the multimedia data.

[0119] The scene data acquisition unit is used to acquire scene data when the user is playing multimedia data, and to extract context features from the scene data.

[0120] The weight allocation unit is used to assign corresponding weights to the fourth probability and the fifth probability based on the context features; The prediction calculation unit is used to calculate the total predicted value of multimedia data by combining the weighted fourth and fifth probabilities.

[0121] The data sorting unit is used to sort multiple multimedia data according to the total predicted value of each multimedia data, and recommend multimedia data with a total predicted value greater than a preset probability threshold to the user.

[0122] The multimedia data playback duration prediction device provided in this disclosure can execute the multimedia data playback duration prediction method provided in any embodiment of this disclosure, and has the corresponding functional modules and beneficial effects of the method. Further functional descriptions of the above modules and units are the same as in the corresponding embodiments described above, and will not be repeated here.

[0123] Figure 10 This is a schematic diagram of the structure of an electronic device provided in an embodiment of this disclosure.

[0124] The following is a detailed reference. Figure 10 The diagram illustrates a structural schematic suitable for implementing an electronic device according to embodiments of the present disclosure. The electronic device may include a processor (e.g., a central processing unit, graphics processor, etc.) 1001, which can perform various appropriate actions and processes according to a program stored in read-only memory (ROM) 1002 or a program loaded from memory 1008 into random access memory (RAM) 1003. The RAM 1003 also stores various programs and data required for the operation of the electronic device. The processor 1001, ROM 1002, and RAM 1003 are interconnected via a bus 1004. An input / output (I / O) interface 1005 is also connected to the bus 1004.

[0125] Typically, the following devices can be connected to the I / O interface 1005: input devices 1006 including, for example, a touchscreen, touchpad, keyboard, mouse, camera, microphone, accelerometer, gyroscope, etc.; output devices 1007 including, for example, a liquid crystal display (LCD), speaker, vibrator, etc.; memory devices 1008 including, for example, magnetic tape, hard disk, etc.; and communication devices 1009. Communication device 1009 allows electronic devices to exchange data via wireless or wired communication with other devices. Although Figure 10 Electronic devices with various devices are shown, but it should be understood that it is not required to implement or have all of the devices shown, and more or fewer devices may be implemented or have instead.

[0126] In particular, according to embodiments of this disclosure, the processes described above with reference to the flowcharts can be implemented as computer software programs. For example, embodiments of this disclosure include a computer program product comprising a computer program carried on a non-transitory computer-readable medium, the computer program containing program code for performing the methods shown in the flowcharts. In such embodiments, the computer program can be downloaded and installed from a network via communication device 1009, or installed from memory 1008, or installed from ROM 1002. When the computer program is executed by processor 1001, it performs the functions defined in a method for predicting the duration of multimedia data playback according to embodiments of this disclosure.

[0127] Figure 10 The electronic device shown is merely an example and should not be construed as limiting the functionality and scope of the embodiments disclosed herein.

[0128] This disclosure also provides a computer-readable storage medium in which the methods described in this disclosure can be implemented in hardware or firmware, or implemented as recordable on a storage medium, or implemented as computer code downloaded over a network and originally stored on a remote storage medium or a non-transitory machine-readable storage medium and subsequently stored on a local storage medium. Thus, the methods described herein can be processed by software stored on a storage medium using a general-purpose computer, a dedicated processor, or programmable or dedicated hardware. The storage medium can be a magnetic disk, optical disk, read-only memory, random access memory, flash memory, hard disk, or solid-state drive, etc.; further, the storage medium can also include combinations of the above types of memory. It is understood that computers, processors, microprocessor controllers, or programmable hardware include storage components capable of storing or receiving software or computer code. When the software or computer code is accessed and executed by the computer, processor, or hardware, it implements the multimedia data playback duration prediction method shown in the above embodiments.

[0129] A portion of this disclosure can be applied to computer program products, such as computer program instructions, which, when executed by a computer, can invoke or provide methods and / or technical solutions according to this disclosure through the operation of the computer. Those skilled in the art will understand that the forms in which computer program instructions exist in a computer-readable medium include, but are not limited to, source files, executable files, and installation package files. Accordingly, the ways in which computer program instructions are executed by a computer include, but are not limited to: the computer directly executing the instructions; the computer compiling the instructions and then executing the corresponding compiled program; the computer reading and executing the instructions; or the computer reading and installing the instructions and then executing the corresponding installed program. Here, the computer-readable medium can be any available computer-readable storage medium or communication medium accessible to a computer.

[0130] Although embodiments of the present disclosure have been described in conjunction with the accompanying drawings, those skilled in the art can make various modifications and variations without departing from the spirit and scope of the present disclosure, and such modifications and variations all fall within the scope defined by the appended claims.

Claims

1. A method for predicting a playtime of multimedia data, characterized in that, The method comprises: inputting user data and multimedia data of a multimedia item to be recommended as original data into a pre-trained prediction model, the prediction model comprising a deep session interest network layer, a residual network layer and a multi-stage prediction tower; performing feature extraction on the original data in the deep session interest network layer to obtain a multi-dimensional deep feature vector; and performing feature extraction on the original data in the residual network layer to obtain a multi-dimensional shallow feature vector; performing compensation processing on the deep feature vector based on the shallow feature vector to generate a target feature vector, wherein the compensation processing is used to compensate for feature loss caused by the deep session interest network layer in the feature extraction process; predicting the target feature vector according to the multi-stage prediction tower to obtain a multi-stage play duration probability of the user when playing the multimedia data.

2. The method of claim 1, wherein, The method comprises: dividing the original data into a plurality of sessions according to a preset time interval in the deep session interest network layer, wherein the session represents a user behavior sequence; obtaining deviation information between any two sessions and injecting the deviation information into an embedded representation of the user behavior sequence; extracting a session interest feature vector from the user behavior sequence with the injected deviation information, and taking the session interest feature vector as a deep feature vector.

3. The method of claim 1, wherein, The method comprises: inputting the original data into a feature extraction network; performing at least one non-linear transformation on the original data through the feature extraction network to extract a shallow feature vector of the original data; wherein the dimension of the shallow feature is associated with the dimension of the original data; the feature extraction network is a residual network, and the at least one non-linear transformation is realized through a residual block in the residual network.

4. The method of claim 3, wherein, The method comprises: obtaining original data features of the original data; inputting the original data features into a convolution layer of the residual block for convolution processing to obtain convolution features, wherein the residual block is an identity residual block; inputting the convolution features into an activation function for non-linear mapping to obtain a first feature vector; adding the first feature vector and the original data features to obtain a shallow feature vector of the original data.

5. The method of claim 1, wherein, The method comprises: inputting the target feature vector into the multi-stage prediction tower to predict a play behavior target of the user playing the multimedia data with increasing duration; wherein the play behavior target represents whether the play duration of the multimedia data exceeds a plurality of incremental thresholds.

6. The method of claim 5, wherein, The method comprises: inputting the target feature vector into a first-stage prediction tower of the multi-stage prediction tower to predict a first probability that a user plays the multimedia data for a time duration greater than a first preset time duration; inputting the target feature vector and the first probability into a second-stage prediction tower of the multi-stage prediction tower to predict a second probability that the user plays the multimedia data for a time duration greater than a second preset time duration; inputting the target feature vector and the second probability into a third-stage prediction tower of the multi-stage prediction tower to predict a third probability that the user plays the multimedia data for a time duration greater than a third preset time duration; inputting the target feature vector and the third probability into a fourth-stage prediction tower of the multi-stage prediction tower to predict a fourth probability that the user plays the multimedia data for a time duration greater than a fourth preset time duration; inputting the target feature vector into a fifth-stage prediction tower of the multi-stage prediction tower to predict a fifth probability that the user performs a collection operation on the multimedia data.

7. The method of claim 6, wherein, The method further comprises: obtaining scene data of the user when playing the multimedia data, and performing feature extraction on the scene data to obtain context features; assigning corresponding weights to the fourth probability and the fifth probability according to the context features; combining the fourth probability and the fifth probability after the weights are assigned to calculate a total prediction value of the multimedia data; sorting a plurality of multimedia data according to the total prediction value of each multimedia data, and recommending multimedia data with a total prediction value greater than a preset probability threshold to the user.

8. A device for predicting the duration of multimedia data playback, characterized in that, The device comprises: a data acquisition module configured to input user data and multimedia data of multimedia items to be recommended into a pre-trained prediction model as original data, the prediction model comprising a deep session interest network layer, a residual network layer, and a multi-stage prediction tower; a deep feature extraction module configured to perform feature extraction on the original data in the deep session interest network layer to obtain a multi-dimensional deep feature vector; and a shallow feature extraction module configured to perform feature extraction on the original data in the residual network layer to obtain a multi-dimensional shallow feature vector; a feature compensation module configured to perform compensation processing on the deep feature vector based on the shallow feature vector to generate a target feature vector, wherein the compensation processing is used to compensate for feature loss caused by the deep session interest network layer in the feature extraction process; a probability prediction module configured to predict a multi-stage playing time duration probability of the user when playing the multimedia data according to the target feature vector and the multi-stage prediction tower.

9. An electronic device, comprising: comprise: a memory and a processor, which are communicatively connected, and the memory stores computer instructions, and the processor executes the computer instructions to perform the prediction method of the playing time duration of the multimedia data according to any one of claims 1 to 7.

10. A computer program product, characterised in that, comprise computer instructions for causing a computer to perform the prediction method of the playing time duration of the multimedia data according to any one of claims 1 to 7.

Images