Video feature extraction model training method, video recommendation method and device

By using user click behavior data to build triplets and train video feature extraction models, the problems of high manual labeling cost and low recommendation accuracy in the existing technology are solved, and more efficient video recommendation effects are achieved.

CN112149604BActive Publication Date: 2024-11-22NETEASE MEDIA TECH BEIJING
View PDF 0 Cites 0 Cited by

Patent Information

Application Number
CN202011064441.X
Authority / Receiving Office
CN · China
Patent Type
Patents(China)
Current Assignee / Owner
Filing Date
2020-09-30
Publication Date
2024-11-22
Estimated Expiration
2040-09-30

AI Technical Summary

Technical Problem

The existing video feature extraction model based on artificial intelligence requires a large amount of manual annotation data for training, resulting in high labeling costs and inability to effectively capture the potential interests of users, resulting in low video recommendation accuracy.

Method used

By analyzing the user's click behavior data, building triples (target video, related video and unrelated video) to train the video feature extraction model, and automatically label related videos using user click behavior to reduce the cost of training data collection, and enable model learning Implicit feature representations between the relevant videos that the user considers.

Benefits of technology

It improves the accuracy of video recommendations, makes the recommended content more in line with the actual interests of users, reduces the cost and time of training data annotation, and enhances the compatibility between the model and application scenarios.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN112149604B_ABST
    Figure CN112149604B_ABST
Patent Text Reader

Abstract

The present application relates to the field of video processing technology, and discloses a training method for a video feature extraction model, a video recommendation method and a device. The training method for the video feature extraction model includes: determining related videos based on the click behaviors of multiple users on videos; constructing multiple triplets based on the determined related videos, wherein each triplet includes a target video, a first video related to the target video, and a second video unrelated to the target video; extracting a feature representation of each video in each triplet based on the video feature extraction model; for each triplet, determining a loss value corresponding to each triplet based on the similarity between the feature representation of the target video and the feature representation of the first video in each triplet, and the similarity between the feature representation of the target video and the feature representation of the second video; and updating the parameters of the video feature extraction model based on the loss values ​​respectively corresponding to the multiple triplets.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present application relates to the field of video processing technology, and in particular to a video feature extraction model training method, a video recommendation method and a device. Background Art

[0002] This section is intended to provide a background or context to the embodiments of the application that are recited in the claims. No description herein is admitted to be prior art by inclusion in this section.

[0003] With the rapid popularization of the Internet, users post a large number of videos on major network platforms every day. For this reason, network platforms need to use video feature extraction models to extract features from videos posted by users, determine videos with a high degree of correlation based on the feature representation of each video, that is, related videos, and then recommend related videos to each user based on the preferences of each user. At present, the similarity of video content is usually judged manually, and then similar videos are marked to obtain training data. Based on a large amount of training data marked as similar videos, the video feature extraction model based on artificial intelligence is trained, so that the video feature extraction model can learn the implicit feature representation in similar videos, and then identify videos with similar content, and push videos with similar content to users as related videos. Summary of the invention

[0004] The existing AI-based video feature extraction model requires a large amount of training data. However, the cost of collecting training data is high when manually labeling related videos, and manual labeling is generally based on content similarity. Therefore, the video feature extraction model trained based on manually labeled training data can only recommend videos with similar content to users, ignoring the potential interest extension of users, resulting in the video feature learning goal not being consistent with the actual application scenario.

[0005] In response to the above technical problems, an improved method is urgently needed to reduce the labeling cost of relevant videos and make the video feature learning objectives consistent with the actual application scenarios, thereby improving the accuracy of video recommendations.

[0006] In a first aspect, an embodiment of the present application provides a method for training a video feature extraction model, comprising:

[0007] Determine related videos based on the click behaviors of multiple users on the videos;

[0008] Based on the determined related videos, construct a plurality of triplets, wherein each triplet includes a target video, a first video related to the target video, and a second video unrelated to the target video;

[0009] Based on the video feature extraction model, extract the feature representation of each video in each triplet;

[0010] For each triplet, based on the similarity between the feature representation of the target video and the feature representation of the first video in each triplet, and the similarity between the feature representation of the target video and the feature representation of the second video, determine the loss value corresponding to each triplet;

[0011] Based on the loss values ​​respectively corresponding to the multiple triplets, the parameters of the video feature extraction model are updated.

[0012] Optionally, determining related videos based on click behaviors of multiple users on the videos specifically includes:

[0013] Obtain historical click behavior data of multiple users on videos, where each user's historical click behavior data includes the clicked video and the corresponding click time;

[0014] Based on the historical click behavior data of the multiple users, a plurality of co-occurrence video pairs are determined, each co-occurrence video pair comprising: two videos that appear in the historical click behavior data of the same user and have adjacent click times;

[0015] Counting the number of times each co-occurring video pair appears in the historical click behavior data of the multiple users;

[0016] Two videos in a co-occurrence video pair whose number of appearances is greater than a number threshold are determined as related videos.

[0017] Optionally, the video feature extraction model includes multiple feature extraction modules, feature splicing modules and a fully connected layer, wherein each feature extraction module corresponds to a type of content information;

[0018] The feature representation of each video in each triplet is extracted based on the video feature extraction model, specifically including:

[0019] For any video in any triplet, obtaining a plurality of content information from the any video;

[0020] For any content information, based on a feature extraction module corresponding to the any content information, extract a feature vector corresponding to the any content information and output it;

[0021] Based on the feature splicing module, the feature vectors output by the multiple feature extraction modules are spliced ​​to obtain a spliced ​​feature vector and output it;

[0022] Based on the fully connected layer, the concatenated feature vector is converted into a vector of fixed dimension to obtain a feature representation corresponding to any one of the videos.

[0023] Optionally, the types of the content information include at least two of the following: video frames, audio frames and text information.

[0024] Optionally, when the any content information is a plurality of video frames, extracting a feature vector corresponding to the any content information specifically includes:

[0025] Based on the image feature extraction model, extract corresponding image features from the multiple video frames respectively;

[0026] The image features respectively corresponding to the multiple video frames are fused to obtain a video frame feature vector.

[0027] Optionally, when the any content information is a plurality of audio frames, extracting a feature vector corresponding to the any content information specifically includes:

[0028] Based on the audio feature extraction model, extract corresponding audio features from the multiple audio frames respectively;

[0029] The audio features respectively corresponding to the multiple audio frames are fused to obtain an audio feature vector.

[0030] Optionally, when the any content information is text information, extracting a feature vector corresponding to the any content information specifically includes:

[0031] Based on the word vector model, obtain the word vector corresponding to each word in the text information;

[0032] The word vectors corresponding to the words are fused to obtain a text feature vector.

[0033] In a second aspect, an embodiment of the present application provides a video recommendation method, including:

[0034] Get the third video clicked by the user;

[0035] Extracting feature representations of the third video and multiple candidate videos respectively based on a video feature extraction model, wherein the video feature extraction model is trained based on a training method of a video feature extraction model;

[0036] Determining the similarity between the feature representation of the third video and the feature representation of each candidate video respectively;

[0037] Based on the similarities corresponding to the multiple candidate videos, a video recommended to the user is determined from the multiple candidate videos.

[0038] In a third aspect, an embodiment of the present application provides a training device for a video feature extraction model, including:

[0039] A related video determination unit, used to determine related videos based on historical click behavior data of multiple users on videos;

[0040] A training data construction module, configured to construct a plurality of triplets based on the determined related videos, wherein each triplet includes a target video, a first video related to the target video, and a second video unrelated to the target video;

[0041] A feature extraction unit, used for extracting a feature representation of each video in each triplet based on a video feature extraction model;

[0042] A loss calculation unit, configured to determine, for each triplet, a loss value corresponding to each triplet based on a similarity between a feature representation of the target video and a feature representation of the first video in each triplet, and a similarity between a feature representation of the target video and a feature representation of the second video;

[0043] An updating unit is used to update the parameters of the video feature extraction model based on the loss values ​​respectively corresponding to the multiple triplets.

[0044] Optionally, the related video determination unit is specifically configured to:

[0045] Obtain historical click behavior data of multiple users on videos, where each user's historical click behavior data includes the clicked video and the corresponding click time;

[0046] Based on the historical click behavior data of the multiple users, a plurality of co-occurrence video pairs are determined, each co-occurrence video pair comprising: two videos that appear in the historical click behavior data of the same user and have adjacent click times;

[0047] Counting the number of times each co-occurring video pair appears in the historical click behavior data of the multiple users;

[0048] Two videos in a co-occurrence video pair whose number of appearances is greater than a number threshold are determined as related videos.

[0049] Optionally, the video feature extraction model includes multiple feature extraction modules, feature splicing modules and a fully connected layer, wherein each feature extraction module corresponds to a type of content information;

[0050] The feature extraction unit is specifically used for:

[0051] For any video in any triplet, obtaining a plurality of content information from the any video;

[0052] For any content information, based on a feature extraction module corresponding to the any content information, extract a feature vector corresponding to the any content information and output it;

[0053] Based on the feature splicing module, the feature vectors output by the multiple feature extraction modules are spliced ​​to obtain a spliced ​​feature vector and output it;

[0054] Based on the fully connected layer, the concatenated feature vector is converted into a vector of fixed dimension to obtain a feature representation corresponding to any one of the videos.

[0055] Optionally, the types of the content information include at least two of the following: video frames, audio frames and text information.

[0056] Optionally, when any of the content information is a plurality of video frames, the feature extraction unit is specifically configured to:

[0057] Based on the image feature extraction model, extract corresponding image features from the multiple video frames respectively;

[0058] The image features respectively corresponding to the multiple video frames are fused to obtain a video frame feature vector.

[0059] Optionally, when any of the content information is a plurality of audio frames, the feature extraction unit is specifically configured to:

[0060] Based on the audio feature extraction model, extract corresponding audio features from the multiple audio frames respectively;

[0061] The audio features respectively corresponding to the multiple audio frames are fused to obtain an audio feature vector.

[0062] Optionally, when any of the content information is text information, the feature extraction unit is specifically used to:

[0063] Based on the word vector model, obtain the word vector corresponding to each word in the text information;

[0064] The word vectors corresponding to the words are fused to obtain a text feature vector.

[0065] In a fourth aspect, an embodiment of the present application provides a video recommendation device, including:

[0066] An acquisition unit, used to acquire a third video clicked by a user;

[0067] A feature extraction unit, used to extract feature representations of the third video and multiple candidate videos respectively based on a video feature extraction model, wherein the video feature extraction model is trained based on a training method of a video feature extraction model;

[0068] a similarity calculation unit, used to respectively determine the similarity between the feature representation of the third video and the feature representation of each candidate video;

[0069] The recommendation unit is used to determine a video recommended to the user from the multiple candidate videos based on the similarities corresponding to the multiple candidate videos.

[0070] In a fifth aspect, an embodiment of the present application provides an electronic device, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor implements the steps of any of the above methods when executing the computer program.

[0071] In a sixth aspect, an embodiment of the present application provides a computer-readable storage medium having computer program instructions stored thereon, which implement the steps of any of the above methods when executed by a processor.

[0072] In a seventh aspect, an embodiment of the present application provides a computer program product, wherein the computer program product comprises a computer program stored on a computer-readable storage medium, wherein the computer program comprises program instructions, and when the program instructions are executed by a processor, the steps of any of the above methods are implemented.

[0073] The video feature extraction model training method and device, video recommendation method and device, electronic device and storage medium provided in the embodiments of the present application complete the labeling of related videos based on the user's click behavior on the video, thereby improving the labeling efficiency of training data and reducing the training cost; in addition, the training data constructed based on the user's click behavior enables the model to learn the implicit feature representations between the related videos that the user considers, and therefore will not recommend videos to the user based solely on the similarity of the video content, so that the video feature learning goals can be consistent with the actual application scenarios, thereby improving the accuracy of video recommendations. BRIEF DESCRIPTION OF THE DRAWINGS

[0074] By reading the detailed description below with reference to the accompanying drawings, the above and other purposes, features and advantages of the exemplary embodiments of the present application will become readily understood. In the accompanying drawings, several embodiments of the present application are shown in an exemplary and non-limiting manner, wherein:

[0075] Figure 1 A schematic diagram of an application scenario of the video recommendation method provided in an embodiment of the present application;

[0076] Figure 2 A flowchart of a method for training a video feature extraction model provided in one embodiment of the present application;

[0077] Figure 3 A schematic diagram of a process for determining related views based on user click behavior provided in an embodiment of the present application;

[0078] Figure 4 A schematic diagram of the structure of a video feature extraction model provided in one embodiment of the present application;

[0079] Figure 5 A schematic diagram of a process for obtaining a feature representation of a video based on a video feature extraction model provided in an embodiment of the present application;

[0080] Figure 6 A schematic diagram of a video recommendation method according to an embodiment of the present application;

[0081] Figure 7 A schematic diagram of the structure of a training device for a video feature extraction model provided in one embodiment of the present application;

[0082] Figure 8 A schematic diagram of the structure of a video recommendation device provided in one embodiment of the present application;

[0083] Fig. 9 A schematic diagram of the structure of an electronic device provided in one embodiment of the present application. DETAILED DESCRIPTION

[0084] The principles and spirit of the present application will be described below with reference to several exemplary embodiments. It should be understood that these embodiments are provided only to enable those skilled in the art to better understand and implement the present application, and are not intended to limit the scope of the present application in any way. On the contrary, these embodiments are provided to make the present disclosure more thorough and complete, and to fully convey the scope of the present disclosure to those skilled in the art.

[0085] Those skilled in the art know that the embodiments of the present application can be implemented as a system, device, equipment, method or computer program product. Therefore, the present disclosure can be specifically implemented in the following forms, namely: complete hardware, complete software (including firmware, resident software, microcode, etc.), or a combination of hardware and software.

[0086] It should be understood herein that any number of elements in the drawings is for illustration rather than limitation, and any naming is only for distinction rather than having any limiting meaning.

[0087] For ease of understanding, the terms involved in the embodiments of the present application are explained below:

[0088] Click behavior, in this application, refers to the behavior of users clicking to watch videos in the client. The client will record the videos clicked by the user and the click time and send them to the server. The server stores the videos and corresponding click times of each user's history clicks and arranges them in order by click time to obtain the historical click behavior data corresponding to each user.

[0089] A co-occurring video pair refers to two videos that appear in the historical click behavior data of the same user and have adjacent click times.

[0090] Related videos in this application refer to videos with a certain relevance, specifically two videos that are frequently clicked by users in succession, such as video A and video B. If a user clicks to watch video A and is very likely to click to watch video B, then video A and video B are related videos, otherwise they are unrelated videos. This can be determined based on the user's historical click behavior data on the videos, and this process will be described in detail later.

[0091] Multimodality: In actual application scenarios, the video itself carries content information in multiple dimensions, such as title, video frame, audio, etc. These content information constitute different modes of the video. In this application, the features of these modes will be learned separately, and the feature representation of the common video will be obtained by fusing the features of these modes.

[0092] Artificial Intelligence (AI) is the theory, method, technology and application system that uses digital computers or machines controlled by digital computers to simulate, extend and expand human intelligence, perceive the environment, acquire knowledge and use knowledge to obtain the best results. In other words, artificial intelligence is a comprehensive technology in computer science that attempts to understand the essence of intelligence and produce a new intelligent machine that can respond in a similar way to human intelligence. Artificial intelligence is to study the design principles and implementation methods of various intelligent machines so that machines have the functions of perception, reasoning and decision-making.

[0093] Deep learning, the concept of deep learning originated from the study of artificial neural networks. Multilayer perceptrons with multiple hidden layers are a type of deep learning structure. Deep learning combines low-level features to form more abstract high-level representations of attribute categories or features to discover distributed feature representations of data. Deep learning is a new field in machine learning research. Its motivation is to establish and simulate neural networks that analyze and learn in the human brain, and to interpret data such as images, sounds, and text by imitating the mechanisms of the human brain. Commonly used deep learning models include: Convolutional Neural Networks (CNN), Recurrent Neural Networks (RNN), Long Short-Term Memory Networks (LSTM), Deep Neural Networks (DNN), Deep Belief Nets (DBNs) and other neural networks. There are two ways for data to propagate in a neural network. One is along the path from input to output, which is called forward propagation, and the other is from output back to input, which is called back propagation. In the forward propagation process, the input information is processed layer by layer by the neural network and transmitted to the output layer. The error between the output value and the expectation is described by the loss function, and then it enters the backward propagation. The partial derivative of the loss function with respect to the weight of each neuron is calculated layer by layer to form the weight gradient data of the loss function with respect to the weight vector. This data serves as the basis for updating the weight parameters, and the training of the neural network is completed in the process of continuously updating the weight parameters.

[0094] Fully connected layers (FC): They act as "classifiers" in the entire convolutional neural network. Each node in the fully connected layer is connected to all nodes in the previous layer to combine the features extracted previously. If the operations of convolutional layers, pooling layers, and activation function layers are to map the original data to the hidden feature space, the fully connected layer plays the role of mapping the learned "distributed feature representation" to the sample label space.

[0095] The loss function is a function that maps the value of a random event or its related random variables to a non-negative real number to represent the "risk" or "loss" of the random event. In applications, the loss function is usually associated with the optimization problem as a learning criterion, that is, the model is solved and evaluated by minimizing the loss function. For example, in machine learning, the loss function is used for parameter estimation of the model, and the loss value obtained based on the loss function can be used to describe the degree of difference between the model's predicted value and the actual value. Common loss functions include the mean square error loss function, SVM (Support Vector Machine) hinge loss function, cross entropy loss function, etc.

[0096] The principles and spirit of the present application are explained in detail below with reference to several representative implementations of the present application. SUMMARY OF THE INVENTION

[0098] The existing AI-based video feature extraction model requires a large amount of training data. However, the cost of collecting training data is high when manually labeling related videos, and manual labeling is generally based on content similarity. Therefore, the video feature extraction model trained based on manually labeled training data can only recommend videos with similar content to users, ignoring the potential interest extension of users, resulting in the video feature learning goal not being consistent with the actual application scenario.

[0099] In order to solve the above problems, the present application provides a training method for a video feature extraction model, which determines related videos based on users' click behaviors on videos, and constructs triplets for training models based on the determined related videos, so that the video feature learning objectives are more consistent with the actual application scenarios. The method specifically includes the following steps: determining related videos based on the click behaviors of multiple users on videos; constructing multiple triplets based on the determined related videos, wherein each triplet includes a target video, a first video related to the target video, and a second video unrelated to the target video; extracting a feature representation of each video in each triplet based on the video feature extraction model; for each triplet, determining the loss value corresponding to each triplet based on the similarity between the feature representation of the target video and the feature representation of the first video in each triplet, and the similarity between the feature representation of the target video and the feature representation of the second video; and updating the parameters of the video feature extraction model based on the loss values ​​corresponding to the multiple triplets. The training method of the video feature extraction model of the present application completes the labeling of related videos based on the user's click behavior on the video, thereby improving the labeling efficiency of the training data and reducing the training cost; in addition, the training data constructed based on the user's click behavior enables the model to learn the implicit feature representations between the videos that the user considers to be related, and therefore will not recommend videos to the user based solely on the similarity of the video content, so that the video feature learning goal can be consistent with the actual application scenario, thereby improving the accuracy of video recommendations.

[0100] After introducing the basic principles of the present application, various non-limiting implementation methods of the present application are described in detail below.

[0101] Application Scenario Overview

[0102] refer to Figure 1 , which is a schematic diagram of an application scenario of the video recommendation method provided in the embodiment of the present application. Figure 1 The application scenario shown includes a terminal device 101 and a server 102. The terminal device 101 and the server 102 may be connected via a wireless communication network or a wired communication network, and the terminal device 101 includes but is not limited to electronic devices such as desktop computers, mobile phones, mobile computers, tablet computers, media players, smart wearable devices, smart TVs, vehicle-mounted devices, and personal digital assistants (PDAs). The server 102 may be an independent physical server, or a server cluster or distributed system consisting of multiple physical servers, or a cloud server that provides basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communications, middleware services, domain name services, security services, CDN, and big data and artificial intelligence platforms.

[0103] The server 102 is used to provide video services, such as live video, online video playback, video download, video publishing and other services. The terminal device 101 is installed with a video client, and the user can obtain the video service provided by the server 102 through the video client, or the user can also access the video website through the browser in the terminal device 101 to obtain the video service provided by the server 102. For example, any user can upload a video to the server 102 corresponding to the video playback platform through the video client to publish the recorded video through the video playback platform, so that other users can watch the video published by the user. The server 102 analyzes and identifies the video uploaded by the user, and determines the related video based on the video analysis and identification results. The server 102 determines the related videos corresponding to the historically watched videos based on the historically watched videos of other users, and determines the videos recommended to the user from the obtained related videos, and sends these videos to the user's terminal device.

[0104] Combine the following Figure 1 The training method of the video feature extraction model and the video recommendation method according to the exemplary embodiment of the present application are described by using the application scenario. It should be noted that the above application scenario is only shown to facilitate understanding of the spirit and principle of the present application, and the implementation of the present application is not limited in this respect. On the contrary, the implementation of the present application can be applied to any applicable scenario.

[0105] Exemplary Methods

[0106] refer to Figure 2 The embodiment of the present application provides a training method for a video feature extraction model, which can be applied to a server and specifically includes the following steps:

[0107] S201. Determine related videos based on click behaviors of multiple users on videos.

[0108] In specific implementation, the client will record the videos clicked by the user and the corresponding click time, and send them to the server. The server stores the videos clicked by each user and the corresponding click time, and arranges them in order by click time to obtain the historical click behavior data corresponding to each user, and stores them in the server. The server can obtain the click behavior of each user on the video from the historical click behavior data corresponding to each user, and based on the obtained click behavior, determine two videos that are frequently clicked by the user in succession from the videos clicked by these multiple users, and determine these two videos as related videos. Each video can be a related video with one or more videos.

[0109] S202: construct a plurality of triplets based on the determined related videos, wherein each triplet includes a target video, a first video related to the target video, and a second video unrelated to the target video.

[0110] In specific implementation, multiple triplets can be constructed based on the videos involved in the historical click behavior data. Taking the construction of a triplet as an example, firstly, a video is selected from the determined related videos as the target video, then a video related to the target video is obtained as the first video, and then a video unrelated to the target video is obtained as the second video, so that a triplet can be constructed.

[0111] The target video and the first video in the triplet are taken as positive samples, and the target video and the second video in the triplet are taken as negative samples, aiming to learn the feature representation of the video based on the positive samples and the negative samples so that the distance between the feature representations of related videos is close and the distance between the feature representations of unrelated videos is far away.

[0112] S203: Extract feature representation of each video in each triplet based on a video feature extraction model.

[0113] Specifically, the target video a, the first video p, and the second video n in a triplet are respectively input into the video feature extraction model to obtain the feature representation corresponding to the target video a. The feature representation corresponding to the first video p The feature representation corresponding to the second video n

[0114] S204. For each triplet, based on the similarity between the feature representation of the target video and the feature representation of the first video in each triplet, and the similarity between the feature representation of the target video and the feature representation of the second video, determine the loss value corresponding to each triplet.

[0115] In specific implementation, calculation methods such as Euclidean distance and cosine similarity can be used to calculate the similarity between the feature representation of the target video and the feature representation of the first video, as well as the similarity between the feature representation of the target video and the feature representation of the second video, which is not limited in the embodiments of the present application.

[0116] In specific implementation, the existing loss function can be used to determine the loss value corresponding to each triple. Taking the triploss loss function as an example, the loss value corresponding to each triple can be expressed as:

[0117]

[0118] S205. Update the parameters of the video feature extraction model based on the loss values ​​corresponding to the multiple triplets.

[0119] The video feature extraction model can be any neural network used to extract video features. The extracted video features are a multi-dimensional vector. The neural networks used in the video feature extraction model include but are not limited to: Convolutional Neural Networks (CNN), Recurrent Neural Networks (RNN), Long Short-Term Memory Networks (LSTM), Deep Neural Networks (DNN), Deep Belief Nets (DBNs), etc. Among them, updating the parameters of the video feature extraction model means updating the weight parameters in each layer of the neural network.

[0120] The training method of the video feature extraction model of the present application completes the labeling of related videos based on the user's click behavior on the video, thereby improving the labeling efficiency of the training data and reducing the training cost; in addition, the training data constructed based on the user's click behavior enables the model to learn the implicit feature representations between the videos that the user considers to be related, and therefore will not recommend videos to the user based solely on the similarity of the video content, so that the video feature learning goal can be consistent with the actual application scenario, thereby improving the accuracy of video recommendations.

[0121] When implementing it, refer to Figure 3 Step S201 specifically includes the following steps:

[0122] S301. Obtain historical click behavior data of multiple users on videos, wherein the historical click behavior data of each user includes the clicked video and the corresponding click time.

[0123] S302. Determine a plurality of co-occurring video pairs based on historical click behavior data of a plurality of users, wherein each co-occurring video pair includes: two videos that appear in the historical click behavior data of the same user and have adjacent click times.

[0124] In specific implementation, the videos in each user's historical click behavior data can be sorted according to the order of the click moments corresponding to each video to obtain each user's historical click sequence, that is, the data in the historical click sequence is sorted according to the order of the click moments, and then the co-occurring video pairs can be quickly determined based on the historical click sequence.

[0125] In actual applications, in the historical click sequence, there will be two videos with adjacent click times but a long interval between them. There is no correlation between such videos. For example, the last video clicked the day before and the first video clicked today are not related videos. Another example is that a user watched a video at 10 a.m. and did not watch other videos until 5 p.m.

[0126] To this end, a time difference threshold can be set to exclude two videos whose click moments are adjacent but whose interval is long in the historical click sequence. Specifically, based on the historical click behavior data of each user, the corresponding historical click sequence can be obtained; for any two adjacent videos in the historical click sequence, it is determined whether the difference between the click moments corresponding to the two videos is less than the time difference threshold. If it is less than, the two videos are determined as a co-occurring video pair. If it is not less than, it indicates that the two videos are not a co-occurring video pair. In this case, each co-occurring video pair includes: two videos whose click moments are adjacent in the historical click behavior data of the same user, and whose difference between the click moments is less than the time difference threshold. Among them, the time difference threshold can be predetermined according to actual application requirements. For example, the time difference threshold can be 1 hour, 2 hours, etc. Alternatively, the time difference threshold can be dynamically adjusted according to the playback duration of the video before the click moment of the two adjacent videos. The time difference threshold can be a value that is not less than the playback duration of the video before the click moment, or a value that is slightly larger than the playback duration of the video before the click moment. For example, if the playback duration of the video before the click moment of the two adjacent videos is 10 minutes, the time difference threshold can be 10 minutes, 11 minutes, or 15 minutes.

[0127] Alternatively, the server may also record the first moment when the user logs into the video client and the second moment when the user closes the video client each time, and obtain the usage period of the user's use of the video client based on the first moment and the second moment. If two videos with adjacent click moments belong to the same usage period, the two videos are determined to be a co-occurring video pair; if two videos with adjacent click moments belong to different usage periods, the two videos are not a co-occurring video pair.

[0128] S303: Count the number of times each co-occurring video pair appears in the historical click behavior data of multiple users.

[0129] For example, video A and video B are a co-occurring video pair. This co-occurring video pair appears 3 times in the historical click behavior data of user 1, once in the historical click behavior data of user 2, user 3, and user 4 respectively, and does not appear in the historical click behavior data of other users. Then the total number of times this co-occurring video appears is 6 times.

[0130] S304: Determine two videos in a co-occurrence video pair whose number of occurrences is greater than a number threshold as related videos.

[0131] Among them, the number threshold can be predetermined according to actual application requirements, and the embodiment of the present application does not limit the specific value of the number threshold.

[0132] Alternatively, the number threshold may be dynamically adjusted according to the number of videos included in the historical click behavior data of multiple users. For example, the number threshold is positively correlated with the number of videos included in the historical click behavior data of multiple users, that is, the more videos are included in the historical click behavior data of multiple users, the larger the number threshold. Specifically, a ratio value may be pre-set, and the number threshold is equal to the total number of videos multiplied by the ratio value, and the total number of videos is the number of videos included in the historical click behavior data of multiple users.

[0133] Figure 3 The implementation method shown defines related videos and related video feature learning targets based on the user's click behavior on the video, that is, videos that are frequently watched continuously by the user are defined as related videos, so that the related video labeling process does not require human intervention and can tap into the extension of the user's potential interests, rather than recommending videos to users based solely on the similarity of the video content, so that the video feature learning targets can be consistent with the actual application scenarios, thereby improving the accuracy of video recommendations.

[0134] by Figure 4 For example, the video feature extraction model includes multiple feature extraction modules, feature concatenation modules and fully connected layers. Each feature extraction module corresponds to a content information, and each feature extraction module is used to extract a feature vector from the corresponding content information.

[0135] Based on this, reference Figure 5 Step S203 specifically includes the following steps:

[0136] S501. For any video in any triplet, obtain a variety of content information from any video.

[0137] The types of content information include at least two of the following: video frames, audio frames and text information.

[0138] Specifically, the audio and video of the video can be separated to obtain the audio part and the video part; then, multiple audio frames are extracted from the audio part at a certain sampling frequency, and multiple video frames are extracted from the video part at a certain sampling frequency, and the text information corresponding to the video is obtained, and the text information includes but is not limited to: video title, video introduction, etc.

[0139] S502 . For any content information, based on a feature extraction module corresponding to any content information, extract a feature vector corresponding to any content information and output it.

[0140] When the content information is multiple video frames, based on the image feature extraction model, corresponding image features are extracted from the multiple video frames respectively, and the image features corresponding to the multiple video frames are fused to obtain a video frame feature vector.

[0141] Specifically, the image feature extraction model may be an InceptionV3 model, which is obtained by training based on the ImageNet training set. The NextVlad network structure may be used to fuse the image features corresponding to multiple video frames into a video frame feature vector to reduce the dimension of the video features.

[0142] When the content information is multiple audio frames, based on the audio feature extraction model, corresponding audio features are extracted from the multiple audio frames respectively, and the audio features corresponding to the multiple audio frames are fused to obtain an audio feature vector.

[0143] Specifically, the audio features corresponding to each audio frame can be extracted based on the trained vggish network, and based on the NextVlad network structure, the audio features corresponding to multiple audio frames can be fused into an audio feature vector to reduce the dimension of the audio features.

[0144] When the content information is text information, the word vectors corresponding to each word in the text information are obtained based on the word vector model; the word vectors corresponding to each word are fused to obtain a text feature vector.

[0145] Specifically, the word vector model can be a word2vec model. The text information can be first segmented using a word segmenter (such as JIEBA) to obtain each word in the text information, and then the word vector of each word is extracted using a pre-trained word2vec model, and then the word vector of each word is feature aggregated based on the NextVlad network structure to obtain a text feature vector.

[0146] S503: Based on the feature splicing module, the feature vectors output by the multiple feature extraction modules are spliced ​​to obtain a spliced ​​feature vector and output it.

[0147] S504: Based on the fully connected layer, the concatenated feature vector is converted into a vector of fixed dimension to obtain a feature representation corresponding to any video.

[0148] In the above implementation, the feature representation of the video adopts end-to-end joint learning of multimodal features, and finally the feature representation of each video is obtained by aggregating multimodal features. The fusion of multimodal features has the advantage of information complementarity, so that the obtained video feature representation is more comprehensive.

[0149] The video feature extraction model obtained by the training method in any of the above embodiments is referred to as Figure 6The embodiment of the present application also provides a video recommendation method, which specifically includes the following steps:

[0150] S601. Obtain a third video clicked by a user.

[0151] In a specific implementation, a user browses videos through a video client. When the user clicks on a certain video, namely, a third video, the client sends a request to the server to obtain the third video. The server sends the third video to the video client, and the video client plays the third video. At the same time, the server determines a video recommended to the user based on the third video clicked by the user.

[0152] S602: Extract feature representations of the third video and multiple candidate videos respectively based on a video feature extraction model.

[0153] The candidate videos are videos waiting to be recommended to the user, and may include newly added videos or previous videos.

[0154] Among them, the video feature extraction model is obtained by the training method in any of the above-mentioned implementations, and the specific process will not be repeated here.

[0155] S603: Determine the similarity between the feature representation of the third video and the feature representation of each candidate video respectively.

[0156] During specific implementation, calculation methods such as Euclidean distance and cosine similarity may be used to calculate the similarity between the feature representation of the third video and the feature representations of each candidate video, which is not limited in the embodiment of the present application.

[0157] S604: Determine a video recommended to the user from the multiple candidate videos based on the similarities corresponding to the multiple candidate videos.

[0158] The higher the similarity between the feature representations of two videos, the more related the two videos are, and vice versa, the less related the two videos are.

[0159] In specific implementation, the candidate videos may be sorted in descending order according to the similarity corresponding to each candidate video, the top N candidate videos are obtained, and the N candidate videos are sent to the video client corresponding to the user, where N is an integer not less than 1.

[0160] The video recommendation method of the present application completes the annotation of related videos based on the user's click behavior on the video, which improves the annotation efficiency of training data and reduces the training cost. In addition, the training data constructed based on the user's click behavior enables the model to learn the implicit feature representation between the related videos that the user believes, so it will not recommend videos to users based only on the similarity of the video content, so that the video feature learning goal can be consistent with the actual application scenario, thereby improving the accuracy of video recommendation. In addition, for newly added videos, the feature representation of the newly added videos can be obtained in time based on the video feature extraction model, and then the newly added videos can be quickly and effectively pushed to suitable users.

[0161] Exemplary Devices

[0162] After introducing the method of the exemplary embodiment of the present application, the training device of the video feature extraction model, the video recommendation device, etc. of the exemplary embodiment of the present application are introduced next.

[0163] like Figure 7 , which is a schematic diagram of the structure of the training device of the video feature extraction model provided in the embodiment of the present application. In one embodiment, the training device 70 of the video feature extraction model includes: a related video determination unit 701, a training data construction module 702, a feature extraction unit 703, a loss calculation unit 704 and an update unit 705.

[0164] A related video determination unit 701 is used to determine related videos based on historical click behavior data of multiple users on videos;

[0165] A training data construction module 702, configured to construct a plurality of triplets based on the determined related videos, wherein each triplet includes a target video, a first video related to the target video, and a second video unrelated to the target video;

[0166] A feature extraction unit 703, used to extract a feature representation of each video in each triplet based on a video feature extraction model;

[0167] A loss calculation unit 704 is used to determine, for each triplet, a loss value corresponding to each triplet based on the similarity between the feature representation of the target video and the feature representation of the first video in each triplet, and the similarity between the feature representation of the target video and the feature representation of the second video;

[0168] The updating unit 705 is used to update the parameters of the video feature extraction model based on the loss values ​​corresponding to the multiple triplets.

[0169] Optionally, the related video determination unit 701 is specifically used to: obtain historical click behavior data of multiple users for videos, the historical click behavior data of each user including the clicked videos and the corresponding click times; determine multiple co-occurring video pairs based on the historical click behavior data of multiple users, each co-occurring video pair including: two videos with adjacent click times that appear in the historical click behavior data of the same user; count the number of times each co-occurring video pair appears in the historical click behavior data of multiple users; and determine two videos in a co-occurring video pair whose number of appearances is greater than a number threshold as related videos.

[0170] Optionally, the video feature extraction model includes a plurality of feature extraction modules, a feature concatenation module and a fully connected layer, wherein each feature extraction module corresponds to a type of content information;

[0171] Accordingly, the feature extraction unit 703 is specifically used to: for any video in any triplet, obtain a variety of content information from any video; for any content information, based on the feature extraction module corresponding to any content information, extract and output the feature vector corresponding to any content information; based on the feature splicing module, splice the feature vectors output by multiple feature extraction modules to obtain and output the spliced ​​feature vector; based on the fully connected layer, convert the spliced ​​feature vector into a vector of fixed dimension to obtain the feature representation corresponding to any video.

[0172] Optionally, the types of content information include at least two of the following: video frames, audio frames and text information.

[0173] Optionally, when any content information is multiple video frames, the feature extraction unit 703 is specifically used to: extract corresponding image features from the multiple video frames based on the image feature extraction model; and fuse the image features corresponding to the multiple video frames to obtain a video frame feature vector.

[0174] Optionally, when any content information is multiple audio frames, the feature extraction unit 703 is specifically used to: extract corresponding audio features from the multiple audio frames based on the audio feature extraction model; and fuse the audio features corresponding to the multiple audio frames to obtain an audio feature vector.

[0175] Optionally, when any content information is text information, the feature extraction unit 703 is specifically used to: obtain the word vectors corresponding to each word in the text information based on the word vector model; and fuse the word vectors corresponding to each word to obtain a text feature vector.

[0176] The training device for the video feature extraction model provided in the embodiment of the present application adopts the same inventive concept as the training method for the above-mentioned video feature extraction model and can achieve the same beneficial effects, which will not be repeated here.

[0177] like Figure 8 , which is a schematic diagram of the structure of a video recommendation device provided by an embodiment of the present application. In one embodiment, the video recommendation device 80 includes: an acquisition unit 801, a feature extraction unit 802, a similarity calculation unit 803 and a recommendation unit 804.

[0178] An acquisition unit 801 is used to acquire a third video clicked by a user;

[0179] A feature extraction unit 802 is used to extract feature representations of the third video and the plurality of candidate videos respectively based on a video feature extraction model, wherein the video feature extraction model is obtained by training based on a training method of the video feature extraction model;

[0180] A similarity calculation unit 803, used to respectively determine the similarity between the feature representation of the third video and the feature representation of each candidate video;

[0181] The recommendation unit 804 is used to determine a video recommended to the user from the multiple candidate videos based on the similarities corresponding to the multiple candidate videos.

[0182] The video recommendation device provided in the embodiment of the present application adopts the same inventive concept as the above-mentioned video recommendation method and can achieve the same beneficial effects, which will not be described in detail here.

[0183] Based on the same inventive concept as the training method and video recommendation method of the above-mentioned video feature extraction model, the embodiment of the present application also provides an electronic device, which can be a single physical server, a server cluster or a distributed system composed of multiple physical servers, or a cloud server that provides basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communications, middleware services, domain name services, security services, CDN, and big data and artificial intelligence platforms. Fig. 9 As shown, the electronic device 90 may include at least one processor 901 and at least one memory 902. The memory 902 stores program codes. When the program codes are executed by the processor 901, the processor 901 executes various steps in the video feature extraction model training method or video recommendation method according to various exemplary embodiments of the present application described in the above “Exemplary Method” section of this specification. For example, the processor 901 may execute the following steps: Figure 2As shown in S201, based on the click behavior of multiple users on the video, related videos are determined; S202, based on the determined related videos, multiple triplets are constructed, wherein each triplet includes a target video, a first video related to the target video, and a second video unrelated to the target video; S203, for extracting the feature representation of each video in each triplet based on the video feature extraction model; S204, for determining, for each triplet, the loss value corresponding to each triplet based on the similarity between the feature representation of the target video and the feature representation of the first video in each triplet, and the similarity between the feature representation of the target video and the feature representation of the second video; S205, for updating the parameters of the video feature extraction model based on the loss values ​​corresponding to the multiple triplets respectively. For example, the processor 901 can also execute the following Figure 6 As shown in S601, obtain the third video clicked by the user; S602, extract the feature representation of the third video and multiple candidate videos respectively based on the video feature extraction model; S603, determine the similarity between the feature representation of the third video and the feature representation of each candidate video respectively; S604, based on the corresponding similarities of the multiple candidate videos, determine the video recommended to the user from the multiple candidate videos.

[0184] Processor 901 can be a general-purpose processor, such as a central processing unit (CPU), a digital signal processor (DigitalSignal Processor, DSP), an application-specific integrated circuit (Application Specific Integrated Circuit, ASIC), a field programmable gate array (Field Programmable Gate Array, FPGA) or other programmable logic devices, discrete gates or transistor logic devices, discrete hardware components, and can implement or execute the various methods, steps and logic block diagrams disclosed in the embodiments of the present application. A general-purpose processor can be a microprocessor or any conventional processor, etc. The steps of the method disclosed in conjunction with the embodiments of the present application can be directly embodied as a hardware processor to be executed, or can be executed by a combination of hardware and software modules in the processor.

[0185] Memory 902, as a non-volatile computer-readable storage medium, can be used to store non-volatile software programs, non-volatile computer executable programs and modules. Memory can include at least one type of storage medium, for example, can include flash memory, hard disk, multimedia card, card-type memory, random access memory (Random Access Memory, RAM), static random access memory (Static Random Access Memory, SRAM), programmable read-only memory (Programmable Read Only Memory, PROM), read-only memory (Read Only Memory, ROM), electrically erasable programmable read-only memory (Electrically Erasable Programmable Read-Only Memory, EEPROM), magnetic memory, disk, optical disk, etc. Memory is any other medium that can be used to carry or store a desired program code in the form of an instruction or data structure and can be accessed by a computer, but is not limited thereto. The memory 902 in the embodiment of the present application can also be a circuit or any other device that can realize a storage function, for storing program instructions and / or data.

[0186] Exemplary Program Products

[0187] An embodiment of the present application provides a computer-readable storage medium for storing computer program instructions used by the above-mentioned electronic device, which includes a program for executing a training method for a video feature extraction model or a video recommendation method in any exemplary embodiment of the present application.

[0188] The above-mentioned computer storage media can be any available media or data storage devices that can be accessed by a computer, including but not limited to magnetic storage (such as floppy disks, hard disks, magnetic tapes, magneto-optical disks (MO), etc.), optical storage (such as CD, DVD, BD, HVD, etc.), and semiconductor storage (such as ROM, EPROM, EEPROM, non-volatile memory (NAND FLASH), solid-state drive (SSD)), etc.

[0189] In some possible implementations, various aspects of the present application may also be implemented as a computer program product, which includes a program code. When the computer program product is run on a server device, the computer program product is used to enable the server device to execute the steps of the video feature extraction model training method or video recommendation method according to various exemplary implementations of the present application described in the above “Exemplary Method” section of this specification. For example, the server device may execute the following steps: Figure 2As shown in S201, based on the click behavior of multiple users on the video, related videos are determined; S202, based on the determined related videos, multiple triplets are constructed, wherein each triplet includes a target video, a first video related to the target video, and a second video unrelated to the target video; S203, based on the video feature extraction model, extracting the feature representation of each video in each triplet; S204, based on the similarity between the feature representation of the target video and the feature representation of the first video in each triplet, and the similarity between the feature representation of the target video and the feature representation of the second video, determining the loss value corresponding to each triplet; S205, based on the loss values ​​corresponding to the multiple triplets, respectively.

[0190] The computer program product may use any combination of one or more readable media. The readable medium may be a readable signal medium or a readable storage medium. The readable storage medium may be, for example, but not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, device or device, or any combination of the above. More specific examples of readable storage media (a non-exhaustive list) include: an electrical connection with one or more wires, a portable disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the above.

[0191] According to the computer program product for training a video feature extraction model or video recommendation according to the embodiment of the present application, it can adopt a portable compact disk read-only memory (CD-ROM) and include program code, and can be run on a server device. However, the program product of the present application is not limited to this. In this document, a computer-readable storage medium can be any tangible medium containing or storing a program, which can be used by or in combination with an instruction execution system, an apparatus or a device.

[0192] A readable signal medium may include a data signal propagated in baseband or as part of a carrier wave, wherein readable program code is carried. Such propagated data signals may take a variety of forms, including, but not limited to, electromagnetic signals, optical signals, or any suitable combination of the foregoing. A readable signal medium may also be any readable medium other than a readable storage medium, which may transmit, propagate, or transfer a program for use by or in conjunction with an instruction execution system, apparatus, or device.

[0193] The program code embodied on the readable medium may be transmitted using any appropriate medium, including but not limited to wireless, wireline, optical fiber cable, RF, etc., or any suitable combination of the foregoing.

[0194] Program code for performing the operations of the present application may be written in any combination of one or more programming languages, including object-oriented programming languages, such as Java, C++, etc., and conventional procedural programming languages, such as "C" or similar programming languages. The program code may be executed entirely on the user computing device, partially on the user computing device, partially on a remote computing device, or entirely on a remote computing device or server. In the case of a remote computing device, the remote computing device may be connected to the user computing device via any type of network, including a local area network (LAN) or a wide area network (WAN), or may be connected to an external computing device (e.g., via the Internet using an Internet service provider).

[0195] It should be noted that, although several units or subunits of the device are mentioned in the above detailed description, this division is merely exemplary and not mandatory. In fact, according to the embodiments of the present application, the features and functions of two or more units described above can be embodied in one unit. Conversely, the features and functions of one unit described above can be further divided into multiple units to be embodied.

[0196] In addition, although the operations of the method of the present application are described in a specific order in the drawings, this does not require or imply that the operations must be performed in this specific order, or that all the operations shown must be performed to achieve the desired results. Additionally or alternatively, some steps may be omitted, multiple steps may be combined into one step, and / or one step may be decomposed into multiple steps.

[0197] Although the spirit and principle of the present application have been described with reference to several specific embodiments, it should be understood that the present application is not limited to the disclosed specific embodiments, and the division of various aspects does not mean that the features in these aspects cannot be combined to benefit, and such division is only for the convenience of expression. The present application is intended to cover various modifications and equivalent arrangements included in the spirit and scope of the attached claims.

Claims

1. A training method for a video feature extraction model, characterized in that: include: Determine related videos based on the click behaviors of multiple users on the videos; Based on the determined related videos, construct a plurality of triplets, wherein each triplet includes a target video, a first video related to the target video, and a second video unrelated to the target video; Based on the video feature extraction model, extract the feature representation of each video in each triplet; For each triplet, based on the similarity between the feature representation of the target video and the feature representation of the first video in each triplet, and the similarity between the feature representation of the target video and the feature representation of the second video, determine the loss value corresponding to each triplet; Based on the loss values ​​respectively corresponding to the multiple triplets, updating the parameters of the video feature extraction model; The determining of related videos based on the click behaviors of multiple users on the videos specifically includes: Obtain historical click behavior data of multiple users on videos, where each user's historical click behavior data includes the clicked video and the corresponding click time; Based on the historical click behavior data of the multiple users, a plurality of co-occurrence video pairs are determined, each co-occurrence video pair comprising: two videos that appear in the historical click behavior data of the same user and have adjacent click times; Counting the number of times each co-occurring video pair appears in the historical click behavior data of the multiple users; Two videos in a co-occurrence video pair whose number of appearances is greater than a number threshold are determined as related videos.

2. The method according to claim 1, characterized in that The video feature extraction model includes multiple feature extraction modules, feature splicing modules and a fully connected layer, wherein each feature extraction module corresponds to a type of content information; The feature representation of each video in each triplet is extracted based on the video feature extraction model, specifically including: For any video in any triplet, obtaining a plurality of content information from the any video; For any content information, based on a feature extraction module corresponding to the any content information, extract a feature vector corresponding to the any content information and output it; Based on the feature splicing module, the feature vectors output by the multiple feature extraction modules are spliced ​​to obtain a spliced ​​feature vector and output it; Based on the fully connected layer, the concatenated feature vector is converted into a vector of fixed dimension to obtain a feature representation corresponding to any one of the videos.

3. The method according to claim 2, characterized in that The types of the content information include at least two of the following: video frames, audio frames and text information.

4. The method according to claim 3, characterized in that When the any content information is a plurality of video frames, extracting a feature vector corresponding to the any content information specifically includes: Based on the image feature extraction model, extract corresponding image features from the multiple video frames respectively; The image features respectively corresponding to the multiple video frames are fused to obtain a video frame feature vector.

5. The method according to claim 3, characterized in that: When the any content information is a plurality of audio frames, extracting a feature vector corresponding to the any content information specifically includes: Based on the audio feature extraction model, extract corresponding audio features from the multiple audio frames respectively; The audio features respectively corresponding to the multiple audio frames are fused to obtain an audio feature vector.

6. The method according to claim 3, characterized in that When the any content information is text information, extracting a feature vector corresponding to the any content information specifically includes: Based on the word vector model, obtain the word vectors corresponding to each word in the text information; The word vectors corresponding to the words are fused to obtain a text feature vector.

7. A video recommendation method, characterized in that: include: Get the third video clicked by the user; Extracting feature representations of the third video and multiple candidate videos respectively based on a video feature extraction model, wherein the video feature extraction model is trained based on the method according to any one of claims 1 to 6; Determining the similarity between the feature representation of the third video and the feature representation of each candidate video respectively; Based on the similarities corresponding to the multiple candidate videos, a video recommended to the user is determined from the multiple candidate videos.

8. A training device for a video feature extraction model, characterized in that: include: A related video determination unit, used to determine related videos based on historical click behavior data of multiple users on videos; A training data construction module, configured to construct a plurality of triplets based on the determined related videos, wherein each triplet includes a target video, a first video related to the target video, and a second video unrelated to the target video; A feature extraction unit, used for extracting a feature representation of each video in each triplet based on a video feature extraction model; A loss calculation unit, configured to determine, for each triplet, a loss value corresponding to each triplet based on a similarity between a feature representation of the target video and a feature representation of the first video in each triplet, and a similarity between a feature representation of the target video and a feature representation of the second video; An updating unit, configured to update parameters of the video feature extraction model based on the loss values ​​respectively corresponding to the plurality of triplets; The related video determination unit is specifically used to obtain historical click behavior data of multiple users for videos, where the historical click behavior data of each user includes the clicked videos and the corresponding click times; based on the historical click behavior data of the multiple users, multiple co-occurring video pairs are determined, where each co-occurring video pair includes: two videos that appear in the historical click behavior data of the same user and have adjacent click times; counting the number of times each co-occurring video pair appears in the historical click behavior data of the multiple users; and determining two videos in the co-occurring video pair whose number of appearances is greater than a number threshold as related videos.

9. The device according to claim 8, characterized in that The video feature extraction model includes multiple feature extraction modules, feature splicing modules and a fully connected layer, wherein each feature extraction module corresponds to a type of content information; The feature extraction unit is specifically used for: For any video in any triplet, obtaining a plurality of content information from the any video; For any content information, based on a feature extraction module corresponding to the any content information, extract a feature vector corresponding to the any content information and output it; Based on the feature splicing module, the feature vectors output by the multiple feature extraction modules are spliced ​​to obtain a spliced ​​feature vector and output it; Based on the fully connected layer, the concatenated feature vector is converted into a vector of fixed dimension to obtain a feature representation corresponding to any one of the videos.

10. The device according to claim 9, characterized in that The types of the content information include at least two of the following: video frames, audio frames and text information.

11. The device according to claim 10, characterized in that When any of the content information is a plurality of video frames, the feature extraction unit is specifically used to: Based on the image feature extraction model, extract corresponding image features from the multiple video frames respectively; The image features respectively corresponding to the multiple video frames are fused to obtain a video frame feature vector.

12. The device according to claim 10, characterized in that When any of the content information is a plurality of audio frames, the feature extraction unit is specifically used to: Based on the audio feature extraction model, extract corresponding audio features from the multiple audio frames respectively; The audio features respectively corresponding to the multiple audio frames are fused to obtain an audio feature vector.

13. The device according to claim 10, characterized in that When any of the content information is text information, the feature extraction unit is specifically used to: Based on the word vector model, obtain the word vector corresponding to each word in the text information; The word vectors corresponding to the words are fused to obtain a text feature vector.

14. A video recommendation device, characterized in that: include: An acquisition unit, used to acquire a third video clicked by a user; A feature extraction unit, configured to extract feature representations of the third video and a plurality of candidate videos respectively based on a video feature extraction model, wherein the video feature extraction model is trained based on the method according to any one of claims 1 to 6; a similarity calculation unit, used to respectively determine the similarity between the feature representation of the third video and the feature representation of each candidate video; The recommendation unit is used to determine a video recommended to the user from the multiple candidate videos based on the similarities corresponding to the multiple candidate videos.

15. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that: When the processor executes the computer program, the steps of the method according to any one of claims 1 to 6 are implemented.

16. A computer-readable storage medium having computer program instructions stored thereon, characterized in that: When the computer program instructions are executed by a processor, the steps of the method according to any one of claims 1 to 6 are implemented.