Video recommendation method, device and electronic device
By dividing the video into segments and using a video recommendation model based on positive and negative feedback, the problem of inaccuracy of interest caused by the modeling of the entire video in the prior art is solved, and the accuracy of video recommendation is improved.
Patent Information
- Application Number
- CN202211154166.X
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2022-09-21
- Publication Date
- 2024-11-05
- Estimated Expiration
- 2042-09-21
AI Technical Summary
In the existing video recommendation technology, the entire video as a modeling unit causes the user's real interests to be not accurately captured, resulting in low recommendation accuracy.
The videos to be recommended are divided into preset numbers of video clips, extract user attribute information and video clip characteristics, and use a video recommendation model based on the sample positive and negative feedback video clips to calculate the user's interest in each video clip, thereby determining the Target recommended video.
It improves the accuracy of video recommendations, captures the differences in users' interest in different parts of the content in a whole video, and recommends videos that are more in line with user interests.
Smart Images

Figure CN115630188B_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to the field of information processing technology, and in particular to a video recommendation method, device and electronic equipment. Background Art
[0002] Urban computing is a process of solving the challenges faced by cities by continuously acquiring, integrating and analyzing various heterogeneous big data in cities. In urban computing, social media data can be used to explore people's preferences and patterns, and can be combined with recommendation systems to recommend things of interest to users, provide help for the daily lives of urban people, and promote the healthy operation of cities.
[0003] Short videos can be used as a part of urban computing to provide users with life services such as news hotspots, food recommendations, and travel suggestions. They can characterize users' behavior patterns and interest preferences by mining their interactions with videos, and recommend videos of interest to users, making it easier for users to quickly capture content of interest and improve their quality of life.
[0004] Currently, video recommendation uses the entire video as a unit to model user interest preferences to evaluate the user's interest in the video. However, users often have different preferences for the content of different parts of a video, and the entire video may not necessarily be watched by the user. This can easily lead to more videos that do not match the user's real interests being recommended to the user, and the recommendation accuracy is not high. Summary of the invention
[0005] The present invention provides a video recommendation method, device and electronic device, which are used to solve the defect of low accuracy of video recommendation in the prior art and improve the accuracy of video recommendation.
[0006] The present invention provides a video recommendation method, comprising:
[0007] Obtain user attribute information and a collection of videos to be recommended;
[0008] Dividing each to-be-recommended video in the to-be-recommended video set into a preset number of video segments;
[0009] Extracting features from the user attribute information and the video clip respectively to obtain user features and video clip visual features;
[0010] Inputting the user features and the visual features of the video clip into a video recommendation model to obtain the interest of the user corresponding to the user attribute information output by the video recommendation model in the video to be recommended;
[0011] Determining a target recommended video from the set of videos to be recommended according to the interest level, and outputting the target recommended video to the user;
[0012] Among them, the video recommendation model is trained based on the sample original video corresponding to the user, the sample positive feedback video clip set and the sample negative feedback video clip set, the sample positive feedback video clip set is a set of video clips of the sample original video watched completely by the user, and the sample negative feedback video clip set is a set of video clips at the moment when the user's skipping behavior occurs.
[0013] According to a video recommendation method provided by the present invention, the step of inputting the user features and the visual features of the video clip into a video recommendation model to obtain the user's interest in the video to be recommended corresponding to the user attribute information output by the video recommendation model includes:
[0014] Inputting the user features and the video clip visual features into the graph convolutional neural network layer of the video recommendation model, obtaining the positive interest embedding features, negative interest embedding features and video clip embedding features of the user output by the graph convolutional neural network layer, wherein the graph convolutional neural network layer is used to perform feature node embedding propagation on the user features and the video clip visual features;
[0015] The positive interest embedding feature, the negative interest embedding feature and the video clip embedding feature are input into the fusion prediction layer of the video recommendation model to obtain the interest level output by the fusion prediction layer. The fusion prediction layer is used to fuse the first interest level of the positive interest embedding feature and the second interest level of the negative interest embedding feature calculated based on the video clip embedding feature.
[0016] According to a video recommendation method provided by the present invention, the user features and the video clip visual features are input into the graph convolutional neural network layer of the video recommendation model, and the positive interest embedding features, negative interest embedding features and video clip embedding features of the user output by the graph convolutional neural network layer are obtained, including:
[0017] Inputting the user features and the video clip visual features into a user aggregation layer of a graph convolutional neural network layer of the video recommendation model, and obtaining positive interest embedding features and negative interest embedding features of the user output by the user aggregation layer;
[0018] Inputting the positive interest embedding feature and the negative interest embedding feature into the video aggregation layer of the graph convolutional neural network layer to obtain the video segment embedding feature output by the video aggregation layer;
[0019] Among them, the user aggregation layer is used to perform positive interest clustering and negative interest clustering of the visual features of the video clip based on the user characteristics; the video aggregation layer is used to embed and propagate the user into the visual features of the video clip based on the positive interest embedding features and the negative interest embedding features.
[0020] According to a video recommendation method provided by the present invention, the step of inputting the positive interest embedding feature, the negative interest embedding feature and the video clip embedding feature into a fusion prediction layer of the video recommendation model to obtain the interest level output by the fusion prediction layer includes:
[0021] Inputting the video segment embedding feature into a first prediction layer of a fusion prediction layer of the video recommendation model to obtain a target video segment embedding feature output by the first prediction layer, wherein the first prediction layer is used to perform a weighted combination of the video segment embedding feature and the video segment visual feature;
[0022] Inputting the target video segment embedding feature, the positive interest embedding feature and the negative interest embedding feature into the second prediction layer of the fusion prediction layer, obtaining the first interest degree and the second interest degree output by the second prediction layer, wherein the second prediction layer is used to splice the positive interest embedding feature and the negative interest embedding feature with the target video segment embedding feature respectively and then perform multi-layer perceptual mapping;
[0023] The first interest degree and the second interest degree are input into the fusion layer of the fusion prediction layer to obtain the interest degree output by the fusion layer, and the fusion layer is used to perform average pooling processing on the first interest degree and the second interest degree.
[0024] According to a video recommendation method provided by the present invention, before inputting the user features and the video clip visual features into the graph convolutional neural network layer of the video recommendation model, the method further includes:
[0025] Inputting the visual features of the video clip into a feature enhancement embedding layer of the video recommendation model to obtain enhanced representation features of the video clip output by the feature enhancement embedding layer, wherein the feature enhancement embedding layer is used to enhance the visual features of the video clip based on a transformation matrix;
[0026] The step of inputting the user features and the video clip visual features into the graph convolutional neural network layer of the video recommendation model to obtain the positive interest embedding features, negative interest embedding features and video clip embedding features of the user output by the graph convolutional neural network layer comprises:
[0027] The user features and the video clip enhanced representation features are input into the graph convolutional neural network layer of the video recommendation model to obtain the user's positive interest embedding features, negative interest embedding features and video clip embedding features output by the graph convolutional neural network layer.
[0028] According to a video recommendation method provided by the present invention, the loss function during the training of the video recommendation model is obtained by weighted summing of the user preference loss function L1 and the video clip loss function L2;
[0029] The user preference loss function L1 is:
[0030]
[0031] The video clip loss function L2 is:
[0032]
[0033] Among them, U represents the user attribute information set, u represents a user attribute information in U, and N c represents the number of sample video segments divided from the sample original video, j represents the segment index, The jth sample video segment set among all the video segments in the complete sample video in the sample positive feedback video segment set of user u, represents the set of sample negative feedback video clips of user u, ω j is the penalty coefficient and ω j =j / N c , represents the interest of user u in sample video clip c, σ() represents the Sigmoid function, represents the set of sample videos that user u has not watched completely, and v represents An element in Represents the user u's interest in the positive feedback video clips in the sample video that has not been watched completely, Represents the interest of user u in the negative feedback video clips in the sample videos that have not been watched completely.
[0034] The present invention also provides a video recommendation device, comprising:
[0035] An acquisition module is used to acquire user attribute information and a set of videos to be recommended;
[0036] A division module, used for dividing each to-be-recommended video in the to-be-recommended video set into a preset number of video segments;
[0037] A feature extraction module, used to extract features from the user attribute information and the video clip respectively to obtain user features and video clip visual features;
[0038] A processing module, configured to input the user features and the visual features of the video clip into a video recommendation model, and obtain the user's interest in the video to be recommended corresponding to the user attribute information output by the video recommendation model;
[0039] A determination module, configured to determine a target recommended video from the set of videos to be recommended according to the interest level;
[0040] An output module, used for outputting the target recommended video to the user;
[0041] Among them, the video recommendation model is trained based on the sample original video corresponding to the user, the sample positive feedback video clip set and the sample negative feedback video clip set, the sample positive feedback video clip set is a set of video clips of the sample original video watched completely by the user, and the sample negative feedback video clip set is a set of video clips at the moment when the user's skipping behavior occurs.
[0042] The present invention also provides an electronic device, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor implements any of the above-mentioned video recommendation methods when executing the computer program.
[0043] The present invention also provides a non-transitory computer-readable storage medium, on which a computer program is stored. When the computer program is executed by a processor, the video recommendation method described above is implemented.
[0044] The present invention also provides a computer program product, comprising a computer program, wherein when the computer program is executed by a processor, the computer program implements any of the above-mentioned video recommendation methods.
[0045] The video recommendation method, device and electronic device provided by the present invention divide each video to be recommended into a preset number of video clips, and input the user features and video clip visual features extracted from the user attribute information and the video clip into a video recommendation model, so as to obtain the user's interest in the recommended video corresponding to the user attribute information output by the video recommendation model, and then determine the target recommended video from the set of videos to be recommended according to the interest, and recommend it to the user, thereby realizing video recommendation based on the user's interest. The video recommendation model is trained based on the sample original video corresponding to the user, the sample positive feedback video clip set and the sample negative feedback video clip set. The sample positive feedback video clip set is the set of video clips of the sample original video that the user has watched in full, and the sample negative feedback video clip set is the set of video clips at the moment when the user skipped. The samples used in model training fully consider the fine-grained interaction characteristics between the user and the entire video at the segment level, and can capture the user's interest differences in different parts of a whole video. Therefore, the trained video recommendation model can capture the user's interest in the video to be recommended from a fine-grained perspective at the video segment level, and the calculation of the interest is more accurate. Therefore, when recommending videos to users based on the interest output by the video recommendation model, videos that are more in line with the user's interests can be recommended, thereby improving the accuracy of video recommendations. BRIEF DESCRIPTION OF THE DRAWINGS
[0046] In order to more clearly illustrate the technical solutions in the present invention or the prior art, the following briefly introduces the drawings required for use in the embodiments or the description of the prior art. Obviously, the drawings described below are some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.
[0047] Figure 1 It is a flowchart of the video recommendation method provided by the present invention;
[0048] Figure 2 It is a flow chart of a method for obtaining user attribute information corresponding to the user's interest in the recommended video based on a video recommendation model provided by the present invention;
[0049] Figure 3 It is a structural schematic diagram of the video recommendation model provided by the present invention;
[0050] Figure 4 It is a schematic diagram of the principle of the training method of the video recommendation model provided by the present invention;
[0051] Figure 5 is a structural schematic diagram of a video recommendation device provided by the present invention;
[0052] Figure 6It is a structural schematic diagram of the electronic device provided by the present invention. DETAILED DESCRIPTION
[0053] In order to make the purpose, technical solution and advantages of the present invention clearer, the technical solution of the present invention will be clearly and completely described below in conjunction with the drawings of the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.
[0054] The video recommendation system can infer the user's interest based on the user's interaction record with the video, and provide the user with a personalized video recommendation list based on the user's interest. For example, it can be applied to short video platforms for video recommendation. The current video recommendation is mainly based on an immersive interface. Users can watch videos continuously and switch videos by sliding the screen, which reduces the cost of selecting content and watching. Unlike the user clicking to enter the video they want to watch, the interaction between the user and the video is continuous in this way. The user's browsing behavior can end at any position in the browsed video content, that is, the user can choose to skip at any video frame position in the browsed video content to end the viewing of the video content. The video recommendation system can characterize the user's behavior pattern and interest preferences by mining the interactive behaviors between the user and the video, such as watching, liking, commenting, and following. According to the user's interest preference, a candidate set that better meets the user's preference is selected from the candidate video set. The parameterized recommendation model estimates the probability that the user will watch the video, and then sorts it according to the probability and pushes the top-ranked videos to the user.
[0055] Among the various interactive behaviors between users and videos, watching and skipping behaviors are the most important user feedback. On the one hand, users will watch longer for content that they are interested in, while they will skip content that they are not interested in. On the other hand, other interactive behaviors, such as likes and comments, are relatively sparse and few in number, which are not enough to complete the learning of user interest preferences. Therefore, how to evaluate users' preferences for videos based on their watching and skipping behaviors and realize personalized video recommendations is of great significance in improving the accuracy of video recommendations and optimizing user experience. The single-column full-screen mode is the mainstream mode of video recommendation. In the single-column full-screen mode, users can watch videos continuously and switch video content by sliding up and down. Different from the display feedback mode in which users select the video content they are interested in and click to watch it, users tend to watch the video sequence recommended by the platform in a more accepting manner. The user's viewing behavior is a weak feedback type that reflects user preferences, which also brings new challenges to modeling user interests.
[0056] In the related art, the video recommendation system can obtain the representation of user interests by focusing on the dynamics and diversity of user interests, distinguishing and aggregating the videos of user historical interactions according to content and timing, or modeling user interests by using different modes of videos, and modeling user interests by using historical interaction sequences and possible future interaction sequences. These methods use relatively coarse-grained positive and negative samples when modeling user interests, that is, the entire video is regarded as a whole, and it is assumed that the user's content preferences for different parts of a video are consistent. However, users often have different preferences for different elements or different clips of the video, and not all the pictures of the video will be watched completely by the user. Therefore, using the entire video as a unit for modeling user interests will have certain deviations, and the result is a coarse-grained user portrait. The recommendation system based on coarse-grained user interests will also lead to more videos that do not meet the user's real interests being recommended to users, and the accuracy of video recommendations will deteriorate, affecting the user experience.
[0057] Unlike users who select the video content they are interested in and click to watch it, in single-column full-screen mode, the video content that users watch is mainly determined by the video recommendation platform. Users tend to watch the video sequence recommended by the platform in a receptive manner. In this process, the interaction between users and videos is continuous and fine-grained, that is, the interaction between users and videos is at the frame level or segment level. Based on this fine-grained interaction feature between users and videos, if the videos that users interact with are divided into fine-grained groups to capture the fine-grained changes in users' interests, and a video recommendation model is built based on this, it will help improve the accuracy of the recommendation model.
[0058] Based on this, an embodiment of the present invention provides a video recommendation method, which can divide each to-be-recommended video in a to-be-recommended video set into a preset number of video clips, perform feature extraction on user attribute information and video clips, obtain user features and video clip visual features, then input the user features and video clip visual features into a video recommendation model, obtain the user attribute information output by the video recommendation model and the user's interest in the recommended video, then determine a target recommended video from the to-be-recommended video set according to the interest, and output the target recommended video to the user; wherein the video recommendation model can be trained based on a sample original video corresponding to the user, a sample positive feedback video clip set, and a sample negative feedback video clip set, wherein the sample positive feedback video clip set is a set of video clips of the sample original video that the user has watched completely, and the sample negative feedback video clip set is a set of video clips at which the user's skipping behavior occurs.
[0059] Combine the following Figure 1-Figure 4The video recommendation method of the present invention is described. The video recommendation method can be applied to electronic devices such as servers, mobile phones, and computers, and can also be applied to video recommendation devices set in electronic devices such as servers, mobile phones, and computers. The video recommendation device can be implemented by software or a combination of software and hardware.
[0060] Figure 1 The schematic diagram of the process of the video recommendation method provided by the embodiment of the present invention is exemplified. Figure 1 As shown, the video recommendation method may include the following steps 110 to 150.
[0061] Step 110: Obtain user attribute information and a set of videos to be recommended.
[0062] User attribute information may include user identity information and other characteristic information of the user, such as gender, age and other characteristic information, but is not limited thereto.
[0063] Step 120: Divide each to-be-recommended video in the to-be-recommended video set into a preset number of video segments.
[0064] The set of videos to be recommended may include multiple videos to be recommended. Each video to be recommended in the set may be divided into a preset number of video segments, for example, 20 video segments.
[0065] Step 130: extracting features from the user attribute information and the video clip respectively to obtain user features and video clip visual features.
[0066] After obtaining the user attribute information, feature extraction can be performed on the user attribute information to obtain user features that characterize the user attribute information. For the divided video segments, visual features of the video segments can be obtained through feature extraction.
[0067] Exemplarily, for each video segment, video frame sampling may be performed, and visual features may be extracted from each captured frame of video image to obtain video segment visual features of the video segment.
[0068] For example, a pre-trained convolutional neural network (CNN) model can be used as a visual encoder, such as the deep residual network model ResNet-50. Specifically, the visual encoder can retain the first 5 CNN layers of the pre-trained ResNet-50 model and add a pooling layer to obtain the feature vector of the video clip, and add a fully connected layer to reduce the dimension of the visual features. For each video clip, a fusion layer can be used to aggregate the frames in the video clip to obtain the original features of the video clip, which are visual features. The fusion layer can be used for average pooling.
[0069] Step 140: Input the user features and the visual features of the video clip into the video recommendation model to obtain the user attribute information output by the video recommendation model corresponding to the user's interest in the recommended video.
[0070] After obtaining the user features and the visual features of each video clip, the user features and the visual features of all video clips can be input into the video recommendation model. The visual recommendation model can calculate the score of each user's interest in each video clip based on the user features. For each user, the scores of video clips from the same video to be recommended are finally fused, such as weighted fusion, to obtain the user's interest in the video to be recommended. In this way, the interest of each user in each video to be recommended represented by the user features can be obtained.
[0071] Exemplarily, weighted fusion may be to assign a first weight to scores less than a score threshold, and to assign a second weight to scores greater than the score threshold, and then to fuse the weighted scores, such as summing them up and performing average pooling, to obtain the user's interest in the video to be recommended.
[0072] The video recommendation model can be trained based on the sample original video, sample positive feedback video clip set and sample negative feedback video clip set corresponding to the user. The sample positive feedback video clip set is a set of video clips of the sample original video that the user has watched completely, and the sample negative feedback video clip set is a set of video clips at the moment when the user skipped.
[0073] Exemplarily, for the sample original video, the historical interactive video data of the user corresponding to the user attribute information can be obtained. The historical interactive video data is the original video file data of the video of the user interaction, which carries all the information of the video and can support fine-grained division processing. While recording the sample original video, the user's interactive record information can also be recorded. The interactive record information may include the user's attribute information, the identification information of the sample original video, the user's viewing time information for each sample original video, the user's viewing behavior for each sample original video, and the duration of each sample original video. According to the interactive record information, the user's viewing situation of each sample original video in the sample original video can be determined, so as to divide the video clips and classify the positive and negative sample video clips.
[0074] For example, each sample original video can be uniformly divided into 5 video segments. When the user watches the sample original video A, if he chooses to skip when watching the second video segment of A, the second video segment can be used as a sample negative feedback video segment. If all 5 video segments of the sample original video A are watched, the 5 video segments are used as sample positive feedback video segments. It can be understood that for any sample original video, as long as it has an unwatched video segment, the 5 video segments of the sample original video will not be used as sample positive feedback video segments; for any video segment, as long as a skipping behavior occurs on the video segment, the video segment will be used as a sample negative feedback video segment. In this way, a sample positive feedback video segment set and a sample negative feedback video segment set can be determined from the sample original video. In this way, a fine-grained training sample can be obtained. The training sample fully considers the fine-grained interaction characteristics of the segment level between the user and the entire sample original video, and can capture the user's interest difference in different parts of the content of a whole video, so that the trained video recommendation model can capture the user's interest in the video to be recommended from a fine-grained perspective at the video segment level.
[0075] Step 150: Determine a target recommended video from the set of videos to be recommended according to the interest level, and output the target recommended video to the user.
[0076] The interest level can represent the probability that the user has finished watching the video to be recommended, and reflects the user's interest level in the video to be recommended.
[0077] In an exemplary embodiment, after obtaining the user attribute information corresponding to the user's interest in the recommended videos, for each user, the user's interest in the recommended videos can be sorted, such as sorting from large to small, and then the recommended videos corresponding to the first set number of interest levels in the sorting result are determined as target recommended videos, such as determining the recommended videos corresponding to the first 10 interest levels as target recommended videos.
[0078] In another example embodiment, after obtaining the user's interest in the recommended video corresponding to the user attribute information, for each user, an interest level greater than an interest level threshold can be determined from the user's interest in the recommended video to obtain a target interest level, and the video to be recommended corresponding to the target interest level in the set of videos to be recommended is determined as the target recommended video.
[0079] The video recommendation method provided by the embodiment of the present invention divides each video to be recommended into a preset number of video segments, and inputs the user features and video segment visual features extracted from the user attribute information and the video segment into a video recommendation model. The user attribute information output by the video recommendation model can obtain the user's interest in the recommended video corresponding to the user, and then according to the interest, a target recommended video can be determined from the set of videos to be recommended and recommended to the user, thereby realizing video recommendation based on the user's interest. The video recommendation model is trained based on the sample original video corresponding to the user, the sample positive feedback video clip set and the sample negative feedback video clip set. The sample positive feedback video clip set is the set of video clips of the sample original video that the user has watched in full, and the sample negative feedback video clip set is the set of video clips at the moment when the user skipped. The samples used in model training fully consider the fine-grained interaction characteristics between the user and the entire video at the segment level, and can capture the user's interest differences in different parts of a whole video. Therefore, the trained video recommendation model can capture the user's interest in the video to be recommended from a fine-grained perspective at the video segment level, and the calculation of the interest is more accurate. Therefore, when recommending videos to users based on the interest output by the video recommendation model, videos that are more in line with the user's interests can be recommended, thereby improving the accuracy of video recommendations.
[0080] based on Figure 1 The video recommendation method of the corresponding embodiment, Figure 2 A flow chart of a method for obtaining user attribute information corresponding to a user's interest in a recommended video based on a video recommendation model provided in an embodiment of the present invention is exemplified. The method may include the following steps 210 to 220, that is, step 140 may include the following steps 210 to 220.
[0081] Step 210: Input the user features and the video clip visual features into the graph convolutional neural network layer of the video recommendation model to obtain the user's positive interest embedding features, negative interest embedding features, and video clip embedding features output by the graph convolutional neural network layer.
[0082] Among them, the graph convolutional neural network layer can be used to embed and propagate feature nodes for user features and video clip visual features. The graph convolutional neural network layer can be a graph convolutional neural network (GCN) model, which can study the representation of graph nodes and extract features from graph data. The visual features of video clips contain the video image data of the video clips. The graph convolutional neural network model can be used to aggregate the information of the neighboring nodes of the video image data. The embedding propagation mechanism of the graph convolutional neural network model can be used to obtain the user's positive interest embedding features and negative interest embedding features. The positive interest embedding features and negative interest embedding features can respectively reflect the user's positive preferences and negative preferences.
[0083] At the same time, the graph convolutional neural network layer can also embed and propagate the visual features of video clips to users based on positive interest embedding features and negative interest embedding features, so as to explore the similarity characteristics between video clips, capture high-order information transmission between video clips, and obtain video clip embedding features.
[0084] Exemplarily, step 210 may include the following steps: inputting user features and video clip visual features into the user aggregation layer of the graph convolutional neural network layer of the video recommendation model, and obtaining the user's positive interest embedding features and negative interest embedding features output by the user aggregation layer; inputting the positive interest embedding features and negative interest embedding features into the video aggregation layer of the graph convolutional neural network layer, and obtaining the video clip embedding features output by the video aggregation layer. The user aggregation layer is used to cluster positive interests and negative interests of the visual features of the video clips based on the user features, wherein the positive interests represent the user's interested features and the negative interests represent the user's uninterested features; the video aggregation layer is used to embed and propagate the user into the visual features of the video clips based on the positive interest embedding features and the negative interest embedding features.
[0085] Step 220: input the positive interest embedding features, the negative interest embedding features and the video clip embedding features into the fusion prediction layer of the video recommendation model to obtain the interest level output by the fusion prediction layer.
[0086] Among them, the fusion prediction layer can be used to fuse the first interest level of the positive interest embedding feature and the second interest level of the negative interest embedding feature calculated based on the video clip embedding feature.
[0087] Specifically, for each user, the fusion prediction layer can splice the video clip embedding features and the user's positive interest embedding features together, and calculate the first interest level at the video clip level through a multilayer perceptron (MLP). At the same time, the video clip embedding features and the user's negative interest embedding features can be spliced together, and the second interest level at the video clip level can be calculated through a multilayer perceptron. Then, the first interest level and the second interest level can be weightedly summed and averaged pooled to obtain the interest level output by the fusion prediction layer.
[0088] Exemplarily, step 220 can be implemented by the following steps: inputting the video segment embedding feature into the first prediction layer of the fusion prediction layer of the video recommendation model to obtain the target video segment embedding feature output by the first prediction layer; inputting the target video segment embedding feature, the positive interest embedding feature and the negative interest embedding feature into the second prediction layer of the fusion prediction layer to obtain the first interest level and the second interest level output by the second prediction layer; inputting the first interest level and the second interest level into the fusion layer of the fusion prediction layer to obtain the interest level output by the fusion layer. Among them, the first prediction layer can be used to perform a weighted combination of the video segment embedding feature and the video segment visual feature; the second prediction layer can be used to perform multi-layer perceptual mapping after splicing the positive interest embedding feature and the negative interest embedding feature with the target video segment embedding feature; the fusion layer can be used to average pool the first interest level and the second interest level.
[0089] according to Figure 2 The method of the corresponding embodiment can obtain the positive interest preference features and negative interest preference features of each user for each video clip through the graph convolutional neural network layer, and can capture the high-order information transmission between video clips to obtain the video clip embedding features. The fusion prediction layer can then fuse each user's interest in each video to be recommended from the video clip level based on these features output by the graph convolutional neural network layer, and can predict the probability of the user finishing watching the video to be recommended from the perspective of fine-grained video content, thereby improving the accuracy of the prediction.
[0090] In an exemplary embodiment of the present invention, the video recommendation model may further include a feature enhancement embedding layer, which may be used to enhance the visual features of the video clip based on a transformation matrix. In video recommendation, user interaction behavior may not be entirely affected by visual features. For example, the background scene of a frame of video image, such as audio, text, etc., may also affect user behavior. In this regard, the extracted visual features of the video clip can be input into the feature enhancement embedding layer while the embedding dimension remains unchanged. The extracted visual features of the video clip are multiplied by a transformation matrix through the feature enhancement embedding layer. The transformation matrix is used to add some text, voice and other information to the visual features of the video clip to obtain embedding in the user preference space, thereby further improving the accuracy of video recommendation.
[0091] Specifically, the following feature enhancement formula can be used to enhance the visual features of the video clip while keeping the embedding dimension unchanged. The feature enhancement formula can be expressed as:
[0092] E c =W t f c .
[0093] Among them, f c The visual features of the video segment c, E c Represents the visual features of the video clip f c The enhanced representation feature of the video clip obtained after enhancement, W t represents the transformation matrix, and W t It is learnable and can be trained when training the video recommendation model.
[0094] Based on this, before inputting the user features and the video clip visual features into the graph convolutional neural network layer of the video recommendation model, the video recommendation method provided by the embodiment of the present invention may also include: inputting the video clip visual features into the feature enhancement embedding layer of the video recommendation model to obtain the video clip enhanced representation features output by the feature enhancement embedding layer. Accordingly, step 210 may be inputting the user features and the video clip enhanced representation features into the graph convolutional neural network layer of the video recommendation model to obtain the user's positive interest embedding features, negative interest embedding features, and video clip embedding features output by the graph convolutional neural network layer.
[0095] Based on the methods of the above embodiments, Figure 3 The schematic diagram of the structure of the video recommendation model provided by the embodiment of the present invention is exemplified, and the video recommendation model and the video recommendation method described above can be compared with each other for reference. Figure 3As shown, the video recommendation model may include a graph convolutional neural network layer and a fusion prediction layer. The graph convolutional neural network layer can be used to embed and propagate feature nodes of user features and video clip visual features to obtain the user's positive interest embedding features, negative interest embedding features and video clip embedding features; the fusion prediction layer can be used to fuse the first interest degree of the positive interest embedding feature and the second interest degree of the negative interest embedding feature calculated based on the video clip embedding features to obtain the user's interest degree in the recommended video.
[0096] Exemplarily, the graph convolutional neural network layer may include a user aggregation layer and a video aggregation layer. The user aggregation layer may be used to cluster positive interests and negative interests of visual features of video clips based on user features; the video aggregation layer may be used to embed and propagate users into visual features of video clips based on positive interest embedding features and negative interest embedding features. The fusion prediction layer may include a first prediction layer, a second prediction layer, and a fusion layer. The first prediction layer may be used to perform a weighted combination of video clip embedding features and video clip visual features. The second prediction layer may be used to perform multi-layer perceptual mapping after splicing positive interest embedding features and negative interest embedding features with target video clip embedding features. The fusion layer may be used to perform average pooling processing on the first interest level and the second interest level.
[0097] Exemplarily, the video recommendation model may further include a feature enhancement embedding layer, and the visual features of the video clip may be first input into the feature enhancement embedding layer, which enhances the visual features of the video clip based on a transformation matrix, and then the user features and the enhanced representation features of the video clip output by the feature enhancement embedding layer are input into the graph convolutional neural network layer.
[0098] Based on the methods of the above embodiments, Figure 4 The schematic diagram of the principle of the training method of the video recommendation model is shown as an example. Figure 4 As shown, the video recommendation model can include a feature enhancement embedding layer, a graph convolutional neural network layer, and a fusion prediction layer.
[0099] Before training the video recommendation model, each sample original video can be divided, for example, into N (N is a positive integer greater than 1) sample video clips, and each sample video clip is sampled to obtain the video frame corresponding to each sample video clip, and then the video frame is visually encoded to extract the frame-level visual features of each sample video clip, and the visual features of each video frame corresponding to each sample video clip can be fused by the time series fusion method to obtain the sample video clip visual features corresponding to each sample video clip. In this way, the fine-grained original features of each sample original video can be extracted.
[0100] The sample original video can be obtained by collecting the user's historical interactive video data. The historical interactive video data is the original video file data of the video of the user's interaction, which carries all the information of the video and can support fine-grained segmentation processing. For example, before the sample original video is segmented, data preprocessing can be performed to filter out invalid sample original videos and corresponding interaction records to improve the quality of training samples. The interaction record information can represent the user's viewing of each sample original video.
[0101] At the same time, based on the user's viewing behavior of the sample original video, the sample video clips that the user has interacted with can be divided into a sample positive feedback video clip set and a sample negative feedback video clip set according to whether they are completed or skipped, and label information is added to each. Specifically, the sample positive feedback video clip set is a set of video clips of the sample original video that the user has watched completely, and the sample negative feedback video clip set is a set of video clips at the moment when the user skipped. It can be understood that in the process of obtaining the sample positive feedback video clip set and the sample negative feedback video clip set, the user attribute information corresponding to the sample is obtained.
[0102] Afterwards, the extracted visual features of the sample video clips can be input into the feature enhancement embedding layer, and the set of sample positive feedback video clips and the set of sample negative feedback video clips with added label information can be input into the graph convolutional neural network layer, and the video recommendation model can be trained using a hybrid supervised learning training method.
[0103] Specifically, in the feature enhancement embedding layer, the visual features of the sample video clip can be enhanced based on the above feature enhancement formula to obtain the enhanced representation features of the sample video clip: Assume that the sample positive feedback video clip set of user u is The set of sample negative feedback video clips of user u is based on and In the graph convolutional neural network layer, the positive interest interaction relationship matrix R between users and sample video clips can be constructed p And negative interest interaction matrix R n , the elements in the matrix can be defined as follows:
[0104]
[0105]
[0106] in, Represents R p The elements in Represents R n The elements in c j represents the jth sample video clip, Represents the i-th user u i The corresponding set of sample positive feedback video clips, Represents the i-th user u i The set of corresponding sample negative feedback video clips.
[0107] Furthermore, we can use R p The adjacency matrix A p and R n The adjacency matrix A n Characterize the connection relationship between users and sample video clips.
[0108] Specifically, Among them, (R p ) T For R p The transpose of (R n ) T For R n The transpose of .
[0109] Furthermore, we can p and A n Perform normalization processing respectively to obtain the corresponding normalization processing results and
[0110]
[0111]
[0112] Among them, D p Representative A p The degree matrix, D n Representative A n The degree matrix of Represents R p The normalized result is Represents R n The normalized result is and Respectively represent and The transpose of .
[0113] Next, based on the normalized results, the GCN model can be used to aggregate neighbor node information to learn the user's positive interest representation, negative interest representation, and embedded representation of sample video clips. At the same time, the user's sample positive interest embedding features can be obtained through the embedding propagation mechanism in the GCN model. And the sample negative interest embedding feature
[0114]
[0115]
[0116] in, is the enhanced representation feature of the sample video clip output by the feature enhancement embedding layer; σ() represents the Sigmoid function, which is used as the activation function of the graph convolutional neural network layer; and is a trainable weight matrix. Based on this, we can achieve Figure 4 The video clip-user propagation in the video clip can capture the user's positive and negative interest preferences for the sample video clips.
[0117] In addition to the association between users and sample video clips, there is also high-order transmission information between sample video clips. For example, two sample video clips may be connected by the same user in a positive interest or negative interest relationship graph. In this case, the two sample video clips may have similar characteristics. In this case, the relationship between sample video clips can be mined through two-hop embedding propagation in the graph convolutional neural network layer. Figure 4 The user-video clip propagation shown in FIG. 4 can improve the effect of video recommendation model training and the accuracy of video recommendation.
[0118] Specifically, the embedding features of the first sub-sample video clip can be obtained through user-video clip propagation and the second sub-sample video clip embedding features
[0119]
[0120]
[0121] in, and is a trainable weight matrix, σ() represents the Sigmoid function, which is used as the activation function of the graph convolutional neural network layer.
[0122] Next, the graph convolutional neural network layer can be used to and Perform average pooling to obtain the final representation of the sample video clip, that is, obtain the embedding features of the sample video clip For example, the average pooling formula can be get The Mean() here represents average pooling.
[0123] At this point, the enhanced representation features of the sample video clip can be obtained and sample video clip embedding features They carry different information of the sample video clips and can be input into the fusion prediction layer to form the final content representation of the sample video clips.
[0124] Furthermore, in the fusion prediction layer, the coefficients can be weighted and adaptively combined. and Get the target video segment embedding feature H c Specifically, we can use the following weighted combination formula to and Perform a weighted combination:
[0125]
[0126] Among them, α0 and α1 are weighting coefficients respectively.
[0127] The fusion prediction layer simultaneously receives the sample positive interest embedding features output by the graph convolutional neural network layer And the sample negative interest embedding feature These two features can be embedded into the target video clip feature H c spliced together, and then the multi-layer perceptron is used to predict the positive and negative interests at the segment level to obtain the first interest level and the second interest level of each sample video segment. Specifically, the following multi-layer perceptron mechanism can be used to obtain the user u's interest in the i-th sample video segment c in the sample video c i First interest and user u for the i-th sample video segment c in the sample video c i Second interest
[0128]
[0129]
[0130] in, express Features of a sample video clip in express Features of a sample video clip in Indicates H c The i-th sample video clip c in i Features, Representatives and To splice, Representatives and For splicing, MLP p () and MLP n () represent multi-layer perceptual mappings respectively.
[0131] Afterwards, the fusion prediction layer can be used to and Perform average pooling to obtain the user u’s interest in sample video c Specifically, the fusion prediction layer can first pass a set of weight coefficients α p and α n right and Fusion is performed to obtain the i-th sample video segment c in sample video c for user u i The final prediction result can be recorded as the estimated interest Exemplarily, it can be obtained based on the following interest fusion formula The interest fusion formula is:
[0132]
[0133] Using subtraction operations during fusion can reduce the negative impact of negative interest preferences on the final prediction results and improve the accuracy of the prediction.
[0134] Afterwards, the fusion prediction layer can fuse the final prediction results of each sample video clip to obtain the interest of user u in sample video c. Specifically, the fusion prediction layer can obtain the probability that user u has finished watching sample video c through the following average pooling formula, that is, obtain the interest of user u in sample video c The average pooling formula is:
[0135]
[0136] Where N c The number of sample video segments into which the sample video c is divided.
[0137] It is understandable that the sample video clips used in the training of the video recommendation model may be historical interaction videos from multiple users. According to the above principle, the video recommendation model may learn the interest of each user in each sample video.
[0138] Furthermore, based on the above Figure 4 Description of the principle: When training a video recommendation model, the user preference loss function L1 and the video clip loss function L2 can be weighted and summed to obtain the loss function L during video recommendation model training. Exemplarily, the loss function L can be expressed as:
[0139] L = αL1 + βL2 + λ||θ||2;
[0140] Among them, α, β and λ are weighting coefficients respectively, and ||θ||2 represents the regularization term.
[0141] Specifically, based on the user preference loss function L1, the user's interest preferences can be learned from fine-grained sample video clips. The sample videos that the user has watched in full reflect the user's positive interest preferences to a certain extent, and the sample video clips of these sample videos can be used as positive samples; while the sample video clips where the user skips and the sample video clips nearby reflect the user's negative interest preferences to a certain extent, and the sample video clips where the skipping behavior occurs can be used as negative samples. As for the sample video clips after the skipping behavior occurs, the user's interest tendency cannot be determined and can be discarded. However, this method still ignores the difference in the strength of the positive samples. Based on this, different penalty coefficients can be set for sample video clips at different positions of the same sample video, and the strength of the positive samples can be distinguished by the penalty coefficients. Specifically, the penalty coefficient ω of the jth sample video clip can be set j ω j =j / N c , which indicates that as the user's browsing time increases, when the jth sample video clip is defined as a positive sample, it can obtain a higher confidence.
[0142] In addition, in the same interactive sample video, the sample video clips skipped by the user always reflect more negative interest preferences than the part watched by the user. The part watched by the user may be composed of several sample video clips, which can be merged as a whole to represent the relatively positive part. Exemplarily, the Bayesian personalized ranking (BPR) loss can be used to process the loss between sample video clips, so that the video recommendation model can distinguish between sample video clips with positive interest preferences and sample video clips with negative interest preferences in the same interactive sample video. That is, for each user fine-grained interaction record (u, v, r), there is a pair of positive and negative samples, where u represents the user, v represents the sample video watched by user u, and r represents the percentage of the part of video v watched by user u, r<1.
[0143] On this basis, the user preference loss function L1 and the video clip loss function L2 can be defined as:
[0144]
[0145]
[0146] Among them, U represents the user attribute information set, u represents a user attribute information in U, and N c represents the number of sample video segments divided from the sample original video, j represents the segment index, The jth sample video segment set among all the video segments in the complete sample video in the sample positive feedback video segment set of user u, represents the set of sample negative feedback video clips of user u, ωj is a punishment system and ω j =j / N c , represents the interest of user u in sample video clip c, σ() represents the Sigmoid function, represents the set of sample videos that user u has not watched completely, and v represents An element in Represents the user u's interest in the positive feedback video clips in the sample video that has not been watched completely, Represents the interest of user u in the negative feedback video clips in the sample videos that have not been watched completely.
[0147] During the training process of the video recommendation model, the loss function L can be continuously optimized until the value of L reaches the minimum.
[0148] The video recommendation device provided by the present invention is described below. The video recommendation device described below and the video recommendation method described above can be referred to each other.
[0149] Figure 5 The schematic diagram of the structure of the video recommendation device provided by the embodiment of the present invention is exemplarily shown. Figure 5 As shown, the video recommendation apparatus 500 may include an acquisition module 510 , a division module 520 , a feature extraction module 530 , a processing module 540 , a determination module 550 and an output module 560 . Among them: the acquisition module 510 can be used to obtain user attribute information and a set of videos to be recommended; the division module 520 can be used to divide each video to be recommended in the set of videos to be recommended into a preset number of video clips; the feature extraction module 530 can be used to extract features from the user attribute information and video clips respectively to obtain user features and video clip visual features; the processing module 540 can be used to input the user features and video clip visual features into the video recommendation model to obtain the user's interest in the recommended video corresponding to the user attribute information output by the video recommendation model; the determination module 550 can be used to determine the target recommended video from the set of videos to be recommended according to the interest; the output module 560 can be used to output the target recommended video to the user; wherein the video recommendation model is trained based on the sample original video corresponding to the user, the sample positive feedback video clip set and the sample negative feedback video clip set, the sample positive feedback video clip set is a set of video clips of the sample original video that the user has watched completely, and the sample negative feedback video clip set is a set of video clips at the moment when the user skipped.
[0150] In an example embodiment, the processing module 540 may include: a first processing unit, which can be used to input user features and video clip visual features into a graph convolutional neural network layer of a video recommendation model to obtain the user's positive interest embedding features, negative interest embedding features and video clip embedding features output by the graph convolutional neural network layer, and the graph convolutional neural network layer is used to embed and propagate feature nodes of the user features and video clip visual features; a second processing unit, which can be used to input the positive interest embedding features, negative interest embedding features and video clip embedding features into a fusion prediction layer of the video recommendation model to obtain the interest level output by the fusion prediction layer, and the fusion prediction layer is used to fuse the first interest level of the positive interest embedding feature and the second interest level of the negative interest embedding feature calculated based on the video clip embedding feature.
[0151] In an example embodiment, the first processing unit may include: a first processing sub-unit, which can be used to input user features and video clip visual features into the user aggregation layer of the graph convolutional neural network layer of the video recommendation model, and obtain the user's positive interest embedding features and negative interest embedding features output by the user aggregation layer; a second processing sub-unit, which can be used to input the positive interest embedding features and the negative interest embedding features into the video aggregation layer of the graph convolutional neural network layer, and obtain the video clip embedding features output by the video aggregation layer; wherein the user aggregation layer is used to perform positive interest clustering and negative interest clustering of video clip visual features based on user features; and the video aggregation layer is used to embed and propagate users into video clip visual features based on the positive interest embedding features and the negative interest embedding features.
[0152] In an example embodiment, the second processing unit may include: a third processing sub-unit, which can be used to input the video clip embedding feature into the first prediction layer of the fusion prediction layer of the video recommendation model to obtain the target video clip embedding feature output by the first prediction layer, and the first prediction layer is used to perform weighted combination of the video clip embedding feature and the video clip visual feature; a fourth processing sub-unit, which can be used to input the target video clip embedding feature, the positive interest embedding feature and the negative interest embedding feature into the second prediction layer of the fusion prediction layer to obtain the first interest degree and the second interest degree output by the second prediction layer, and the second prediction layer is used to splice the positive interest embedding feature and the negative interest embedding feature with the target video clip embedding feature respectively and then perform multi-layer perceptual mapping; a fifth processing sub-unit, which can be used to input the first interest degree and the second interest degree into the fusion layer of the fusion prediction layer to obtain the interest degree output by the fusion layer, and the fusion layer is used to perform average pooling processing on the first interest degree and the second interest degree.
[0153] In an exemplary embodiment, the processing module 540 may further include a third processing unit, which may be used to input the visual features of the video clip into a feature enhancement embedding layer of a video recommendation model to obtain the video clip enhanced representation features output by the feature enhancement embedding layer, and the feature enhancement embedding layer is used to enhance the visual features of the video clip based on a transformation matrix; the first processing unit may be specifically used to input the user features and the video clip enhanced representation features into a graph convolutional neural network layer of the video recommendation model to obtain the user's positive interest embedding features, negative interest embedding features and video clip embedding features output by the graph convolutional neural network layer.
[0154] In an exemplary embodiment, the loss function during video recommendation model training is obtained by weighted summing of the user preference loss function L1 and the video segment loss function L2; wherein the user preference loss function L1 is:
[0155]
[0156] The video clip loss function L2 is:
[0157]
[0158] Among them, U represents the user attribute information set, u represents a user attribute information in U, and N c represents the number of sample video segments divided from the sample original video, j represents the segment index, The jth sample video segment set among all the video segments in the complete sample video in the sample positive feedback video segment set of user u, represents the set of sample negative feedback video clips of user u, ω j is the penalty coefficient and ω j =j / N c , represents the interest of user u in sample video clip c, σ() represents the Sigmoid function, represents the set of sample videos that user u has not watched completely, and v represents An element in Represents the user u's interest in the positive feedback video clips in the sample video that has not been watched completely, Represents the interest of user u in the negative feedback video clips in the sample videos that have not been watched completely.
[0159] Figure 6 An example of a physical structure diagram of an electronic device is shown in FIG. Figure 6As shown, the electronic device 600 may include: a processor (processor) 610, a communication interface (Communication Interface) 620, a memory (memory) 630 and a communication bus 640, wherein the processor 610, the communication interface 620, and the memory 630 communicate with each other through the communication bus 640. The processor 610 can call the logic instructions in the memory 630 to execute the video recommendation method provided by the above-mentioned method embodiments, which may include: obtaining user attribute information and a set of videos to be recommended; dividing each video to be recommended in the set of videos to be recommended into a preset number of video clips; performing feature extraction on the user attribute information and the video clips respectively to obtain user features and visual features of the video clips; inputting the user features and the visual features of the video clips into a video recommendation model to obtain the user's interest in the recommended video corresponding to the user attribute information output by the video recommendation model; determining a target recommended video from the set of videos to be recommended based on the interest, and outputting the target recommended video to the user; wherein the video recommendation model is trained based on the sample original video corresponding to the user, the sample positive feedback video clip set and the sample negative feedback video clip set, the sample positive feedback video clip set is a set of video clips of the sample original video that the user has watched completely, and the sample negative feedback video clip set is a set of video clips at the moment when the user skipped.
[0160] In addition, the logic instructions in the above-mentioned memory 630 can be implemented in the form of a software functional unit and can be stored in a computer-readable storage medium when it is sold or used as an independent product. Based on such an understanding, the technical solution of the present invention, in essence, or the part that contributes to the prior art or the part of the technical solution, can be embodied in the form of a software product, and the computer software product is stored in a storage medium, including a number of instructions for a computer device (which can be a personal computer, a server, or a network device, etc.) to perform all or part of the steps of the method described in each embodiment of the present invention. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (ROM, Read-Only Memory), random access memory (RAM, Random Access Memory), disk or optical disk and other media that can store program codes.
[0161] On the other hand, the present invention also provides a computer program product, which includes a computer program, which can be stored on a non-transitory computer-readable storage medium. When the computer program is executed by a processor, the computer can execute the video recommendation method provided by the above-mentioned method embodiments, which may include: obtaining user attribute information and a set of videos to be recommended; dividing each video to be recommended in the set of videos to be recommended into a preset number of video clips; performing feature extraction on the user attribute information and the video clips respectively to obtain user features and visual features of the video clips; inputting the user features and the visual features of the video clips into a video recommendation model to obtain the user's interest in the recommended video corresponding to the user attribute information output by the video recommendation model; determining a target recommended video from the set of videos to be recommended based on the interest, and outputting the target recommended video to the user; wherein the video recommendation model is trained based on the sample original video corresponding to the user, the sample positive feedback video clip set and the sample negative feedback video clip set, the sample positive feedback video clip set is a set of video clips of the sample original video that the user has watched completely, and the sample negative feedback video clip set is a set of video clips at the moment when the user skipped.
[0162] On the other hand, the present invention also provides a non-transitory computer-readable storage medium having a computer program stored thereon, which is implemented by a processor to execute the video recommendation method provided by the above-mentioned method embodiments when the computer program is executed, and the method may include: obtaining user attribute information and a set of videos to be recommended; dividing each video to be recommended in the set of videos to be recommended into a preset number of video clips; performing feature extraction on the user attribute information and the video clips respectively to obtain user features and visual features of the video clips; inputting the user features and the visual features of the video clips into a video recommendation model to obtain the user's interest in the recommended video corresponding to the user attribute information output by the video recommendation model; determining a target recommended video from the set of videos to be recommended based on the interest, and outputting the target recommended video to the user; wherein the video recommendation model is trained based on the sample original video corresponding to the user, the sample positive feedback video clip set and the sample negative feedback video clip set, the sample positive feedback video clip set is a set of video clips of the sample original video that the user has watched completely, and the sample negative feedback video clip set is a set of video clips at the moment when the user skipped.
[0163] The device embodiments described above are merely illustrative, wherein the units described as separate components may or may not be physically separated, and the components displayed as units may or may not be physical units, that is, they may be located in one place, or they may be distributed on multiple network units. Some or all of the modules may be selected according to actual needs to achieve the purpose of the scheme of this embodiment. Ordinary technicians in this field can understand and implement it without paying creative labor.
[0164] Through the description of the above implementation methods, those skilled in the art can clearly understand that each implementation method can be implemented by means of software plus a necessary general hardware platform, and of course, it can also be implemented by hardware. Based on this understanding, the above technical solution is essentially or the part that contributes to the prior art can be embodied in the form of a software product, and the computer software product can be stored in a computer-readable storage medium, such as ROM / RAM, a disk, an optical disk, etc., including a number of instructions for a computer device (which can be a personal computer, a server, or a network device, etc.) to execute the methods described in each embodiment or some parts of the embodiments.
[0165] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit it. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or make equivalent replacements for some of the technical features therein. However, these modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the embodiments of the present invention.
Claims
1. A video recommendation method, characterized in that: include: Obtain user attribute information and a collection of videos to be recommended; Dividing each to-be-recommended video in the to-be-recommended video set into a preset number of video segments; Extracting features from the user attribute information and the video clip respectively to obtain user features and video clip visual features; Inputting the user features and the visual features of the video clip into a video recommendation model to obtain the interest of the user corresponding to the user attribute information output by the video recommendation model in the video to be recommended; Determining a target recommended video from the set of videos to be recommended according to the interest level, and outputting the target recommended video to the user; Among them, the video recommendation model is trained based on the sample original video corresponding to the user, the sample positive feedback video clip set and the sample negative feedback video clip set, the sample positive feedback video clip set is a set of video clips of the sample original video watched completely by the user, and the sample negative feedback video clip set is a set of video clips at the moment when the user's skipping behavior occurs.
2. The video recommendation method according to claim 1, characterized in that: The step of inputting the user features and the visual features of the video clip into a video recommendation model to obtain the interest of the user in the video to be recommended corresponding to the user attribute information output by the video recommendation model, comprises: Inputting the user features and the video clip visual features into the graph convolutional neural network layer of the video recommendation model, obtaining the positive interest embedding features, negative interest embedding features and video clip embedding features of the user output by the graph convolutional neural network layer, wherein the graph convolutional neural network layer is used to perform feature node embedding propagation on the user features and the video clip visual features; The positive interest embedding feature, the negative interest embedding feature and the video clip embedding feature are input into the fusion prediction layer of the video recommendation model to obtain the interest level output by the fusion prediction layer. The fusion prediction layer is used to fuse the first interest level of the positive interest embedding feature and the second interest level of the negative interest embedding feature calculated based on the video clip embedding feature.
3. The video recommendation method according to claim 2, characterized in that: The step of inputting the user features and the video clip visual features into the graph convolutional neural network layer of the video recommendation model to obtain the positive interest embedding features, negative interest embedding features and video clip embedding features of the user output by the graph convolutional neural network layer comprises: Inputting the user features and the video clip visual features into a user aggregation layer of a graph convolutional neural network layer of the video recommendation model, and obtaining positive interest embedding features and negative interest embedding features of the user output by the user aggregation layer; Inputting the positive interest embedding feature and the negative interest embedding feature into the video aggregation layer of the graph convolutional neural network layer to obtain the video segment embedding feature output by the video aggregation layer; Among them, the user aggregation layer is used to perform positive interest clustering and negative interest clustering of the visual features of the video clip based on the user characteristics; the video aggregation layer is used to embed and propagate the user into the visual features of the video clip based on the positive interest embedding features and the negative interest embedding features.
4. The video recommendation method according to claim 2 or 3, characterized in that: The step of inputting the positive interest embedding feature, the negative interest embedding feature, and the video clip embedding feature into a fusion prediction layer of the video recommendation model to obtain the interest level output by the fusion prediction layer includes: Inputting the video segment embedding feature into a first prediction layer of a fusion prediction layer of the video recommendation model to obtain a target video segment embedding feature output by the first prediction layer, wherein the first prediction layer is used to perform a weighted combination of the video segment embedding feature and the video segment visual feature; Inputting the target video segment embedding feature, the positive interest embedding feature and the negative interest embedding feature into the second prediction layer of the fusion prediction layer, obtaining the first interest degree and the second interest degree output by the second prediction layer, wherein the second prediction layer is used to splice the positive interest embedding feature and the negative interest embedding feature with the target video segment embedding feature respectively and then perform multi-layer perceptual mapping; The first interest degree and the second interest degree are input into the fusion layer of the fusion prediction layer to obtain the interest degree output by the fusion layer, and the fusion layer is used to perform average pooling processing on the first interest degree and the second interest degree.
5. The video recommendation method according to claim 2 or 3, characterized in that: Before inputting the user features and the video clip visual features into the graph convolutional neural network layer of the video recommendation model, the method further includes: Inputting the visual features of the video clip into a feature enhancement embedding layer of the video recommendation model to obtain enhanced representation features of the video clip output by the feature enhancement embedding layer, wherein the feature enhancement embedding layer is used to enhance the visual features of the video clip based on a transformation matrix; The step of inputting the user features and the video clip visual features into the graph convolutional neural network layer of the video recommendation model to obtain the positive interest embedding features, negative interest embedding features and video clip embedding features of the user output by the graph convolutional neural network layer comprises: The user features and the video clip enhanced representation features are input into the graph convolutional neural network layer of the video recommendation model to obtain the user's positive interest embedding features, negative interest embedding features and video clip embedding features output by the graph convolutional neural network layer.
6. The video recommendation method according to any one of claims 1 to 3, characterized in that: The loss function during the training of the video recommendation model is obtained by weighted summing of the user preference loss function L1 and the video clip loss function L2; The user preference loss function L1 is: The video clip loss function L2 is: Among them, U represents the user attribute information set, u represents a user attribute information in U, and N c represents the number of sample video segments divided from the sample original video, j represents the segment index, The jth sample video segment set among all the video segments in the complete sample video in the sample positive feedback video segment set of user u, represents the set of sample negative feedback video clips of user u, ω j is the penalty coefficient and ω j =j / N c , represents the interest of user u in sample video clip c, σ() represents the Sigmoid function, represents the set of sample videos that user u has not watched completely, and v represents An element in Represents the user u's interest in the positive feedback video clips in the sample video that has not been watched completely, Represents the interest of user u in the negative feedback video clips in the sample videos that have not been watched completely.
7. A video recommendation device, characterized in that: include: An acquisition module is used to acquire user attribute information and a set of videos to be recommended; A division module, used for dividing each to-be-recommended video in the to-be-recommended video set into a preset number of video segments; A feature extraction module, used to extract features from the user attribute information and the video clip respectively to obtain user features and video clip visual features; A processing module, configured to input the user features and the visual features of the video clip into a video recommendation model, and obtain the user's interest in the video to be recommended corresponding to the user attribute information output by the video recommendation model; A determination module, configured to determine a target recommended video from the set of videos to be recommended according to the interest level; An output module, used for outputting the target recommended video to the user; Among them, the video recommendation model is trained based on the sample original video corresponding to the user, the sample positive feedback video clip set and the sample negative feedback video clip set, the sample positive feedback video clip set is a set of video clips of the sample original video watched completely by the user, and the sample negative feedback video clip set is a set of video clips at the moment when the user's skipping behavior occurs.
8. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein: When the processor executes the computer program, the video recommendation method according to any one of claims 1 to 6 is implemented.
9. A non-transitory computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the video recommendation method according to any one of claims 1 to 6 is implemented.
10. A computer program product, comprising a computer program, characterized in that When the computer program is executed by a processor, the video recommendation method according to any one of claims 1 to 6 is implemented.
