Debiased short video recommendation method, device and medium based on multimodal knowledge graph

Abstract

The present application provides a debiased short video recommendation method, device and medium based on a multimodal knowledge graph, which relates to the field of short video recommendation. The method includes: obtaining behavior data; constructing a multimodal user knowledge graph based on the behavior data; constructing a video viewing trajectory based on the behavior data; constructing a multimodal variational autoencoder model, learning the video viewing trajectory through the multimodal variational autoencoder model, training to obtain bias parameters and form debiasing terms; merging each of the multimodal knowledge graphs into a connection network composed of multiple entity connections; based on the connection network, using a multi-head cross-attention network to learn the relationship between click-through rate and video popularity; predicting the user's popularity score for the video through the relationship, the feature vector of the connection network and the debiasing term, and obtaining the video recommendation result. Using the multimodal knowledge graph, the video that the user is interested in is predicted, which improves the recommendation efficiency.

Description

Technical Field

[0001] The present application relates to the field of short video recommendation, and in particular to a debiased short video recommendation method, device and medium based on a multimodal knowledge graph. Background Art

[0002] With the popularity of smartphones and the rise of social media platforms, short videos have attracted more and more attention and love in recent years. Some well-known short video platforms, such as TikTok and Kuaishou, have become social platforms with a large number of users worldwide. As an emerging form of entertainment and communication, short videos not only have broad application prospects in the entertainment industry, but also play an important role in advertising marketing, education and other fields. The strong charm of short videos is due to its unique UI innovation interface: instead of providing options for users to click, short video platforms actively recommend content to users, allowing users to watch only one video at a time. This also makes it easier for short videos to recommend short videos of interest to the user through relevant algorithms.

[0003] In the existing technology, short videos are recommended based on content. When some videos are liked by more and more users, the number of likes and comments on the videos increases. Most of the existing short video recommendation technologies judge whether users are interested in the content based on the user's likes, comments, and attention data, and do not make recommendations based on the video viewing time. In addition, they rely on simple database information for recommendation, without considering the relationship between videos, and the recommended content is single. Summary of the invention

[0004] The purpose of the present invention is to provide a debiased short video recommendation method, device and medium based on a multimodal knowledge graph in order to solve the problems of low accuracy and single recommended content in existing short video recommendation technologies.

[0005] The above-mentioned purpose of the present application is achieved through the following technical solutions:

[0006] S1: Obtaining behavior data of short video users, the behavior data including likes, comments, and viewing time;

[0007] S2: Build a multimodal user knowledge graph based on behavioral data;

[0008] S3: Construct video viewing trajectory based on behavioral data;

[0009] S4: Construct a multimodal variational autoencoder model, learn the video viewing trajectory through the multimodal variational autoencoder model, train the bias parameters and form the debiasing term;

[0010] S5: merging each of the multimodal knowledge graphs into a connection network consisting of multiple entity connections;

[0011] S6: Based on the connection network, a multi-head cross attention network is used to learn the relationship between click rate and video popularity; through the relationship, the feature vector of the connection network and the de-biasing term, the user's popularity score for the video is predicted to obtain the video recommendation result.

[0012] Optionally, step S1 includes: establishing a data collection system, collecting and storing the behavior data of each user watching short videos, and obtaining the behavior data from the data collection system.

[0013] Optionally, step S2 includes:

[0014] S21: Collect metadata required for constructing the knowledge graph, where the metadata includes: tags of video content and categories of video content;

[0015] S22: Based on metadata, define the entities and attributes of the knowledge graph;

[0016] S23: establishing associations between likes, comments, and viewing time and users and video content based on the behavior data, using likes and comments as directed edges between users and video content, and viewing time as the weight of the directed edges;

[0017] According to the number of likes of the video content by the user, the number of likes of the user in a specific time period is counted to obtain the characteristic value of the number of likes of the user

[0018] According to the user's comments on the video content, the sentiment analysis method is used to obtain the emotional level of the comments.

[0019] According to the user's viewing time of the video content, the characteristic value of the user's viewing time is obtained

[0020] According to the characteristic value Emotional level And the eigenvalue Indicates the user's interest in watching the video V u,t , the formula is as follows:

[0021]

[0022] in Indicates the number of likes by the user; It represents the emotional level of the comment, the closer to 1, the more interested, and the closer to 0, the less interested; represents the viewing time of the user, w1, w2, w3 represent the weight coefficients;

[0023] S24: Integrate the defined entities, attributes and relationships to build a multimodal user knowledge graph; use a graph database to store the multimodal user knowledge graph.

[0024] Optionally, step S23 includes:

[0025] Define n users and m video contents, number the users from 1 to n, and the video contents from 1 to m;

[0026] According to the association between each user i and video content j, a directed edge E(i,j) is defined, where likes and comments can represent two different directed edges respectively, and the viewing time is used as the weight of edge E(i,j), as follows:

[0027] The relationship between users’ likes on video content: if user i likes content j, then the directed edge E(i,j) indicates that this relationship exists;

[0028] User comment relationship on video content: If user i comments on video content j, the directed edge E(i,j) indicates that this relationship exists;

[0029] Watching time as the weight of directed edges: If user i watches content j and the watching time is t, then t is used as the weight of directed edge E(i,j).

[0030] Optionally, step S3 includes:

[0031] S31: Sort the videos watched by each user by timestamp to construct a video trajectory sequence; based on the behavior data, project the likes, comments, and viewing time into the same space to form the user's feature vector; each element in the video trajectory sequence represents a video watched, and the feature vector of the video is used as the element of the trajectory;

[0032] S32: constructing a training set of a video viewing trajectory model based on the user's historical behavior data and historical video content;

[0033] S33: extracting a first visual feature from the video content of the training set using a convolutional neural network; extracting a first text feature of the video from the video content of the training set using a statistical method based on word frequency and inverse document frequency; converting the first visual feature and the first text feature into a vector form;

[0034] S34: Construct the user's video viewing trajectory. The formula is as follows:

[0035]

[0036] Where T is the video trajectory sequence, represents the eigenvector f from i=1 to i=n i The sequence composed of.

[0037] Optionally, step S4 includes:

[0038] S41: Construct a dataset based on video titles, video keywords, video tags, and behavior data;

[0039] S42: according to the data set, using a multi-layer perceptron model to learn a second visual feature of the video in the data set, and using a statistical method based on word frequency and inverse document frequency to extract a second text feature;

[0040] S43: Convert the second visual feature and the second text feature into a vector form, as follows:

[0041] The second visual feature and the second text feature are connected to obtain the comprehensive feature vector of the video. The formula is as follows:

[0042] V=Concatenate(E video , E text )

[0043] Among them, V represents the comprehensive feature vector, E video represents the second visual feature vector extracted using the pre-trained multi-layer perceptron model, E text represents the second text feature vector extracted using the statistical method for word frequency and inverse document frequency; Concatenate() represents concatenation;

[0044] S44: Introducing a bias parameter β into the multi-layer perceptron model structure; adding the bias parameter β as an additional bias parameter to the output layer of the multimodal variational autoencoder model; automatically learning the value of the bias parameter by minimizing the loss function during the training process of the multimodal variational autoencoder model, and updating the bias parameter using a gradient descent optimization algorithm;

[0045] S45: The multimodal variational autoencoder model is trained through the data set, and the model parameters are learned by maximizing the likelihood function, so that the model learns the video viewing trajectory, obtains the bias parameters and forms the debiasing term.

[0046] Optionally, step S44 includes:

[0047] The deviation parameter is updated according to the gradient of the loss function. The formula is as follows:

[0048]

[0049] Where b is the bias parameter, α is the learning rate, It represents the gradient.

[0050] Optionally, step S6 includes:

[0051] S61: Acquire connection network data, the connection network data including: user information, video information and click rate; construct a connection network according to the connection network data;

[0052] S62: aligning entities in each of the multimodal knowledge graphs with entities in the connection network;

[0053] S63: The likes and comments of users in the connection network are used as directed edges between users and video content, and the viewing time is used as the weight of the directed edge;

[0054] S64: using a multi-head cross-attention network to learn the relationship between the click rate, the behavior data, and the video popularity; and in the multi-head cross-attention network, adding a de-biasing term to the loss function by regularization to reduce the model's preference for specific videos or users;

[0055] S65: Use the attention mechanism to model the relationship; extract the video features, text features and user features in the graph neural network and fuse them into feature vectors; calculate the similarity based on the feature vectors of the behavior data and the comprehensive feature vectors of the watched videos; sort the videos based on the similarity, and perform a weighted combination with the heat scores of the videos, and recommend the video scores to the user in descending order, as follows:

[0056] Similarity is used to measure users’ preferences and interests in videos. The formula is as follows:

[0057] EuclideanDistance(V u,t ,V u,c )=sqrt(sum((V u,t -V u,c ) 2 ))

[0058] Where EuclideanDistance(V u,t ,V u,c ) represents the Euclidean distance, sum represents the sum operation, and sqrt represents the square root operation. u,t and V u,c They represent the user's interest in watching the video and the user's liking for the recommended short video respectively;

[0059] The calculation steps of multi-head attention are as follows:

[0060] Construct the attention weight matrix a ij ,as follows:

[0061] a ij =attention(f(User),g(Video))

[0062] Among them, f and g are functions for mapping user and video features, User represents user; Video represents video; attention represents multi-head attention;

[0063] Calculate the output of multi-head cross attention:

[0064] H = softmax(a ij )*User

[0065] Among them, a ij is the attention weight matrix, H is the output of multi-head cross attention;

[0066] Use the output H of the multi-head cross attention network to predict the click rate as follows:

[0067] CTR=f(H)

[0068] Among them, CTR is the predicted result of click-through rate, f is the fully connected layer and activation function;

[0069] The popularity score is defined as the final score of click-through rate, viewing time and comment interaction data. The video recommendation result is determined based on the final score.

[0070] The popularity score includes: click rate V ctr , viewing time V t , comment interaction data V l , and user feedback V f , the heat score calculation formula is as follows:

[0071] V H =V ctr *W1+V t *W2+V l *W3+V f *W4

[0072] Among them, W1~W5 are weight coefficients, V H It's the heat score.

[0073] An electronic device includes a processor, a memory, a user interface and a network interface, wherein the memory is used to store instructions, the user interface and the network interface are used to communicate with other devices, and the processor is used to execute the instructions stored in the memory so that the electronic device executes a debiased short video recommendation method based on a multimodal knowledge graph.

[0074] A computer-readable storage medium stores instructions, and when the instructions are executed, a debiased short video recommendation method based on a multimodal knowledge graph is executed.

[0075] The beneficial effects of the technical solution provided by this application are:

[0076] 1. We built a knowledge graph of user behavior data to obtain the correlation between videos watched by users and provide a variety of recommended videos. We built the user's video viewing trajectory, learned the video viewing trajectory through a multimodal variational autoencoder model, trained the bias parameters and formed a debiasing term to offset the user's bias in watching video time. We analyzed the user's popularity score for the video in combination with the current video popularity to avoid blindly pushing videos with high popularity, which would reduce the user experience.

[0077] 2. Use a multi-head cross-attention network to learn the relationship between click-through rate, the behavioral data, and video popularity, and perform cross-attention calculations on different features to capture the interactions between them and improve the prediction accuracy of the model.

[0078] 3. Use the attention mechanism to model relationships and predict popular videos that users may be interested in, taking into account the interactive relationship between users and videos. BRIEF DESCRIPTION OF THE DRAWINGS

[0079] The present application will be further described below with reference to the accompanying drawings and embodiments, in which:

[0080] Figure 1 It is a step diagram of a debiased short video recommendation method based on a multimodal knowledge graph in an embodiment of the present application;

[0081] Figure 2 It is a schematic diagram of the electronic device structure of the debiased short video recommendation method based on the multimodal knowledge graph in the embodiment of the present application. DETAILED DESCRIPTION

[0082] In order to have a clearer understanding of the technical features, purposes and effects of the present application, the specific implementation methods of the present application are now described in detail with reference to the accompanying drawings.

[0083] An embodiment of the present application provides a debiased short video recommendation method based on a multimodal knowledge graph.

[0084] Please refer to Figure 1 , Figure 1 This is a step diagram of a method for debiasing short video recommendation based on a multimodal knowledge graph in an embodiment of the present application, including:

[0085] S1: Obtaining behavior data of short video users, the behavior data including likes, comments, and viewing time;

[0086] S2: Build a multimodal user knowledge graph based on behavioral data;

[0087] S3: Construct video viewing trajectory based on behavioral data;

[0088] S4: Construct a multimodal variational autoencoder model, learn the video viewing trajectory through the multimodal variational autoencoder model, train the bias parameters and form the debiasing term;

[0089] S5: merging each of the multimodal knowledge graphs into a connection network consisting of multiple entity connections;

[0090] S6: Based on the connection network, a multi-head cross attention network is used to learn the relationship between click rate and video popularity; through the relationship, the feature vector of the connection network and the de-biasing term, the user's popularity score for the video is predicted to obtain the video recommendation result.

[0091] Step S1 includes: establishing a data collection system, collecting and storing the behavior data of each user watching short videos, and obtaining the behavior data from the data collection system.

[0092] Step S2 includes:

[0093] S21: Collect metadata required for constructing the knowledge graph, where the metadata includes: tags of video content and categories of video content;

[0094] S22: Based on metadata, define the entities and attributes of the knowledge graph;

[0095] S23: establishing associations between likes, comments, and viewing time and users and video content based on the behavior data, using likes and comments as directed edges between users and video content, and viewing time as the weight of the directed edges;

[0096] According to the number of likes of the video content by the user, the number of likes of the user in a specific time period is counted to obtain the characteristic value of the number of likes of the user

[0097] According to the user's comments on the video content, the sentiment analysis method is used to obtain the emotional level of the comments.

[0098] According to the user's viewing time of the video content, the characteristic value of the user's viewing time is obtained

[0099] According to the characteristic value Emotional level And the eigenvalue Indicates the user's interest in watching the video V u,t , the formula is as follows:

[0100]

[0101] in Indicates the number of likes by the user; It represents the emotional level of the comment, the closer to 1, the more interested, and the closer to 0, the less interested; represents the viewing time of the user, w1, w2, w3 represent the weight coefficients;

[0102] S24: Integrate the defined entities, attributes and relationships to build a multimodal user knowledge graph; use a graph database to store the multimodal user knowledge graph.

[0103] Step S23 includes:

[0104] Define n users and m video contents, number the users from 1 to n, and the video contents from 1 to m;

[0105] According to the association between each user i and video content j, a directed edge E(i,j) is defined, where likes and comments can represent two different directed edges respectively, and the viewing time is used as the weight of edge E(i,j), as follows:

[0106] The relationship between users’ likes on video content: if user i likes content j, then the directed edge E(i,j) indicates that this relationship exists;

[0107] User comment relationship on video content: If user i comments on video content j, the directed edge E(i,j) indicates that this relationship exists;

[0108] Watching time as the weight of directed edges: If user i watches content j and the watching time is t, then t is used as the weight of directed edge E(i,j).

[0109] Step S3 includes:

[0110] S31: Sort the videos watched by each user by timestamp to construct a video trajectory sequence; based on the behavior data, project the likes, comments, and viewing time into the same space to form the user's feature vector; each element in the video trajectory sequence represents a video watched, and the feature vector of the video is used as the element of the trajectory;

[0111] S32: constructing a training set of a video viewing trajectory model based on the user's historical behavior data and historical video content;

[0112] S33: extracting a first visual feature from the video content of the training set using a convolutional neural network; extracting a first text feature of the video from the video content of the training set using a statistical method based on word frequency and inverse document frequency; converting the first visual feature and the first text feature into a vector form;

[0113] S34: Construct the user's video viewing trajectory. The formula is as follows:

[0114]

[0115] Where T is the video trajectory sequence, represents the eigenvector f from i=1 to i=n i The sequence composed of.

[0116] Step S4 includes:

[0117] S41: Construct a dataset based on video titles, video keywords, video tags, and behavior data;

[0118] S42: according to the data set, using a multi-layer perceptron model to learn a second visual feature of the video in the data set, and using a statistical method based on word frequency and inverse document frequency to extract a second text feature;

[0119] S43: Convert the second visual feature and the second text feature into a vector form, as follows:

[0120] The second visual feature and the second text feature are connected to obtain the comprehensive feature vector of the video. The formula is as follows:

[0121] V=Concatenate(E video , E text )

[0122] Among them, V represents the comprehensive feature vector, E video represents the second visual feature vector extracted using the pre-trained multi-layer perceptron model, E text represents the second text feature vector extracted using the statistical method for word frequency and inverse document frequency; Concatenate() represents concatenation;

[0123] S44: Introducing a bias parameter β into the multi-layer perceptron model structure; adding the bias parameter β as an additional bias parameter to the output layer of the multimodal variational autoencoder model; automatically learning the value of the bias parameter by minimizing the loss function during the training process of the multimodal variational autoencoder model, and updating the bias parameter using a gradient descent optimization algorithm;

[0124] S45: The multimodal variational autoencoder model is trained through the data set, and the model parameters are learned by maximizing the likelihood function, so that the model learns the video viewing trajectory, obtains the bias parameters and forms the debiasing term.

[0125] Specifically, after the deviation parameter training is completed, a de-biasing term is formed, and the de-biasing term is related to the trajectory deviation to offset the user's deviation in video viewing time.

[0126] Step S44 includes:

[0127] The deviation parameter is updated according to the gradient of the loss function. The formula is as follows:

[0128]

[0129] Where b is the bias parameter, α is the learning rate, It represents the gradient.

[0130] Specifically, the gradient descent algorithm is based on calculating the gradient of the mean square error loss function with respect to the bias term parameters, and gradually reduces the value of the loss function by updating the parameters in the opposite direction of the gradient.

[0131] Step S6 includes:

[0132] S61: Acquire connection network data, the connection network data including: user information, video information and click rate; construct a connection network according to the connection network data;

[0133] S62: aligning entities in each of the multimodal knowledge graphs with entities in the connection network;

[0134] S63: The likes and comments of users in the connection network are used as directed edges between users and video content, and the viewing time is used as the weight of the directed edge;

[0135] S64: using a multi-head cross-attention network to learn the relationship between the click rate, the behavior data, and the video popularity; and in the multi-head cross-attention network, adding a de-biasing term to the loss function by regularization to reduce the model's preference for specific videos or users;

[0136] S65: Use the attention mechanism to model the relationship; extract the video features, text features and user features in the graph neural network and fuse them into feature vectors; calculate the similarity based on the feature vectors of the behavior data and the comprehensive feature vectors of the watched videos; sort the videos based on the similarity, and perform a weighted combination with the heat scores of the videos, and recommend the video scores to the user in descending order, as follows:

[0137] Similarity is used to measure users’ preferences and interests in videos. The formula is as follows:

[0138] EuclideanDistance(V u,t ,V u,c )=sqrt(sum((V u,t -V u,c ) 2 ))

[0139] Where EuclideanDistance(Vu,t ,V u,c ) represents the Euclidean distance, sum represents the sum operation, and sqrt represents the square root operation. u,t and V u,c They represent the user's interest in watching the video and the user's liking for the recommended short video respectively;

[0140] The calculation steps of multi-head attention are as follows:

[0141] Construct the attention weight matrix a ij ,as follows:

[0142] a ij =attention(f(User),g(Video))

[0143] Among them, f and g are functions for mapping user and video features, User represents user; Video represents video; attention represents multi-head attention;

[0144] Calculate the output of multi-head cross attention:

[0145] H = softmax(a ij )*User

[0146] Among them, a ij is the attention weight matrix, H is the output of multi-head cross attention;

[0147] Use the output H of the multi-head cross attention network to predict the click rate as follows:

[0148] CTR=f(H)

[0149] Among them, CTR is the predicted result of click-through rate, f is the fully connected layer and activation function;

[0150] The popularity score is defined as the final score of click-through rate, viewing time and comment interaction data. The video recommendation result is determined based on the final score.

[0151] The popularity score includes: click rate V ctr , viewing time V t , comment interaction data V l , and user feedback V f , the heat score calculation formula is as follows:

[0152] V H =V ctr *W1+V t *W2+V l *W3+V f *W4

[0153] Among them, W1~W5 are weight coefficients, V H It's the heat score.

[0154] Specifically, according to the number of likes of the video in a specific time period, the video click rate characteristic value V is obtained. ctr ; Based on the data of the comments and interactions under the video, the video interaction data V is obtained l ; Based on the feedback from the comments under the video, use sentiment analysis to obtain the user feedback feature value V of the video f .

[0155] The present application also discloses an electronic device. Figure 2 , Figure 2 The electronic device 500 may include: at least one processor 501 , at least one network interface 504 , a user interface 503 , a memory 505 , and at least one communication bus 502 .

[0156] The communication bus 502 is used to realize the connection and communication between these components.

[0157] The user interface 503 may include a display screen, and the optional user interface 503 may also include a standard wired interface or a wireless interface.

[0158] The network interface 504 may optionally include a standard wired interface or a wireless interface (such as a WI-FI interface).

[0159] The above are only exemplary embodiments of the present disclosure and cannot be used to limit the scope of the present disclosure. That is, any equivalent changes and modifications made according to the teachings of the present disclosure are still within the scope of the present disclosure. After considering the disclosure of the specification and the truth of practice, those skilled in the art will easily think of other embodiments of the present disclosure.

[0160] This application is intended to cover any variation, use or adaptation of the present disclosure, which follows the general principles of the present disclosure and includes common knowledge or customary technical means in the art not described in the present disclosure. The description and examples are to be regarded as exemplary only, and the scope and spirit of the present disclosure are defined by the claims.

Claims

1. A debiased short video recommendation method based on multimodal knowledge graph, characterized in that: The method comprises the following steps: S1: Obtaining behavior data of short video users, the behavior data including likes, comments, and viewing time; S2: Build a multimodal user knowledge graph based on behavioral data; S3: Construct video viewing trajectory based on behavioral data; Step S3 includes: S31: Sort the videos watched by each user by timestamp to construct a video trajectory sequence; based on the behavior data, project the likes, comments, and viewing time into the same space to form the user's feature vector; each element in the video trajectory sequence represents a video watched, and the feature vector of the video is used as the element of the trajectory; S32: constructing a training set of a video viewing trajectory model based on the user's historical behavior data and historical video content; S33: extracting a first visual feature from the video content of the training set using a convolutional neural network; extracting a first text feature of the video from the video content of the training set using a statistical method based on word frequency and inverse document frequency; converting the first visual feature and the first text feature into a vector form; S34: Construct the user's video viewing trajectory. The formula is as follows: in is the video trajectory sequence, Representative from arrive The eigenvector of The sequence of components; S4: Construct a multimodal variational autoencoder model, learn the video viewing trajectory through the multimodal variational autoencoder model, train the bias parameters and form the debiasing term; Step S4 includes: S41: Construct a dataset based on video titles, video keywords, video tags, and behavior data; S42: according to the data set, using a multi-layer perceptron model to learn a second visual feature of the video in the data set, and using a statistical method based on word frequency and inverse document frequency to extract a second text feature; S43: Convert the second visual feature and the second text feature into a vector form, as follows: The second visual feature and the second text feature are connected to obtain the comprehensive feature vector of the video. The formula is as follows: in, represents the comprehensive feature vector, represents the second visual feature vector extracted using the pre-trained multi-layer perceptron model. represents a second text feature vector extracted using the statistical method for word frequency and inverse document frequency; Indicates connection; S44: Introducing bias parameters into the multilayer perceptron model structure ; Set the deviation parameter Add as an additional bias parameter to the output layer of the multimodal variational autoencoder model; automatically learn the value of the bias parameter by minimizing the loss function during the training process of the multimodal variational autoencoder model, and update the bias parameter using the gradient descent optimization algorithm; S45: Train the multimodal variational autoencoder model through the data set, learn the model parameters by maximizing the likelihood function, so that the model learns the video viewing trajectory, obtains the bias parameters and forms the debiasing term; S5: merging each of the multimodal knowledge graphs into a connection network consisting of multiple entity connections; S6: Based on the connection network, a multi-head cross attention network is used to learn the relationship between click rate and video popularity; through the relationship, the feature vector of the connection network and the de-biasing term, the user's popularity score for the video is predicted to obtain the video recommendation result.

2. The method for debiasing short video recommendation based on multimodal knowledge graph according to claim 1, characterized in that: Step S1 includes: establishing a data collection system, collecting and storing the behavior data of each user watching short videos, and obtaining the behavior data from the data collection system.

3. The method for debiasing short video recommendation based on multimodal knowledge graph according to claim 1, characterized in that: Step S2 includes: S21: Collect metadata required for constructing the knowledge graph, where the metadata includes: tags of video content and categories of video content; S22: Based on metadata, define the entities and attributes of the knowledge graph; S23: establishing associations between likes, comments, and viewing time and users and video content based on the behavior data, using likes and comments as directed edges between users and video content, and viewing time as the weight of the directed edges; According to the number of likes of the video content by the user, the number of likes of the user in a specific time period is counted to obtain the characteristic value of the number of likes of the user ; According to the user's comments on the video content, the sentiment analysis method is used to obtain the emotional level of the comments. ; According to the user's viewing time of the video content, the characteristic value of the user's viewing time is obtained ; According to the characteristic value , emotional level And the eigenvalue , indicating the user's interest in watching the video , the formula is as follows: in Indicates the number of likes by the user; It represents the emotional level of the comment, the closer to 1, the more interested, and the closer to 0, the less interested; Indicates the time the user watched. , , represents the weight coefficient; S24: Integrate the defined entities, attributes and relationships to build a multimodal user knowledge graph; use a graph database to store the multimodal user knowledge graph.

4. The method for debiasing short video recommendation based on multimodal knowledge graph as claimed in claim 3, characterized in that: Step S23 includes: definition Users and video content, and number users from 1 to , video content numbered from 1 to ; According to each user and video content The relationship between them defines a directed edge , where likes and comments can represent two different directed edges respectively, and viewing time is used as an edge The weights are as follows: The relationship between users’ likes on video content: If users Liked the content , then there is a directed edge Indicates that this relationship exists; User comments on video content: If the user Commented on the video content , then there is a directed edge Indicates that this relationship exists; Watching time as the weight of the directed edge: If the user Viewed content , and watched for , then As a directed edge The weight of .

5. The method for debiasing short video recommendation based on multimodal knowledge graph according to claim 1, characterized in that: Step S44 includes: The deviation parameter is updated according to the gradient of the loss function. The formula is as follows: in is the bias parameter, is the learning rate, It represents the gradient.

6. The method for debiasing short video recommendation based on multimodal knowledge graph according to claim 3, characterized in that: Step S6 includes: S61: Acquire connection network data, the connection network data including: user information, video information and click rate; construct a connection network according to the connection network data; S62: aligning entities in each of the multimodal knowledge graphs with entities in the connection network; S63: The likes and comments of users in the connection network are used as directed edges between users and video content, and the viewing time is used as the weight of the directed edge; S64: using a multi-head cross-attention network to learn the relationship between the click rate, the behavior data, and the video popularity; and in the multi-head cross-attention network, adding a de-biasing term to the loss function by regularization to reduce the model's preference for specific videos or users; S65: Use the attention mechanism to model the relationship; extract the video features, text features and user features in the graph neural network and fuse them into feature vectors; calculate the similarity based on the feature vectors of the behavior data and the comprehensive feature vectors of the watched videos; sort the videos based on the similarity, and perform a weighted combination with the heat scores of the videos, and recommend the video scores to the user in descending order, as follows: Similarity is used to measure users’ preferences and interests in videos. The formula is as follows: in represents the Euclidean distance, represents the sum operation, Represents the square root operation; and They represent the user's interest in watching the video and the user's liking for the recommended short video respectively; The calculation steps of multi-head attention are as follows: Constructing the attention weight matrix ,as follows: in, and is a function that maps user and video features, Represents a user; Indicates video; Indicates multiple attentions; Calculate the output of multi-head cross attention: in, is the attention weight matrix, The output of multi-head cross attention; Using the output of a multi-head crisscross attention network The click-through rate prediction is as follows: in, is the prediction result of click rate. is the fully connected layer and activation function; The popularity score is defined as the final score of click-through rate, viewing time and comment interaction data. The video recommendation result is determined based on the final score. The popularity score includes: click-through rate , watch time , comment interaction data , and user feedback , the heat score calculation formula is as follows: = in, ~ is the weight coefficient, It's the heat score.

7. An electronic device, characterized in that: The electronic device (500) comprises a processor (501), a memory (505), a user interface (503) and a network interface (504), wherein the memory (505) is used to store instructions, the user interface (503) and the network interface (504) are used to communicate with other devices, and the processor (501) is used to execute the instructions stored in the memory (505) so that the electronic device (500) executes the method according to any one of claims 1 to 6.

8. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores instructions, and when the instructions are executed, the method steps according to any one of claims 1 to 6 are performed.

Images