Short video recommendation method and system based on hypercycle network and multi-task training

Abstract

The invention discloses a short video recommendation method and system based on a hypercycle network and multi-task training, and the method comprises the steps: firstly constructing a super node in a hypercycle neural network for integrating a plurality of user interaction behaviors, thereby carrying out the modeling of the semantic association of different user interaction behaviors; inputting the obtained historical behavior sequences of the user and a target short video into a trained short video recommendation model, and capturing a sequence dependency relationship among the historical behavior sequences by a hyper-recurrent neural network based on super-node design, modeling directed connection between the click node and the favorite node through a high-speed neural network; then calculating user preferences in different interactive behavior spaces based on an addition attention mechanism of the target short video; and finally, fusing the user preferences in different interactive behavior spaces, and outputting the probability of clicking the target short video by the user to finish the recommendation of the short video.

Description

Technical Field

[0001] The present invention belongs to the technical field of Internet services, and in particular to a short video recommendation method and system based on a hyperloop network and multi-task training. Background Art

[0002] Methods based on deep learning have achieved great success in the problem of short video sequence recommendation. These sequence recommendation methods are based on neural networks, self-attention mechanisms, and graph neural networks. However, these methods only focus on the behavior patterns in the "click" behavior sequence, while ignoring other user behaviors, such as "skip" and "like" behaviors. In fact, short video platforms record various behaviors of users. These behaviors indicate the degree of like or dislike of the user for the short video. For example, if a user watches a short video for a few seconds and then slides past it, it is a "skip" behavior. If the user is attracted by the video and lets the short video continue to play, it is a "click" behavior. If the user likes the short video very much, plays the video repeatedly or likes the video, it is a "like" behavior.

[0003] Existing short video recommendation methods based on multi-behavior sequences take into account the differences between different behaviors and use the same framework but different model parameters to model different behavior sequences. These methods ignore the semantic association between different behaviors and the temporal dependency between short videos of different interaction types in the user behavior sequence. The semantic association between different behaviors means that the user satisfaction expressed by different types of behaviors is in order of magnitude. The "skip" behavior conveys the user's negative emotions, indicating that the user is not interested in the short video. On the contrary, the user's "click" and "like" behaviors represent the user's positive emotions towards the short video, and the "like" behavior indicates that the short video has a deeper appeal. In summary, the semantic association means that "skip" and "click" / "like" behaviors express opposite semantics, and there is a conditional dependency between "click" and "like" behaviors. "Click" behavior is a "pre-like" behavior, which is consistent with the relationship between "browse" and "buy" behaviors in e-commerce. In addition, there is a temporal dependency between short videos of different interaction types in the user behavior sequence. The temporal dependency here means that the items that the user is currently interacting with are similar to the items that he has interacted with before. Most multi-behavior sequence recommendation methods only capture the temporal dependencies between different positive interaction items when modeling different types of user behaviors, because these positive interaction items all contain content that users like and are naturally similar. They ignore the temporal dependencies between positive feedback behaviors and negative feedback behaviors in user interaction behavior sequences. In actual user multi-behavior interaction sequences, some short videos that are swiped past may be similar to short videos that are clicked. Summary of the invention

[0004] The purpose of the present invention is to solve the problems existing in the prior art and provide a short video recommendation method and system based on a hyperloop network and multi-task training. The method of the present invention is to capture the semantic associations between different behaviors of users in the short video platform and the temporal dependencies between short videos of different interaction types in the user behavior sequence, thereby improving the model recommendation effect.

[0005] In order to achieve the above-mentioned invention object, the present invention specifically adopts the following technical solutions:

[0006] In a first aspect, the present invention provides a short video recommendation method based on a hyperloop network and multi-task training, comprising the following steps:

[0007] S1. Construct a supernode in a hypercyclic neural network to integrate multiple user interaction behaviors, thereby modeling the semantic associations of different user interaction behaviors; each supernode contains three nodes, corresponding to three types of user interaction behaviors: skip, click, and like;

[0008] S2. Input the acquired user's historical behavior sequence and target short video into the trained short video recommendation model. First, the input historical behavior sequence is processed by a hypercyclic neural network designed based on super nodes to capture the sequence dependency between historical behavior sequences and obtain the hidden state of each unit of the hypercyclic neural network. The number of units in the hypercyclic neural network is the same as the length of the user's historical behavior sequence. When the input is a user interaction behavior of the like type, an additional layer of high-speed neural network is added to each unit of the hypercyclic neural network to model the directed connection between the click node and the like node.

[0009] S3. Divide the user's historical behavior sequence into sub-behavior sequences under different interactive behavior spaces according to the user's interactive behavior type, and use the additive attention mechanism based on the target short video to process the sub-behavior sequence, the hidden state of the hypercyclic neural network unit and the target short video to calculate the user preferences under different interactive behavior spaces;

[0010] S4. Integrate user preferences in different interactive behavior spaces, output the probability of users clicking on the target short video, and complete the recommendation of short videos.

[0011] Based on the above scheme, each step can be implemented in the following preferred specific manner.

[0012] As a preferred embodiment of the first aspect, in step S1, there is a directed connection between the internal click node and the like node, and both the internal click node and the like node indicate the user's positive preference for the target short video, and the like node indicates that the user is more satisfied with the target short video;

[0013] The nodes inside the supernode are activated according to the input user interaction behavior category: if the input user interaction behavior is skip, the internal skip node is activated, otherwise the internal click node is activated; if the input user interaction behavior is like, the internal click and like nodes are activated at the same time.

[0014] As a preferred embodiment of the above-mentioned first aspect, in a cycle process of the tth unit of the hypercyclic neural network, the hidden state of the t-1th unit is processed by the first unit update function and the unit copy function to obtain the hidden state of the tth unit.

[0015] Furthermore, in step S2, the one-cycle process of the t-th unit of the hypercyclic neural network is:

[0016]

[0017] Among them, p∈{+,-} is a placeholder; It is the logical NOT operation of p; represents the hidden state of the t-th unit; F(·) is the first unit update function; C(·) is the unit replication function; represents the hidden state of the t-1th unit; x t is the input short video representation at the t-th time step; represents the indicator function of the t-th unit.

[0018] As a preferred embodiment of the first aspect, in step S2, when the indicator function of the tth unit When , the unit replication function is expressed as When the indicator function of the tth unit When , the first unit update function of the t-th unit is represented by the indicator function of the t-1-th unit, the output of the hidden layer of the t-th unit, the output of the update gate, and the candidate activation vector.

[0019] Furthermore, when the indicator function of the tth unit When , the first unit update function of the tth unit is expressed as:

[0020]

[0021] in, Both represent the indicator function of the t-1th unit; h t-1 represents the output of the hidden layer of the t-1th unit; T represents the vector transposition operator; represents the output of the update gate; σ is the sigmoid activation function; represents the output of the reset gate; tanh represents the tanh activation function; represents the candidate activation vector; ⊙ is the Hadamard product; Both represent trainable parameters in the short video recommendation model.

[0022] As a preferred embodiment of the first aspect, the high-speed neural network uses a second unit to update the function Update the hidden state of the tth unit.

[0023] Furthermore, the specific process of updating the hidden state of the t-th unit is:

[0024]

[0025] Among them, g t is the output of the transfer gate; W g , W hh Both represent the trainable parameters in the short video recommendation model; h′ t Represents an intermediate variable.

[0026] As a preferred embodiment of the above-mentioned first aspect, in step S3, the sub-behavior sequences under different interactive behavior spaces are respectively a skip sub-behavior sequence, a click sub-behavior sequence, a like sub-behavior sequence, and a click but not like sub-behavior sequence, wherein the click sub-behavior sequence contains all elements in the like sub-behavior sequence, and the click but not like sub-behavior sequence is composed of all elements belonging to the click sub-behavior sequence but not belonging to the like sub-behavior sequence.

[0027] As a preferred embodiment of the above-mentioned first aspect, in step S3, the hidden state of the corresponding sub-behavior sequence is extracted from the final output of the hypercyclic neural network according to the subscript of the sub-behavior sequence, and the user preferences in different interactive behavior spaces are obtained using an additive attention mechanism based on the target short video.

[0028] Furthermore, user preferences in different interactive behavior spaces are specifically expressed as:

[0029] p b =Attention(x new ,S b )

[0030] Among them, p b represents the user preference in b behavior space; S b represents the sub-behavior sequence in the b behavior space; p b and S b The subscript b in the is a placeholder for skip action skip, click action click, like action like, and click but not like action click_notlike; new It is the target short video representation.

[0031] As a preferred embodiment of the first aspect, the specific process of step S4 is as follows:

[0032] S41. Perform a weighted sum of the user's skip preference and click preference to obtain a user preference related to the user's click rate;

[0033] S42. Connect the user preference related to the user click rate and the target short video representation to obtain a first connected representation and input it into the first prediction network to obtain the output probability of the user clicking the target short video; the first prediction network is implemented by two layers of MLP.

[0034] As a preferred embodiment of the first aspect, during the training process of the short video recommendation model, the cross entropy loss between the probability of a user clicking on a target short video and the true value of the user clicking on the short video is calculated as the click-through rate prediction loss, the cross entropy loss between the probability of a user liking a target short video and the true value of the user liking the short video is calculated as the like rate prediction loss, the click rate prediction loss and the like rate prediction loss are weighted as the total loss, the short video recommendation model parameters are updated based on minimizing the total loss, and the trained short video recommendation model is obtained by continuous iteration;

[0035] The probability that a user likes a target short video is obtained by: performing a weighted summation of the user's click-but-don't-like preference and the like preference to obtain a user preference related to the user's like rate; connecting the user preference related to the user's like rate and the target short video representation to obtain a second connected representation and input it into a second prediction network to obtain an output probability that the user likes the target short video; the second prediction network is implemented by a two-layer MLP.

[0036] In a second aspect, the present invention provides a short video recommendation system based on a hyperloop network and multi-task training, comprising:

[0037] The supernode construction module is used to construct supernodes in the hypercyclic neural network, which is used to integrate multiple user interaction behaviors, so as to model the semantic associations of different user interaction behaviors; each supernode contains three nodes, corresponding to the three types of user interaction behaviors: skip, click and like;

[0038] The dependency acquisition module is used to input the acquired user's historical behavior sequence and target short video into the trained short video recommendation model. The input historical behavior sequence is first processed by a hypercyclic neural network designed based on super nodes to capture the sequence dependency between the historical behavior sequences and obtain the hidden state of each unit of the hypercyclic neural network. The number of units in the hypercyclic neural network is the same as the length of the user's historical behavior sequence. When the input is a user interaction behavior of the like type, an additional layer of high-speed neural network is added to each unit of the hypercyclic neural network to model the directed connection between the click node and the like node.

[0039] The user preference acquisition module is used to divide the user's historical behavior sequence into sub-behavior sequences under different interactive behavior spaces according to the user's interactive behavior type. The sub-behavior sequence, the hidden state of the hypercyclic neural network unit and the target short video are processed by the additive attention mechanism based on the target short video to calculate the user preference under different interactive behavior spaces;

[0040] The result acquisition module is used to integrate user preferences in different interactive behavior spaces, output the probability of users clicking on the target short video, and complete the recommendation of short videos.

[0041] Compared with the prior art, the present invention has the following beneficial effects:

[0042] The method of the present invention proposes the concept of a supernode, and integrates multiple behaviors of users into a supernode to capture the semantic associations between different behaviors.

[0043] The method of the present invention further designs a hyper-recurrent neural network based on super nodes to capture the temporal dependencies between short videos of different interaction types in the user behavior sequence.

[0044] The short video recommendation model proposed in the method of the present invention is a multi-task training model. It proposes to use the like rate prediction task to assist the click rate prediction main task. The like rate prediction task helps the "like" node in the super node to learn the user's preference, making the "like" node and "click" node in the super node more distinguishable. BRIEF DESCRIPTION OF THE DRAWINGS

[0045] Figure 1 It is a schematic diagram of the process of the present invention;

[0046] Figure 2 It is a schematic diagram of the structure of a supernode in the method of the present invention;

[0047] Figure 3 Schematic diagram of the calculation process of the hypercyclic neural network in the method of the present invention;

[0048] Figure 4 Schematic diagram of the training process of the short video recommendation model in the method of the present invention;

[0049] Figure 5 The system block diagram of the method of the present invention. DETAILED DESCRIPTION

[0050] In order to make the above-mentioned purpose, features and advantages of the present invention more obvious and easy to understand, the specific implementation mode of the present invention is described in detail below in conjunction with the accompanying drawings. In the following description, many specific details are set forth to facilitate a full understanding of the present invention. However, the present invention can be implemented in many other ways different from those described herein, and those skilled in the art can make similar improvements without violating the connotation of the present invention. Therefore, the present invention is not limited to the specific embodiments disclosed below. The technical features in each embodiment of the present invention can be combined accordingly without conflicting with each other.

[0051] In the description of the present invention, it should be understood that the terms "first" and "second" are only used for the purpose of distinguishing descriptions, and should not be understood as indicating or implying relative importance or implicitly indicating the number of technical features indicated. Therefore, the features defined as "first" and "second" may explicitly or implicitly include at least one of the features.

[0052] The method of the present invention uses multiple user behavior data as input to obtain user preferences and predict short videos. Let u∈U and v∈V represent users and short videos respectively, and U and V represent user sets and short video sets respectively. For ease of understanding, short videos are sometimes referred to as items. Assume that there are n types of user interaction behaviors in the short video platform, represented as {b1, b2, …, b n}, such as "skip", "click" and "like". A user's historical behavior sequence is represented as S = {(v1,b1),(v2,b2),…,(v j ,b j ),…,(v |S| ,b |S| )}, where (v1,b1),(v2,b2),…,(v j ,b j ),…,(v |S| ,b |S| ) represent the interaction behaviors between the 1st, 2nd, …, j, …, |S|th users and the 1st, 2nd, …, j, …, |S|th users respectively, (v j ,b j ) represents the jth user v j The interaction behavior is b j According to the user interaction behavior type in the historical behavior sequence S, the historical behavior sequence S can be further divided into sub-behavior sequences under different user interaction behavior spaces: For example, the sub-behavior sequences in the “skip”, “click”, and “like” behavior spaces are S skip , S click and S like .

[0053] The problem of the present invention is expressed as follows: Input the user's historical behavior sequence S and the target short video v new , training a short video recommendation model can output the target short video v that the user clicks on new In order to solve this problem, the present invention proposes a short video recommendation method based on a hyperloop network and multi-task training, which consists of four parts: the first part is to design a supernode to integrate multiple behaviors, so as to model the semantic association of different user behaviors; the second part is to design a hyperloop neural network based on supernodes to capture the sequence dependency between multiple behavior sequences; the third part is to use an additive attention mechanism based on the target short video to calculate the user's preference in different behavior spaces; the fourth part is to predict the probability and loss of users clicking and liking the target short video, and then train the short video recommendation model.

[0054] like Figure 1 As shown, in a preferred implementation of the present invention, the short video recommendation method based on hyperloop network and multi-task training includes the following steps S1 to S4. The specific implementation process is described below.

[0055] S1. Construct a supernode in a hypercyclic neural network to integrate multiple user interaction behaviors, thereby modeling the semantic associations of different user interaction behaviors; each supernode contains three nodes, corresponding to the three types of user interaction behaviors: skip, click, and like.

[0056] It should be noted that, in the present invention, Figure 2 As shown, in step S1, there is a directed connection between the internal click node and the like node, and both the internal click node and the like node indicate the user's positive preference for the target short video, and the like node indicates that the user is more satisfied with the target short video;

[0057] The nodes inside the supernode are activated according to the input user interaction behavior category: if the input user interaction behavior is skip, the internal skip node is activated, otherwise the internal click node is activated; if the input user interaction behavior is like, the internal click and like nodes are activated at the same time. Because the internal skip and click nodes are not activated at the same time, the opposite semantics of skip and click behaviors can be captured.

[0058] It should also be noted that in the present invention, if there are more behaviors that characterize user satisfaction, such as long-time viewing and complete viewing behaviors, the supernode can naturally be expanded into more layers by adding directed connections.

[0059] S2. Input the acquired user's historical behavior sequence and target short video into the trained short video recommendation model. First, the input historical behavior sequence S is processed by a hyper-cyclic neural network designed based on super nodes to capture the sequence dependencies between historical behavior sequences and obtain the hidden state of each unit of the hyper-cyclic neural network. The number of units in the hyper-cyclic neural network is the same as the length of the user's historical behavior sequence. When the input is a user interaction behavior of the like type, an additional layer of high-speed neural network is added to each unit of the hyper-cyclic neural network to model the directed connection between the click node and the like node.

[0060] It should be noted that in the present invention, the hypercyclic neural network treats all historical behavior sequences of users as a whole, rather than modeling each sub-behavior sequence independently. This overall operation enables the short video recommendation model to capture the complex temporal dependencies between different types of behaviors.

[0061] It should be noted that in step S2, and represents the hidden state of the t-th unit, The hidden state representing the user's positive interest flow, Represents the hidden state of the user's negative interest flow. Indicator function and Used to indicate which internal node is activated. Once an internal node is activated, the corresponding hidden state will absorb the context information to update; otherwise, the state of the previous time step is directly copied. Indicates that the click node inside the tth unit is activated. Correspondingly, Indicates that the click node inside the tth unit is suppressed. Indicates that the skip node inside the tth unit is activated, Indicates that the skipped nodes inside the t-th unit are suppressed. Equation It is always true, and a like behavior is also a click behavior. The cycle process of the t-th unit of the hypercyclic neural network is:

[0062]

[0063] Among them, p∈{+,-} is a placeholder; is the logical NOT operation of p; F(·) is the first unit update function, which is used to absorb the input short video representation x at the t-th time step t and the hidden state of the t-1th unit To update the hidden state of the tth unit C(·) is the unit replication function; represents the indicator function of the t-th unit.

[0064] Furthermore, when the indicator function of the tth unit When , the cycle process of the tth unit of the hypercyclic neural network Can be organized into The unit replication function C(·) can be defined as

[0065] Furthermore, when the indicator function of the tth unit When , the cycle process of the tth unit of the hypercyclic neural network Can be organized into The first unit update function of the tth unit can be defined as:

[0066]

[0067] in, Both represent the indicator function of the t-1th unit; h t-1 represents the output of the hidden layer of the t-1th unit; T represents the vector transposition operator; represents the output of the update gate; σ is the sigmoid activation function; represents the output of the reset gate; tanh represents the tanh activation function; represents the candidate activation vector; ⊙ is the Hadamard product; Both represent trainable parameters in the short video recommendation model.

[0068] It should also be noted that in step S2, in order to model the directed connection between the click node and the like node, a layer of high-speed neural network is integrated into the recurrent neural network iteration process. The high-speed neural network uses the second unit to update the function Update the hidden state of the tth unit:

[0069]

[0070] Among them, g t is the output of the transfer gate; W g , W hh Both represent the trainable parameters in the short video recommendation model; h′ t Represents an intermediate variable.

[0071] If there are more types of behaviors in the user's historical behavior sequence, the hypercyclic neural network can be expanded with more layers of high-speed neural networks. Each deeper layer of high-speed neural network represents the user's deeper preferences. Taking the user behavior sequence S = {(v1, click), (v2, skip), (v3, click), (v4, like), (v5, skip)} as an example, the calculation process of the hypercyclic neural network is as follows: Figure 3 shown.

[0072] S3. The user's historical behavior sequence is divided into sub-behavior sequences under different interactive behavior spaces according to the user's interactive behavior type. The sub-behavior sequence, the hidden state of the hyper-recurrent neural network unit and the target short video are processed by the additive attention mechanism based on the target short video to calculate the user preference under different interactive behavior spaces.

[0073] It should be noted that in step S3, the subsequences in different interactive behavior spaces are respectively skip subbehavior sequence S skip , click sub-behavior sequence S click And like sub-behavior sequence S like and click but don't like sub-behavior sequence S click_notlike , the click sub-behavior sequence S click Contains the like sub-behavior sequence S like All elements in the clicked but not liked sub-behavior sequence S click_notlike The click sub-behavior sequence S click But it does not belong to the like sub-behavior sequence S like All elements of .

[0074] It should be noted that in step S3, the hidden state of the corresponding sub-behavior sequence is extracted from the final output of the hypercyclic neural network according to the subscript of the sub-behavior sequence, and the user preference in different interactive behavior spaces is obtained by using the additive attention mechanism based on the target short video:

[0075] p b =Attention(x new ,S b )

[0076] Among them, p b represents the user preference in b behavior space; S b represents the sub-behavior sequence in the b behavior space; p b and S b The subscript b in the is a placeholder for skip action skip, click action click, like action like, and click but not like action click_notlike; new Is the target short video v new Representation.

[0077] Specifically, in this embodiment, the above-mentioned additive attention mechanism Attention form is as follows:

[0078]

[0079] Among them, α trepresents the tth attention value; b1 and q1 are both trainable vectors of the short video recommendation model; W1 and W2 are both transfer matrices; h t Represents the output of the hidden layer of the tth unit of the hypercyclic neural network; represents the normalized attention value; α k represents the th attention value; |S b | indicates S b In particular, when S b YesS skip When t is the output of the hypercyclic neural network On the contrary, h t is the output of the hypercyclic neural network

[0080] Through the above-mentioned additive attention mechanism based on the target short video, the user preferences under different interactive behavior spaces are:

[0081] p skip =Attention(x new ,S skip )

[0082] p click =Attention(x new ,S click )

[0083] p like =Attention(x new ,S like )

[0084] p click_notlike =Attention(x new ,S click_notlike )

[0085] Among them, p skip 、p click 、p like and p click_notlike They represent the user's skip preference, click preference, like preference, and click but not dislike preference respectively.

[0086] S4. Integrate user preferences in different interactive behavior spaces, output the probability of users clicking on the target short video, and complete the recommendation of short videos.

[0087] It should be noted that, in step S4 of the present invention, the specific process is as follows:

[0088] S41. Set the user's skip preference p skip and click on Preferences click Perform weighted summation to obtain user preferences related to user click rate

[0089]

[0090] Among them, the structure of the Merge() function is the weighted sum of vectors; skip ∈(0,1) is p skip The weight of .

[0091] S42. Connect the user preference related to the user click rate and the target short video representation to obtain a first connection representation and input it into the first prediction network Predict() to obtain the probability of the output user clicking the target short video.

[0092]

[0093] In this embodiment, the first prediction network is implemented by two layers of MLP, and the specific processing process is:

[0094]

[0095] Among them, W3, W4, b3 and b4 all represent trainable parameters in the short video recommendation model; φ is the RELU activation function.

[0096] It should be noted that, in the training process of the short video recommendation model in step S4, Figure 4 As shown, the cross entropy loss between the probability of a user clicking on a target short video and the true value of the user clicking on the short video is calculated as the click-through rate prediction loss, the cross entropy loss between the probability of a user liking a target short video and the true value of the user liking the short video is calculated as the like rate prediction loss, the click rate prediction loss and the like rate prediction loss are weighted as the total loss, the short video recommendation model parameters are updated based on minimizing the total loss, and the trained short video recommendation model is obtained by continuous iteration.

[0097] In this embodiment, the main task of the present invention is to predict the click-through rate. In order to enhance the distinguishability of the short video recommendation model for user preferences in different behavior spaces, the present invention further designs an auxiliary task to predict the probability of users liking the target short video. During the training process of the short video recommendation model, the main task and the auxiliary task are used to update the parameters of the short video recommendation model, and the probability of users clicking on the target short video and the probability of liking the target short video are output; during the prediction process of the short video recommendation model, only the main task predicts the click-through rate, that is, only the probability of users clicking on the target short video is output. The present invention takes into account the user's preference for the target short video v new The click-through rate of the target short video v is related to the user's skip preference and click preference. Therefore, the user's skip preference and click preference are integrated to predict the user's click on the target short video v newThe probability of click-through rate prediction loss is obtained Similarly, users’ responses to the target short video v new The like probability is related to the user's click, but the dislike preference is related to the like preference. Therefore, the two preferences are combined to predict the user's like for the target short video v new The probability of , and get the prediction loss of the like rate The calculation process is the same as the process of calculating the click-through rate prediction loss, and then the short video recommendation model is trained.

[0098] Specifically, we first calculate the probability of a user clicking on the target short video and the probability of a user clicking on the short video v new The cross entropy loss between the true values ​​is used as the click rate prediction loss

[0099]

[0100] Among them, y click ∈{0,1} represents the user clicking on the short video v new The true value of; Loss() function represents the cross entropy loss function.

[0101] Then, the user clicked but did not like the preference p click_notlike and like preference p like Perform weighted summation to obtain user preferences related to user like rate

[0102]

[0103] Connect the user preferences related to the user's like rate and the target short video representation to obtain the second connected representation and input it into the second prediction network Predict() to obtain the output probability that the user likes the target short video

[0104]

[0105] In this embodiment, the second prediction network is implemented by two layers of MLP, and the specific processing process is the same as that of the first prediction network, which will not be repeated.

[0106] Next, calculate the probability that the user likes the short video and the probability that the user likes the short video v new The cross entropy loss between the true values ​​is used as the like rate prediction loss

[0107]

[0108] Among them, y like ∈{0,1} represents the true value of the user's liking for the short video.

[0109] The click rate prediction loss and the like rate prediction loss are weighted as the total loss:

[0110]

[0111] Among them, β like is the weight of the loss of the like rate prediction. In this embodiment, considering that click rate prediction is the main task of the method of the present invention and like probability prediction is the auxiliary task of the method of the present invention, β like Usually it is a relatively small value. In the embodiment of the present invention, β like =0.1, technicians can adjust β based on different data sets like Adjust parameters.

[0112] Finally, based on minimizing the total loss, the short video recommendation model parameters are updated, and Adam is used for optimization. After continuous iteration, a trained short video recommendation model is obtained.

[0113] It should also be noted that the short video recommendation method based on hyperloop network and multi-task training in the above embodiment can essentially be executed by a computer program or module. Therefore, similarly, based on the same inventive concept, another preferred embodiment of the present invention also provides a short video recommendation system based on hyperloop network and multi-task training corresponding to the short video recommendation method based on hyperloop network and multi-task training provided in the above embodiment, such as Figure 5 As shown, it includes:

[0114] The supernode construction module is used to construct supernodes in the hypercyclic neural network, which is used to integrate multiple user interaction behaviors, so as to model the semantic associations of different user interaction behaviors; each supernode contains three nodes, corresponding to the three types of user interaction behaviors: skip, click and like;

[0115] The dependency acquisition module is used to input the acquired user's historical behavior sequence and target short video into the trained short video recommendation model. The input historical behavior sequence is first processed by a hypercyclic neural network designed based on super nodes to capture the sequence dependency between the historical behavior sequences and obtain the hidden state of each unit of the hypercyclic neural network. The number of units in the hypercyclic neural network is the same as the length of the user's historical behavior sequence. When the input is a user interaction behavior of the like type, an additional layer of high-speed neural network is added to each unit of the hypercyclic neural network to model the directed connection between the click node and the like node.

[0116] The user preference acquisition module is used to divide the user's historical behavior sequence into sub-behavior sequences under different interactive behavior spaces according to the user's interactive behavior type. The sub-behavior sequence, the hidden state of the hypercyclic neural network unit and the target short video are processed by the additive attention mechanism based on the target short video to calculate the user preference under different interactive behavior spaces;

[0117] The result acquisition module is used to integrate user preferences in different interactive behavior spaces, output the probability of users clicking on the target short video, and complete the recommendation of short videos.

[0118] The above-described embodiment is only a preferred solution of the present invention, but it is not intended to limit the present invention. A person skilled in the relevant technical field may make various changes and modifications without departing from the spirit and scope of the present invention. Therefore, any technical solution obtained by equivalent replacement or equivalent transformation falls within the protection scope of the present invention.

Claims

1. A short video recommendation method based on hyperloop network and multi-task training, characterized in that: The following steps are involved: S1. Construct a supernode in a hypercyclic neural network to integrate multiple user interaction behaviors, thereby modeling the semantic associations of different user interaction behaviors; each supernode contains three nodes, corresponding to three types of user interaction behaviors: skip, click, and like; S2. Input the acquired user's historical behavior sequence and target short video into the trained short video recommendation model. First, the input historical behavior sequence is processed by a hypercyclic neural network designed based on super nodes to capture the sequence dependency between historical behavior sequences and obtain the hidden state of each unit of the hypercyclic neural network. The number of units in the hypercyclic neural network is the same as the length of the user's historical behavior sequence. When the input is a user interaction behavior of the like type, an additional layer of high-speed neural network is added to each unit of the hypercyclic neural network to model the directed connection between the click node and the like node. S3. Divide the user's historical behavior sequence into sub-behavior sequences under different interactive behavior spaces according to the user's interactive behavior type, and use the additive attention mechanism based on the target short video to process the sub-behavior sequence, the hidden state of the hypercyclic neural network unit and the target short video to calculate the user preferences under different interactive behavior spaces; S4. Integrate user preferences in different interactive behavior spaces, output the probability of users clicking on the target short video, and complete the recommendation of short videos.

2. A short video recommendation method based on hyperloop network and multi-task training as claimed in claim 1, characterized in that: In step S1, there is a directed connection between the internal click node and the like node, and both the internal click node and the like node indicate the user's positive preference for the target short video, and the like node indicates that the user is more satisfied with the target short video; The nodes inside the supernode are activated according to the input user interaction behavior category: if the input user interaction behavior is skip, the internal skip node is activated, otherwise the internal click node is activated; if the input user interaction behavior is like, the internal click and like nodes are activated at the same time.

3. The short video recommendation method based on hyperloop network and multi-task training as claimed in claim 1, characterized in that: In step S2, during one cycle of the t-th unit of the hypercyclic neural network, the hidden state of the t-1-th unit is processed by the first unit update function and the unit copy function to obtain the hidden state of the t-th unit.

4. A short video recommendation method based on hyperloop network and multi-task training as claimed in claim 3, characterized in that: In step S2, when the indicator function of the tth unit When , the unit replication function is expressed as When the indicator function of the tth unit When , the first unit update function of the t-th unit is represented by the indicator function of the t-1-th unit, the output of the hidden layer of the t-th unit, the output of the update gate, and the candidate activation vector.

5. A short video recommendation method based on hyperloop network and multi-task training as claimed in claim 4, characterized in that: Fast neural network update function with second unit Update the hidden state of the tth unit.

6. The short video recommendation method based on hyperloop network and multi-task training as claimed in claim 1, characterized in that: In step S3, the sub-behavior sequences under different interactive behavior spaces are respectively a skip sub-behavior sequence, a click sub-behavior sequence, a like sub-behavior sequence, and a click but not like sub-behavior sequence, wherein the click sub-behavior sequence contains all elements in the like sub-behavior sequence, and the click but not like sub-behavior sequence is composed of all elements that belong to the click sub-behavior sequence but not to the like sub-behavior sequence.

7. A short video recommendation method based on hyperloop network and multi-task training as claimed in claim 6, characterized in that: In step S3, the hidden state of the corresponding sub-behavior sequence is extracted from the final output of the hypercyclic neural network according to the subscript of the sub-behavior sequence, and the user preferences in different interactive behavior spaces are obtained using an additive attention mechanism based on the target short video.

8. The short video recommendation method based on hyperloop network and multi-task training as claimed in claim 1, characterized in that: The specific process of step S4 is as follows: S41. Perform a weighted sum of the user's skip preference and click preference to obtain a user preference related to the user's click rate; S42. Connect the user preference related to the user click rate and the target short video representation to obtain a first connected representation and input it into the first prediction network to obtain the output probability of the user clicking the target short video; the first prediction network is implemented by two layers of MLP.

9. The short video recommendation method based on hyperloop network and multi-task training as claimed in claim 1, characterized in that: In the training process of the short video recommendation model, the cross entropy loss between the probability of the user clicking on the target short video and the true value of the user clicking on the short video is calculated as the click-through rate prediction loss, the cross entropy loss between the probability of the user liking the target short video and the true value of the user liking the short video is calculated as the like rate prediction loss, the click rate prediction loss and the like rate prediction loss are weighted as the total loss, the short video recommendation model parameters are updated based on minimizing the total loss, and the trained short video recommendation model is obtained by continuous iteration; The probability that a user likes a target short video is obtained by: performing a weighted summation of the user's click-but-don't-like preference and the like preference to obtain a user preference related to the user's like rate; connecting the user preference related to the user's like rate and the target short video representation to obtain a second connected representation and input it into a second prediction network to obtain an output probability that the user likes the target short video; the second prediction network is implemented by a two-layer MLP.

10. A short video recommendation system based on hyperloop network and multi-task training, characterized in that: include: The supernode construction module is used to construct supernodes in the hypercyclic neural network, which is used to integrate multiple user interaction behaviors, so as to model the semantic associations of different user interaction behaviors; each supernode contains three nodes, corresponding to the three types of user interaction behaviors: skip, click and like; The dependency acquisition module is used to input the acquired user's historical behavior sequence and target short video into the trained short video recommendation model. The input historical behavior sequence is first processed by a hypercyclic neural network designed based on super nodes to capture the sequence dependency between the historical behavior sequences and obtain the hidden state of each unit of the hypercyclic neural network. The number of units in the hypercyclic neural network is the same as the length of the user's historical behavior sequence. When the input is a user interaction behavior of the like type, an additional layer of high-speed neural network is added to each unit of the hypercyclic neural network to model the directed connection between the click node and the like node. The user preference acquisition module is used to divide the user's historical behavior sequence into sub-behavior sequences under different interactive behavior spaces according to the user's interactive behavior type. The sub-behavior sequence, the hidden state of the hypercyclic neural network unit and the target short video are processed by the additive attention mechanism based on the target short video to calculate the user preference under different interactive behavior spaces; The result acquisition module is used to integrate user preferences in different interactive behavior spaces, output the probability of users clicking on the target short video, and complete the recommendation of short videos.

Images