Hierarchical attention micro-video sequence recommendation method and device based on multi-scale modeling

Abstract

The present invention proposes a hierarchical attention micro-video sequence recommendation method and device based on multi-scale modeling. By predicting the user's click-through rate for micro-videos and then determining whether to recommend them to the user, the recommendation performance of the micro-video platform can be effectively improved. The sequence recommendation method for the micro-video sharing platform includes: step 1, using a time window to capture the user's short-term dynamic interest representation; step 2, using the time window information obtained in step 1, using the category-level attention mechanism to describe the user's coarse-grained interest representation; step 3, using the time window information obtained in step 1, using the item-level attention mechanism to capture the fine-grained user interest representation; step 4, using the user interest representation obtained in steps 2 and 3, using a forward multi-head self-attention mechanism to identify and integrate the unified representation of the long-term correlation between the previously segmented time windows, and using vector information to recommend micro-video sequences.

Description

Technical Field

[0001] The present invention relates to the field of micro-video sharing platform recommendation technology, and in particular to a hierarchical attention micro-video sequence recommendation method and device based on multi-scale modeling. Background Art

[0002] Recently, micro-video sharing services have become extremely popular. Users can shoot short videos (usually tens of seconds) with their smartphones and then upload them to online platforms (such as Kuaishou and Douyin). Many silent users are attracted to the micro-videos on online platforms and browse or interact with them, but their contributions are relatively small. On popular micro-video sharing platforms in China, there are more than 60 million active users every day, and users upload more than 10 million micro-videos every day. This shows that there is a clear phenomenon of information overload. Therefore, there is an urgent need to design efficient sequential recommendation algorithms for micro-video sharing platforms to help users find their favorite micro-videos and thus improve the user experience of sharing service platforms.

[0003] Specifically in the micro-video recommendation scenario, some progress has been made in recent years, but this task is still very difficult and cannot solve the information overload problem of micro-video sharing platforms, resulting in poor recommendation performance of micro-video sharing platforms. Summary of the Invention

[0004] The present invention proposes a hierarchical attention micro-video sequence recommendation method and device based on multi-scale modeling, which is used to solve or at least partially solve the technical problem of poor recommendation performance of micro-video sharing platforms in the prior art.

[0005] To achieve the above objectives, the first aspect of the present invention provides a hierarchical attention micro-video sequence recommendation method based on multi-scale modeling, comprising:

[0006] S1: Use time windows to capture users' short-term dynamic interest representations, where user behavior sequences are divided into multiple time windows, each of which contains multiple micro-videos;

[0007] S2: Based on the micro-videos contained in the time window, the category-level attention mechanism is used to describe the user's coarse-grained interest representation for short-term dynamic interest representation;

[0008] S3: Based on the micro-videos contained in the time window, the item-level attention mechanism is used to describe the user's fine-grained interest representation for short-term dynamic interest representation;

[0009] S4: Integrate the user's coarse-grained interest representation and the user's fine-grained interest representation, and use a forward multi-head self-attention mechanism to identify and integrate the unified representation vector of the long-term correlation between the segmented time windows;

[0010] S5: Based on the user's coarse-grained interest representation, the user's fine-grained interest representation, and the unified representation vector, the interaction probability between the user and the item is estimated by using a prediction function, and sequential recommendation is performed using the user's coarse-grained interest representation, the user's fine-grained interest representation, and the unified representation vector.

[0011] In one embodiment, step S1 is implemented in the following manner:

[0012]

[0013] in, is the t-th visual feature vector, representing the user's short-term dynamic interest representation, f represents the visual feature vector; E f is an embedding matrix obtained through learning, Is the use of micro-videos i The visual feature vector extracted from the cover image by the Inception-v3 model pre-trained on ImageNet, s i represents the i-th micro-video.

[0014] In one embodiment, step S2 is implemented in the following manner:

[0015]

[0016] Where ⊙ represents element-by-element multiplication; k represents the order of the micro-video in the micro-video sequence; represents the user's coarse-grained interest representation, t represents the time window sorting number, and c represents the category level; c tj represents the category-level vector representation of the j-th micro-video in the t-th time window; Represents the normalized attention weight of the category-level attention score.

[0017] In one embodiment, step S3 is implemented in the following manner:

[0018]

[0019] Among them, ⊙ represents element-by-element multiplication, represents the fine-grained interest representation of the user, t represents the sorting number of the time window, and f represents the visual feature vector; f tj represents the j-th micro-video item-level vector representation in the t-th time window, Represents the normalized attention weight of the attention score at the visual feature level.

[0020] In one embodiment, step S4 includes:

[0021] S4.1: Fusing the user's coarse-grained interest representation with the user's fine-grained interest representation to obtain the user's local interest representation;

[0022] S4.2: Representing the user's local interests {l t ,t=1,2…,m} are copied three times and recorded as {q t},{k t} and {v t}, {q t},{k t} and {v t} respectively represent query items, key items and value items, subscript t represents the sort number of the time window, and {q t} and {k t}Calculate the attention score, that is, {q t} and {k t} is called the attention score, and then the calculated attention score is combined with {v t}Weighted aggregation obtains global interest representation;

[0023] S4.3: Based on the global interest representation obtained in S4.2, {q t},{k t} and {v t} linearly map z times, each time using a different projection matrix to map it to d z In dimensional space, set d z =d q =d k =d v =d / z,d z d k d v , d represent the dimension of query item, the dimension of key item, the dimension of value item and the dimension of user global interest representation respectively; wherein, in each mapping process, between the tth and jth time windows, {q t} and {k j} multiplication operation to obtain a d v Attention score of the global interest representation of the dimension;

[0024] S4.4: Based on the attention score of the global interest representation obtained in S4.3, the attention score tensor is calculated And add a directed mask H to the obtained attention score tensor;

[0025] S4.5: Based on the sum of the attention score tensor obtained in S4.4 and a directed mask, the softmax function is normalized to obtain the attention weight of the global interest representation;

[0026] S4.6: Calculate the attention weight tensor L based on the global interest representation obtained in S4.5. Each element of L is a d z dimensional vector, and obtain the user's global interest representation g based on the tensor L t ;

[0027] S4.7: Obtain the user's final interest representation based on the user's local interest representation and the user's global interest representation.

[0028] In one embodiment, during each mapping process in step S4.3, d is calculated. v The attention score of the global interest representation of the dimension is:

[0029]

[0030] where r = 1, 2, ..., z; is the weight matrix; and is {q t} and {k j}Projection matrix parameters for both; is the bias vector.

[0031] In one embodiment, the elements of the directional mask H in step S4.4 are:

[0032]

[0033] When the attention score of the global interest representation is normalized by the softmax function through a directed mask, if j ≥ t, the position information in the j time window contributes to the position weight in the t time window to 0.

[0034] In one embodiment, the elements of the tensor L in step S4.6 are:

[0035]

[0036] where i=1,2,…,z, Indicates v j The projection matrix parameters, where v j Represents the jth value item, each element of L is a d z dimensional vector, representing the tensor of attention weights. After attention weighting, the tensor is reorganized along the r index, and the spliced ​​shape of this dimension is Finally, sum along the dimension t to obtain {g1,g2,…,g m},in

[0037] In one embodiment, the calculation method of the user's final interest representation in step S4.7 is:

[0038] i t =LN(l t +g t )

[0039] Among them, LN() is a normalization method, l t represents the user's local interest representation, g t Represents the user's global interest representation.

[0040] Based on the same inventive concept, the second aspect of the present invention provides a hierarchical attention micro-video sequence recommendation device based on multi-scale modeling, comprising:

[0041] The information acquisition module is used to capture the user's short-term dynamic interest representation using time windows, where the user behavior sequence is divided into multiple time windows, each of which contains multiple micro-videos;

[0042] The category representation module is used to describe the user's coarse-grained interest representation based on the micro-videos contained in the time window and the short-term dynamic interest representation using the category-level attention mechanism;

[0043] The item representation module is used to describe the user's fine-grained interest representation based on the micro-videos contained in the time window and the short-term dynamic interest representation using the item-level attention mechanism;

[0044] The vector integration module is used to integrate the user's coarse-grained interest representation and the user's fine-grained interest representation, and adopts a forward multi-head self-attention mechanism to identify and integrate the unified representation vector of the long-term correlation between the segmented time windows;

[0045] The sequential recommendation module is used to estimate the interaction probability between users and items based on the user's coarse-grained interest representation, the user's fine-grained interest representation, and the unified representation vector by using a prediction function, and then perform sequential recommendation using the user's coarse-grained interest representation, the user's fine-grained interest representation, and the unified representation vector;

[0046] The control unit is respectively connected to the information acquisition module, the category representation module, the item representation module, the vector integration module, and the sequence recommendation module to control the operation of the cutting module.

[0047] Compared with the prior art, the advantages and beneficial technical effects of the present invention are as follows:

[0048] The present invention provides a hierarchical attention micro-video sequence recommendation method and device based on multi-scale modeling, and constructs a new micro-video sharing platform sequence recommendation model MMTHA (Multi-scale Modeling TemporalHierarchical Attention, including an information acquisition module, a category representation module, a project representation module, a vector integration module and a sequence recommendation module. First, a time window is used to capture the user's short-term dynamic interests; second, a category-level attention mechanism is used to describe the user's coarse-grained interests, and a project-level attention mechanism captures the fine-grained user interests; finally, a forward multi-head self-attention mechanism is used to identify and integrate the long-term correlations between previously segmented time windows; therefore, under the premise of predicting the user's click-through rate, the present invention can mine the best user preference representation from the user's historical behavior sequence to meet the user's personalized needs for items of interest, help users discover information that is valuable to them, and at the same time enable the information to be displayed to users who are interested in it, effectively improving the recommendation performance of the micro-video sharing platform. BRIEF DESCRIPTION OF THE DRAWINGS

[0049] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the following briefly introduces the drawings required for use in the embodiments or the description of the prior art. Obviously, the drawings described below are some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative work.

[0050] Figure 1 A flowchart of a hierarchical attention micro-video sequence recommendation method based on multi-scale modeling provided by an embodiment of the present invention;

[0051] Figure 2 This is a structural diagram of the micro-video sharing platform sequence recommendation model MMTHA involved in an embodiment of the present invention;

[0052] Figure 3 This is a module diagram of the forward multi-head self-attention mechanism involved in an embodiment of the present invention (corresponding to step S4). DETAILED DESCRIPTION

[0053] The purpose of the present invention is to provide a hierarchical attention micro-video sequence recommendation method and device based on multi-scale modeling, which can solve the information overload problem of the micro-video sharing platform and effectively improve the recommendation performance of the micro-video sharing platform.

[0054] To make the objectives, technical solutions, and advantages of the embodiments of the present invention more clear, the technical solutions in the embodiments of the present invention will be clearly and completely described below in conjunction with the accompanying drawings in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without making creative efforts shall fall within the scope of protection of the present invention.

[0055] Example 1

[0056] The embodiment of the present invention provides a hierarchical attention micro-video sequence recommendation method based on multi-scale modeling, including:

[0057] S1: Use time windows to capture users' short-term dynamic interest representations, where user behavior sequences are divided into multiple time windows, each of which contains multiple micro-videos;

[0058] S2: Based on the micro-videos contained in the time window, the category-level attention mechanism is used to describe the user's coarse-grained interest representation for short-term dynamic interest representation;

[0059] S3: Based on the micro-videos contained in the time window, the item-level attention mechanism is used to describe the user's fine-grained interest representation for short-term dynamic interest representation;

[0060] S4: Integrate the user's coarse-grained interest representation and the user's fine-grained interest representation, and use a forward multi-head self-attention mechanism to identify and integrate the unified representation vector of the long-term correlation between the segmented time windows;

[0061] S5: Based on the user's coarse-grained interest representation, the user's fine-grained interest representation, and the unified representation vector, the interaction probability between the user and the item is estimated by using a prediction function, and sequential recommendation is performed using the user's coarse-grained interest representation, the user's fine-grained interest representation, and the unified representation vector.

[0062] See Figure 1 , which is a flowchart of the hierarchical attention micro-video sequence recommendation method based on multi-scale modeling provided in an embodiment of the present invention.

[0063] In the specific implementation process, the user behavior sequence is first divided into m time windows, each time window contains k micro-videos (m×k=n), and n represents the number of user behavior sequences.

[0064] In one embodiment, step S1 is implemented in the following manner:

[0065]

[0066] in, is the t-th visual feature vector, representing the user's short-term dynamic interest representation, f represents the visual feature vector; E f is an embedding matrix obtained through learning, Is the use of micro-videos i The visual feature vector extracted from the cover image by the Inception-v3 model pre-trained on ImageNet, s i represents the i-th micro-video.

[0067] Specifically, a time window is used to capture the user's short-term dynamic interest representation, and a micro-video sequence clicked by a user is recorded as {s1, s2, ...s n}, the purpose of which is to map it into a feature vector sequence {x1,x2,…,x n},in Get the feature vector representation of each item. For each micro video s i , you can use the Inception-v3 model pre-trained on ImageNet to extract the corresponding visual feature vector from the cover image because is a high-dimensional vector, which is embedded by learning an embedding matrix E f Projecting it into the low-dimensional space, its expression is shown above.

[0068] In the specific implementation process, each micro video has only one category attribute. Category information can be represented by a one-hot vector. Therefore, micro video s t The category video one-hot encoding vector is denoted as because It is also a high-dimensional vector, which can be obtained by learning an embedding matrix E c Mapping it into a low-dimensional space, its expression is as follows:

[0069]

[0070] in is the category feature vector, E c represents an embedding matrix obtained through learning, Indicates micro videos t The one-hot encoded vector of the category video.

[0071] In one embodiment, step S2 is implemented in the following manner:

[0072]

[0073] Where ⊙ represents element-by-element multiplication; k represents the order of the micro-video in the micro-video sequence; represents the user's coarse-grained interest representation, t represents the time window sorting number, and c represents the category level; c tj represents the category-level vector representation of the j-th micro-video in the t-th time window; Represents the normalized attention weight of the category-level attention score.

[0074] Specifically, in step S2, based on the short-term dynamic interest representation obtained in step S1, the j-th micro-video in the t-th time window is recorded as s tj , whose eigenvector is denoted as x tj , where t=1,2,…,m,j=1,2,…k. The category-level attention score is calculated as follows:

[0075] β c (c tj )=W c σ(W1c tj +W2f tj +b1)+b2,

[0076] in is the weight parameter matrix to be learned; and is the bias vector; σ(·) is the element activation function; c tj represents the category-level vector representation in the t-th time window; f tj Represents the item-level vector representation in the t-th time window. Then, the softmax function is used for normalization and the attention score is calculated as follows:

[0077]

[0078] Then, after the final category-level representation is normalized by the softmax function, the normalized attention weight is used in the t-th time window. It is used to aggregate the category-level information of items, that is, to aggregate them into the coarse-grained interest representation of users.

[0079] In one embodiment, step S3 is implemented in the following manner:

[0080]

[0081] Among them, ⊙ represents element-by-element multiplication, represents the fine-grained interest representation of the user, t represents the sorting number of the time window, and f represents the visual feature vector; f tj represents the j-th micro-video item-level vector representation in the t-th time window, Represents the normalized attention weight of the attention score at the visual feature level.

[0082] Specifically, in step S3, based on the time window information obtained in step 1, the item-level attention mechanism is used to capture fine-grained user interest representation for short-term dynamic interest representation. After the item-level representation is normalized by the softmax function, the normalized attention weight is used to aggregate item-level (i.e., micro-video) information within the t-th time window, that is, aggregated into the user's fine-grained interest representation.

[0083] In the specific implementation process, based on the time window information and short-term dynamic interest representation obtained by S1, the j-th micro video in the t-th time window is recorded as s tj , whose eigenvector is denoted as x tj , where t=1,2,…,m,j=1,2,…k. Similarly, the item-level attention score is calculated using the following method:

[0084] β f (f tj )=W f σ(W ′ 1c tj +W ′ 2f tj +b ′ 1)+b ′ 2

[0085] It should be noted that the values ​​of t and j and the embedded representation of items specified here are only for illustrating the specific implementation process of the present invention and are not intended to limit the present invention.

[0086] in is the weight parameter matrix to be learned; and is the bias vector; σ(·) is the element activation function. Then, the attention score is calculated by using the softmax function for normalization, which is calculated as follows:

[0087]

[0088] However, after the final item-level representation is normalized by the softmax function, the normalized attention weight is used to aggregate the item-level (i.e., micro-video) information within the t-th time window, that is, aggregated into the user's fine-grained interest representation, which is calculated as follows:

[0089]

[0090] Where ⊙ represents element-by-element multiplication, f tj represents the item-level vector representation in the t-th time window, β c Represents the attention score. Finally, the category-level vector representation of the item and item-level vector representations Splice them together and finally get the final user interest vector representation in the t-th time window

[0091] In one embodiment, step S4 includes:

[0092] S4.1: Fuse the user's coarse-grained interest representation with the user's fine-grained interest representation to obtain the user's local interest representation;

[0093] S4.2: Representing the user's local interests {l t ,t=1,2…,m} are copied three times and recorded as {q t},{k t} and {v t}, {q t},{k t} and {v t} respectively represent query items, key items and value items, subscript t represents the sort number of the time window, and {q t} and {k t}Calculate the attention score, that is, {q t} and {k t} is called the attention score, and then the calculated attention score is combined with {v t}Weighted aggregation obtains global interest representation;

[0094] S4.3: Based on the global interest representation obtained in S4.2, {q t},{k t} and {v t} linearly map z times, each time using a different projection matrix to map it to d z In dimensional space, set d z =d q =d k =d v =d / z,d z d k d v , d represent the dimension of query item, the dimension of key item, the dimension of value item and the dimension of user global interest representation respectively; wherein, in each mapping process, between the tth and jth time windows, {q t} and {k j} multiplication operation to obtain a d v Attention score of the global interest representation of the dimension;

[0095] S4.4: Based on the attention score of the global interest representation obtained in S4.3, the attention score tensor is calculated And add a directed mask H to the obtained attention score tensor;

[0096] S4.5: Based on the sum of the attention score tensor obtained in S4.4 and a directed mask, perform softmax function normalization to obtain the attention weights of the global interest representation;

[0097] S4.6: Calculate the tensor L based on the attention weights of the global interest representation obtained in S4.5. Each element of L is a d z -dimensional vector, and obtain the global interest representation g of the user based on the tensor L t ;

[0098] S4.7: Obtain the final interest representation of the user according to the local interest representation and the global interest representation of the user. Specifically, the expression of the global interest representation obtained in step S4.2 is:

[0099] g t {t = 1, 2…, m}

[0100] Please refer to Figure 3 , which is a schematic diagram of the forward multi-head self-attention mechanism module involved in the embodiments of the present invention.

[0101] In order to capture the long-range dependencies in the user behavior sequence, S4.4 adds a directed mask H to the obtained attention score tensor.

[0102] In S4.5, based on the attention score tensor obtained in step S4.4, when the attention scores of the global interest representation obtained in step S4.3 are normalized by the softmax function, if j ≥ t, the contribution of the position information at the j position to the weight at the t position is 0. Thus, when the attention model is at the t position, it only focuses on the position information where j < t. The obtained softmax function-normalized attention weights will be used for

[0103] Please refer to Figure 2 , which is a schematic diagram of the structure of the micro-video sharing platform sequence recommendation model MMTHA involved in the embodiments of the present invention.

[0104] In one implementation, during each mapping process in step S4.3, the method for calculating the attention scores of the d v -dimensional global interest representation is:

[0105]

[0106] where r = 1, 2,…, z; is the weight matrix; and is {q t} and kj The projection matrix parameters of both; is the bias vector.

[0107] Specifically, taking the r-th mapping as an example, between the t-th and j-th time windows, through q t and k j generate an attention score of dimension d v The calculation method is as shown in the above formula.

[0108] In one embodiment, the elements of the directed mask H in step S4.4 are:

[0109]

[0110] When the attention scores of the global interest representation are normalized by the softmax function through the directed mask, if j≥t, the contribution of the position information in the j-th time window to the position weight in the t-th time window is 0.

[0111] In one embodiment, the elements of the tensor L in step S4.6 are:

[0112]

[0113] where i = 1, 2,..., z, represents the projection matrix parameter of v j where v j represents the j-th value item, and each element of L is a d z dimensional vector, representing the tensor of attention weights. After attention weighting, the tensor is reorganized. Along the r index, this dimension is concatenated with a shape of Finally, summing along the t dimension, we get {g1, g2,..., g m}, where

[0114] Specifically, in step S4.5, using the attention score tensor obtained in step S4.4, when the attention scores are normalized by the softmax function, if j≥t, the contribution of the j position information to the t position weight is 0. Thus, when the attention model is at the t position, it only focuses on the position information where j < t. The obtained softmax function-normalized attention weights will be used for

[0115] where d = d z ×z, so a tensor L of shape m×m×d will be formed.

[0116] In one embodiment, the calculation method of the user's final interest representation in step S4.7 is:

[0117] i t =LN(l t +g t )

[0118] Among them, LN() is a normalization method, l t represents the user's local interest representation, g t Represents the user's global interest representation.

[0119] In the implementation, we use stochastic batch gradient descent to optimize the MMTHA model in mini-batch mode. This is a widely applicable optimization method for neural network models. When constructing a batch unit, we sample a batch of historical user-item interaction pairs (u,i).

[0120] This paper builds on existing sequential recommendation methods and proposes a hierarchical attention-based micro-video sequence recommendation method based on multi-scale modeling. First, time windows are used to capture users' short-term dynamic interests. Second, a category-level attention mechanism is used to describe users' coarse-grained interests. Third, an item-level attention mechanism is used to capture fine-grained user interests. Finally, a forward multi-head self-attention mechanism is used to identify and integrate long-term correlations between previously segmented time windows, and the integrated embedding vector information is used for personalized recommendation services.

[0121] Example 2

[0122] Based on the same inventive concept, this embodiment provides a hierarchical attention micro-video sequence recommendation device based on multi-scale modeling, including:

[0123] The information acquisition module is used to capture the user's short-term dynamic interest representation using time windows, where the user behavior sequence is divided into multiple time windows, each of which contains multiple micro-videos;

[0124] The category representation module is used to describe the user's coarse-grained interest representation based on the micro-videos contained in the time window and the short-term dynamic interest representation using the category-level attention mechanism;

[0125] The item representation module is used to describe the user's fine-grained interest representation based on the micro-videos contained in the time window and the short-term dynamic interest representation using the item-level attention mechanism;

[0126] The vector integration module is used to integrate the user's coarse-grained interest representation and the user's fine-grained interest representation, and adopts a forward multi-head self-attention mechanism to identify and integrate the unified representation vector of the long-term correlation between the segmented time windows;

[0127] The sequential recommendation module is used to estimate the interaction probability between users and items based on the user's coarse-grained interest representation, the user's fine-grained interest representation, and the unified representation vector by using a prediction function, and then perform sequential recommendation using the user's coarse-grained interest representation, the user's fine-grained interest representation, and the unified representation vector;

[0128] The control unit is respectively connected to the information acquisition module, the category representation module, the item representation module, the vector integration module, and the sequence recommendation module to control the operation of the cutting module.

[0129] Specifically, the information acquisition module uses a time window to capture the user's short-term dynamic interest representation according to the content described in step S1 of the first embodiment;

[0130] The category representation module uses the category-level attention mechanism to describe the user's coarse-grained interest representation based on the content described in step S2 of the first embodiment, based on the user's short-term dynamic interest representation information obtained by the information acquisition module;

[0131] The item representation module uses the item-level attention mechanism to describe the user's fine-grained interest representation based on the short-term dynamic interest representation information of the user obtained by the information acquisition module according to the content described in step S3 of the first embodiment;

[0132] The vector integration module integrates the user interest representations obtained by the category representation module and the item representation module according to the content described in step S4 of Example 1, and uses a forward multi-head self-attention mechanism to identify and integrate a unified representation vector of the long-term correlation between the previously segmented time windows;

[0133] The sequential recommendation module uses the user interest representation information obtained by the vector fusion module and the representation module, estimates the interaction probability between users and items by using the prediction function, and uses the embedding vector information for sequential recommendation.

[0134] Preferably, the hierarchical attention micro-video sequence recommendation device based on multi-scale modeling provided by the present invention may also include: an input display unit, which is communicatively connected to the information acquisition module category representation module, item representation module, vector integration module, sequence recommendation module, and control unit, allowing the user to input control instructions and perform corresponding display according to the control instructions.

[0135] The control unit is connected to the information acquisition module, category representation module, item representation module, vector integration module, and sequence recommendation module to control their operation.

[0136] Functions and effects of the invention

[0137] The present invention provides a method and device for recommending micro-video sequences based on multi-scale modeling and hierarchical attention. This method is used to construct a new sequence recommendation model for micro-video sharing platforms, MMTHA (Multi-scale Modeling Temporal Hierarchical Attention). First, a time window is used to capture users' short-term dynamic interests. Second, a category-level attention mechanism is used to describe users' coarse-grained interests, while an item-level attention mechanism captures fine-grained user interests. Finally, a forward multi-head self-attention mechanism is used to identify and integrate long-term correlations between previously segmented time windows. Therefore, while predicting user click-through rates, the present invention can also mine the best user preference representation from the user's historical behavior sequence to meet the user's personalized needs for items of interest, helping users discover valuable information while allowing the information to be presented to interested users, effectively improving the recommendation performance of the micro-video sharing platform.

[0138] Since the device described in Example 2 of the present invention is used to implement the hierarchical attention micro-video sequence recommendation method based on multi-scale modeling in Example 1 of the present invention, the specific structure and variations of the device are readily understood by those skilled in the art based on the method described in Example 1 of the present invention, and thus will not be further described here. All devices used in the method of Example 1 of the present invention fall within the scope of protection of the present invention.

[0139] It will be understood by those skilled in the art that embodiments of the present invention may be provided as methods, systems, or computer program products. Thus, the present invention may take the form of an entirely hardware embodiment, an entirely software embodiment, or an embodiment combining software and hardware. Furthermore, the present invention may take the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to magnetic disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0140] The present invention is described with reference to flowcharts and / or block diagrams of methods, devices (systems), and computer program products according to embodiments of the present invention. It should be understood that each process and / or block in the flowcharts and / or block diagrams, as well as combinations of processes and / or blocks in the flowcharts and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to produce a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the processes in the flowcharts and / or block diagrams. Figure 1 a process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0141] Although the preferred embodiments of the present invention have been described, those skilled in the art may make additional changes and modifications to these embodiments once they have learned the basic creative concept. Therefore, the appended claims are intended to be interpreted as including the preferred embodiments and all changes and modifications that fall within the scope of the present invention.

[0142] Obviously, those skilled in the art may make various changes and modifications to the embodiments of the present invention without departing from the spirit and scope of the embodiments of the present invention. Thus, if such changes and modifications of the embodiments of the present invention fall within the scope of the claims and their equivalents, the present invention is intended to include such changes and modifications.

Claims

1. A hierarchical attention micro-video sequence recommendation method based on multi-scale modeling, characterized by: include: S1: Use time windows to capture users' short-term dynamic interest representations, where user behavior sequences are divided into multiple time windows, each of which contains multiple micro-videos; S2: Based on the micro-videos contained in the time window, the category-level attention mechanism is used to describe the user's coarse-grained interest representation for short-term dynamic interest representation; S3: Based on the micro-videos contained in the time window, the item-level attention mechanism is used to describe the user's fine-grained interest representation for short-term dynamic interest representation; S4: Integrate the user's coarse-grained interest representation and the user's fine-grained interest representation, and use a forward multi-head self-attention mechanism to identify and integrate the unified representation vector of the long-term correlation between the segmented time windows; S5: Based on the user's coarse-grained interest representation, the user's fine-grained interest representation, and the unified representation vector, the interaction probability between the user and the item is estimated by using a prediction function, and sequential recommendation is performed using the user's coarse-grained interest representation, the user's fine-grained interest representation, and the unified representation vector; Step S2 is implemented in the following manner: Where ⊙ represents element-by-element multiplication; k represents the order of the micro-video in the micro-video sequence; represents the user's coarse-grained interest representation, t represents the time window sorting number, and c represents the category level; c tj represents the category-level vector representation of the j-th micro-video in the t-th time window; represents the normalized attention weight of the category-level attention score; Step S3 is implemented in the following manner: Among them, ⊙ represents element-by-element multiplication, represents the fine-grained interest representation of the user, t represents the sorting number of the time window, and f represents the visual feature vector; f tj represents the j-th micro-video item-level vector representation in the t-th time window, Represents the normalized attention weight of the attention score at the visual feature level.

2. The hierarchical attention micro-video sequence recommendation method based on multi-scale modeling according to claim 1, characterized in that: Step S1 is implemented in the following manner: in, is the t-th visual feature vector, representing the user's short-term dynamic interest representation, f represents the visual feature vector; E f is an embedding matrix obtained through learning, Is the use of micro-videos i The visual feature vector extracted from the cover image by the Inception-v3 model pre-trained on ImageNet, s i represents the i-th micro-video.

3. The hierarchical attention micro-video sequence recommendation method based on multi-scale modeling according to claim 1, characterized in that: Step S4 includes: S4.1: Fuse the user's coarse-grained interest representation with the user's fine-grained interest representation to obtain the user's local interest representation; S4.2: Representing the user's local interests {l t ,t=1,2…,m} are copied three times and recorded as {q t },{k t } and {v t }, {q t },{k t } and {v t } respectively represent query items, key items and value items, subscript t represents the sort number of the time window, and {q t } and {k t }Calculate the attention score, that is, {q t } and {k t } is called the attention score, and then the calculated attention score is combined with {v t }Weighted aggregation obtains global interest representation; S4.3: Based on the global interest representation obtained in S4.2, {q t },{k t } and {v t } linearly map z times, each time using a different projection matrix to map it to d z In dimensional space, set d z =d q =d k =d v =d / z,d q d k d v , d represent the dimension of query item, the dimension of key item, the dimension of value item and the dimension of user global interest representation respectively; wherein, in each mapping process, between the tth and jth time windows, {q t } and {k j } multiplication operation to obtain a d v Attention score of the global interest representation of the dimension; S4.4: Based on the attention score of the global interest representation obtained in S4.3, the attention score tensor is calculated And add a directed mask H to the obtained attention score tensor; S4.5: Based on the sum of the attention score tensor obtained in S4.4 and a directed mask, the softmax function is normalized to obtain the attention weight of the global interest representation; S4.6: Calculate the attention weight tensor L based on the global interest representation obtained in S4.

5. Each element of L is a d z dimensional vector, and obtain the user's global interest representation g based on the tensor L t ; S4.7: Obtain the user's final interest representation based on the user's local interest representation and the user's global interest representation.

4. The hierarchical attention micro-video sequence recommendation method based on multi-scale modeling according to claim 3 is characterized in that: In each mapping process in step S4.3, calculate d v The attention score of the global interest representation of the dimension is: in is the weight matrix; is {q t } and {k j }Projection matrix parameters for both; is the bias vector.

5. The hierarchical attention micro-video sequence recommendation method based on multi-scale modeling according to claim 3 is characterized in that: The elements of the directional mask H in step S4.4 are: When the attention score of the global interest representation is normalized by the softmax function through a directed mask, if j ≥ t, the position information in the j time window contributes to the position weight in the t time window to 0.

6. The hierarchical attention micro-video sequence recommendation method based on multi-scale modeling according to claim 3, characterized in that: The elements of the tensor L in step S4.6 are: where i=1,2,…,z, Indicates v j The projection matrix parameters, where v j Represents the jth value item, each element of L is a d z dimensional vector, representing the tensor of attention weights. After attention weighting, the tensor is reorganized along the r index, and the spliced ​​shape of this dimension is Finally, sum along the dimension t to obtain {g1,g2,…,g m },in 7. The hierarchical attention micro-video sequence recommendation method based on multi-scale modeling according to claim 3, characterized in that: The calculation method of the user's final interest representation in step S4.7 is: i t =LN(l t +g t ) Among them, LN() is a normalization method, l t represents the user's local interest representation, g t Represents the user's global interest representation.

8. A hierarchical attention micro-video sequence recommendation device based on multi-scale modeling, characterized by: The device for implementing the hierarchical attention micro-video sequence recommendation method based on multi-scale modeling as claimed in claim 1 comprises: The information acquisition module is used to capture the user's short-term dynamic interest representation using time windows, where the user behavior sequence is divided into multiple time windows, each of which contains multiple micro-videos; The category representation module is used to describe the user's coarse-grained interest representation based on the micro-videos contained in the time window and the short-term dynamic interest representation using the category-level attention mechanism; The item representation module is used to describe the user's fine-grained interest representation based on the micro-videos contained in the time window and the short-term dynamic interest representation using the item-level attention mechanism; The vector integration module is used to integrate the user's coarse-grained interest representation and the user's fine-grained interest representation, and adopts a forward multi-head self-attention mechanism to identify and integrate the unified representation vector of the long-term correlation between the segmented time windows; The sequential recommendation module is used to estimate the interaction probability between users and items based on the user's coarse-grained interest representation, the user's fine-grained interest representation, and the unified representation vector by using a prediction function, and then perform sequential recommendation using the user's coarse-grained interest representation, the user's fine-grained interest representation, and the unified representation vector; The control unit is respectively connected to the information acquisition module, category representation module, item representation module, vector integration module, and sequence recommendation module to control the operation of each module.

Images