An interest frame recommendation method based on high-definition image generation technology

Abstract

The application belongs to the technical field of Internet application, and particularly relates to an interest frame recommendation method based on high-definition image generation technology, which comprises the following steps: obtaining historical interaction behavior data of a user, extracting a user behavior sequence from the historical interaction behavior data, embedding the user behavior sequence, fusing the embedded user behavior sequence with corresponding granularity information, and obtaining an input vector; constructing a prediction model, wherein an Item2Frame layer of the prediction model acquires a multi-interest frame matrix according to the input vector, and a high-definition interest portrait generation layer of the prediction model performs noise compensation on the obtained multi-interest frame matrix to obtain a vector representation of an overall high-definition interest portrait of the user; calculating the similarity between the vector representation of the overall high-definition interest portrait of the user and an item vector, and recommending N items with the highest similarity to the user; and the application is applied to an e-commerce platform to provide better services for users, and the e-commerce platform can more accurately mine user interests, so as to recommend more content of interest to users to generate profits for the platform.

Description

TECHNICAL FIELD

[0001] The application belongs to the technical field of Internet application, and particularly relates to an interest frame recommendation method based on high-definition image generation technology. BACKGROUND

[0002] Providing users with better and more accurate recommendation services is the focus of recommendation systems all the time. The research directions mainly include traditional recommendation algorithms and deep learning recommendation algorithms: in the traditional recommendation algorithms, the modeling of user interest preferences is mainly focused on, and these methods have achieved good results in practice; in the deep learning recommendation algorithms, the mainstream mining methods are divided into recurrent neural networks and Markov chains, which both mine user interests by analyzing the context correlation in behavior data. Essentially, these studies are all for more accurately capturing user interests and realizing high-quality recommendations.

[0003] At the same time, mining user interests plays an important role in the process of enterprises participating in market competition and obtaining relative competitive advantages. E-commerce platforms have been committed to accurately mining user interests to rapidly improve the growth of enterprise brands or sales, and effectively mining user interest points is more conducive to platforms attracting consumer behavior to achieve profitability. Under such a background, the research on e-commerce platform user behavior data analysis has obvious practical significance and application value, and e-commerce platforms can provide better recommendation services for users by analyzing user historical behavior data and accurately mining user interests, thereby making profits for the platform.

[0004] In recent years, researchers have conducted a lot of research on user interest mining and commodity characteristics, mainly based on collaborative filtering algorithms, traditional machine learning algorithms and deep learning algorithms. Although these studies can improve the problem of data sparsity in recommendation algorithms, most of them are limited to low-dimensional vectors, and high-dimensional sparse vectors are still not applicable.

[0005] With the continuous expansion of e-commerce platform data and the gradual maturity of deep learning technology, neural networks, deep learning and other nonlinear models have gradually become the mainstream choice of marketing data analysis and prediction models. Some scholars use neural networks to model user portraits and apply the modeled portraits to recommendation systems.

[0006] In the process of more accurate recommendation, some challenges are encountered in the prior art:

[0007] 1. Pixelization of fine-grained atomic user interests. User behavior data presents unevenness, which leads to differences in granularity and atomicity of each user interest, which brings difficulties to capturing user interests at the pixel level.

[0008] 2. How to frame multiple interests. Users' interest points are often multi-dimensional, how to capture multi-interest frames from the fine-grained interest pixel matrix is obviously a problem.

[0009] 3. How to generate a high-definition interest portrait of the user. The noise points between multi-interest frames are inconsistent, how to optimize and compensate at the pixel level according to the similarity between the interest frames, and then generate a high-definition interest portrait is challenging. SUMMARY

[0010] In view of the above challenges, the present application proposes an interest frame recommendation method based on high-definition image generation technology, which specifically includes the following steps:

[0011] Obtain the user's historical interaction behavior data in a period of time, and the historical interaction behavior data includes the time when the interaction occurs and the order of the item interaction;

[0012] According to the user's historical interaction behavior data, extract the granularity information existing in the user's interaction data;

[0013] The objects in the user's interaction behavior data in a period of time are arranged in time sequence to obtain a user behavior sequence, and the sequence is converted into an embedding matrix;

[0014] Each embedding vector in the embedding matrix is fused with its corresponding granularity information to obtain an input vector;

[0015] A prediction model is constructed, which includes an Item2Frame layer and a high-definition interest portrait generation layer. The Item2Frame layer obtains a multi-interest frame matrix according to the input vector, and the high-definition interest portrait generation layer performs noise compensation on the obtained multi-interest frame matrix to obtain a vector representation of the user's overall high-definition interest portrait;

[0016] Calculate the similarity between the vector representation of the user's overall high-definition interest portrait and the item vector, and recommend the N items with the highest similarity to the user's overall high-definition interest portrait to the user.

[0017] Further, the process of fusing the granularity information extracted by the feature extraction layer from the user's historical interaction behavior data with the embedding matrix of the user's historical interaction items as an input vector includes:

[0018] Given a set of user interaction sequences sorted in time increasing order Denote the item of the nth interaction of user u in a period of time, n = {1, 2, …, N};

[0019] Then the granularity information vector corresponding to each interaction of the user in the period of time is:

[0020] Granularity = {Granularity1, Granularity2, …, Granularity N};

[0021] The user interaction sequence is embedded to obtain an item embedding matrix, denoted as:

[0022] E = Embedding(S u );

[0023] Granularity information is integrated into the item embedding matrix, denoted as:

[0024]

[0025] where N is the number of user interactions in a period of time; Granularity i represents the i-th element of the granularity information vector, i = {1, 2, …, N} and Granularity i The value is less than the value of the array Granularity i+1 E i represents the i-th row vector of the item embedding matrix; IPE is the input vector.

[0026] Further, the Item2Frame layer obtains a multi-interest frame matrix according to the input vector, that is, the multi-interest frame of the user is obtained through an improved self-attention function, and the multi-interest frame matrix of the user is obtained by multiplying the multi-interest frame of the user and the input vector. The process of obtaining the multi-interest frame of the user by the improved self-attention function includes:

[0027]

[0028] LeakyRelu = max(0, x) + negative_slope * min(0, x);

[0029] where A is the multi-interest frame of the user; negative_slope is a negative slope control factor; W1 and W2 are trainable parameter matrices, the size of W1 is d a × d, and W2 has a size of d a × K, d a is the dimension of the multi-interest frame, d is the dimension of the embedding matrix, and K is the number of interest frames; x represents the variable input to leakyRelu; H is the historical behavior sequence of the user.

[0030] Further, the high-definition interest portrait generation layer performs noise compensation on the obtained multi-interest frame matrix to obtain a vector representation of the user's overall high-definition interest portrait, including the following process:

[0031] The input vector is taken as the input of the high-definition interest portrait generation layer, and the high-definition interest portrait generation layer includes an attention layer, and a multi-interest frame compensation matrix representation of the user is obtained through the attention layer;

[0032] The multiple attention layers are stacked, and residual connection, layer normalization and dropout are applied to each attention layer in the stacked layers;

[0033] The last layer of attention layer outputs a vector representation of the overall high-definition interest portrait of the user.

[0034] Further, the obtaining of the multi-interest frame compensation matrix representation of the user includes:

[0035] A i =Concat(head1,…,hea h )W o

[0036]

[0037] Wherein, A i is a multi-interest frame compensation matrix representation of the user; Concat() represents a splicing operation; is a trainable matrix corresponding to the query value, key and value in the attention mechanism, W o ∈R h ×d×d is; K is the number of interest frames, and d is the dimension of the embedding matrix; d w =d / h; h is the number of heads of the multi-head attention mechanism; I u is a multi-interest frame matrix of the user.

[0038] Further, a non-linear function is added to each layer of attention layer, and a feedforward neural network adopting Leaky ReLU as an activation function is used to process the multi-interest frame compensation matrix representation of the user, including:

[0039] FFN(A i )=LeakyReLU(A i W1+b1)W2+b2;

[0040] Wherein, FFN() represents a feedforward neural network; LeakyReLU() represents a Leaky ReLU activation function; W1,W2∈R d×d , b1, b2∈R d are network parameters that can be learned in the feedforward neural network, the dimensions of W1 and W2 are dxd, and the dimensions of b1 and b2 are d.

[0041] Further, the attentions between layers are cascaded, and the last layer of attention layer outputs the vector representation of the overall high-definition interest portrait of the user, including:

[0042]

[0043] wherein, is the output of the k+1 layer of attention layer; LayerNorm() represents the normalization operation; Dropout() represents the Dropout operation.

[0044] The present application designs a fine-grained pixel-level user interest mining method PE, which mines atomic-level user interest from multiple dimensions such as time, space, and frequency; an Item2Frame method for generating user multi-interest frames based on atomic-level interest pixels is proposed, which maps multiple interest frames through each interest pixel cluster; an FNC interest frame noise compensation method is proposed, which uses a multi-head attention mechanism to optimize and complement noise between multi-interest frames at the pixel level, and more completely generates a global high-definition user interest portrait; the present application can be applied to an e-commerce platform to provide better services for users, and the e-commerce platform can more accurately mine user interests, thereby recommending more user-interested content to generate profits for the platform. It is also helpful for the e-commerce platform to master the distribution of consumer group behavior characteristics, analyze potential customer groups, and obtain good economic benefits. BRIEF DESCRIPTION OF DRAWINGS

[0045] Figure 1 is a flowchart of the interest frame recommendation method based on the high-definition image generation technology of the present application;

[0046] Figure 2 is a schematic diagram of the PE algorithm used in the interest frame recommendation method based on the high-definition image generation technology of the present application;

[0047] Figure 3 is a schematic diagram of the Item2Frame algorithm used in the interest frame recommendation method based on the high-definition image generation technology of the present application. DETAILED DESCRIPTION

[0048] The technical solutions in the embodiments of the present application will be described clearly and completely below with reference to the drawings in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, not all. Based on the embodiments in the present application, all other embodiments obtained by those skilled in the art without creative labor are within the scope of protection of the present application.

[0049] The present application provides an interest frame recommendation method based on high-definition image generation technology, comprising the following steps:

[0050] Acquire historical interaction behavior data of a user in a period of time, the historical interaction behavior data including time of interaction occurrence and order of item interaction;

[0051] According to the user historical interaction behavior data, extract granularity information existing in the user interaction data;

[0052] Arrange objects in the interaction behavior data of the user in a period of time in time sequence to obtain a user behavior sequence, and convert the sequence into an embedding matrix;

[0053] Fuse each embedding vector in the embedding matrix with corresponding granularity information respectively to obtain an input vector;

[0054] Construct a prediction model, the model including an Item2Frame layer and a high-definition interest portrait generation layer, the Item2Frame layer acquiring a multi-interest frame matrix according to the input vector, and the high-definition interest portrait generation layer performing noise compensation on the obtained multi-interest frame matrix to obtain vector representation of an overall high-definition interest portrait of the user;

[0055] Calculate similarity between the vector representation of the overall high-definition interest portrait of the user and an item vector, and recommend N items with the highest similarity to the vector representation of the overall high-definition interest portrait of the user to the user.

[0056] In order to better illustrate the technical solutions of the present application and make the advantages more concise and clear, the problems to be solved by the present application will be explained in detail, and then the specific embodiments of the present application will be further described with reference to the accompanying drawings.

[0057] As Figure 1 The overall flow of the present application is shown, indicating that the input of the present application is historical behavior data of a user under an e-commerce platform, time information and basic information of an interacted item, and the output after the high-definition interest generation model is a high-definition interest portrait of the user, a multi-interest frame of the user and a commodity recommendation list. In the present embodiment, the present application mainly includes the following three steps:

[0058] (1) Online data acquisition.

[0059] There are two ways to acquire data, one is to acquire through an online public data set, and the other is to directly query historical interaction data in a database provided by an enterprise. Here, what needs to be acquired is the interaction of a user in a specified period of time (such as browsing, collecting, adding to a shopping cart, ordering, etc.), and the trigger time, the order before and after the interaction behavior. And pre-process the data.

[0060] In this embodiment, the original data can be obtained from the publicly available dataset provided by the network or directly from the enterprise database. Usually, the obtained original data is unstructured and cannot be directly used for data analysis. Most of the unstructured data can be structured through simple data cleaning. For example, duplicate data can be deleted, invalid messages can be cleared, etc.

[0061] (2) Extract relevant attribute information.

[0062] According to the obtained user historical interaction data, in this embodiment, according to the differences in each user behavior data, the granularity information existing in their respective interaction data is extracted.

[0063] If there is a set of users u∈U and a set of items i∈I, where u represents a certain user in the user set U, and i represents a certain item in the item set I. For each user u, its historical behavior sequence can be expressed as:

[0064]

[0065] Among them, History u represents the historical behavior sequence of user u; represents that the item interacted by user u at time t j is at any 0 < n < N, t n < t n+1 ; Preprocess the historical interaction data of the user, sort it according to time, and obtain the input of the entire model. The user behavior sequence S u can be expressed as:

[0066]

[0067] Such as Figure 2 , due to the problem of inconsistent interest frame granularity caused by uneven user interaction data, the item granularity matrix Granularity is proposed. Given a set of user interaction sequences sorted in ascending order of time This application believes that the granularity of an item increases as time gets closer. In other words, the item that a user recently operates represents their strongest desire in the recent period and has a finer granularity, with a higher weight in extracting the user's interest frame. Then, the item granularity matrix Granularity can be expressed as:

[0068] Granularity = {Granularity1, Granularity2,..., Granularity [[ID=​​​n represents the granularity of the nth behavior in the user behavior sequence, and if for any 0 < n < N, there exists Granularity n <Granularity n+1 .

[0070] (3) Establish a model.

[0071] As Figure 1 , in this embodiment, establishing a model includes three stages: fine-grained information mining, interest frame mining, and high-definition interest portrait generation, namely:

[0072] In the fine-grained information mining stage, starting from the user's historical interaction behavior, mine the granularity information existing in the user behavior data, and at the same time use embedding to embed the item into the low-dimensional dense vector space;

[0073] In the interest frame mining stage, through the obtained interest pixel clusters, propose the Item2Frame algorithm to obtain multiple interest frames from the interest pixel clusters;

[0074] In the high-definition interest portrait generation stage, generate the user's high-definition interest portrait through the high-definition interest portrait generation algorithm.

[0075] In the process of fine-grained interest pixel mining, the user's historical behavior sequence is used as the input, and the items are sorted according to the time information in the sequence to obtain the user's behavior sequence This process is expressed as:

[0076] S u = sort(History u )

[0077] S u = S u .length>20? truncate(S u ) : padding(S u )

[0078] where sort() is the sorting function; S u .length represents the length of the user's behavior sequence S u ; truncate() is the truncation function, padding() is the padding function, 20? truncate(S u ) : padding(S u ) means to truncate S uTruncation or padding makes its length 20, this embodiment adopts the length of 20, but the present application does not limit the length, and those skilled in the art can truncate or pad according to actual needs.

[0079] The obtained user behavior sequence S u , all items are converted into item vector matrix by using the conventional embedding operation, and then the user interaction sequence is converted into embedding matrix E ∈ R n×d , which is expressed as:

[0080] E=Embedding(S u )

[0081] Wherein, E represents the embedding matrix of the user interaction sequence, and Embedding() represents the embedding operation.

[0082] In view of the problem of inconsistent granularity caused by uneven user interaction data, it is determined that the interest intensity of the user is determined by two factors: interaction time and density of the time period. The closer the interaction time is, the denser the time period is, and the more detailed the granularity of the item contains, and the higher the weight in extracting the user interest frame. Based on this, the granularity information is integrated into the item embedding matrix, and the final input is obtained:

[0083]

[0084] Wherein, IPE is the final input, that is, the input vector; E n represents the embedding matrix of the n-th row vector.

[0085] In the interest frame mining stage, considering that the user behavior sequence often contains multiple categories of interest points, a user multi-interest frame generation method Item2Frame is proposed. In the Item2Frame layer, the similarity score of each item is calculated by the attention function to fill the multi-interest pixel cluster, and the interest frame is mapped according to each interest pixel cluster, as shown in Figure 3 .

[0086] The attention function can be described as mapping a query and a set of key-value pairs to an output, where the query, key, value and output are all vectors. The output is calculated as a weighted sum of values, where the weight assigned to each value is computed by an attention function from the query and the corresponding key.

[0087] In this embodiment, an interest vector a ∈ R N is obtained by processing the user historical behavior sequence by the improved self-attention function, which is defined as follows:

[0088]

[0089] LeakyRelu form as follows:

[0090] LeakyRelu(x) = max(0, x) + negative_slope * min(0, x)

[0091] Wherein, negative_slope controls the angle of negative slope, the default setting negative_slope is: 0.01; W1, w2 are trainable parameters, the size is respectively: d a And d a , d a is the dimension of the frame of interest;() T Indicates the transpose of the matrix vector.

[0092] In this embodiment, first, atomic-level user interest is mined from multiple dimensions such as time, space, and frequency; then the Item2Frame algorithm is proposed, which maps multiple interest frames according to each interest pixel cluster; finally, the FNC algorithm is proposed to optimize and complement noise at the pixel level between multiple interest frames, and to generate a more complete global high-definition user interest portrait.

[0093] Then, the embodiment obtains a tensor representation of a user interest frame by inputting the vector IPE and the interest vector a, that is:

[0094] v u = IPE * a

[0095] Wherein, v u is the tensor representation of the interest frame of the user u.

[0096] v u This frame only represents a single interest frame of the user, and in actual situations, the user often has multiple interest frames. In order to represent the multiple interest frames of the user, w2 is converted into a W2 matrix of d a × K, K is the number of interest frames, and the interest frame vector a ∈ R N becomes a multiple interest frame matrix Indicated as:

[0097]

[0098] The final multiple interest frame matrix of the user can be represented as:

[0099] I u = IPE * A

[0100] The task in the high-definition interest portrait generation stage is to optimize and compensate noise at the pixel level between multiple interest frames, and to generate a more comprehensive high-definition interest portrait of the user. The multi-head attention mechanism is used to complete the above operation.

[0101] The attention function used in this embodiment is scaled dot-product attention, which can be expressed as:

[0102]

[0103] where Attention(Q, K, V) denotes the attention coefficients; Q (Query) represents the query, K (key) represents the key, and V (value) represents the value; by calculating the dot product of the query Q and all keys K, then dividing each value by where is a scaling factor to prevent the inner product from being too large when the dimension is too high, and finally the softmax function is applied to obtain the weight of the value. For self-attention mechanism, the query Q, the key K and the value V come from the same object.

[0104] The self-attention mechanism uses the feature matrix IPE as input, which is transformed linearly and input into our multi-head attention layer. Similar to multiple convolution kernels in convolutional neural networks, multi-head attention mechanism allows the model to focus on information from different representation subspaces at different positions, and its formula is as follows:

[0105] A i =Concat(head1,…,head h )W o

[0106]

[0107] where the multi-interest frame compensation matrix representation A i ∈R K×d of the user can be obtained, in the formula, W o ∈R h×d×d , h is the number of heads.

[0108] The multi-head attention mechanism layer mainly depends on linear projection, and we can stack multiple self-attention layers to expand the model capacity. In order to give each layer nonlinearity, a feedforward neural network with Leaky ReLU as the activation function is used:

[0109] FFN(A i )=LeakyReLU(A i W1+b1)W2+b2

[0110] where W1,W2∈R d×d ,b1,b2∈R dare learnable parameters in the feed-forward neural network, FFN( ) represents the feed-forward neural network; LeakyReLU( ) represents the Leaky ReLU activation function.

[0111] The same linear transformation is used in different self-attention layers to realize parameter sharing. At the same time, stacking self-attention layers may cause problems such as overfitting and gradient disappearance, therefore, in the embodiment, residual connection, layer normalization and dropout are applied to each module, after the application, the output of the k-th layer self-attention layer is represented as:

[0112]

[0113] wherein, is the output of the k+1-th layer attention layer; LayerNorm( ) represents the normalization operation; Dropout( ) represents the Dropout operation.

[0114] In order to alleviate the problem of overfitting, we apply dropout on the output of the feed-forward neural network, then propagate the bottom layer features to the higher layer through the residual connection, finally we apply layer normalization to stabilize and accelerate the training of the neural network.

[0115] The output of the last layer of attention layer is taken as the vector representation Interest of the overall high-definition interest portrait of the user u , the similarity between the item vector of the item to be recommended and the vector representation Interest of the overall high-definition interest portrait of the user calculated is calculated, the similarity can be calculated in the form of cosine distance, Euclidean distance and the like, the similarity between the two vectors, the vector representation Interest of the overall high-definition interest portrait of the user u The most similar item is recommended to the user. u

[0116] Although the embodiments of the present application have been shown and described, it can be understood by those skilled in the art that various changes, modifications, replacements and variations can be made to the embodiments without departing from the principles and spirits of the present application, the scope of the present application is defined by the appended claims and their equivalents.​

Claims

1. A method for recommending an interesting frame based on a high-definition image generation technique, characterized by, The method comprises the following steps: obtaining historical interaction behavior data of a user in a period of time, the historical interaction behavior data comprising time of interaction and order of item interaction; extracting granularity information existing in the user interaction data according to the historical interaction behavior data of the user; arranging objects in the interaction behavior data of the user in a period of time in time sequence to obtain a user behavior sequence, and converting the sequence into an embedding matrix; fusing each embedding vector in the embedding matrix with corresponding granularity information to obtain an input vector; constructing a prediction model, the model comprising an Item2Frame layer and a high-definition interest portrait generation layer, the Item2Frame layer obtaining a multi-interest frame matrix according to the input vector, i.e. obtaining a multi-interest frame of the user through an improved self-attention function, and multiplying the multi-interest frame of the user with the input vector to obtain a multi-interest frame matrix of the user, wherein the process of the improved self-attention function for obtaining the multi-interest frame of the user comprises: LeakyRelu = max(0, x) + negative_slope * min(0, x); wherein A is a multi-interest frame of the user; negative_slope is a negative slope control factor; W1 and W2 are trainable parameter matrices, the size of W1 is d a x d, and W2 is of size d a x K, d a is the dimension of the multi-interest frame, d is the dimension of the embedding matrix, K is the number of interest frames; x represents the variable input to the leakyRelu; H is the historical behavior sequence of the user; the high-definition interest portrait generation layer performing noise compensation on the obtained multi-interest frame matrix to obtain a vector representation of an overall high-definition interest portrait of the user, comprising: taking the input vector as input of the high-definition interest portrait generation layer, the high-definition interest portrait generation layer comprising an attention layer, and obtaining a multi-interest frame compensation matrix representation of the user through the attention layer; stacking multiple attention layers, and applying residual connection, layer normalization and dropout to each attention layer in the stacked layers; the last layer of attention layer outputs the vector representation of the overall high-definition interest portrait of the user; calculating similarity between the vector representation of the overall high-definition interest portrait of the user and an item vector, and recommending N items with the highest similarity to the vector representation of the overall high-definition interest portrait of the user to the user.

2. The method of claim 1, wherein the method is based on a high-definition image generation technique. The process of the feature extraction layer fusing the granularity information with an embedding matrix of historical interaction items of the user as an input vector after extracting the granularity information from the historical interaction behavior data of the user comprises: Given a set of user interaction sequences sorted in increasing order of time Item representing the nth interaction of user u in a time period, n = {1, 2, …, N} then the granularity information vector representation corresponding to each interaction item of the user in the period of time is: Granularity = {Granularity1, Granularity2,..., Granularity N}; performing embedding operation on the user interaction sequence to obtain an item embedding matrix, denoted as: E = Embedding(S u ); fusing the granularity information into the item embedding matrix, denoted as: where N is the number of interactions of a user in a period of time; Granularity i represents the i-th element of the granularity information vector, i = {1, 2, …, N} and Granularity i has a value less than the value of the array Granularity i+1 ; E i represents the i-th row vector of the item embedding matrix; IPE is an input vector. 3.The method of claim 1, wherein, the acquisition of the multi-interest frame compensation matrix representation of the user comprises: A i = Concat(head1,..., head h )W o head i = Attention(I u W i Q ,I u W i K ,I u W i V ), i = {1, 2,... h}; wherein A i is the user's multi-interest frame compensation matrix representation; Concat() represents the concatenation operation; is the trainable matrix corresponding to the query, key, and value in the attention mechanism, W o ∈R h ×d×d is ; K is the number of interest frames, and d is the dimension of the embedding matrix; d w =d / h; h is the number of heads of the multi-head attention mechanism; I u is the user's multi-interest frame matrix.

4. The method of claim 3, wherein the method is characterized by: applying a nonlinearity to each layer of attention layer, and adopting a Leaky ReLU as an activation function of a feedforward neural network to process the multi-interest frame compensation matrix representation of the user, comprising: FFN(A i ) = LeakyReLU(A i W1+b1)W2+b2; where FFN() represents a feed-forward neural network; LeakyReLU() represents a Leaky ReLU activation function; W1, W2 e R d×d , b1, b2 e R d are learnable network parameters in the feed-forward neural network, the dimensions of W1, W2 are d x d, and the dimensions of b1, b2 are d. 5.The method of claim 1, wherein, concatenating the attention layers, and the last layer of attention layer outputs the vector representation of the overall high-definition interest portrait of the user, comprising: wherein, is the output of the k+1 layer of attention; LayerNorm() denotes a normalization operation; Dropout() denotes a Dropout operation.

Images