Sequence recommendation method and system for long-short term interest capture and fine-grained time fusion

Abstract

The invention relates to a sequence recommendation method and system integrating long and short time interest capture and fine-grained time. The method comprises the following steps: collecting behavior data of a user; a deep learning network model with long and short time interest capture and fine-grained time fusion is constructed, the deep learning network model carries out modeling on timestamps according to change frequency characteristics of fine-grained time information, and a self-attention mechanism and a convolutional neural network are used to extract long time interest and short time interest of a user respectively. Using a comparative learning mechanism to extract associated interests in long-term and short-term interests, and introducing associated interest information into an interest fusion module to realize accurate recommendation; constructing a training set through the collected data, and then training the deep learning network model by using the training set; and inputting the user, the article and the interaction data into the trained deep learning network model, and outputting the click probability of the current user on the target article. The method and the system are beneficial to improving the accuracy of sequence recommendation prediction.

Description

Technical Field

[0001] The present invention belongs to the field of recommendation systems, and in particular relates to a sequence recommendation method and system for long- and short-term interest capture and fine-grained time fusion. Background Art

[0002] The sequential recommendation system is a key module of the recommendation system. The sequential recommendation system can integrate multi-source information, including the user's click item sequence information, behavior time series information, etc., so as to have a more comprehensive understanding of user interests. By considering the time sequence of user behavior, the sequential recommendation system can better capture the user's evolving interests and provide more accurate and personalized recommendations. This means that the system can identify the user's changing needs for different content at different time points, adjust the recommendation strategy in a timely manner, and avoid pushing outdated or no longer interesting information to the user. At the same time, the user's long-term interests represent the user's stable interests, while the user's short-term interests represent the user's instantaneous interests caused by the current surrounding environment and the influence of long-term interests. Therefore, how to effectively model the user's long-term and short-term interests and time series is crucial to the performance of the sequential recommendation system, which is also an important topic in the sequential recommendation system.

[0003] However, although the existing models have achieved performance improvements to a certain extent, they still face some major weaknesses. First, the existing models often ignore the complex relationship between user preferences and temporal context. Although they process the relative position information of interactions in the sequence, they often ignore the time-related information. Even some models pay attention to time-related information and model the fine-grained time information contained in the timestamp information, such as year, month, day, etc., but they do not perform effective fusion. Most of them simply use operations such as splicing and addition to fuse fine-grained information. Secondly, although the existing long-term and short-term preference modeling work has better modeled the user's long-term and short-term preferences. However, most of the work often ignores the intrinsic correlation between the user's current preference and the long-term preference, and the user's current preference is often associated with multiple long-term preferences, so it is necessary to model the intrinsic correlation between the user's short-term preference and the long-term preference. Finally, both long-term interests and short-term interests play a vital role in the recommendation task, so the model must be able to distinguish which interest is more important in different situations. Existing models usually rely on a simple weighted sum gating mechanism to integrate long-term interests and short-term interests, which limits the model's capabilities. The similarity information between long-term preferences and short-term preferences often reflects the degree to which the user's current preferences are affected by the surrounding environment. Specifically, when long-term preferences are not similar to short-term preferences, it means that the user's current preferences are more affected by the surrounding environment, and short-term preferences should be given a greater weight. If there is a certain similarity between long-term preferences and short-term preferences, a certain weight should be given to long-term preferences. Summary of the invention

[0004] The purpose of the present invention is to provide a sequence recommendation method and system that combines long-term and short-term interest capture with fine-grained time fusion, which is conducive to improving the accuracy of sequence recommendation prediction.

[0005] In order to achieve the above object, the technical solution adopted by the present invention is: a sequence recommendation method for long-term and short-term interest capture and fine-grained time fusion, comprising the following steps:

[0006] Step A: Collect user behavior data, including user ID, target item ID, and interaction timestamp;

[0007] Step B: construct a deep learning network model G that captures long-term and short-term interests and fuses fine-grained time. The deep learning network model G models the timestamp according to the change frequency characteristics of the fine-grained time information, uses the self-attention mechanism and the convolutional neural network to extract the user's long-term interests and short-term interests respectively, uses the contrastive learning mechanism to extract the related interests in the long-term and short-term interests, and introduces the related interest information into the interest fusion module to achieve accurate recommendation; construct a training set through the data collected in step A, and then use the training set to train the deep learning network model G;

[0008] Step C: Input the user, item, and interaction data into the trained deep learning network model G, and output the current user's click probability for the target item.

[0009] Furthermore, the step B specifically includes the following steps:

[0010] Step B1: Process the data collected in step A to obtain the item interaction sequence S of each user u , the corresponding initial position code P init , the corresponding interaction timestamp sequence S t , target item ID, target item timestamp, target item initial position code, user ID;

[0011] Step B2: The item interaction sequence S obtained in step B1 u Input to the item embedding layer to get the item interaction sequence matrix Input the user ID and target item ID into the user embedding layer and item embedding layer respectively to obtain the user vector e u and e target ; Encode the initial position obtained in step B1 as P init Input to the position embedding layer to get the position embedding matrix E p ; Input the target object initial position encoding obtained in step B1 into the position embedding layer to obtain the position embedding vector p target ; The target item timestamp and user vector e obtained in step B1u Input the fine-grained time fusion network for training to obtain the target object timestamp vector t target ; The above obtained t target , e target , p target The three are vector added to get the target item vector The interaction timestamp sequence S obtained in step B1 t and user vector e u Input the fine-grained time fusion network for training to obtain the fine-grained time series matrix Through the position embedding matrix E p To represent the corresponding item position information in the item interaction sequence; the item interaction sequence matrix The corresponding fine-grained time series matrix and the position embedding matrix E p According to the last dimension of the matrix, we can concatenate and obtain the fine-grained time-item matrix E S ;

[0012] Step B3: The fine-grained time-item matrix E obtained in step B2 S Input into the long-term interest extraction network constructed by the multi-head self-attention mechanism to extract the user's long-term interest matrix U long ;

[0013] Step B4: The fine-grained time-item matrix E obtained in step B2 S Input into the local interest extraction network constructed by the convolutional neural network to extract the user's short-term interest matrix U short ;

[0014] Step B5: The user short-term interest matrix U obtained in step B4 short As a query, the user long-term interest matrix U obtained in step B3 long Execute the cross attention mechanism to extract the correlation interest matrix U associated with the short-term interests in the user's long-term interests link , through U long with U link Perform a subtraction operation to obtain the user's long-term interests that are not related to short-term interests U except ; U from the same user link with U short As a positive sample pair, U except with U short As a negative sample pair, contrastive learning is performed to bring the user's short-term interests closer to the interests related to the long-term interests and the short-term interests, and push away the interests that are not related to the short-term interests and the long-term interests, and obtain the contrast loss L cl ;

[0015] Step B6: The user association interest matrix U obtained in step B5 link and the user's short-term interest matrix U short Input into the long-term and short-term similarity fusion network, and use the similarity information of long-term preference and short-term preference to obtain the fused interest matrix U fuse ; The similarity information between long-term preference and short-term preference reflects the degree to which the user's current preference is affected by the surrounding environment. If the long-term preference is not similar to the short-term preference, it means that the user's current preference is greatly affected by the surrounding environment, then the weight of the short-term preference is increased. If the long-term preference and the short-term preference have a set similarity, then the weight of the long-term preference is increased.

[0016] Step B7: The target object vector obtained in steps B2 and B6 And the fused interest matrix U fuse , take out the interest matrix U fuse The last line in and the target object vector Perform vector dot multiplication, use the sigmoid function to predict click-through rate, and use the cross entropy function to calculate the prediction loss; then, according to the target loss function, calculate the gradient of each parameter in the deep learning network model through the back propagation method, and use the stochastic gradient descent method to update each parameter;

[0017] Step B8: When the loss value generated by the deep learning network model is less than the set threshold or reaches the maximum number of iterations, the training of the deep learning network model G is terminated.

[0018] Furthermore, the step B1 specifically includes the following steps:

[0019] Step B11: For each user interaction sequence containing item ID and timestamp information, firstly filter out users and items that do not meet the conditions according to preset conditions;

[0020] Step B12: Based on step B11, the user interaction sequences are first sorted in time order, and a corresponding data set is generated for each user's user interaction sequence according to the preset sequence length l+1. For the l+1th item in the user interaction sequence, the first l item sequence is constructed to predict the l+1th item; if the number of user interaction items is less than l+1, the beginning of the sequence is filled with zero values ​​according to the time order;

[0021] Step B13: Split the data obtained in step B12 into the user's item interaction sequence S u , the corresponding interaction timestamp sequence S t, the target item ID, the target item timestamp, and the user ID; for user u, (e1, t1) represents that user u interacts with the item with item ID e1 at timestamp t1. Then, user u has a user interaction sequence {(e1, t1), (e2, t2), (e3, t3), (e4, t4)}, where {t1 < t2 < t3 < t4}. According to the preset length of 4, the user interaction sequence is divided. The initial position encoding rule is from 0 to the length of the user interaction sequence - 1, thus obtaining the item interaction sequence S u = {e1, e2, e3}, the interaction timestamp sequence S t = {t1, t2, t3}, S u corresponds to the initial position encoding as P init = {0, 1, 2}, the target item ID e4, the target item timestamp t4, and the target item initial position encoding {3}.

[0022] Further, the specific steps of step B2 are as follows:

[0023] Step B21: According to the interaction timestamp sequence S obtained in step B13 t , from each timestamp t in the interaction timestamp sequence S t extract various fine-grained time information, including year information yuer, season information quarter, month information month, the day of the month dayofmonth, the day of the week dayofweek, and hour information hour; input the user ID and the target item ID into the user embedding layer and the item embedding layer to obtain e i and e u and e target ;

[0024] Step B22: Process the various fine-grained time information obtained in step B21;

[0025] The processing method for the day of the week dayofweek is as follows: Given that there are 7 days in a week, the week sequence S week = {1, …, 7}, and after mapping through the embedding layer, the week sequence matrix To make the mapping vectors with closer time distances more similar, perform a one-dimensional CNN convolution operation on the week sequence matrix with a window size of the preset value w 1 , and the initial value of the window weights decreases from the center to both ends, obtaining the updated vector matrix Similarly, obtain other updated time sequence matrices, including: the year sequence matrix the season sequence matrix the month sequence matrix the month-day sequence matrix the hour sequence matrix

[0026]

[0027] Step B23: The interactive timestamp sequence S obtained in step B13 t , query the fine-grained time vector corresponding to each timestamp from the update time series matrix obtained in step B22, that is, for S t Each timestamp t in i There are corresponding fine-grained time vector groups Taking into account the different frequency characteristics of fine-grained time information, the fine-grained time vector group Divide into fast-changing fine-grained time information and slowly changing fine-grained temporal information User vector e u As a query, the fine-grained time vector group Two types of fine-grained time information in and Perform a cross-attention mechanism to obtain the updated time vectors and Then for the two vectors and Perform the splicing operation to get the timestamp t i The corresponding fine-grained time vector Thus we get the interaction timestamp sequence S t The corresponding fine-grained time series vector matrix

[0028]

[0029]

[0030]

[0031]

[0032]

[0033] in, and represents the attention coefficient, W1 and W2 represent trainable parameters, N s N represents the number of slowly changing fine-grained time information in the fine-grained time vector group. f Indicates the number of fast-changing fine-grained time information in the fine-grained time vector group;

[0034] Step B24: Embed the vector matrix E by position p To represent the corresponding item position information in the item interaction sequence; the item interaction sequence vector matrix The corresponding fine-grained time series vector matrix and position embedding vector matrix E p According to the last dimension of the vector matrix, we can concatenate and obtain the fine-grained time-item vector matrix E S ;

[0035]

[0036] Furthermore, the specific implementation method of step B3 is: the fine-grained time-item vector matrix E obtained in step B24 S , where each row vector represents the fine-grained temporal item information e represented by the corresponding item in the sequence s , by fine-grained time item vector matrix E S Perform multi-head self-attention mechanism learning to capture the user's long-term interest vector U in the interaction sequence long ;

[0037]

[0038]

[0039] Among them, head h represents the output of the h-th attention function, d h Indicates the dimension of each head, n head represents the number of attention functions, and E3 represents the trainable parameters.

[0040] Furthermore, the specific implementation method of step B4 is: the fine-grained time-item vector matrix E obtained in step B24 is S Input into the local interest extraction network constructed by a one-dimensional convolutional neural network to extract the user's short-term interest vector matrix U short ;

[0041]

[0042] Among them, w1 is the preset value of the window size.

[0043] Furthermore, the step B5 specifically includes the following steps:

[0044] Step B51: The user short-term interest matrix U obtained in step B4 short As a query, each row vector represents a short-term interest e short , the user long-term interest matrix U obtained in step B3 longPerform a cross-attention mechanism, where each row represents a long-term interest e long , to extract the interest vector e associated with the short-term interest in the user's long-term interest link , and then form the long-term and short-term correlation interest matrix U link ;

[0045]

[0046]

[0047] in, represents the attention coefficient, W4 represents the trainable parameter, N s Represents the short-term interest matrix U short number of rows;

[0048] Step B52: Based on the user's long-term interest vector matrix U obtained in step B4 long , the long-short time associated interest vector U obtained in step B51 link Perform a subtraction operation to obtain the user's long-term interests that are not related to short-term interests U except ;

[0049] U except =U long -U link

[0050] Step B53: The interest U that is not related to the short-term interest among the user's long-term interests obtained in step B52 except , for the long-term and short-term correlation interest matrix U from the same user link and the user's short-term interest matrix U short As a positive sample pair, the long-term irrelevant interest matrix U except and the short-term interest matrix U short As a negative sample pair, comparative learning is performed to bring the user's short-term interests closer to the interests related to the long-term interests and the short-term interests, and push away the interests that are not related to the short-term interests and the long-term interests, and obtain the correlation interest contrast loss L cl ;

[0051] S p =Sim(U short ,U link )

[0052] S n =Sim(U short ,U except )

[0053]

[0054] Among them, τ is the temperature coefficient and sim represents the cosine similarity function.

[0055] Furthermore, the step B6 specifically includes the following steps:

[0056] Step B61: Associate the user interest vector U obtained in step B52 link The user short-term interest matrix U obtained in step B4 short Use the cosine similarity function to characterize the similarity between long-term preference and short-term preference to generate λ;

[0057] λ=Sigmoid(Sim(U short ,U link ))

[0058] Step B62: Obtain the similarity λ between the long-term preference and the short-term preference according to step B52, and fuse the user long-term interest matrix U obtained in step B3 long The user short-term interest matrix U obtained in step B4 short , generate the fused interest matrix U fuse ;

[0059] U fuse =λ*U short +(1-λ)*U long .

[0060] Furthermore, the step B7 specifically includes the following steps:

[0061] Step B71: The vector e of the target object obtained in steps B21 and B62 is t And the fused interest matrix U fuse , take out the interest matrix U fuse In and the target object vector Perform vector dot multiplication and use the sigmoid function to predict the click rate to get the predicted click rate:

[0062]

[0063] Step B72: Use the cross entropy function L target Calculate the prediction loss, update the learning rate through the gradient optimization algorithm Adam, and use back propagation to iteratively update the model parameters to minimize the loss function to train the model; the total model loss is the weighted addition of the above losses:

[0064]

[0065] L all =L target +βL cl

[0066] Where N represents the number of samples, L represents the length of a single training sample, is the corresponding label, β is the hyperparameter, L target is the loss function for click rate prediction, L cl is the associated interest contrast loss function.

[0067] The present invention also provides a sequence recommendation system for long- and short-term interest capture and fine-grained time fusion, including a memory, a processor, and computer program instructions stored in the memory and capable of being executed by the processor. When the processor executes the computer program instructions, the above-mentioned method steps can be implemented.

[0068] Compared with the prior art, the present invention has the following beneficial effects:

[0069] 1. The present invention proposes a user-adaptive fine-grained time modeling method, which is used to convert a timestamp into a one-dimensional vector representing time information. According to the frequency of change characteristics of the fine-grained time information itself, the fine-grained time information is divided into two parts (two parts with fast and slow change frequencies) and processed separately using user vectors, thereby effectively extracting the fine-grained time information contained in the timestamp.

[0070] 2. This paper proposes a novel comparative learning mechanism to compare the long-term and short-term related interests U from the same user. link and the user's short-term interest matrix U short As a positive sample pair, the long-term irrelevant interest matrix U except and the short-term interest matrix U short As negative sample pairs, comparative learning is performed to bring the user's short-term interests closer to the interests in the long-term interests that are related to the short-term interests, and push away the short-term interests from the interests in the long-term interests that are not related to the short-term interests.

[0071] 3. The present invention proposes a novel long-term and short-term interest fusion module. The similarity between long-term preference and short-term preference reflects the degree to which the user's current preference is affected by the surrounding environment. When the long-term preference is not similar to the short-term preference, it means that the user's current preference is greatly affected by the surrounding environment, and a greater weight is given to the short-term preference. If there is a certain similarity between the long-term preference and the short-term preference, a certain weight is given to the long-term preference. By introducing the similarity between the long-term preference and the short-term preference into the interest fusion module, the effective fusion of long-term and short-term interests is achieved. BRIEF DESCRIPTION OF THE DRAWINGS

[0072] Figure 1 is a flow chart of a method implementation of an embodiment of the present invention;

[0073] Figure 2It is an architecture diagram of the deep learning network model G for long- and short-term interest capture and fine-grained time fusion in an embodiment of the present invention. DETAILED DESCRIPTION

[0074] The present invention will be further described below in conjunction with the accompanying drawings and embodiments.

[0075] It should be noted that the following detailed descriptions are exemplary and are intended to provide further explanation of the present application. Unless otherwise specified, all technical and scientific terms used herein have the same meanings as those commonly understood by those skilled in the art to which the present application belongs.

[0076] It should be noted that the terms used herein are only for describing specific embodiments and are not intended to limit the exemplary embodiments according to the present application. As used herein, unless the context clearly indicates otherwise, the singular form is also intended to include the plural form. In addition, it should be understood that when the terms "comprise" and / or "include" are used in this specification, it indicates the presence of features, steps, operations, devices, components and / or combinations thereof.

[0077] like Figure 1 As shown, this embodiment provides a sequence recommendation method for long-term and short-term interest capture and fine-grained time fusion, including the following steps:

[0078] Step A: Collect user behavior data, including user ID, target item ID, interaction timestamp, etc.

[0079] Step B: Construct a deep learning network model G that captures long-term and short-term interests and fuses fine-grained time. The deep learning network model G models the timestamp according to the frequency of change characteristics of the fine-grained time information itself, uses a self-attention mechanism and a convolutional neural network to extract the user's long-term interests and short-term interests respectively, uses a contrastive learning mechanism to extract related interests in long-term and short-term interests, and introduces the related interest information into the interest fusion module to achieve accurate recommendation; construct a training set through the data collected in step A, and then use the training set to train the deep learning network model G;

[0080] Step C: Input the user, item, and interaction data into the trained deep learning network model G, and output the current user's click probability for the target item.

[0081] Figure 2 It is an architecture diagram of the deep learning network model G for long-term and short-term interest capture and fine-grained time fusion in this embodiment. In this embodiment, the specific implementation steps of step B are as follows.

[0082] Step B1: Process the data collected in step A to obtain the item interaction sequence S of each user u, the corresponding initial position encoding P init , the corresponding interaction timestamp sequence S t , the target item ID, the target item timestamp, the target item initial position encoding, the user ID.

[0083] In this embodiment, the step B1 specifically includes the following steps:

[0084] Step B11: For each user interaction sequence (including item ID and timestamp information), first screen out the users and items that do not meet the conditions according to the preset conditions.

[0085] Step B12: On the basis of step B11, first sort the user interaction sequences in chronological order, generate a corresponding data set for each user's user interaction sequence according to the preset sequence length l + 1. For the (l + 1)-th item in the user interaction sequence, construct the first l item sequences to predict the (l + 1)-th item; if the number of interaction items of the user is less than l + 1, then according to the chronological order, use zero values to fill in at the beginning of the sequence.

[0086] Step B13: Split the data obtained in step B12 into the user's item interaction sequence S u , the corresponding interaction timestamp sequence S t , the target item ID and the target item timestamp, the user ID; for example: for a certain user u, (e1, t1) means that user u interacts with the item with item ID e1 at timestamp t1 (for example, likes, collects, etc.), then user u has a user interaction sequence {(e1, t1), (e2, t2), (e3, t3), (e4, t4)}, where {t1 < t2 < t3 < t4}, divide the user interaction sequence according to the preset length 4, and the initial position encoding rule is from 0 to the length of the user interaction sequence - 1, so as to obtain the item interaction sequence S u ={e1, e2, e3}, the interaction timestamp sequence S t ={t1, t2, t3}, S u The corresponding initial position encoding is P init ={0, 1, 2}, the target item ID e4, the target item timestamp t4, the target item initial position encoding {3}.

[0087] Step B2: Input the item interaction sequence S obtained in step B1 u into the item embedding layer to obtain the item interaction sequence matrix Input the user ID and the target item ID into the user embedding layer and the item embedding layer respectively to obtain the user vector e u and e target ; the initial position encoding obtained in step B1 is P initInput to the position embedding layer to get the position embedding matrix E p ; Input the target object initial position encoding obtained in step B1 into the position embedding layer to obtain the position embedding vector p target ; The target item timestamp and user vector e obtained in step B1 u Input the fine-grained time fusion network for training to obtain the target object timestamp vector t target ; The above obtained t target , e target , p target The three are vector added to get the target item vector The interaction timestamp sequence S obtained in step B1 t and the user vector e u Input the fine-grained time fusion network for training to obtain the fine-grained time series matrix Through the position embedding matrix E p To represent the corresponding item position information in the item interaction sequence; the item interaction sequence matrix The corresponding fine-grained time series matrix and the position embedding matrix E p According to the last dimension of the matrix, we can concatenate and obtain the fine-grained time-item matrix E S .

[0088] In this embodiment, step B2 specifically includes the following steps:

[0089] Step B21: The interaction timestamp sequence S obtained in step B13 t , from the interaction timestamp sequence S t Each timestamp t in i Extract various fine-grained time information, including year information yuer, season information quarter, month information month, day of month, day of week, hour information hour; input the user ID and target item ID into the user embedding layer and item embedding layer to obtain e u and e target .

[0090] Step B22: Process the various fine-grained time information obtained in step B21.

[0091] The method for processing the day of the week is: according to the 7 days in a week, the weekly sequence S week ={1,…,7}, and the week sequence matrix is ​​obtained after the embedding layer mapping In order to make the mapping vectors with closer time distance more similar, the week sequence matrix Perform a one-dimensional CNN convolution operation with a window size of the preset value w 1 , the initial value of the window weight decreases from the center to both ends, and the updated vector matrix is ​​obtained Similarly, other updated time series matrices are obtained, including: year series matrix Seasonal series matrix Monthly sequence matrix Month-day sequence matrix Hourly sequence matrix

[0092]

[0093] Step B23: The interactive timestamp sequence S obtained in step B13 t , query the fine-grained time vector corresponding to each timestamp from the update time series matrix obtained in step B22, that is, for S t Each timestamp t in i There are corresponding fine-grained time vector groups Taking into account the different frequency characteristics of fine-grained time information, the fine-grained time vector group Divide into fast-changing fine-grained time information and slowly changing fine-grained temporal information User vector e u As a query, the fine-grained time vector group Two types of fine-grained time information in and Perform a cross-attention mechanism to obtain the updated time vectors and Then for the two vectors and Perform the splicing operation to get the timestamp t i The corresponding fine-grained time vector Thus we get the interaction timestamp sequence S t The corresponding fine-grained time series vector matrix

[0094]

[0095]

[0096]

[0097]

[0098]

[0099] in, and represents the attention coefficient, W1 and W2 represent trainable parameters, N s N represents the number of slowly changing fine-grained time information in the fine-grained time vector group. f Indicates the number of fast-changing fine-grained time information in the fine-grained time vector group.

[0100] Step B24: Embed the vector matrix E by position p To represent the corresponding item position information in the item interaction sequence; the item interaction sequence vector matrix The corresponding fine-grained time series vector matrix and position embedding vector matrix E p According to the last dimension of the vector matrix, we can concatenate and obtain the fine-grained time-item vector matrix E S ;

[0101]

[0102] Step B3: The fine-grained time-item matrix E obtained in step B2 S Input into the long-term interest extraction network constructed by the multi-head self-attention mechanism to extract the user's long-term interest matrix U long .

[0103] In this embodiment, the specific implementation method of step B3 is: the fine-grained time item vector matrix E obtained in step B24 S , where each row vector represents the fine-grained temporal item information e represented by the corresponding item in the sequence s , by the fine-grained time item vector matrix E S Perform multi-head self-attention mechanism learning to capture the user's long-term interest vector U in the interaction sequence long ;

[0104]

[0105]

[0106] Among them, head h represents the output of the h-th attention function, d h Indicates the dimension of each head, n head represents the number of attention functions, and W3 represents the trainable parameters.

[0107] Step B4: The fine-grained time-item matrix E obtained in step B2 S Input into the local interest extraction network constructed by the convolutional neural network to extract the user's short-term interest matrix Ushort .

[0108] In this embodiment, the specific implementation method of step B4 is: the fine-grained time item vector matrix E obtained in step B24 is S Input into the local interest extraction network constructed by a one-dimensional convolutional neural network to extract the user's short-term interest vector matrix U short ;

[0109]

[0110] Among them, w1 is the preset value of the window size.

[0111] Step B5: The user short-term interest matrix U obtained in step B4 short As a query, the user long-term interest matrix U obtained in step B3 long Execute the cross attention mechanism to extract the correlation interest matrix U associated with the short-term interests in the user's long-term interests link , through U long with U link Perform a subtraction operation to obtain the user's long-term interests that are not related to short-term interests U except ; U from the same user link with U short As a positive sample pair, U except with U short As a negative sample pair, contrastive learning is performed to bring the user's short-term interests closer to the interests related to the long-term interests and the short-term interests, and push away the interests that are not related to the short-term interests and the long-term interests, and obtain the contrast loss L cl .

[0112] In this embodiment, step B5 specifically includes the following steps:

[0113] Step B51: The user short-term interest matrix U obtained in step B4 short As a query, each row vector represents a short-term interest e short , the user long-term interest matrix U obtained in step B3 long Perform a cross-attention mechanism, where each row represents a long-term interest e loing , to extract the interest vector e associated with the short-term interest in the user's long-term interest link , and then form the long-term and short-term correlation interest matrix U link ;

[0114]

[0115]

[0116] in, represents the attention coefficient, W4 represents the trainable parameter, N s Represents the short-term interest matrix U short number of rows.

[0117] Step B52: Based on the user's long-term interest vector matrix U obtained in step B4 long , the long-short time associated interest vector U obtained in step B51 link Perform a subtraction operation to obtain the user's long-term interests that are not related to short-term interests U except ;

[0118] U except =U long -U link

[0119] Step B53: The interest U that is not related to the short-term interest among the user's long-term interests obtained in step B52 except , for the long-term and short-term correlation interest matrix U from the same user link and the user's short-term interest matrix U short As a positive sample pair, the long-term irrelevant interest matrix U except and the short-term interest matrix U short As a negative sample pair, comparative learning is performed to bring the user's short-term interests closer to the interests related to the long-term interests and the short-term interests, and push away the interests that are not related to the short-term interests and the long-term interests, and obtain the correlation interest contrast loss L cl ;

[0120] S p =Sim(U short ,U link )

[0121] S n =Sim(U short ,U except )

[0122]

[0123] Among them, τ is the temperature coefficient and sim represents the cosine similarity function.

[0124] Step B6: The user association interest matrix U obtained in step B5 link and the user's short-term interest matrix U short Input into the long-term and short-term similarity fusion network, and use the similarity information of long-term preference and short-term preference to obtain the fused interest matrix U fuse; The similarity information between long-term preference and short-term preference reflects the degree to which the user's current preference is affected by the surrounding environment. If the long-term preference is not similar to the short-term preference, it means that the user's current preference is greatly affected by the surrounding environment, and a greater weight is given to the short-term preference. If there is a certain similarity between the long-term preference and the short-term preference, a certain weight is given to the long-term preference.

[0125] In this embodiment, step B6 specifically includes the following steps:

[0126] Step B61: Associate the user interest vector U obtained in step B52 link The user short-term interest matrix U obtained in step B4 short Use the cosine similarity function to characterize the similarity between long-term preference and short-term preference to generate λ;

[0127] λ=Sigmoid(Sim(U short ,U link ))

[0128] Step B62: Obtain the similarity λ between the long-term preference and the short-term preference according to step B52, and fuse the user long-term interest matrix U obtained in step B3 long The user short-term interest matrix U obtained in step B4 short , generate the fused interest matrix U fuse ;

[0129] U fuse =λ*U short +(1-λ)*U long .

[0130] Step B7: The target object vector obtained in steps B2 and B6 And the fused interest matrix U fuse , take out the interest matrix U fuse The last line in With the target object vector Perform vector dot multiplication, use the sigmoid function to predict the click rate, and use the cross entropy function to calculate the prediction loss; then, based on the target loss function, calculate the gradient of each parameter in the deep learning network model through the back propagation method, and use the stochastic gradient descent method to update each parameter.

[0131] In this embodiment, step B7 specifically includes the following steps:

[0132] Step B71: The vector e of the target object obtained in steps B21 and B62 is t And the fused interest matrix U fuse , take out the interest matrix U fuse In and the target object vector Perform vector dot multiplication and use the sigmoid function to predict the click rate to get the predicted click rate:

[0133]

[0134] Step B72: Use the cross entropy function L target Calculate the prediction loss, update the learning rate through the gradient optimization algorithm Adam, and use back propagation to iteratively update the model parameters to minimize the loss function to train the model; the total model loss is the weighted addition of the above losses:

[0135]

[0136] L all =L target +βL cl

[0137] Where N represents the number of samples, L represents the length of a single training sample, is the corresponding label, β is the hyperparameter, L target is the loss function for click rate prediction, L cl is the correlation interest contrast loss function.

[0138] Step B8: When the loss value generated by the deep learning network model is less than the set threshold or reaches the maximum number of iterations, the training of the deep learning network model G is terminated.

[0139] This embodiment also provides a sequence recommendation system that captures long- and short-term interests and fuses fine-grained time, which is characterized by comprising a memory, a processor, and computer program instructions stored in the memory and capable of being executed by the processor. When the processor executes the computer program instructions, the above-mentioned method steps can be implemented.

[0140] Those skilled in the art will appreciate that the embodiments of the present application may be provided as methods, systems, or computer program products. Therefore, the present application may adopt the form of a complete hardware embodiment, a complete software embodiment, or an embodiment in combination with software and hardware. Moreover, the present application may adopt the form of a computer program product implemented in one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) that include computer-usable program code.

[0141] The present application is described with reference to the flowcharts and / or block diagrams of the methods, devices (systems), and computer program products according to the embodiments of the present application. It should be understood that each process and / or box in the flowchart and / or block diagram, as well as the combination of the processes and / or boxes in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to generate a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the processes in the flowchart and / or block diagram. Figure 1 A process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0142] These computer program instructions may also be stored in a computer-readable memory capable of directing a computer or other programmable data processing device to operate in a specific manner, so that the instructions stored in the computer-readable memory produce an article of manufacture comprising an instruction device, which implements the process Figure 1 A process or multiple processes and / or boxes Figure 1 A function specified in one or more boxes.

[0143] These computer program instructions can also be loaded onto a computer or other programmable data processing device so that a series of operating steps are executed on the computer or other programmable device to produce a computer-implemented process, thereby providing instructions for implementing the process. Figure 1 A process or multiple processes and / or boxes Figure 1 The steps for the functions specified in one or more boxes.

[0144] The above is only a preferred embodiment of the present invention, and does not limit the present invention in other forms. Any technician familiar with the profession may use the above disclosed technical content to change or modify it into an equivalent embodiment with equivalent changes. However, any simple modification, equivalent change and modification made to the above embodiment according to the technical essence of the present invention without departing from the technical solution of the present invention still belongs to the protection scope of the technical solution of the present invention.

Claims

1. A sequence recommendation method that combines long-term and short-term interest capture with fine-grained time fusion, characterized in that: The following steps are involved: Step A: Collect user behavior data, including user ID, target item ID, and interaction timestamp; Step B: construct a deep learning network model G that captures long-term and short-term interests and fuses fine-grained time. The deep learning network model G models the timestamp according to the change frequency characteristics of the fine-grained time information, uses the self-attention mechanism and the convolutional neural network to extract the user's long-term interests and short-term interests respectively, uses the contrastive learning mechanism to extract the related interests in the long-term and short-term interests, and introduces the related interest information into the interest fusion module to achieve accurate recommendation; construct a training set through the data collected in step A, and then use the training set to train the deep learning network model G; Step C: Input the user, item, and interaction data into the trained deep learning network model G, and output the current user's click probability for the target item.

2. The sequence recommendation method of long- and short-term interest capture and fine-grained time fusion according to claim 1 is characterized in that: The step B specifically comprises the following steps: Step B1: Process the data collected in step A to obtain the item interaction sequence S of each user u , the corresponding initial position code P init , the corresponding interaction timestamp sequence S t , target item ID, target item timestamp, target item initial position code, user ID; Step B2: The item interaction sequence S obtained in step B1 u Input to the item embedding layer to get the item interaction sequence matrix Input the user ID and target item ID into the user embedding layer and item embedding layer respectively to obtain the user vector e u and e target ; Encode the initial position obtained in step B1 as P init Input to the position embedding layer to get the position embedding matrix E p ; Input the target object initial position encoding obtained in step B1 into the position embedding layer to obtain the position embedding vector p target ; The target item timestamp and user vector e obtained in step B1 u Input the fine-grained time fusion network for training to obtain the target object timestamp vector t target ; The above obtained t target , e target , p target The three are vector added to get the target item vector The interaction timestamp sequence S obtained in step B1 t and the user vector e u Input the fine-grained time fusion network for training to obtain the fine-grained time series matrix Through the position embedding matrix E p To represent the corresponding item position information in the item interaction sequence; the item interaction sequence matrix The corresponding fine-grained time series matrix and the position embedding matrix E p According to the last dimension of the matrix, we can concatenate and obtain the fine-grained time-item matrix E S ; Step B3: The fine-grained time-item matrix E obtained in step B2 S Input into the long-term interest extraction network constructed by the multi-head self-attention mechanism to extract the user's long-term interest matrix U long ; Step B4: The fine-grained time-item matrix E obtained in step B2 S Input into the local interest extraction network constructed by the convolutional neural network to extract the user's short-term interest matrix U short ; Step B5: The user short-term interest matrix U obtained in step B4 short As a query, the user long-term interest matrix U obtained in step B3 long Execute the cross attention mechanism to extract the correlation interest matrix U associated with the short-term interests in the user's long-term interests link , through U long with U link Perform a subtraction operation to obtain the user's long-term interests that are not related to short-term interests U except ; U from the same user link with U short As a positive sample pair, U except with U short As a negative sample pair, contrastive learning is performed to bring the user's short-term interests closer to the interests related to the long-term interests and the short-term interests, and push away the interests that are not related to the short-term interests and the long-term interests, and obtain the contrast loss L cl ; Step B6: The user association interest matrix U obtained in step B5 link and the user's short-term interest matrix U short Input into the long-term and short-term similarity fusion network, and use the similarity information of long-term preference and short-term preference to obtain the fused interest matrix U fuse ; The similarity information between long-term preference and short-term preference reflects the degree to which the user's current preference is affected by the surrounding environment. If the long-term preference is not similar to the short-term preference, it means that the user's current preference is greatly affected by the surrounding environment, then the weight of the short-term preference is increased. If the long-term preference and the short-term preference have a set similarity, then the weight of the long-term preference is increased. Step B7: The target object vector obtained in steps B2 and B6 And the fused interest matrix U fuse , take out the interest matrix U fuse The last line in and the target object vector Perform vector dot multiplication, use the sigmoid function to predict click-through rate, and use the cross entropy function to calculate the prediction loss; then, according to the target loss function, calculate the gradient of each parameter in the deep learning network model through the back propagation method, and use the stochastic gradient descent method to update each parameter; Step B8: When the loss value generated by the deep learning network model is less than the set threshold or reaches the maximum number of iterations, the training of the deep learning network model G is terminated.

3. The sequence recommendation method of long- and short-term interest capture and fine-grained time fusion according to claim 2 is characterized in that: The step B1 specifically comprises the following steps: Step B11: For each user interaction sequence containing item ID and timestamp information, firstly filter out users and items that do not meet the conditions according to preset conditions; Step B12: Based on step B11, the user interaction sequences are first sorted in time order, and a corresponding data set is generated for each user's user interaction sequence according to the preset sequence length l+1. For the l+1th item in the user interaction sequence, the first l item sequence is constructed to predict the l+1th item; if the number of user interaction items is less than l+1, the beginning of the sequence is filled with zero values ​​according to the time order; Step B13: Split the data obtained in Step B12 into the user's item interaction sequence S u , the corresponding interaction timestamp sequence S t , the target item ID, the target item timestamp, and the user ID; for user u, (e1, t1) represents that user u interacts with the item with item ID e1 at timestamp t1. Then, user u has a user interaction sequence {(e1, t1), (e2, t2), (e3, t3), (e4, t4)}, where {t1 < t2 < t3 < t4}. According to the preset length of 4, divide the user interaction sequence. The initial position encoding rule is from 0 to the length of the user interaction sequence - 1, so as to obtain the item interaction sequence S u = {e1, e2, e3}, the interaction timestamp sequence S t = {t1, t2, t3}, S u corresponds to the initial position encoding as P init = {0, 1, 2}, the target item ID e4, the target item timestamp t4, and the initial position encoding of the target item {3}.

4. The sequence recommendation method of long- and short-term interest capture and fine-grained time fusion according to claim 3 is characterized in that: The step B2 specifically comprises the following steps: Step B21: The interaction timestamp sequence S obtained in step B13 t , from the interaction timestamp sequence S t Each timestamp t in i Extract various fine-grained time information, including year information yuer, season information quarter, month information month, day of month, day of week, hour information hour; input the user ID and target item ID into the user embedding layer and item embedding layer to obtain e u and e target ; Step B22: Processing various fine-grained time information obtained in step B21; The method for processing the day of the week is: according to the 7 days in a week, the weekly sequence S week ={1,…,7}, and the week sequence matrix is ​​obtained after the embedding layer mapping In order to make the mapping vectors with closer time distance more similar, the week sequence matrix Perform a one-dimensional CNN convolution operation with a window size of the preset value w 1 , the initial value of the window weight decreases from the center to both ends, and the updated vector matrix is ​​obtained Similarly, other updated time series matrices are obtained, including: year series matrix Seasonal series matrix Monthly sequence matrix Month-day sequence matrix Hourly sequence matrix Step B23: The interactive timestamp sequence S obtained in step B13 t , query the fine-grained time vector corresponding to each timestamp from the update time series matrix obtained in step B22, that is, for S t Each timestamp t in i There are corresponding fine-grained time vector groups Taking into account the different frequency characteristics of fine-grained time information, the fine-grained time vector group Divide into fast-changing fine-grained time information and slowly changing fine-grained temporal information User vector e u As a query, the fine-grained time vector group Two types of fine-grained time information in and Perform a cross-attention mechanism to obtain the updated time vectors and Then for the two vectors and Perform the splicing operation to get the timestamp t i The corresponding fine-grained time vector Thus we get the interaction timestamp sequence S t The corresponding fine-grained time series vector matrix in, and represents the attention coefficient, W1 and W2 represent trainable parameters, N s N represents the number of slowly changing fine-grained time information in the fine-grained time vector group. f Indicates the number of fast-changing fine-grained time information in the fine-grained time vector group; Step B24: Embed the vector matrix E by position p To represent the corresponding item position information in the item interaction sequence; the item interaction sequence vector matrix The corresponding fine-grained time series vector matrix and position embedding vector matrix E p According to the last dimension of the vector matrix, we can concatenate and obtain the fine-grained time-item vector matrix E S ; 5. The sequence recommendation method of long- and short-term interest capture and fine-grained time fusion according to claim 4 is characterized in that: The specific implementation method of step B3 is: the fine-grained time-item vector matrix E obtained in step B24 S , where each row vector represents the fine-grained temporal item information e represented by the corresponding item in the sequence s , by fine-grained time item vector matrix E S Perform multi-head self-attention mechanism learning to capture the user's long-term interest vector U in the interaction sequence long ; Among them, head h represents the output of the h-th attention function, d h Indicates the dimension of each head, n head represents the number of attention functions, and W3 represents the trainable parameters.

6. The sequence recommendation method of long- and short-term interest capture and fine-grained time fusion according to claim 5 is characterized in that: The specific implementation method of step B4 is: the fine-grained time-item vector matrix E obtained in step B24 is S Input into the local interest extraction network constructed by a one-dimensional convolutional neural network to extract the user's short-term interest vector matrix U short ; Among them, w1 is the preset value of the window size.

7. The sequence recommendation method of long- and short-term interest capture and fine-grained time fusion according to claim 6 is characterized in that: The step B5 specifically comprises the following steps: Step B51: The user short-term interest matrix U obtained in step B4 short As a query, each row vector represents a short-term interest e short , the user long-term interest matrix U obtained in step B3 long Perform a cross-attention mechanism, where each row represents a long-term interest e long , to extract the interest vector e associated with the short-term interest in the user's long-term interest link , and then form the long-term and short-term correlation interest matrix U link ; in, represents the attention coefficient, W4 represents the trainable parameter, N s Represents the short-term interest matrix U short number of rows; Step B52: Based on the user's long-term interest vector matrix U obtained in step B4 long , the long-short time associated interest vector U obtained in step B51 link Perform a subtraction operation to obtain the user's long-term interests that are not related to short-term interests U except ; IN except =U long -IN link Step B53: The interest U that is not related to the short-term interest among the user's long-term interests obtained in step B52 except , for the long-term and short-term correlation interest matrix U from the same user link and the user's short-term interest matrix U short As a positive sample pair, the long-term irrelevant interest matrix U except and the short-term interest matrix U short As a negative sample pair, comparative learning is performed to bring the user's short-term interests closer to the interests related to the long-term interests and the short-term interests, and push away the interests that are not related to the short-term interests and the long-term interests, and obtain the correlation interest contrast loss L cl ; S p =Yes(U short ,U link ) S n =Yes(U short ,U except ) Among them, τ is the temperature coefficient and sim represents the cosine similarity function.

8. The sequence recommendation method of long- and short-term interest capture and fine-grained time fusion according to claim 7 is characterized in that: The step B6 specifically comprises the following steps: Step B61: Associate the user interest vector U obtained in step B52 link The user short-term interest matrix U obtained in step B4 short Use the cosine similarity function to characterize the similarity between long-term preference and short-term preference to generate λ; λ=Sigmoid(Sim(U short ,IN link )) Step B62: Obtain the similarity λ between the long-term preference and the short-term preference according to step B52, and fuse the user long-term interest matrix U obtained in step B3 long The user short-term interest matrix U obtained in step B4 short , generate the fused interest matrix U fuse ; IN fuse =λ*U short +(1-λ)*U long 。 9. The sequence recommendation method of long- and short-term interest capture and fine-grained time fusion according to claim 8 is characterized in that: The step B7 specifically includes the following steps: Step B71: The vector e of the target object obtained in steps B21 and B62 is t And the fused interest matrix U fuse , take out the interest matrix U fuse In and the target object vector Perform vector dot multiplication and use the sigmoid function to predict the click rate to get the predicted click rate: Step B72: Use the cross entropy function L target Calculate the prediction loss, update the learning rate through the gradient optimization algorithm Adam, and use back propagation to iteratively update the model parameters to minimize the loss function to train the model; the total model loss is the weighted addition of the above losses: L all =L target +βL cl Where N represents the number of samples, : represents the length of a single training sample, is the corresponding label, β is the hyperparameter, L target is the loss function for click rate prediction, L cl is the associated interest contrast loss function.

10. A sequence recommendation system that combines long-term and short-term interest capture with fine-grained time fusion, characterized in that: The method comprises a memory, a processor, and computer program instructions stored in the memory and executable by the processor. When the processor executes the computer program instructions, the method steps as described in any one of claims 1 to 9 can be implemented.

Images