A personalized recommendation method and device for dynamic time scene applications

By preprocessing user data and using a time-aware self-attention mechanism, we generate short sequences of pseudo-historical items, solving the problem of inaccurate recommendation results in dynamic time scenarios. This enables a personalized and scalable recommendation system that can update recommendation results over time.

CN115329194BActive Publication Date: 2025-09-09HOHAI UNIV
View PDF 2 Cites 0 Cited by

Patent Information

Application Number
CN202210968240.5
Authority / Receiving Office
CN · China
Patent Type
Patents(China)
Current Assignee / Owner
Filing Date
2022-08-12
Publication Date
2025-09-09
Estimated Expiration
2042-08-12

AI Technical Summary

Technical Problem

Existing personalized recommendation methods lack adaptability in dynamic time scenarios, resulting in inaccurate recommendation results and the inability to update over time, limiting the performance of the recommendation system and user experience.

Method used

By preprocessing user data, dividing long-sequence and short-sequence user behaviors, and using the reversed long-sequence model to generate short-sequence pseudo-history items, combined with the time-aware self-attention mechanism, the time-aware self-attention recommendation model is trained to generate a time interval matrix to improve recommendation accuracy.

Benefits of technology

The recommendation results are personalized and scalable, and can be continuously updated over time, improving the adaptability and accuracy of the recommendation system in dynamic time scenarios.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN115329194B_ABST
    Figure CN115329194B_ABST
Patent Text Reader

Abstract

The present invention discloses a personalized recommendation method and device for dynamic temporal scenarios, comprising steps such as data preprocessing, training a reverse prediction model, generating pseudo-historical items, enhancing short sequences, and generating a time-aware self-attention recommendation model. The present invention can improve the ability of recommendation systems to adapt to dynamic temporal scenarios. It has the advantages of high accuracy and strong scalability. Compared with current personalized recommendation methods, the present invention alleviates the data sparsity problem of recommendation systems, improves the accuracy of recommendation results, makes recommendation results more personalized, and can continuously update recommendation results over time, thus enhancing practicality.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present invention relates to the technical field of data mining and recommendation systems, and in particular to a personalized recommendation method and device for dynamic time scenario applications. Background Art

[0002] In recent years, personalized recommendation methods have been widely used across major online platforms and mobile applications both domestically and internationally. However, with the rapid development of the internet, people are increasingly demanding more accurate and personalized recommendations. Using traditional, non-personalized recommendation methods will limit the performance of recommendation systems, leading to user churn and financial losses for businesses. Summary of the Invention

[0003] Objective: To overcome the shortcomings of the existing technology, the present invention aims to provide a personalized recommendation method for dynamic time scenario applications, which can improve the ability of the recommendation system to adapt to dynamic time scenarios, improve the accuracy of recommendation results and the scalability of the recommendation system, make the recommendation results more personalized, and continuously update the recommendation results over time, making them more practical.

[0004] Technical solution: To solve the above technical problems, the technical solution adopted by the present invention is:

[0005] In a first aspect, a personalized recommendation method for dynamic time scenario applications is provided, comprising:

[0006] (1) Preprocess the user's original data set to obtain a long sequence of user behavior sequences and short user behavior sequences The users are divided into long sequence users and short sequence users;

[0007] (2) Reverse the long sequence of user behavior sequences and use the reversed long sequence of user behavior sequences to train a reverse prediction model

[0008] (3) Migrating the reverse prediction model to short-sequence user behavior sequences On the other hand, pseudo historical items of short-sequence users are generated to obtain enhanced short-sequence user behavior sequences

[0009] (4) Sequence long sequence user behavior and enhanced short sequence user behavior sequence Together as the input of the time-aware self-attention recommendation model, the user's time interval matrix R is generated. u , where the time interval matrix R u Include time interval information;

[0010] (5) Train and obtain a time-aware self-attention recommendation model; input the time interval information, user behavior sequence and absolute position information of the item into the time-aware self-attention module for training, and finally obtain the time-aware self-attention recommendation model to realize the recommendation function.

[0011] In some embodiments, in step (1), preprocessing the original data set includes:

[0012] (1a) Divide the original data set by different users to obtain a historical behavior list for each user; for each item in the user behavior list, arrange it in the order of the time when the user interacts with it, and obtain the behavior sequence S of user u. u ={o1,o2,...,o w}, where o w represents the wth behavior;

[0013] (1b) The obtained user behavior sequence S u According to the length of the sequence, it is divided into long sequence user behavior sequence and short user behavior sequences That is, if the length of a user behavior sequence |S u If | is less than L, the sequence is considered a short-sequence user behavior sequence; otherwise, the sequence is considered a long-sequence user behavior sequence.

[0014] In some embodiments, step (3) comprises:

[0015] (3a) The reverse prediction model obtained in step (2) Migrate to short user behavior sequences And use the reverse prediction model to generate pseudo historical items of short sequence users, which can be expressed as:

[0016]

[0017] (3b) Place the generated pseudo-history item before the initial item o1 of the original short sequence to form a new enhanced short sequence user behavior sequence, which is expressed as:

[0018]

[0019] Among them, q represents the total number of pseudo history items generated, and n represents the number of original short sequence items.

[0020] In some embodiments, step (4) comprises:

[0021] (4a) Combining long sequence of user behavior and enhanced short sequence user behavior sequence Convert them together into a fixed-length sequence S:

[0022]

[0023] Among them, m represents the maximum sequence length of the input model; if or If the length of S is greater than m, only the latest m items are considered; otherwise, padding items are added to the left of the sequence S until its length reaches m;

[0024] (4b) Similarly, for the enhanced short sequence user behavior sequence and long user behavior sequences The corresponding time series and Convert to a fixed sequence t:

[0025]

[0026] (4c) For the time of the pseudo-history items generated in step (3), use the average time interval The calculation method is as follows:

[0027]

[0028] (4d) Get the user's fixed time series t=(t1, t2, ..., t m ), the time interval between the i-th item and the j-th item is defined as Δt=|t i -t j |, and use the relative length of the time interval between items The definition is as follows:

[0029]

[0030] (4e), the time interval matrix of user u is obtained as:

[0031]

[0032] In some embodiments, in step (5), the time interval information, the user behavior sequence, and the absolute position information of the item are input together into the time-aware self-attention module for training, including:

[0033] (5a) Embedding layer: For each input sequence, the user behavior sequence is converted into an item embedding matrix Absolute position information is converted into absolute position embedding matrix Time interval information is converted into time interval embedding matrix The following embedding matrix is ​​obtained:

[0034] E I =[m s1 ,m s2 ,...,m sm ] T

[0035]

[0036]

[0037] Among them, m sm represents the mth user behavior; Represent the key and value of the position embedding information in the self-attention mechanism respectively; Respectively represent the key and value of the relative time interval information embedding in the self-attention mechanism;

[0038] (5b) Model input: After the embedding layer, a new sequence is calculated get:

[0039] Z=[z1,z2,...,z m ] T

[0040] in, Calculated by the following formula:

[0041]

[0042]

[0043]

[0044] Where Z represents the output of the embedding layer, The input matrices representing queries, keys, and values, respectively, are unique per layer and per attention head; α ij 、e ij is the intermediate parameter; is the scaling factor; b i is the deviation term; T is the transposed matrix;

[0045] (5c) Position feedforward network: To make the model nonlinear to consider the interaction between different latent dimensions, a position feedforward network is applied to each output of the self-attention layer: the output z of the embedding layer is transformed into i As the input of the position feedforward network, the output of the position feedforward network FFN (z i ):

[0046] FFN(z i)=G((z i W1+b1)W2)+b2

[0047] Among them, G(.) is the activation function, represents the weight matrix, b1 and b2 represent the bias terms;

[0048] (5d) Stacked self-attention blocks: residual connections, dropout, and layer normalization are used as follows:

[0049] Z i =z i +Dropout(FFN(LN(z i )))

[0050]

[0051] Where ⊙ is the element-wise product; μ, σ represent the mean and variance of x respectively; γ, β represent the learned scale factor and bias term respectively; Dropout(.) represents the dropout process; LN(.) represents the normalization function;

[0052] (5e) Prediction layer: In order to predict the user's next possible action, the SoftMax function is used to calculate the user's response to the candidate item y i,t The interaction probability p(y i,t ), the calculation formula is:

[0053]

[0054] in, represents the embedding vector of item i, Represents the first given sequence of t items (o1, o2, ..., o t ) and the time interval between it and the t+1th item (r 1(t+1) ,r 2(t+1) ,...,r t(t+1) ).

[0055] In some embodiments, in step (5), the following loss function Loss is used to train the optimization model:

[0056]

[0057] Among them, i represents the predicted candidate item, j represents the negative sample, They represent the interaction probability of user u to the predicted candidate items and negative samples, respectively. is the collective representation of the embedding matrix, and λ is the regularization parameter, which is used to prevent the model from overfitting.

[0058] In a second aspect, the present invention provides a personalized recommendation device for dynamic time scenario applications, including a processor and a storage medium;

[0059] The storage medium is used to store instructions;

[0060] The processor is configured to operate according to the instructions to execute the steps of the method according to the first aspect.

[0061] In a third aspect, the present invention provides a storage medium having a computer program stored thereon, which implements the steps of the method described in the first aspect when executed by a processor.

[0062] Beneficial Effects: The personalized recommendation method and device for dynamic time scenario applications provided by the present invention have the following advantages:

[0063] (1) The present invention uses transfer learning technology to transfer the reverse prediction model obtained by training long-sequence users to short-sequence users, generating pseudo-historical items of short-sequence users, thereby achieving short-sequence enhancement and solving the data sparsity problem;

[0064] (2) This paper proposes a new personalized recommendation method based on a time-aware self-attention mechanism model, which can deeply explore the changing trends of user behavior preferences over time and improve the ability of the recommendation system to be applied in dynamic time scenarios;

[0065] (3) The present invention combines the proposed data preprocessing method, short sequence enhancement technology and time-aware self-attention mechanism to improve the accuracy of recommendation results and the scalability of the recommendation system, making the recommendation results more personalized and able to continuously update the recommendation results over time, making them more practical. BRIEF DESCRIPTION OF THE DRAWINGS

[0066] Figure 1 Flowchart of the method in the embodiment of the present invention.

[0067] Figure 2 Schematic diagram of short sequence enhancement in an embodiment of the present invention.

[0068] Figure 3 This is a diagram of the improved time-aware self-attention model in an embodiment of the present invention. DETAILED DESCRIPTION

[0069] In order to make the technical means, creative features, objectives and effects achieved by the present invention easier to understand, the present invention is further described below in conjunction with specific embodiments.

[0070] In the description of the present invention, "several" means more than one, "plurality" means more than two, "greater than," "less than," and "exceed" are understood to exclude the number itself, while "above," "below," and "within" are understood to include the number itself. The use of "first" and "second" in the description is solely for the purpose of distinguishing technical features and should not be construed as indicating or implying relative importance, implicitly specifying the number of the indicated technical features, or implicitly specifying the order of the indicated technical features.

[0071] In the description of the present invention, reference to terms such as "one embodiment," "some embodiments," "illustrative embodiments," "examples," "specific examples," or "some examples" means that the specific features, structures, materials, or characteristics described in conjunction with the embodiment or example are included in at least one embodiment or example of the present invention. In this specification, the exemplary expressions of the above terms do not necessarily refer to the same embodiment or example. Moreover, the specific features, structures, materials, or characteristics described may be combined in any suitable manner in any one or more embodiments or examples.

[0072] Example 1

[0073] In some embodiments, the first aspect Figure 1 and Figure 2 As shown, a personalized recommendation method for dynamic time scenario applications is provided, including:

[0074] (1) Preprocess the user's original data set to obtain a long sequence of user behavior sequences and short user behavior sequences The users are divided into long sequence users and short sequence users;

[0075] (2) Reverse the long sequence of user behavior sequences and use the reversed long sequence of user behavior sequences to train a reverse prediction model

[0076] (3) Migrating the reverse prediction model to short-sequence user behavior sequences On the other hand, pseudo historical items of short-sequence users are generated to obtain enhanced short-sequence user behavior sequences

[0077] (4) Generate a time interval matrix; convert long sequence user behavior sequence and enhanced short sequence user behavior sequence Together as the input of the time-aware self-attention recommendation model, the user's time interval matrix R is generated. u , where the time interval matrix R u including time interval information;

[0078] (5) Train and obtain a time-aware self-attention recommendation model; input the time interval information, user behavior sequence and absolute position information of the item into the time-aware self-attention module for training, and finally obtain the time-aware self-attention recommendation model to realize the recommendation function.

[0079] The preprocessing of the original data set in step (1) includes:

[0080] (1a) Divide the original data set by different users to obtain different historical behavior lists of users; for each item in the behavior list, arrange it in the order of the time when the user interacts with it, and obtain the behavior sequence S of user u. u ={o1,o2,...,o w}, where o w represents the wth behavior;

[0081] (1b) The obtained user behavior sequence S u According to the length of the sequence, it is divided into long sequence user behavior sequence and short sequence user behavior sequence That is, if the length of a user behavior sequence |S u If | is less than L, the sequence is considered a short sequence user; otherwise, the sequence is considered a long sequence user.

[0082] The short sequence enhancement in step (3) refers to:

[0083] (3a) The reverse prediction model obtained in step (2) Migrate to the behavior sequence of short-sequence users And use the reverse prediction model to generate pseudo historical items of short sequence users, expressed as:

[0084]

[0085] (3b) Place the generated pseudo-history item before the initial item o1 of the original short sequence to form a new enhanced short sequence user behavior sequence, which is expressed as:

[0086]

[0087] Among them, q represents the total number of pseudo history items generated, and n represents the number of original short sequence items.

[0088] In some embodiments, step (4) specifically includes:

[0089] (4a) Combining long sequence of user behavior and enhanced short sequence user behavior sequence Convert them together into a fixed-length sequence S:

[0090]

[0091] Among them, m represents the maximum sequence length of the input model; if or If the length of S is greater than m, only the latest m items are considered; otherwise, padding items are added to the left of the sequence S until its length reaches m;

[0092] (4b) Similarly, for the enhanced short sequence user behavior sequence and long user behavior sequences The corresponding time series and Convert to a fixed sequence t:

[0093]

[0094] (4c) For the time of the pseudo-history items generated in step (3), use the average time interval The calculation method is as follows:

[0095]

[0096] (4d) Get the user's fixed time series t=(t1, t2, ..., t m ), the time interval between the i-th item and the j-th item is defined as Δt=|t i -t j |, and use the relative length of the time interval between items The definition is as follows:

[0097]

[0098] (4e), the time interval matrix of user u is obtained as:

[0099]

[0100] In step (5), the time interval information, the user behavior sequence and the absolute position information of the item are input together into the time-aware self-attention module for training, including:

[0101] (5a) Embedding layer: For each input sequence, the user behavior sequence is converted into an item embedding matrix Absolute position information is converted into absolute position embedding matrix Time interval information is converted into time interval embedding matrix The following embedding matrix is ​​obtained:

[0102] E I =[m s1 ,m s2 ,...,m sm ] T

[0103]

[0104]

[0105] Among them, m sm represents the mth user behavior; Represent the key and value of the position embedding information in the self-attention mechanism respectively; Respectively represent the key and value of the relative time interval information embedding in the self-attention mechanism;

[0106] (5b) Model input: After the embedding layer, a new sequence is calculated get:

[0107] Z=[z1,z2,...,z m ] T

[0108] in, Calculated by the following formula:

[0109]

[0110]

[0111]

[0112] Where Z represents the output of the embedding layer, The input matrices representing queries, keys, and values, respectively, are unique per layer and per attention head; α ij 、e ij is the intermediate parameter; is the scaling factor; b i is the deviation term; T is the transposed matrix;

[0113] (5c) Position feedforward network: To make the model nonlinear to consider the interaction between different latent dimensions, a position feedforward network is applied to each output of the self-attention layer: the output z of the embedding layer is transformed into i As the input of the position feedforward network, the output of the position feedforward network FFN (z i ):

[0114] FFN(z i )=G((z i W1+b1)W2)+b2

[0115] Among them, G(.) is the activation function, represents the weight matrix, b1 and b2 represent the bias terms;

[0116] (5d) Stacked self-attention blocks: residual connections, dropout, and layer normalization are used as follows:

[0117] Z i =z i +Dropout(FFN(LN(z i )))

[0118]

[0119] Where ⊙ is the element-wise product; μ, σ represent the mean and variance of x respectively; γ, β represent the learned scale factor and bias term respectively; Dropout(.) represents the dropout process; LN(.) represents the normalization function;

[0120] (5e) Prediction layer: In order to predict the user's next possible action, the SoftMax function is used to calculate the user's response to the candidate item y i,t The interaction probability p(y i,t ), the calculation formula is:

[0121]

[0122] in, represents the embedding vector of item i, Represents the first given sequence of t items (o1, o2, ..., o t ) and the time interval between it and the t+1th item (r 1(t+1) ,r 2(t+1) ,...,r t(t+1) ).

[0123] (5f) Model reasoning: The following loss function is used to train and optimize the model:

[0124]

[0125] Among them, i represents the predicted candidate item, j represents the negative sample, They represent the interaction probability of user u to the predicted candidate items and negative samples, respectively. is the collective representation of the embedding matrix, and λ is the regularization parameter, which is used to prevent the model from overfitting.

[0126] Example 2

[0127] In a second aspect, this embodiment provides a personalized recommendation device for dynamic time scenario applications, including a processor and a storage medium;

[0128] The storage medium is used to store instructions;

[0129] The processor is configured to operate according to the instructions to execute the steps of the method according to embodiment 1.

[0130] Example 3

[0131] In a third aspect, this embodiment provides a storage medium on which a computer program is stored. When the computer program is executed by a processor, the steps of the method described in Example 1 are implemented.

[0132] Those skilled in the art will appreciate that the embodiments of the present application can be provided as methods, systems, or computer program products. Therefore, the present application can adopt the form of a complete hardware embodiment, a complete software embodiment, or an embodiment in combination with software and hardware. Moreover, the present application can adopt the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to magnetic disk storage, CD-ROM, optical storage, etc.) that contain computer-usable program code.

[0133] The present application is described with reference to the flowcharts and / or block diagrams of the methods, devices (systems), and computer program products according to the embodiments of the present application. It should be understood that each process and / or box in the flowchart and / or block diagram, as well as the combination of the processes and / or boxes in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to produce a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the steps in the process. Figure 1 a process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0134] These computer program instructions may also be stored in a computer readable memory that can direct a computer or other programmable data processing device to work in a specific manner, so that the instructions stored in the computer readable memory produce an article of manufacture comprising an instruction device, which implements the process Figure 1 a process or multiple processes and / or boxes Figure 1 The function specified in one or more boxes.

[0135] These computer program instructions can also be loaded onto a computer or other programmable data processing device so that a series of operational steps are executed on the computer or other programmable device to produce a computer-implemented process, thereby providing the instructions executed on the computer or other programmable device for implementing the process. Figure 1 a process or multiple processes and / or boxes Figure 1 A step that specifies a function in one or more boxes.

[0136] It is understood from common technical knowledge that the present invention may be implemented by other embodiments that do not depart from its spirit or essential features. Therefore, the embodiments disclosed above are, in all respects, merely illustrative and not exclusive. All modifications within the scope of the present invention or equivalent to the scope of the present invention are intended to be encompassed by the present invention.

Claims

1. A personalized recommendation method for dynamic time scenario applications, characterized by: The method comprises: (1) Preprocess the user's original data set to obtain a long sequence of user behavior sequences and short user behavior sequences The users are divided into long sequence users and short sequence users; (2) Reverse the long sequence of user behavior sequences and use the reversed long sequence of user behavior sequences to train a reverse prediction model (3) Migrating the reverse prediction model to short-sequence user behavior sequences On the other hand, pseudo historical items of short-sequence users are generated to obtain enhanced short-sequence user behavior sequences (4) Sequence long sequence user behavior and enhanced short sequence user behavior sequence Together as the input of the time-aware self-attention recommendation model, the user's time interval matrix R is generated. u , where the time interval matrix R u including time interval information; (5) Train and obtain a time-aware self-attention recommendation model; input the time interval information, user behavior sequence, and absolute position information of the item into the time-aware self-attention module for training, and finally obtain the time-aware self-attention recommendation model to realize the recommendation function; The time interval information, user behavior sequence and absolute position information of the item are input together into the time-aware self-attention module for training, including: (5a), embedding layer: for each input sequence, the user behavior sequence is converted into an item embedding matrix, the absolute position information is converted into an absolute position embedding matrix, and the time interval information is converted into a time interval embedding matrix; (5b), model input: after the embedding layer, a new sequence is calculated; (5c), position feedforward network: in order to make the model have nonlinear characteristics to consider the interaction between different potential dimensions, a position feedforward network is applied to each output of the self-attention layer: the output z of the embedding layer is converted into an item embedding matrix ...d), model input: after the embedding layer, a new sequence is calculated; (5e), ​​position feedforward network: in order to make the model have nonlinear characteristics to consider the interaction between different potential dimensions, the position feedforward network is applied to each output of the self-attention layer: the output z of the embedding layer is converted into an item embedding matrix; (5f), model input: after the embedding layer, a new sequence is calculated; (5g), position feedforward network: in order to make the model have nonlinear characteristics to consider the interaction between different potential dimensions, the position feedforward network is applied to each output of the self-attention layer: the output z of the embedding layer is converted into an item embedding matrix; (5g), model input: after the embedding layer, a new sequence is calculated; (5g), position feedforward network: in order to make the model have nonlinear characteristics to consider the interaction between different potential dimensions, the position feedforward network is applied to each output of the self-attention layer i As the input of the position feedforward network, the output of the position feedforward network FFN (z i ): (5d), stacked self-attention block: residual connection, dropout and layer normalization are used; (5e), ​​prediction layer: in order to predict the user's possible next action, the SoftMax function is used to calculate the user's response to the candidate item y i,t The interaction probability p(y i,t ).

2. The personalized recommendation method for dynamic time scenario applications according to claim 1, characterized in that: In step (1), preprocessing the original data set includes: (1a) Divide the original data set by different users to obtain a historical behavior list for each user; for each item in the user behavior list, arrange it in the order of the time when the user interacts with it, and obtain the behavior sequence S of user u. u ={o1,o2,...,o w }, where o w represents the wth behavior; (1b) The obtained user behavior sequence S u According to the length of the sequence, it is divided into long sequence user behavior sequence and short user behavior sequences That is, if the length of a user behavior sequence |S u If | is less than L, the sequence is considered a short-sequence user behavior sequence; otherwise, the sequence is considered a long-sequence user behavior sequence.

3. The personalized recommendation method for dynamic time scenario applications according to claim 2, characterized in that: The step (3) comprises: (3a) The reverse prediction model obtained in step (2) Migrate to short user behavior sequences And use the reverse prediction model to generate pseudo historical items of short sequence users, which can be expressed as: (3b) Place the generated pseudo-history item before the initial item o1 of the original short sequence to form a new enhanced short sequence user behavior sequence, which is expressed as: Among them, q represents the total number of pseudo history items generated, and n represents the number of original short sequence items.

4. The personalized recommendation method for dynamic time scenario applications according to claim 1 or 3, characterized in that: The step (4) comprises: (4a) Combining long sequence of user behavior and enhanced short sequence user behavior sequence Convert them together into a fixed-length sequence S: Among them, m represents the maximum sequence length of the input model; if or If the length of S is greater than m, only the latest m items are considered; otherwise, padding items are added to the left of the sequence S until its length reaches m; (4b) Similarly, for the enhanced short sequence user behavior sequence and long user behavior sequences The corresponding time series and Convert to a fixed sequence t: (4c) For the time of the pseudo-history items generated in step (3), use the average time interval The calculation method is as follows: (4d) Get the user's fixed time series t=(t1, t2, ..., t m ), the time interval between the i-th item and the j-th item is defined as Δt=|t i -t j |, and use the relative length of the time interval between items The definition is as follows: (4e), the time interval matrix of user u is obtained as:

5. The personalized recommendation method for dynamic time scenario applications according to claim 1, characterized in that: (5a) Embedding layer: For each input sequence, the user behavior sequence is converted into an item embedding matrix Absolute position information is converted into absolute position embedding matrix Time interval information is converted into time interval embedding matrix The following embedding matrix is ​​obtained: E I =[m s1 ,m s2 ,...,m sm ] T Among them, m sm represents the mth user behavior; Represent the key and value of the position embedding information in the self-attention mechanism respectively; Respectively represent the key and value of the relative time interval information embedding in the self-attention mechanism; (5b) Model input: After the embedding layer, a new sequence is calculated get: Z=[z1,z2,...,z m ] T in, Calculated by the following formula: Where Z represents the output of the embedding layer, The input matrices representing queries, keys, and values, respectively, are unique per layer and per attention head; α ij 、e ij is the intermediate parameter; is the scaling factor; b i is the deviation term; T is the transposed matrix; (5c) Position feedforward network: To make the model nonlinear to consider the interaction between different latent dimensions, a position feedforward network is applied to each output of the self-attention layer: the output z of the embedding layer is transformed into i As the input of the position feedforward network, the output of the position feedforward network FFN (z i ): <h2 style=";text-align:left;direction:ltr">FFN(z<h2 style=";text-align:left;direction:ltr"> i <h2 style=";text-align:left;direction:ltr"> )=G((z<h2 style=";text-align:left;direction:ltr"> i <h2 style=";text-align:left;direction:ltr"> W1+b1)W2)+b2 Among them, G(.) is the activation function, represents the weight matrix, b1 and b2 represent the bias terms; (5d) Stacked self-attention blocks: residual connections, dropout, and layer normalization are used as follows: FROM i =from i +Dropout(FFN(LN(z i ))) Where ⊙ is the element-wise product; μ, σ represent the mean and variance of x respectively; γ, β represent the learned scale factor and bias term respectively; Dropout(.) represents the dropout process; LN(.) represents the normalization function; (5e) Prediction layer: In order to predict the user's next possible action, the SoftMax function is used to calculate the user's response to the candidate item y i,t The interaction probability p(y i,t ), the calculation formula is: in, represents the embedding vector of item i, Represents the first given sequence of t items (o1, o2, ..., o t ) and the time interval between it and the t+1th item (r 1(t+1) ,r 2(t+1) ,...,r t(t+1) ).

6. The personalized recommendation method for dynamic time scenario applications according to claim 5, characterized in that: In step (5), the following loss function Loss is used to train the optimization model: Among them, i represents the predicted candidate item, j represents the negative sample, They represent the interaction probability of user u to the predicted candidate items and negative samples, respectively. is the collective representation of the embedding matrix, and λ is the regularization parameter, which is used to prevent the model from overfitting.

7. A personalized recommendation device for dynamic time scenario applications, characterized in that: including processor and storage medium; The storage medium is used to store instructions; The processor is configured to operate according to the instructions to execute the steps of the method according to any one of claims 1 to 6.

8. A storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the steps of the method according to any one of claims 1 to 6 are implemented.

Citation Information

Patent Citations

  • Personalized knowledge recommendation method and system for different scenes

    CN111797321A

  • Recommendation method based on self-attention mechanism

    CN113822742A