A self-attention recommendation method, apparatus, and device based on data lineage features and behavioral semantic features.

Abstract

This application discloses a self-attention recommendation method based on data lineage features and behavioral semantic features, comprising: training a product recommendation model based on a neural network using users' historical interaction sequence information and data lineage relationship information involving products; the product recommendation model including a data lineage feature extraction module, a behavioral semantic modeling module, and a self-attention decoder; acquiring the target user's historical interaction sequence information and the data lineage relationship information involving products and inputting them into the trained product recommendation model, and outputting the corresponding target recommended product. This invention also provides a self-attention recommendation device and apparatus based on data lineage features and behavioral semantic features. The product recommendation model trained by this method can further utilize data lineage features and user behavioral semantic features, thus enabling more accurate item recommendations.

Description

Technical Field

[0001] This invention relates to the field of sequence recommendation, and in particular to a self-attention recommendation method and apparatus based on data lineage features and behavioral semantic features. Background Technology

[0002] With the rapid development of social network services, massive amounts of data are being used to mine user behavior patterns to achieve personalized product recommendations. Recommendation tasks aim to suggest products that users might be interested in based on their historical behavior and basic product information, thereby improving user experience and generating potential revenue for social network operators and businesses.

[0003] Chinese patent document CN114529364A discloses a product sequence recommendation method based on an attention mechanism, comprising: acquiring a user's historical product dataset and performing data preprocessing; using a multi-head attention mechanism to capture the relationship dependencies between items in the user's recent historical behavior to obtain a short-term interest preference vector; adding the similarity coefficient between the user and the item in the implicit space to the weights of the attention mechanism, assigning different weights to long-term interests, and obtaining the final representation of long-term preferences. Finally, the short-term and long-term preferences are weighted and fused to obtain the user's final interest preference. Then, the user interacts with each item in the candidate set, and the relevance score is obtained in the prediction layer. The obtained scores are then sorted from high to low to obtain the recommendation sequence, with a higher relevance score indicating a greater likelihood of the user's next interaction.

[0004] Chinese patent document CN114218475A discloses a method and apparatus for effectively training a deep sequence recommendation model through iterative stacking. The method includes: first training a shallow sequence recommendation model using partial data; then, through iterative stacking, using the parameters of the network structure blocks of the previous shallow model as the initialization of a new top-level structure block to construct a sequence recommendation model with twice the depth; and finally, adding new data to fine-tune the new model, thereby accelerating model training and saving training time and costs. This iterative stacking method allows for gradual expansion from a shallow sequence recommendation model to a sequence recommendation model of ideal depth, as needed.

[0005] Existing technologies cannot effectively capture the long-term and short-term dependencies shown in user interactions with products, and they only process product information by simply splicing and stacking it, without deeply mining the characteristics of product information. As a result, the recommendation effect of existing technologies is not ideal.

[0006] Therefore, how to leverage the short-term and long-term dependencies between users and products, how to deeply mine product information features, and how to improve the accuracy of item recommendations in recommendation models are technical problems that need to be solved by those skilled in the art. Summary of the Invention

[0007] This invention provides a self-attention recommendation method based on data lineage features and behavioral semantic features, which can improve the accuracy of self-attention recommendation; this invention also provides a self-attention recommendation device based on data lineage features and behavioral semantic features, which also has the above-mentioned beneficial effects.

[0008] The technical solution of the present invention is as follows:

[0009] A self-attention recommendation method based on data lineage features and behavioral semantic features includes:

[0010] A product recommendation model is trained based on a neural network by utilizing users’ historical interaction sequence information and data lineage information related to products. The product recommendation model includes a data lineage feature extraction module, a behavioral semantic modeling module, and a self-attention decoder.

[0011] Obtain the target user's historical interaction sequence information and the data lineage information of the products involved, input them into the trained product recommendation model, and output the corresponding target recommended products.

[0012] The product recommendation model includes:

[0013] The data lineage feature extraction module obtains the data lineage features of products by using a multi-layer graph convolutional layer based on the lineage link diagram of products involved in the user's historical interaction sequence.

[0014] The behavioral semantic modeling module extracts the user's behavioral semantic features based on the user's historical interaction sequence;

[0015] The self-attention decoder transforms data lineage features and behavioral semantic features into a probability distribution of the target recommended product.

[0016] The process of extracting data lineage features using the data lineage feature extraction module includes:

[0017] Initialize the feature matrix of the product link nodes based on the product's data lineage link diagram;

[0018] By stacking multiple graph convolutional layers, the information of the feature matrix is ​​passed layer by layer to higher orders, thus obtaining the data lineage features of the product.

[0019] Furthermore, the data lineage characteristics of the product are extracted through the following steps:

[0020] (A1) Define the data lineage graph as follows: in, The data lineage nodes in the diagram are... Let the relationships between nodes be represented by the degree matrix and adjacency matrix of the data lineage graph. make To include A matrix of feature vectors of n nodes, where each row... It is node v i eigenvectors;

[0021] (A2) The initial feature matrix of the product link nodes is obtained through the first layer of graph convolutional layer. The calculation formula is as follows:

[0022]

[0023] In the formula, ρ is the initial feature matrix output by the first convolutional layer; ρ is the activation function; W0 is the weight matrix. This is the normalized symmetric autocorrelation matrix;

[0024] (A3) By passing the information of the initial feature matrix layer by layer through multiple graph convolutional layers, the data lineage feature of the product is obtained. The calculation formula is as follows:

[0025]

[0026] in, W and Wj are the feature matrix and weight matrix of the j-th layer, respectively;

[0027] (A4) At this point, we have obtained the data lineage characteristics of the products involved in the user's historical interaction sequence.

[0028]

[0029] Where GCN is a j-layer graph convolutional network, and X is... The matrix of eigenvectors of n nodes, where [·] represents the slicing operation.

[0030] The process of extracting user behavioral semantic features through the behavioral semantic modeling module includes:

[0031] Generate a fixed-length product embedding matrix for the set of products involved in the user's historical interaction sequence;

[0032] Generate a one-hot code for each product in the user's historical interaction sequence, and perform matrix multiplication between the one-hot code and the product embedding matrix to obtain the user interaction sequence embedding matrix;

[0033] By fusing the user interaction sequence embedding matrix with the corresponding time encoding, the semantic features of user behavior can be obtained.

[0034] Furthermore, the following steps are used to extract semantic features of user behavior:

[0035] (B1) Define the user's historical interaction sequence User historical interaction sequence Collection of products Embedding a matrix for products, for historical interaction sequences Each product in k The embedding vector is E l l k ,in It is a product l k The corresponding one-hot vector;

[0036] (B2) The interaction time is encoded using location coding, and the time feature embedding E is calculated by the following formula. t :

[0037]

[0038] Where i∈{1,2,…,d} m} indicates time embedding The dimension, t k Indicates the timestamp of the interactive activity;

[0039] (B3) User historical interaction sequence Historical Interaction Embedding Matrix Calculated by the following formula:

[0040]

[0041] in, Indicates product l k The set of one-hot vectors, Indicates the transpose operation;

[0042] Historical Interaction Embedded Matrix This refers to behavioral semantic features.

[0043] The self-attention decoder transforms data lineage features and behavioral semantic features into a probability distribution for the target recommendation product, including:

[0044] The self-attention decoder includes a multi-head attention module, a feedforward neural network, and a multilayer perceptron;

[0045] Dependencies between products in a historical interaction sequence are modeled by assigning self-attention scores;

[0046] High-dimensional features in multiple mapping spaces are extracted using a multi-head attention module;

[0047] The high-dimensional features are transformed into a probability distribution using a feedforward neural network and a multilayer perceptron.

[0048] Obtaining the probability distribution of the recommended product through a self-attention decoder includes the following steps:

[0049] (C1) Data lineage characteristics and behavioral semantic features Broadcast summation yields a representation matrix that integrates kinship features and behavioral semantic features.

[0050]

[0051] (C2) Calculate attention output using the following formula

[0052]

[0053] Q = YW Q K = YW K V = YW V

[0054] Where Q, K, and V are the query matrix, key matrix, and value matrix transformed from the representation matrix Y, respectively, and W... Q , and It is a linearly changing weight matrix;

[0055] (C3) By stacking multi-head self-attention layers to learn the depth representation of each product in different trajectories, the output O of the multi-head attention layer is given by the following formula:

[0056] O = [S1,S2,…,S] z W O

[0057] Where z is the number of heads in the multi-head attention layer; It is an aggregate matrix; It is the output of the i-th head;

[0058] (C4) Use the output O of the multi-head attention layer as the input to the position feedforward neural network to obtain the output of the position feedforward neural network. The calculation process is as follows:

[0059]

[0060] in, and These are all learnable parameters in a feedforward neural network, d h yes and the dimension of b1;

[0061] (C5) After performing pooling and batch regularization on F, the user's product preference representation is obtained.

[0062] (C6) Representing user product preferences by stacking multilayer perceptrons Probability distribution of conversion into target recommended products The calculation process is as follows:

[0063] h (j+1) =MLP j (h (j) )

[0064] Among them, MLP j For the j-th multilayer perceptron, the initial hidden representation h (0) h represents the user's product preferences.

[0065] This invention also provides a self-attention recommendation device based on data lineage features and behavioral semantic features, comprising:

[0066] The model training module uses the user's historical interaction sequence information and the data lineage information of the products to train a product recommendation model based on a neural network. The product recommendation model includes a data lineage feature extraction module, a behavioral semantic modeling module, and a self-attention decoder.

[0067] The target product recommendation module obtains the target user's historical interaction sequence information and the data lineage information of the products involved, inputs them into the trained product recommendation model, and outputs the corresponding target recommended products.

[0068] This invention also provides a self-attention recommendation device based on data lineage features and behavioral semantic features, comprising:

[0069] Memory, used to store computer programs;

[0070] A processor, configured to implement the steps of the self-attention recommendation method based on data lineage features and behavioral semantic features as described in any one of claims 1-8 when executing the computer program.

[0071] Compared with the prior art, the beneficial effects of the present invention are as follows:

[0072] Compared to existing technologies that fail to effectively capture the short- and long-term dependencies exhibited in user-product interactions and merely process product information through simple concatenation and stacking without deeply exploring product information features, this invention introduces a self-attention recommendation method based on data lineage features and behavioral semantic features. This method introduces a data lineage link graph to represent the potential relationships between product attribute fields and employs a graph convolutional network to extract lineage features. It utilizes trainable product embedding vectors to represent product features and combines these features with temporal features to represent user behavioral characteristics. A self-attention layer extracts the short- and long-term dependencies exhibited in user-product interactions, and a multilayer perceptron predicts the recommendation probability distribution of candidate products. In other words, the product recommendation model trained by this invention can further utilize data lineage features and user behavioral semantic features, thus enabling more accurate item recommendations.

[0073] The recommended apparatus and recommended equipment provided by the present invention also have the above-mentioned beneficial effects. Attached Figure Description

[0074] Figure 1 This is a diagram of the self-attention recommendation model architecture based on data lineage features and behavioral semantics of the present invention.

[0075] Figure 2 This is a flowchart illustrating the self-attention recommendation method based on data lineage features and behavioral semantics of the present invention.

[0076] Figure 3 This is an example of data lineage, where each node represents a database node and a business code node, etc. Detailed Implementation

[0077] The present invention will be further described in detail below with reference to the accompanying drawings and embodiments. It should be noted that the embodiments described below are intended to facilitate the understanding of the present invention and do not limit it in any way.

[0078] The core of this invention is to provide a self-attention recommendation method based on data lineage features and behavioral semantics, which can improve the accuracy of product recommendations based on data lineage features and behavioral semantics, including: training a product recommendation model based on a neural network using user interaction history and product information.

[0079] It should be noted that, Figure 1 The diagram below illustrates the framework of a self-attention recommendation model based on data lineage features and behavioral semantic features, as provided in this embodiment of the invention. As can be seen, the recommendation model includes: a data lineage feature extraction module, a behavioral semantic modeling module, and a self-attention decoder.

[0080] Figure 2This is a flowchart illustrating a self-attention recommendation method based on data lineage features and behavioral semantic features, provided as an embodiment of the present invention. Figure 2 As shown, a self-attention recommendation method based on data lineage features and behavioral semantic features includes:

[0081] S1. Define related terms: user set, product set, interaction behavior, interaction sequence, data lineage diagram;

[0082] S2. Extract lineage features through the data lineage feature extraction module: Generate the product attribute feature network based on the mined lineage link graph, and obtain product information features based on data lineage relationship by stacking multiple graph convolutional layers to achieve high-order information transfer of lineage nodes in the network.

[0083] S3. Extract user behavior features using the behavior semantic modeling module: Generate a fixed-length product embedding matrix for the product set; generate a one-hot code for each product in the user interaction sequence, and perform matrix multiplication between the one-hot code and the product embedding matrix to obtain the user interaction sequence embedding; fuse the user interaction sequence embedding with the corresponding time code to output the user's behavior semantic representation.

[0084] S4. Use a self-attention decoder to convert lineage features and behavioral features into probability distributions of recommended products: This decoder directly models the dependency relationships between historical interactive products by assigning self-attention scores, extracts high-dimensional features in multiple mapping spaces through a multi-head attention module, and converts the high-dimensional features into probability distributions through a feedforward neural network and a multilayer perceptron.

[0085] S5. Generate training and test sets using user interaction records and product information. Train the model on the training set using the cross-entropy loss function and make product recommendations on the test set.

[0086] The specific implementation steps of S1 are as follows:

[0087] S11. Define the user set as... Indicates the number of users; the product set is Represents the quantity of products;

[0088] S12. Define the interaction between the user and the product as... in, This indicates that user u interacted with product l at time t, such as by purchasing, rating, or clicking.

[0089] S13. Define the interaction sequence in, This represents the historical interaction sequence between user u and related products. Indicates the number of interactions;

[0090] S14. Define product embedding as Among them, E / Represents the product embedding matrix. d represents the set of real numbers. m Let e ​​be the dimension of the embedded vector; i For product l i Embedding vector; define l i For product l i The unique heat vector;

[0091] The optimal dimension d of the embedding vector m Take 512.

[0092] S15. Define the data lineage feature map as follows: in, The data lineage nodes in the diagram are... For the relationships between nodes, such as Figure 3 As shown. Meanwhile, let the degree matrix and adjacency matrix of the data lineage feature map be respectively... make To include A matrix of eigenvectors of n nodes, where each row... It is node v i eigenvectors.

[0093] The data lineage feature extraction module described in S2 is implemented using the following steps:

[0094] S21. A graph convolutional network is used to extract the lineage features of the data. The feature matrix output by the first layer of the graph convolutional network is: The calculation process is as follows:

[0095]

[0096] in, Normalized symmetric autocorrelation matrix, W0 is a weight matrix, ρ is the activation function, M (1) This is the feature matrix output by the first graph convolutional layer.

[0097] S22. To integrate relevant information over a larger neighborhood, j graph convolutional network layers are stacked, and the formula for feature propagation layer by layer is as follows:

[0098]

[0099] in, and W j These are the feature matrix and weight matrix of the j-th layer, respectively.

[0100] S23. At this point, the interaction sequence... The lineage characteristics of the products can be obtained from the data, and the calculation formula is as follows:

[0101]

[0102] in, for The data lineage feature matrix of the products, GCN is a j-layer graph convolutional network, and X is... The matrix of eigenvectors of n nodes, where [·] represents the slicing operation.

[0103] The specific implementation steps of the behavioral semantic modeling module described in S3 are as follows:

[0104] S31. For product collections The product is embedded in the matrix, therefore for historical interaction sequences Each product in k We represent its embedding vector as E l l k ,in It is a product l k The corresponding one-hot vector.

[0105] S32. Encode the time of the interaction using location encoding, embedding the time feature into E. t Calculated using the following formula:

[0106]

[0107] Where i∈{1,2,…,d} m} indicates time embedding The dimension, t k Indicates the timestamp of the interactive activity.

[0108] S33. Historical Interaction Embedding Matrix It is given by the following formula:

[0109]

[0110] in, This represents the set of one-hot vectors of the products that have been interacted with. This indicates the transpose operation.

[0111] The self-attention decoder described in S4 is implemented using the following steps:

[0112] S41. Representing data lineage characteristics and behavioral semantic representation By performing broadcast summation, we obtain the fused kinship feature and behavioral semantic feature Y:

[0113]

[0114] in, A representation matrix that integrates bloodline characteristics and behavioral semantics.

[0115] S42. Learn long-short-term dependencies through a self-attention mechanism to enhance the features of product embeddings in the representation matrix. Specifically, the attention output is calculated using the following formula.

[0116]

[0117] Q = YW Q K = YW K V = YW V

[0118] Where Q, K, and V are the query matrix, key matrix, and value matrix transformed from matrix Y, respectively, and W... q , and It is a linearly changing weight matrix.

[0119] The optimal dimension d of the weight matrix k and d v All values ​​are 512.

[0120] S43. By stacking multi-head self-attention layers to learn the deep representation of each product in different trajectories, the goal of learning multi-context-aware trajectory data is achieved. The multi-head attention output O is given by the following formula:

[0121] O = [S1,S2,…,S] z W O

[0122] Where z is the number of heads in the multi-head attention layer, with an optimal value of 8; It is an aggregate matrix. It is the output of the i-th head.

[0123] S44. The output O of the multi-head attention layer is used as the input of the position feedforward neural network to obtain the hidden representation F, and its calculation process is as follows:

[0124]

[0125] in, and These are all learnable parameters in a neural network, d h yes The optimal value for the dimension of b1 is 512, and F is the output of the position feedforward neural network.

[0126] S45. After N layers of multi-head self-attention, obtain a representation that includes data lineage features and user behavior semantics. After performing pooling and batch regularization on F, the user's product preference representation can be obtained.

[0127] S46. The user's product preference representation is transformed into a probability distribution for product recommendations by stacking multilayer perceptrons. The calculation process is as follows:

[0128] h (j+1) =MLP j (h (j) )

[0129] Among them, MLP j For the j-th multilayer perceptron, the initial hidden representation h (0) h represents the user's product preferences.

[0130] The specific implementation steps of S5 are as follows:

[0131] S51. Generate user interaction sequences using user product interaction records.

[0132] S52. Use the first 70% of the products in the interaction sequence as input to the training set, and use 70% to 80% of the products as labels for the training set.

[0133] Similarly, the first 80% of the products in the interaction sequence are used as input to the test set, and the remaining 20% ​​are used as labels for the test set.

[0134] S53. This model uses the cross-entropy function as the objective loss function, aiming to minimize the following loss:

[0135]

[0136] in, It is a multi-hot real label vector, meaning that if the user interacts with the product in a real scenario, the value corresponding to the index of the product in the vector is 1, otherwise it is 0.

[0137] S54. Use the test set to perform recommendation tests on the model.

[0138] Another core aspect of this invention is to provide a self-attention recommendation device based on data lineage features and behavioral semantics, comprising:

[0139] By inputting the user's historical access sequence and the product's data lineage diagram, the system outputs recommended products.

[0140] The self-attention recommendation device based on data lineage features and behavioral semantics provided in this embodiment of the invention has the beneficial effects of the self-attention recommendation method based on data lineage features and behavioral semantics described above.

[0141] The foregoing has provided a detailed description of the self-attention recommendation method and apparatus based on data lineage features and behavioral semantics provided by this invention. Specific embodiments have been used to illustrate the principles and implementation methods of this invention. These embodiments are merely illustrative and are intended to aid in understanding the method and its core concepts. It should be noted that those skilled in the art can make various improvements and modifications to this invention without departing from its principles, and these improvements and modifications also fall within the scope of protection of the claims of this invention.

[0142] Example 1: Tobacco Sales Recommendation Business

[0143] The dataset in this embodiment is real cigarette sales data collected by Zhejiang Tobacco Industry Co., Ltd., which includes 11,424 business entities (hereinafter referred to as users), 389 tobacco products, and 875,616 related sales records.

[0144] The goal of this embodiment is to recommend cigarettes to users based on the first 80% of their purchasing behavior in chronological order, and to use the remaining 20% ​​as an accuracy verification.

[0145] Specifically, in this embodiment, for all users, a cigarette purchase sequence is generated based on their cigarette purchase records. Simultaneously, the first 70% of cigarettes in each user's purchase sequence, ordered chronologically, are used as input to the training set, and 70%–80% of the products are used as labels for the training set. Similarly, the first 80% of cigarettes in the purchase sequence are used as input to the test set, and the remaining 20% ​​are used as labels for the test set.

[0146] Furthermore, the specific steps of this embodiment are as follows: Define relevant terms:

[0147] a) Define the user set as Define the set of cigarette types as The number of cigarette varieties;

[0148] b) Define the user's cigarette purchasing behavior as in, This represents the number of cigarettes (l) purchased by user u at time t.

[0149] c) Define the user's purchase sequence in, This represents the purchase sequence of user u;

[0150] d) Define product embedding as Among them, E / The embedding matrix of cigarettes is represented. d represents the set of real numbers. m Let e ​​be the dimension of the embedded vector; i For cigarettes i Embedding vector; define l i For product l i The unique heat vector.

[0151] e) Define the data lineage feature map as follows: in, The data lineage nodes in the diagram are... Let the relationships between nodes be defined. Meanwhile, let the degree matrix and adjacency matrix of the data lineage feature map be respectively... make To include A matrix of eigenvectors of n nodes, where each row... It is node v i eigenvectors.

[0152] 2) Extract bloodline features from the data using the bloodline feature extraction module. The specific implementation steps are as follows:

[0153] a) A graph convolutional network is used to extract the lineage features of the data. The feature matrix output by the first layer of the graph convolutional network is shown in the figure. The calculation process is as follows:

[0154]

[0155] in, The normalized symmetric autocorrelation matrix, W0 is a weight matrix, and ρ is the activation function.

[0156] b) To integrate relevant information over a larger neighborhood, j graph convolutional network layers are stacked, and the formula for feature propagation layer by layer is as follows:

[0157]

[0158] in, and W j These are the feature matrix and weight matrix of the j-th layer, respectively.

[0159] c) At this point, the lineage characteristics of the cigarettes involved in the procurement sequence can be obtained, and the calculation formula is as follows:

[0160]

[0161] in, for The data lineage feature matrix of the products, GCN is a j-layer graph convolutional network, and X is... The matrix of eigenvectors of n nodes, where [·] represents the slicing operation.

[0162] 3) Extract user purchasing behavior features using the behavioral semantic modeling module. The specific implementation steps are as follows:

[0163] a) For cigarette sets For the cigarette embedding matrix, therefore for the historical interaction sequence Each cigarette in k We represent its embedding vector as E l l k ,in It is a product l k The corresponding one-hot vector.

[0164] b) Encode the time of the interaction using location encoding, embedding the time feature into E. t Calculated using the following formula:

[0165]

[0166] Where i∈{1,2,…,d} m} indicates time embedding The dimension, t k This indicates the timestamp of the corresponding user's purchasing activity.

[0167] c) Historical cigarette procurement embedded matrix It is given by the following formula:

[0168]

[0169] in, This represents the set of unique heat vectors for cigarettes purchased by a user throughout their history. This indicates the transpose operation.

[0170] 4) The kinship features and behavioral features are transformed into a probability distribution for recommended cigarettes using a self-attention decoder. The specific implementation steps are as follows:

[0171] a) To integrate kinship features and behavioral semantic features, kinship features are represented as follows: and behavioral semantic representation By performing broadcast summation, we get:

[0172]

[0173] in, A representation matrix that integrates bloodline characteristics and behavioral semantics.

[0174] b) Learn long-short-term dependencies through a self-attention mechanism to enhance the features of cigarette embeddings in the representation matrix. Specifically, the attention output is first calculated using the following formula.

[0175]

[0176] Q = YW q K = YW K V = YW V

[0177] Where Q, K, and V are the query matrix, key matrix, and value matrix transformed from matrix Y, respectively, and W... Q , and It is a linearly changing weight matrix.

[0178] c) By stacking multi-head self-attention layers to learn a deep representation of each cigarette in different trajectories, the goal of learning multi-context-aware trajectory data is achieved. The multi-head attention output O is given by the following formula:

[0179] O = [S] ( ,S2,…,S z W O

[0180] Where z is the number of heads in the multi-head attention layer. It is an aggregate matrix. It is the output of the i-th head.

[0181] d) The multi-head attention layer outputs O to obtain the hidden representation F, and its calculation process is as follows:

[0182]

[0183] in, and These are all learnable parameters in a neural network, d h yes The dimensions are b1 and b1, and F is the output of the position feedforward neural network.

[0184] e) After N layers of multi-head self-attention, a representation containing data lineage features and user behavior semantics can be obtained. After performing pooling and batch regularization on F, the user's product preference representation can be obtained.

[0185] f) Transform the hidden representation into a probability distribution for product recommendations by stacking multilayer perceptrons. The calculation process is as follows:

[0186] h (j+1) =MLP j (h (j) )

[0187] Among them, MLP j For the j-th multilayer perceptron, the initial hidden representation h (0) h represents the user's product preferences.

[0188] 5) Generate training and test sets using user interaction records and cigarette information. Train the model on the training set using the cross-entropy loss function, and make product recommendations on the test set. The specific process includes:

[0189] a) Use the first 70% of the products in the interaction sequence as input to the training set, and 70%–80% of the products as labels for the training set. Similarly, use the first 80% of the products in the interaction sequence as input to the test set, and the remaining 20% ​​as labels for the test set.

[0190] b) The model uses the cross-entropy function as the objective loss function, which aims to minimize the following loss.

[0191]

[0192] in, It is a multi-hot real label vector, meaning that if the user interacts with the product in a real scenario, the value corresponding to the index of the product in the vector is 1, otherwise it is 0.

[0193] c) Use the test set to perform recommendation tests on the model.

[0194] The embodiments described above provide a detailed explanation of the technical solutions and beneficial effects of the present invention. It should be understood that the above descriptions are merely specific embodiments of the present invention and are not intended to limit the present invention. Any modifications, additions, and equivalent substitutions made within the scope of the principles of the present invention should be included within the protection scope of the present invention.

Claims

1. A self-attention recommendation method based on data bloodline features and behavior semantic features, characterized in that, The application relates to a product recommendation method and device. The product recommendation model comprises: The data blood relationship feature extraction module obtains the data blood relationship features of the products through a multilayer graph convolution layer according to a blood link graph of the products involved in the historical interaction sequence of the user, and the data blood relationship feature extraction module comprises the following steps: (A2) the first-layer graph convolution layer is used to obtain an initial feature matrix of the product link node, and the calculation formula is as follows: (A3) the information of the initial feature matrix is sequentially transmitted through a multilayer graph convolution layer, and the data blood relationship features of the products are obtained, and the calculation formula is as follows: (A1) define the data bloodline link graph as wherein, is a data bloodline node in the graph, is a relationship between nodes; let the degree matrix and the adjacency matrix of the data bloodline link graph be , ; let be a matrix containing feature vectors of nodes, wherein each row is a feature vector of a node , is the dimension of the embedding vector; The behavior semantic modeling module extracts the behavior semantic features of the user according to the historical interaction sequence of the user, and the behavior semantic modeling module comprises the following steps: ； In the formula, is an initialized feature matrix output by a first layer convolutional layer; is an activation function; is a weight matrix; is a normalized symmetric autocorrelation matrix; The self-attention decoder converts the data blood relationship features and the behavior semantic features into the probability distribution of the target recommendation product. ； wherein, and are the first layer feature matrix and weight matrix, respectively; (A4) So far, the data bloodline features of the products involved in the historical interaction sequence of the user are obtained : ； wherein, is layer graph convolutional network, is matrix of feature vectors of nodes, is a slicing operation, is a user history interaction sequence; The behavior semantic features of the user are extracted through the following steps: The self-attention decoder converts the data blood relationship features and the behavior semantic features into the probability distribution of the target recommendation product, and the self-attention decoder comprises the following steps:

2. The self-attention recommendation method based on data blood relationship features and behavior semantic features according to claim 1, characterized in that, The self-attention decoder comprises a multi-head attention module, a feedforward neural network and a multilayer perceptron. (B1) defining a user historical interaction sequence , the user historical interaction sequence a set of products , embedding matrix for the products, for each product in the historical interaction sequence is , where ∈ is the one-hot vector corresponding to the product ; (B2) encode the time of the interaction with a position encoding, compute the time feature embedding from : ； wherein, represents a time embedding of dimension, represents a timestamp of when the interaction activity occurred; (B3) the user historical interaction sequence the historical interaction embedding matrix is calculated by the following formula: ； wherein, a set of one-hot vectors representing products , denotes a transpose operation; a history interaction embedding matrix is the behavior semantic feature. 3.The self-attention recommendation method based on data blood relationship features and behavior semantic features according to claim 1, characterized in that, The dependency relationship between the products in the historical interaction sequence is modeled by assigning self-attention scores; The multi-head attention module is used to extract high-dimensional features in a multi-mapping space; The high-dimensional features are converted into the probability distribution through the feedforward neural network and the multilayer perceptron. The probability distribution of the target recommendation product is obtained through the self-attention decoder, and the probability distribution of the target recommendation product is obtained through the following steps: The model training module trains the product recommendation model based on a neural network by using the historical interaction sequence information of the user and the data blood relationship information of the products involved, and the product recommendation model comprises a data blood relationship feature extraction module, a behavior semantic modeling module and a self-attention decoder; 4.The self-attention recommendation method based on data blood relationship features and behavior semantic features according to claim 3, characterized in that, The target product recommendation module inputs the historical interaction sequence information of the target user and the data blood relationship information of the products involved into the trained product recommendation model, and outputs the corresponding target recommendation product; (C1) broadcast adding data bloodline features and behavior semantic features to obtain a representation matrix of the fusion of bloodline features and behavior semantic features : ； (C2) The attention output is calculated by the following equation : ； ； wherein , and are the query matrix, key matrix and value matrix respectively transformed from the representation matrix , , and are linearly transformed weight matrices; (C3) Learning deep representation of each product in different trajectories by stacking multi-head self-attention layers, the multi-head attention layer outputs is given by the following equation: ； wherein, is the number of heads in the multi-headed attention layer; is the aggregation matrix; is the output of the th head. (C4) the multi-headed attention layer output as input to the positional feed-forward neural network to obtain an output of the positional feed-forward neural network The computation process is as follows: ； wherein, , , and are learnable parameters in the feedforward neural network, is , and is the dimension of (C5) to After the performing and pooling and batch normalization, the product preference representation of the user is obtained ; (C6) representing the user's product preference by stacking multiple layers of perceptrons probability distribution of converting into the target recommended product , = , The calculation process is as follows:​ ； wherein, is a first .​​ 5. A self-attention recommendation device based on data bloodline features and behavior semantic features, characterized in that, The product recommendation model comprises: ​ ​ ​ The data bloodline feature extraction module obtains data bloodline features of the product through a multi-layer graph convolution layer according to a bloodline link diagram of the product involved in the historical interaction sequence of the user, including: initializing a feature matrix of a product link node according to the data bloodline link diagram of the product; performing high-order transmission of information of the feature matrix layer by layer through a stacked multi-layer graph convolution layer to obtain the data bloodline features of the product; and extracting the data bloodline features of the product through the following steps: (A1) define the data bloodline link graph as wherein, is a data bloodline node in the graph, is a relationship between nodes; let the degree matrix and the adjacency matrix of the data bloodline link graph be , ; let be a matrix containing the characteristic vectors of the nodes, wherein each row is the characteristic vector of the node , is the dimension of the embedding vector; (A2) obtaining an initialized feature matrix of the product link node through a first-layer graph convolution layer, and the calculation formula is: ； In the formula, is an initialized feature matrix output by a first layer convolutional layer; is an activation function; is a weight matrix; is a normalized symmetric autocorrelation matrix; (A3) sequentially performing high-order transmission of information of the initialized feature matrix layer by layer through a multi-layer graph convolution layer to obtain the data bloodline features of the product, and the calculation formula is: ； wherein, and are the first layer feature matrix and weight matrix, respectively; (A4) So far, the data bloodline features of the products involved in the historical interaction sequence of the user are obtained : ； wherein, is layer graph convolutional network, is matrix of feature vectors of nodes, is a slicing operation, is a user history interaction sequence; The behavior semantic modeling module extracts behavior semantic features of the user according to the historical interaction sequence of the user, including: generating a fixed-length product embedding matrix for a product set involved in the historical interaction sequence of the user; generating a one-hot encoding for each product in the historical interaction sequence of the user, and performing matrix multiplication on the one-hot encoding and the product embedding matrix to obtain a user interaction sequence embedding matrix; and fusing the user interaction sequence embedding matrix and corresponding time encoding to obtain the behavior semantic features of the user; The self-attention decoder converts the data bloodline features and the behavior semantic features into a probability distribution of a target recommended product.

6. A self-attention recommendation device based on data bloodline features and behavior semantic features, characterized in that, The memory is configured to store a computer program. The processor is configured to execute the computer program to implement the steps of the self-attention recommendation method based on the data bloodline features and the behavior semantic features according to any one of claims 1-4. ​

Images