Soft Attention Mechanism Learning Method Based on Positional Encoding

Abstract

The present invention relates to the field of graph neural network session recommendation, and specifically, it is about designing a learning method for a soft attention mechanism based on positional encoding. Session recommendation based on GNN is a method for personalized session recommendation using graph neural network (GNN) technology. In session recommendation based on GNN, a user's session can be modeled as a graph, where nodes represent different items or behaviors, and features regarding the user's interests and behavior patterns can be extracted from the session graph to perform personalized session recommendation. To improve the recommendation effect of session recommendation based on graph neural network, the present invention provides a learning method for a soft attention mechanism based on positional encoding. Positional encoding can help the model better understand the context relationship and sequential information of nodes in session data. The present invention introduces positional encoding into the soft attention mechanism of graph neural network session recommendation. By assigning unique positional encodings to items, the model can accurately capture the position and role of items in the session. In this way, the soft attention mechanism can calculate the attention weights between nodes more accurately. The present invention effectively improves the context modeling ability of the model by introducing positional encoding, and enhances the accuracy and personalization degree of the recommendation results.

Description

Technical Field

[0001] The present invention relates to the field of graph neural network session recommendation, and specifically, it is about designing a learning method of soft attention mechanism based on positional encoding. Background Art

[0002] Session recommendation based on GNN is a method for personalized session recommendation using graph neural network (GNN) technology; traditional personalized recommendation systems usually generate recommendation results based on users' historical behaviors and interests, but for session data containing chronological order and interaction relationships, traditional methods often struggle to capture its characteristics; as a machine learning model specifically designed for processing graph-structured data, GNN can capture the relationships and dependencies between nodes; in session recommendation based on GNN, a user's session can be modeled as a graph, where nodes represent different items or behaviors, and edges represent different interaction relationships, such as clicks, purchases, etc.; through the learning and inference of GNN, features related to users' interests and behavior patterns can be extracted from the session graph, thereby performing personalized session recommendation; however, most of the existing session recommendation schemes currently do not consider the importance of positional encoding for global features. Without introducing positional encoding, it may bring disadvantages such as loss of temporal information, confusion of node relationships, and weakening of context modeling ability, affecting the model's accurate understanding of session data and the quality of recommendation results.

[0003] The soft attention mechanism is a commonly used technique for strengthening the model's attention to and weight allocation for input data; by calculating the attention weights of each input element, the soft attention mechanism enables the model to selectively focus on task-related information; different from the traditional hard attention mechanism, the soft attention mechanism can assign different weights to multiple input elements, and these weights are continuous and differentiable, so that it can be trained end-to-end together with optimization algorithms such as gradient descent; the core idea of the soft attention mechanism is to adjust the representation or weighted sum of input elements through the learned attention weights, so that the model can more accurately focus on the information useful for the current task; by flexibly adjusting the attention weights, the soft attention mechanism can help the model better understand and utilize the relevance of input data, and improve the performance and effect of the model. Summary of the Invention

[0004] In order to improve the recommendation effect of session recommendation based on graph neural network, the present invention provides a learning method of soft attention mechanism based on positional encoding.

[0005] The technical solution adopted by the present invention to solve its technical problems is: a learning method of soft attention mechanism based on positional encoding, including the following steps:

[0006] 1) Generate positional encoding:

[0007] 1.1) Generate a position matrix for learning;

[0008] 1.2) Calculate position encoding based on the distance from the last item;

[0009] 2) Embed the soft attention mechanism:

[0010] 2.1) Calculate the average feature;

[0011] 2.2) Embed the position encoding into the calculation of the soft attention mechanism;

[0012] 2.3) Linearly combine the obtained coefficients with the global features to obtain the inter-group session features;

[0013] Compared with the existing technologies, the beneficial effect of the present invention is that position encoding is introduced into the soft attention mechanism of graph neural network session recommendation. The position encoding can help the model better understand the context relationship and sequential information of nodes in session data; by assigning unique position vectors to nodes, the model can accurately capture the position and role of nodes in the session; in this way, the soft attention mechanism can calculate the attention weights between nodes more accurately; introducing position encoding can improve the context modeling ability of the model and enhance the accuracy and personalization degree of the recommendation results. BRIEF DESCRIPTION OF THE DRAWINGS

[0014] The present invention will be further described below in conjunction with the drawings and embodiments. In the drawings:

[0015] Figure 1 are schematic diagrams of two position encodings embedding session features in an embodiment of the present invention;

[0016] Figure 2 are schematic diagrams of the MRR recommendation effect and loss in an embodiment of the present invention. DETAILED DESCRIPTION OF THE INVENTION

[0017] The embodiments of the present invention include the following steps:

[0018] In session recommendation, since items closer to the end of the session are likely to contribute more to predicting the next interaction, a position encoding is generated based on the distance of the item from the end of the session to balance the importance of each item; however, the position encoding plays different roles for local and global features, so a direct embedding method of the soft attention mechanism is adopted for global features, thereby improving the effect of the model;

[0019] The soft attention mechanism learning method based on position encoding focuses on processing the global features corresponding to inter-group sessions. The specific process is as follows:

[0020] First, generate a position matrix, then generate position information based on the distance between each item and the last item, calculate the position encoding, and adjust the weights of different items in the recommendation;

[0021] Then, generate the global item average factor feature according to the original global item factor feature, so as to reduce the noise of irrelevant items in the inter-group session and enhance the performance ability of the inter-group session feature;

[0022] Next, the position encoding, average feature, and global item factor feature participate in the learning of the soft attention mechanism to obtain the feature weight coefficient;

[0023] Finally, through linear combination, obtain the final feature of the inter-group session;

[0024] Based on the inter-group session feature obtained above for training, effectively embed the position encoding into the soft attention mechanism of the global feature, thereby improving the effect of model recommendation.

[0025] (1) Generate position encoding

[0026] The process of generating position encoding includes setting the position matrix and calculating the position encoding:

[0027] (1.1) Set the position matrix

[0028] Given a session sequence s = [v s,1 , v s,2 ,...., v s,ι , with length ι, generate a position bias for each element v s,i in the session sequence Combine these ι position biases to generate a learned position matrix

[0029] (1.2) Calculate the position encoding

[0030] For each item v s,i in the session sequence, set its position encoding according to its relative distance from the last item v s,ι Thus, all corresponding learnable position encodings in the session sequence are obtained

[0031] (2) Embed the soft attention mechanism

[0032] The embedding of the soft attention mechanism in the position encoding is divided into three steps: generating the average feature, embedding, and linear combination:

[0033] (2.1) Calculate the average feature of each session

[0034] Given a session sequence s = [v s,1 ​, v s,2 ,..., v s,ι , with a length of ι. In session-based recommendation of hierarchical graph neural networks, by obtaining the relationships between the graph structures of different sessions, the global features of each item are obtained The k-th factor of the global features of all items is taken out and summed up, and then the sum result is divided by the session length ι, thus obtaining the average feature factor of all items Finally, they are concatenated to obtain the average feature factor of all items in the session sequence, and the average feature is obtained The corresponding formula is as follows:

[0035]

[0036] (2.2) Embedding

[0037] For each session sequence, the global features of each item are known Average factor feature and positional encoding The k-th factor of the feature of the i-th item The k-th factor of the average feature and the positional encoding of the i-th item in the embedding calculation are respectively multiplied by the learnable matrices and , and their multiplication results are added to the learnable vector b k ; then, the sum result is sigmoid-activated to map into a vector between 0 and 1, and this vector is multiplied by the learnable vector to obtain a constant between 0 and 1 which is the global feature coefficient of each item; the corresponding formula is as follows:

[0038]

[0039] (2.3) Linear combination

[0040] The global features of each item are known and their coefficients are; first, the global factor features of each item are multiplied by the coefficients , and then their multiplication results are summed up, thus obtaining the between-group session feature factor s g,k , and finally, these between-group session feature factors are combined to obtain the between-group session feature s g = [s g,1 , s g,2 , …, s g,K ; the corresponding formula is as follows:

[0041]

Claims

1. A learning method for soft attention mechanism based on position encoding, characterized in that: (1) Generate position encoding The process of generating position encoding includes setting a position matrix and calculating position encoding: (1.1) Set the position matrix Given a session sequence s = [v s,1 , v s,2 ,...., v s,ι , with a length of ι, set a position bias for each item v s,i in the session sequence Combine these ι position biases to generate a learnable position matrix (1.2) Calculate position encoding For each item v in the conversation sequence s,i , calculate its position encoding according to its relative distance from the last item v s,ι . Thus, all corresponding learnable position encodings in the corresponding conversation sequence are obtained (2) Embed the soft attention mechanism The embedding of the soft attention mechanism in position encoding is divided into three steps: generating average features, embedding, and linear combination: (2.1) Calculate the average features of each session Given a session sequence s = [v s,1 , v s,2 ,..., v s,ι , with length ι, in session recommendation of hierarchical graph neural network, by obtaining the relationships between the graph structures of different sessions, the global features of each item are obtained Take out and sum up the k-th factor of the global features of all items , and then divide the sum result by the session length ι, thus obtaining the average feature factor of all items Finally, concatenate them to obtain the average feature factor of all items in the session sequence, and get the average feature The corresponding formula is as follows: (2.2) Embed For each session sequence, the global features of each item are known Average factor feature and position encoding The k-th factor of the i-th item feature The k-th factor of the average feature and the position encoding of the i-th item calculated by embedding are multiplied by learnable matrices and respectively, and their multiplication results are added to the learnable vector b k ; then, the sum result is sigmoid-activated to map into a vector between 0 and 1, and this vector is multiplied by the learnable vector to obtain a constant between 0 and 1 which is the global feature coefficient of each item; the corresponding formula is as follows: (2.3) Linear combination The global features of each item are known and their coefficients are as follows: First, multiply the global factor features of each item by the coefficients through scalar multiplication, and then sum up their scalar multiplication results, thus obtaining the between-group session feature factor s g,k . Finally, combine these between-group session feature factors to obtain the between-group session feature s g = [s g,1 , s g,2 , …, s g,K ; The corresponding formula is as follows:

Images