A movie recommendation method and system based on cross-attention mechanism

Abstract

The application provides a movie recommendation method and system based on a cross attention mechanism, wherein the method comprises the following steps: constructing a user-movie feature matrix according to user information and movie information; extracting attention features in the user-movie feature matrix; extracting user-movie interaction features in the user-movie feature matrix; fusing the attention features and the user-movie interaction features; and performing movie rating prediction by using the fused features, the attention features and the user-movie interaction features, so as to perform movie recommendation according to the predicted rating. The application improves the accuracy of movie rating prediction, improves the accuracy of movie viewing user recommendation results, can more fully mine user information, realizes accurate movie recommendation, and improves user use satisfaction.

Description

Technical Field

[0001] This invention belongs to the field of data recommendation technology, and in particular relates to a movie recommendation method and system based on a cross-attention mechanism. Background Technology

[0002] With the advent of the big data era and the rapid development of related technologies, extracting effective information from massive amounts of data to fulfill a practical role in movie recommendation systems, thereby enhancing overall user stickiness and increasing platform value, is a trend in platform development. The recommendation system, the core of a recommendation platform, is a personalized information system that recommends products and information to users based on their interests. It also faces the crucial task of mining user information needs from massive amounts of data.

[0003] The accuracy and efficiency of a recommender system hinge on the recommender algorithm, with collaborative filtering (CF) being the most widely used. CF algorithms fall into two categories: memory-based and model-based. Memory-based CF uses the similarity between users or items to make recommendations, and its effectiveness and ease of implementation have led to its widespread adoption. Despite the extensive research and application of collaborative recommendation algorithms, as the scale of the recommender system increases, similarity calculation becomes increasingly difficult. Collaborative recommendation algorithms suffer from two significant problems: 1. The cold-start problem: For new users or items in the system, without interaction with the items or users, collaborative filtering algorithms struggle to calculate the similarity between users or items, making it difficult to recommend items to new users or recommend new items to them. 2. Sparsity problem (Cold-start problem): In the recommendation process of big data environment, the recommendation system needs to analyze a large amount of data. However, since the rating of an individual user for an item is limited, this leads to the sparsity of the user-item rating matrix. The sparse user-item rating matrix makes it difficult to calculate the similarity of users or items, which seriously affects the quality of the recommendation system.

[0004] To alleviate the data sparsity and cold-start problems in collaborative recommendation algorithms, various types of auxiliary data are incorporated. Extensively studied and primarily used in current recommendation algorithms include social trust relationships, item content, item reviews, social networks, and user social tags. These algorithms can improve the accuracy of recommendation results and mitigate the data sparsity or cold-start problem. However, in some practical situations, certain auxiliary data is unavailable, significantly reducing the accuracy of recommendation results. Compared to these auxiliary data, basic user data (e.g., user gender, user age, and user ID) and basic item data (e.g., item name, item category) are more readily available on websites. Utilizing basic user and item data as user and item features in the recommendation system can alleviate the data sparsity and cold-start problems and improve the performance of recommendation algorithms. While some recommendation algorithms already use basic user and item data, these algorithms often directly transform the basic user and item data using certain rules or methods (e.g., using fuzzy numbers or interval numbers). These direct transformation methods may lead to information loss, failing to fully extract potential or deeper user and item features from the basic user and item data and provide recommendations accordingly. Furthermore, research on building recommendation systems using both user and item basic data simultaneously is still relatively limited among commonly used recommendation algorithms. Therefore, a key issue is how to extract and construct useful features for recommendation models from different dimensions of user and item basic data to achieve effective recommendations. Summary of the Invention

[0005] This invention addresses the shortcomings of existing technologies by providing a movie recommendation method and system based on a cross-attention mechanism.

[0006] In a first aspect, the present invention provides a movie recommendation method based on a cross-attention mechanism, comprising:

[0007] Construct a user-movie feature matrix based on user information and movie information;

[0008] Extract attention features from the user-movie feature matrix;

[0009] Extract user-movie interaction features from the user-movie feature matrix;

[0010] Fusion of attention features and user-movie interaction features;

[0011] Movie ratings are predicted using fused features, attention features, and user-movie interaction features, and then movie recommendations are made based on the predicted ratings.

[0012] Furthermore, the extraction of attention features from the user-movie feature matrix includes:

[0013] Construct the expression for the first feature vector calculated by the attention mechanism:

[0014]

[0015] in, The first eigenvector of the λ-th channel of the user-movie feature matrix after being processed by the attention mechanism; softmax(·) is the activation function; Q λ K λ and V λ All are the first intermediate matrices; I λ This is the input to the λ-th channel of the user-movie feature matrix; and All are trainable first parameter matrices of the λ-th channel of the user-movie feature matrix; T denotes the vector transpose operator; d K The first scaling factor;

[0016] Construct the attention feature O0″ expression based on the first feature vector expression:

[0017] O0″=LayerNorm(Add(O0′,Feed-Forward(O0′));

[0018] Where, LayNorm(·) is the normalization operator; Add(·) is the merging operator; Feed-Forward(·) is the feedforward network function; O0′ is the merged feature of normalized channel features and input features in the feature extraction stage; O0′=LayerNorm(Add(O0,I); I is the user-movie feature matrix; O0 is the merged result of feature vectors calculated by the attention mechanism in the feature extraction stage; C represents the total number of channels in the user-movie feature matrix.

[0019] Further, the extraction of user-movie interaction features from the user-movie feature matrix includes:

[0020] Construct user-movie interaction feature expressions:

[0021] H(UF λ ,MF λ )=UF λ odot MF λ ;

[0022] Among them, H(UF) λ ,MF λ ) represents user-movie interaction features; UF λ MF represents the λ-th user feature in the user-movie feature matrix; λλ represents the λ-th movie feature in the user-movie feature matrix; odot represents the element-wise multiplication of vectors.

[0023] Furthermore, the fusion of attention features and user-movie interaction features includes:

[0024] Construct the expression for the second feature vector calculated by the attention mechanism:

[0025]

[0026] Among them, O1 λ Q' is the second eigenvector of the λ-th channel of the user-movie feature matrix after being processed by the attention mechanism; softmax(·) is the activation function; Q' λ K' λ and V' λ All are the second intermediate matrices; I λ For the λ-th channel of the user-movie feature matrix; I = X or I = Y; X is the feature vector O″0; Y is the feature vector of the user-movie interaction features; and All are trainable second parameter matrices of the λ-th channel of the user-movie feature matrix; T denotes the vector transpose operator; d K' This is the second scaling factor;

[0027] Construct the attention feature O″1 expression based on the second feature vector expression:

[0028] O″1=LayerNorm(Add(O′1,Feed-Forward(O′1));

[0029] Where, LayNorm(·) is the normalization operator; Add(·) is the merging operator; Feed-Forward(·) is the feedforward network function; O′1 is the feature fusion stage, where normalized channel features and input features are merged; O′1=LayerNorm(Add(O1,I); I is the user-movie feature matrix; O1 is the merged result of the feature vectors calculated by the attention mechanism in the feature fusion stage; C represents the total number of channels in the user-movie feature matrix.

[0030] Furthermore, the user information includes user ID, user gender, user age, and user occupation; the movie information includes movie ID, movie year, movie genre, and user rating of the movie.

[0031] Secondly, the present invention provides a movie recommendation system based on a cross-attention mechanism, comprising:

[0032] The module is used to construct a user-movie feature matrix based on user information and movie information;

[0033] The first extraction module is used to extract attention features from the user-movie feature matrix;

[0034] The second extraction module is used to extract user-movie interaction features from the user-movie feature matrix;

[0035] The feature fusion module is used to fuse attention features and user-movie interaction features;

[0036] The rating prediction module is used to predict movie ratings using fused features, attention features, and user-movie interaction features, so as to recommend movies based on the predicted ratings.

[0037] Furthermore, the first extraction module includes:

[0038] The first building unit is used to construct the first feature vector expression calculated by the attention mechanism:

[0039]

[0040] in, The first eigenvector of the λ-th channel of the user-movie feature matrix after being processed by the attention mechanism; softmax(·) is the activation function; Q λ K λ and V λ All are the first intermediate matrices; I λ This is the input to the λ-th channel of the user-movie feature matrix; and All are trainable first parameter matrices of the λ-th channel of the user-movie feature matrix; T denotes the vector transpose operator; d K The first scaling factor;

[0041] The second building unit is used to construct the attention feature O0″ expression based on the first feature vector expression:

[0042] O0″=LayerNorm(Add(O0′,Feed-Forward(O0′));

[0043] Where, LayNorm(·) is the normalization operator; Add(·) is the merging operator; Feed-Forward(·) is the feedforward network function; O0′ is the merged feature of normalized channel features and input features in the feature extraction stage; O0′=LayerNorm(Add(O0,I); I is the user-movie feature matrix; O0 is the merged result of feature vectors calculated by the attention mechanism in the feature extraction stage; C represents the total number of channels in the user-movie feature matrix.

[0044] Furthermore, the second extraction module includes:

[0045] The third building block is used to construct the user-movie interaction feature expression:

[0046] H(UF λ ,MF λ )=UF λ odot MF λ ;

[0047] Among them, H(UF) λ ,MF λ ) represents user-movie interaction features; UF λ MF represents the λ-th user feature in the user-movie feature matrix; λ λ represents the λ-th movie feature in the user-movie feature matrix; odot represents the element-wise multiplication of vectors.

[0048] Furthermore, the feature fusion module includes:

[0049] The fourth building unit is used to construct the second feature vector expression calculated by the attention mechanism:

[0050]

[0051] in, Q' is the second eigenvector of the λ-th channel of the user-movie feature matrix after being processed by the attention mechanism; softmax(·) is the activation function; Q' λ K' λ and V' λ All are the second intermediate matrices; I λ For the λ-th channel of the user-movie feature matrix; I = X or I = Y; X is the feature vector O″0; Y is the feature vector of the user-movie interaction features; and All are trainable second parameter matrices of the λ-th channel of the user-movie feature matrix; T denotes the vector transpose operator; d K' This is the second scaling factor;

[0052] The fifth building unit is used to construct the attention feature O″1 expression based on the second feature vector expression:

[0053] O″1=LayerNorm(Add(O′1,Feed-Forward(O′1));

[0054] Where, LayNorm(·) is the normalization operator; Add(·) is the merging operator; Feed-Forward(·) is the feedforward network function; O′1 is the feature fusion stage, where normalized channel features and input features are merged; O′1=LayerNorm(Add(O1,I); I is the user-movie feature matrix; O1 is the merged result of the feature vectors calculated by the attention mechanism in the feature fusion stage; C represents the total number of channels in the user-movie feature matrix.

[0055] This invention provides a movie recommendation method and system based on a cross-attention mechanism. The method includes constructing a user-movie feature matrix based on user information and movie information; extracting attention features from the user-movie feature matrix; extracting user-movie interaction features from the user-movie feature matrix; fusing the attention features and user-movie interaction features; and using the fused features, attention features, and user-movie interaction features to predict movie ratings, and then recommending movies based on the predicted ratings. This invention improves the accuracy of movie rating prediction and the accuracy of movie recommendation results for movie viewers, enabling more comprehensive mining of user information, achieving accurate movie recommendations, and enhancing user satisfaction. Attached Figure Description

[0056] To more clearly illustrate the technical solution of the present invention, the drawings used in the embodiments will be briefly introduced below. Obviously, the drawings described below are only embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0057] Figure 1 A flowchart illustrating a movie recommendation method based on a cross-attention mechanism provided in this embodiment of the invention;

[0058] Figure 2 This is a schematic diagram of the structure of a movie recommendation method based on cross-attention mechanism provided in an embodiment of the present invention;

[0059] Figure 3 This is a schematic diagram of the structure of a movie recommendation system based on a cross-attention mechanism, provided as an embodiment of the present invention. Detailed Implementation

[0060] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0061] In one embodiment, such as Figure 1 As shown, this embodiment of the invention provides a movie recommendation method based on a cross-attention mechanism, including:

[0062] Step 101: Construct a user-movie feature matrix based on user information and movie information.

[0063] In this step, based on the user's movie viewing data (Movielens-1M dataset), the user movie data tables are merged to obtain an information table containing user information, movie information, and ratings. Each row of data includes user ID, user gender, user age, user occupation, movie ID, movie year, movie genre, and the user's rating for that movie.

[0064] The user gender field needs to be converted from "F" to "M" and "0" to "1" respectively; the user occupation needs to be converted from different occupations to 1, 2, 3, ...; the movie genre needs to be converted from different genres to 1, 2, 3, ...; the movie year feature is extracted from the movie name field as an input movie feature; the input integer sequence is converted into a dense vector representation, and all features are mapped to a dense vector representation.

[0065] Step 102: Extract attention features from the user-movie feature matrix.

[0066] In this step, attention features are extracted from the input feature matrix using TransformerEncoder. TransformerEncoder is used to extract the most valuable information from the user-movie feature matrix.

[0067] By employing a multi-head attention mechanism, residual connections, normalization, and a feedforward network function, linear representation of user movie features is achieved, and the weight matrix is ​​optimized by adding several constraint terms. For example,

[0068] Construct the expression for the first feature vector calculated by the attention mechanism:

[0069]

[0070] in, The first eigenvector of the λ-th channel of the user-movie feature matrix after being processed by the attention mechanism; softmax(·) is the activation function; Q λ K λ and V λ All are the first intermediate matrices; I λ This is the input to the λ-th channel of the user-movie feature matrix; and All are trainable first parameter matrices of the λ-th channel of the user-movie feature matrix; T denotes the vector transpose operator; d K This is the first scaling factor. The first scaling factor prevents the gradient of the softmax function from vanishing or saturating.

[0071] Construct the attention feature O″0 expression based on the first feature vector expression:

[0072] O″0=LayerNorm(Add(O′0,Feed-Forward(O′0))).

[0073] Where, LayerNorm(·) is the normalization operator; Add(·) is the merging operator; Feed-Forward(·) is the feedforward network function; the feedforward network function is a linear mapping followed by an activation function; O′0 is the merged feature of the normalized channel features and input features in the feature extraction stage; O′0=LayerNorm(Add(O0,I)); I is the user-movie feature matrix; O0 is the merged result of the feature vectors calculated by the attention mechanism in the feature extraction stage; C represents the total number of channels in the user-movie feature matrix. O″0 is a vector, which is calculated by self-attention over multiple subspaces.

[0074] Step 103: Extract user-movie interaction features from the user-movie feature matrix.

[0075] In this step, user-movie interaction features are extracted using the GMF (generalized matrix factorization model). User-related features such as user ID, gender, age, and occupation are merged in the second dimension to form a complete user feature, denoted as UF. Similarly, movie features such as movie ID, year, and genre are also merged in the second dimension to form a complete movie feature, MF. Element-wise multiplication of vectors is used to simulate user-item interaction. For example,

[0076] Construct user-movie interaction feature expressions:

[0077] H(UF λ,MF λ )=UF λ odot MF λ .

[0078] Among them, H(UF) λ ,MF λ ) represents user-movie interaction features; UF λ MF represents the λ-th user feature in the user-movie feature matrix; λ Let λ represent the λ-th movie feature in the user-movie feature matrix; odot represents the element-wise multiplication of vectors. In this step, the user-movie interaction feature is denoted as O″2, which is used by the GMF recommendation model to extract the feature vector of the user-movie interaction feature in the user-movie feature matrix.

[0079] Step 104: Fuse attention features and user-movie interaction features.

[0080] In this step, based on a cross-attention mechanism network, two different recommendation models are fused to extract features. Following the steps above, the TransformerEncoder extracts attention features from the input feature matrix, and the GMF model extracts user-movie interaction features, thus obtaining high-order user-movie features. Then, the cross-attention mechanism network is used to extract features from the two different recommendation models.

[0081] The vectors from the two recommendation models described above are merged using a cross-attention mechanism, with O″0 as the principal feature vector, and a multi-head attention mechanism is computed. For example,

[0082] Construct the expression for the second feature vector calculated by the attention mechanism:

[0083]

[0084] in, Q' is the second eigenvector of the λ-th channel of the user-movie feature matrix after being processed by the attention mechanism; softmax(·) is the activation function; Q' λ K' λ and V' λ All are the second intermediate matrices; I λ For the λ-th channel of the user-movie feature matrix; I = X or I = Y; X is the feature vector O″0; Y is the feature vector of the user-movie interaction features; and All are trainable second parameter matrices of the λ-th channel of the user-movie feature matrix; T denotes the vector transpose operator; d K'The second scaling factor prevents gradient vanishing or saturation in the softmax function. The Feed-Forward network is a linear mapping followed by an activation function. Unlike step 102, Q, K, and V in this step represent feature vectors from different recommendation systems.

[0085] Construct the attention feature O″1 expression based on the second feature vector expression:

[0086] O″1=LayerNorm(Add(O′1, Feed-Forward(O′1))).

[0087] Where, LayNorm(·) is the normalization operator; Add(·) is the merging operator; Feed-Forward(·) is the feedforward network function; O′1 is the feature fusion stage, where normalized channel features and input features are merged; O′1=LayerNorm(Add(O1,I); I is the user-movie feature matrix; O1 is the merged result of the feature vectors calculated by the attention mechanism in the feature fusion stage; C represents the total number of channels in the user-movie feature matrix. This step yields the result O″1, a vector obtained by fusing feature vectors from different recommendation systems via a cross-attention mechanism. This vector is calculated using self-attention across multiple subspaces. The cross-attention mechanism uses the attention feature O″0 as its output feature vector, serving as one of the inputs to the cross-attention mechanism in this step.

[0088] Step 105: Use the fused features, attention features, and user-movie interaction features to predict movie ratings, and then recommend movies based on the predicted ratings.

[0089] In this step, the fused feature O″1 (the fused feature obtained after the cross-attention mechanism of O″0 and O″2), the attention feature O″0, and the user-movie interaction feature O″2 are combined. This way, both the attention feature and the original feature are preserved. The features are then fed into a linear layer for movie rating prediction. The GELU activation function is used, and the dropout function and L2 regularization are used to avoid overfitting, thereby enabling movie recommendations based on the predicted ratings.

[0090] To demonstrate that the solution of the present invention is superior to the prior art, specific examples are further disclosed in the embodiments of the present invention.

[0091] This invention studies movie viewing data obtained from the Movielens-1M database. The dataset contains 8834 movies, and 100,004 movie ratings were collected from 610 real users.

[0092] In this embodiment of the invention, RMSE and MAE are used to verify the accuracy of the algorithm. Specifically, for each rating prediction task, a 20% test set, a 70% training set, and a 10% validation set are divided.

[0093] The movie recommendation method based on cross-attention mechanism (SCNCF) provided in this embodiment of the invention is compared with the following six methods: 1) NCF: In this method, GMF and MLP are combined to directly merge two different recommendation systems. 2) Autorec: Autorec is a recommendation system model, which is a single hidden layer neural network recommendation model that combines the ideas of autoencoders and collaborative filtering. Its principle is to use the rating matrix of users or items to complete the autoencoding of user or item vectors, and then use the reconstructed vectors for recommendation. 3) GMF+MLP: In this method, GMF is used to interact with user and item features, and finally MLP is used to output the predicted rating. 4) User-CF: In this method, a user-movie rating matrix is ​​constructed, where each element represents a user's rating of a movie. The similarity between users is calculated, and the most similar users are found. Based on the rating data of neighboring users, the rating of the target user for unrated movies is predicted. 5) Item-CF: In this method, a user-movie rating matrix is ​​constructed, where each element represents a user's rating of a movie. The similarity between movies is calculated, and the most similar movies are found. Based on neighboring movie rating data, predict the target user's rating for unrated movies. Table 1 reports the results of different methods in the rating prediction task. 6) MLP is an abbreviation for Multilayer Perceptron, an artificial neural network model that can be used for movie recommendation. The basic idea of ​​MLP is to take the feature vectors of users and movies as input, pass them through several hidden layers, and output a predicted rating value. MLP can learn the non-linear relationship between users and movies, improving the accuracy of recommendations. The results are shown in Table 1.

[0094] Table 1: RMSE of different methods in the rating prediction task

[0095] Model RMSE SCNCF 0.85140 NCF 0.8565 Autorec 0.9129 User-CF 1.3548 Item-CF 1.2257 MLP 0.8661 GMF+MLP 0.8598

[0096] As shown in Table 1, the method of this invention outperforms the comparative methods in the prediction task. First, NCF, Autorec, and GMF+MLP treat feature information indiscriminately, thus failing to capture crucial information. In contrast, the method of this invention improves the interpretability of the model by focusing on key factors between users and movies, generating more persuasive recommendation reasons, and reducing overfitting by focusing on the commonalities and differences between users and movies, thus containing richer information than the comparative methods. Second, the method of this invention uses a cross-attention mechanism to fuse different recommendation models, thereby conveying richer recommendation system features and improving performance.

[0097] This invention provides a movie recommendation method based on a cross-attention mechanism. By processing the Movielens-1M dataset, a TransformerEncoder and a GMF recommendation model are constructed for each user's viewing history. Finally, a cross-attention mechanism is used to fuse the two different recommendation models. This attention mechanism focuses on important information in the input sequence, unlike convolutional neural networks which rely on fixed window sizes and strides. Finally, by merging the features of the three models, movie rating prediction is performed, improving prediction accuracy and significantly enhancing the accuracy of movie recommendations for movie-watching users. This method can more fully leverage user information, achieve more accurate movie recommendations, and improve user satisfaction.

[0098] The method of this invention can improve the interpretability of the model by focusing on key factors between users and movies, generating more persuasive recommendation reasons. It also reduces overfitting by focusing on the commonalities and differences between users and movies, thus containing richer information than comparative methods. Secondly, the method of this invention uses a cross-attention mechanism to fuse different recommendation models, thereby conveying richer recommendation system features and improving performance.

[0099] Based on the same inventive concept, this invention also provides a movie recommendation system based on a cross-attention mechanism. Since the principle of this system in solving the problem is similar to that of the movie recommendation method based on a cross-attention mechanism, the implementation of this system can refer to the implementation of the movie recommendation method based on a cross-attention mechanism, and the repeated parts will not be described again.

[0100] In another embodiment, the movie recommendation system based on the cross-attention mechanism provided in this invention, such as... Figure 3 As shown, it includes:

[0101] Module 10 is used to construct a user-movie feature matrix based on user information and movie information.

[0102] The first extraction module 20 is used to extract attention features from the user-movie feature matrix.

[0103] The second extraction module 30 is used to extract user-movie interaction features from the user-movie feature matrix.

[0104] Feature fusion module 40 is used to fuse attention features and user-movie interaction features.

[0105] The rating prediction module 50 is used to predict movie ratings using fused features, attention features, and user-movie interaction features, so as to recommend movies based on the predicted ratings.

[0106] For example, the first extraction module includes:

[0107] The first building unit is used to construct the first feature vector expression calculated by the attention mechanism:

[0108]

[0109] in, The first eigenvector of the λ-th channel of the user-movie feature matrix after being processed by the attention mechanism; softmax(·) is the activation function; Q λ K λ and V λ All are the first intermediate matrices; I λ This is the input to the λ-th channel of the user-movie feature matrix; and All are trainable first parameter matrices of the λ-th channel of the user-movie feature matrix; T denotes the vector transpose operator; d K This is the first scaling factor.

[0110] The second building unit is used to construct the attention feature O0″ expression based on the first feature vector expression:

[0111] O0″=LayerNorm(Add(O0′,Feed-Forward(O0′))).

[0112] Where, LayNorm(·) is the normalization operator; Add(·) is the merging operator; Feed-Forward(·) is the feedforward network function; O′0 is the merged feature of normalized channel features and input features in the feature extraction stage; O′0=LayerNorm(Add(O0,I); I is the user-movie feature matrix; O0 is the merged result of feature vectors calculated by the attention mechanism in the feature extraction stage; C represents the total number of channels in the user-movie feature matrix.

[0113] For example, the second extraction module includes:

[0114] The third building block is used to construct the user-movie interaction feature expression:

[0115] H(UF λ ,MF λ )=UF λ odot MF λ .

[0116] Among them, H(UF) λ ,MF λ ) represents user-movie interaction features; UF λMF represents the λ-th user feature in the user-movie feature matrix; λ λ represents the λ-th movie feature in the user-movie feature matrix; odot represents the element-wise multiplication of vectors.

[0117] For example, the feature fusion module includes:

[0118] The fourth building unit is used to construct the second feature vector expression calculated by the attention mechanism:

[0119]

[0120] in, Q' is the second eigenvector of the λ-th channel of the user-movie feature matrix after being processed by the attention mechanism; softmax(·) is the activation function; Q' λ K' λ and V' λ All are the second intermediate matrices; I λ For the λ-th channel of the user-movie feature matrix; I = X or I = Y; X is the feature vector O″0; Y is the feature vector of the user-movie interaction features; and All are trainable second parameter matrices of the λ-th channel of the user-movie feature matrix; T denotes the vector transpose operator; d K' This is the second scaling factor.

[0121] The fifth building unit is used to construct the attention feature O″1 expression based on the second feature vector expression:

[0122] O″1=LayerNorm(Add(O′1, Feed-Forward(O′1))).

[0123] Where, LayNorm(·) is the normalization operator; Add(·) is the merging operator; Feed-Forward(·) is the feedforward network function; O′1 is the feature fusion stage, where normalized channel features and input features are merged; O′1=LayerNorm(Add(O1,I); I is the user-movie feature matrix; O1 is the merged result of the feature vectors calculated by the attention mechanism in the feature fusion stage; C represents the total number of channels in the user-movie feature matrix.

[0124] In this embodiment of the invention, the user information includes user ID, user gender, user age, and user occupation; the movie information includes movie ID, movie year, movie genre, and user rating of the movie.

[0125] For more detailed information on the working process of each of the above modules, please refer to the relevant content disclosed in the foregoing embodiments, which will not be repeated here.

[0126] In another embodiment, the present invention provides a computer device including a processor and a memory; wherein the processor executes a computer program stored in the memory to implement the steps of the above-described movie recommendation method based on the cross-attention mechanism.

[0127] For more detailed information on the above methods, please refer to the relevant content disclosed in the foregoing embodiments, which will not be repeated here.

[0128] In another embodiment, the present invention provides a computer-readable storage medium for storing a computer program; when the computer program is executed by a processor, it implements the steps of the movie recommendation method based on the cross-attention mechanism described above.

[0129] For more detailed information on the above methods, please refer to the relevant content disclosed in the foregoing embodiments, which will not be repeated here.

[0130] The various embodiments in this specification are described in a progressive manner, with each embodiment focusing on its differences from other embodiments. Similar or identical parts between embodiments can be referred to interchangeably. The systems, devices, and storage media disclosed in the embodiments are described simply because they correspond to the methods disclosed in the embodiments; relevant details can be found in the method section.

[0131] Those skilled in the art will clearly understand that the techniques in the embodiments of the present invention can be implemented using software plus necessary general-purpose hardware platforms. Based on this understanding, the technical solutions in the embodiments of the present invention, or the parts that contribute to the prior art, can be embodied in the form of a software product. This computer software product can be stored in a storage medium, such as ROM / RAM, magnetic disk, optical disk, etc., and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute the methods described in various embodiments or certain parts of the embodiments of the present invention.

[0132] The present invention has been described in detail above with reference to specific embodiments and exemplary examples; however, these descriptions should not be construed as limiting the present invention. Those skilled in the art will understand that various equivalent substitutions, modifications, or improvements can be made to the technical solutions and embodiments of the present invention without departing from the spirit and scope of the invention, and all such modifications and improvements fall within the scope of the present invention. The scope of protection of the present invention is defined by the appended claims.

Claims

1. A movie recommendation method based on cross-attention mechanism, characterized in that, include: Construct a user-movie feature matrix based on user information and movie information; Extracting attention features from the user-movie feature matrix, specifically including: Construct the expression for the first feature vector calculated by the attention mechanism: ； in, For the user-movie feature matrix, the first... λ The first feature vector of each channel after being calculated by the attention mechanism; softmax(·) is the activation function; Q λ , K λ and V λ All are the first intermediate matrices; ; ; ; I λ For the user-movie feature matrix, the first... λ Input of each channel; , and All are the first in the user-movie feature matrix. λ The trainable first parameter matrix of each channel; T Represents the vector transpose operator; d K The first scaling factor; Construct attention features based on the expression of the first feature vector. expression: ； Where LayNorm(·) is the normalization operator; Add(·) is the merging operator; Feed-Forward(·) is the feedforward network function; This is the combined feature of normalized channel features and input features during the feature extraction stage; ; I User-movie feature matrix; O 0 represents the merged result of the feature vectors calculated by the attention mechanism during the feature extraction stage; ; C The total number of channels in the user-movie feature matrix; Extracting user-movie interaction features from the user-movie feature matrix, specifically including: Construct user-movie interaction feature expressions: H ( UF λ , MF λ ) = UF λ odot MF λ ; in, H ( UF λ , MF λ ) represents the user-movie interaction characteristics; UF λ Represents the first element in the user-movie feature matrix. λ Individual user characteristics; MF λ Represents the first element in the user-movie feature matrix. λ Each movie feature; odot represents element-wise multiplication of vectors; The fusion of attention features and user-movie interaction features includes: Construct the expression for the second feature vector calculated by the attention mechanism: ； in, For the user-movie feature matrix, the first... λ The second feature vector of each channel is calculated by the attention mechanism; softmax(·) is the activation function; Q' λ , K' λ and V' λ All are the second intermediate matrices; ; ; ; X For feature vectors ; Y The feature vector of user-movie interaction features; , and All are the first in the user-movie feature matrix. λ A trainable second parameter matrix for each channel; T Represents the vector transpose operator; d K' This is the second scaling factor; Construct attention features based on the second feature vector expression. expression: ； Where LayNorm(·) is the normalization operator; Add(·) is the merging operator; Feed-Forward(·) is the feedforward network function; In the feature fusion stage, normalized channel features are merged with input features; ; I User-movie feature matrix; O 1 represents the merged result of the feature vectors calculated by the attention mechanism during the feature fusion stage; ; C The total number of channels in the user-movie feature matrix; Movie ratings are predicted using fused features, attention features, and user-movie interaction features, and then movie recommendations are made based on the predicted ratings.

2. The movie recommendation method based on cross-attention mechanism according to claim 1, characterized in that, The user information includes user ID, user gender, user age, and user occupation; the movie information includes movie ID, movie year, movie genre, and user rating of the movie.

3. A movie recommendation system based on a cross-attention mechanism, used to execute the movie recommendation method based on a cross-attention mechanism as described in claim 1 or 2, characterized in that, include: The module is used to construct a user-movie feature matrix based on user information and movie information; The first extraction module is used to extract attention features from the user-movie feature matrix; The second extraction module is used to extract user-movie interaction features from the user-movie feature matrix; The feature fusion module is used to fuse attention features and user-movie interaction features; The rating prediction module is used to predict movie ratings using fused features, attention features, and user-movie interaction features, so as to recommend movies based on the predicted ratings.

Images