Recommendation Method, Device, Computer Equipment and Storage Medium Based on Sorting Objectives

Abstract

The present application relates to a recommendation method, device, computer device and storage medium based on a sorting target. The method includes: obtaining training samples; the training samples include historical interaction data of multiple sample users and corresponding items; constructing an optimization target of an initial recommendation model based on the AUC metric in the recommendation scenario; training the initial recommendation model to obtain an item recommendation model based on the optimization target, the training samples and a preset loss function; predicting the interest scores of users and recommended items according to the user representations and item representations output by the item recommendation model, so as to obtain the recommended sorting results of the recommended items according to the interest scores. Through the present application, it is possible to construct an optimization target of the initial recommendation model based on the AUC metric in the recommendation scenario, making the optimization target more unified and adapted to the sorting target in the recommendation scenario, thereby improving the sorting effect of the recommendation model.

Description

Technical Field

[0001] The present application relates to the field of artificial intelligence technology, and in particular, to a recommendation method, apparatus, computer device, and storage medium based on a sorting target. Background Art

[0002] With the development of Internet technology, more and more users conduct various activities through network platforms such as e-commerce, streaming media, and social networks. The recommendation system aims to provide personalized recommendations to users and plays a core role in the network platform.

[0003] Existing recommendation models based on graph neural networks model users and items as nodes in a graph and model interactions as edges. The graph structure can make full use of the collaborative information in the recommendation system. However, the optimization objective of the current graph neural network is essentially to make the features between nodes connected by edges as smooth as possible, that is, to promote adjacent nodes to have similar representations. The ultimate goal of the recommendation scenario is to provide a sorted item sequence. The optimization objective of the existing recommendation model does not match the goal of the recommendation scenario, resulting in poor performance of the recommendation model.

[0004] In view of the problem that the optimization objective of the existing recommendation model does not match the goal of the recommendation scenario in the related art, no effective solution has been proposed yet. Summary of the Invention

[0005] Based on this, in view of the above technical problems, it is necessary to provide a recommendation method, apparatus, computer device, and storage medium based on a sorting target that can adapt to the recommendation scenario.

[0006] In a first aspect, in the present embodiment, a recommendation method based on a sorting target is provided, including:

[0007] Obtain training samples; the training samples include historical interaction data of multiple sample users and corresponding items;

[0008] Based on the AUC metric in the recommendation scenario, construct an optimization objective for the initial recommendation model;

[0009] Based on the optimization objective, the training samples, and a preset loss function, train the initial recommendation model to obtain an item recommendation model;

[0010] Predict the interest scores of users and recommended items according to the user representations and item representations output by the item recommendation model, so as to obtain the recommended sorting results of the recommended items according to the interest scores.

[0011] In some of these embodiments, the constructing an optimization objective for the initial recommendation model based on the AUC metric in the recommendation scenario includes:

[0012] Construct the initial recommendation model based on the graph Transformer architecture;

[0013] In the initial recommendation model, maximize the AUC metric as the optimization objective.

[0014] In some embodiments, the optimization objective is:

[0015] argmax z E(Z; δ)

[0016]

[0017] where u represents a user, represents the set of users; each element (u, i, j) in the triple set means that user u has interacted with item i and not interacted with item j; represents the set of items that have interacted with user u; and respectively represent the predicted interest scores of user u and item i, item j; δ(·) represents the activation function; Z is a representation, including user representation and item representation; represents the expectation; is the regularization term, and λ is the regularization coefficient.

[0018] In some embodiments, training the initial recommendation model to obtain an item recommendation model based on the optimization objective, the training samples, and a preset loss function includes:

[0019] In the Rankformer layer iteratively in the initial recommendation model, encode the initial user representation and the initial item representation with the optimization objective to obtain the predicted representation output by the initial recommendation model;

[0020] Determine the predicted interest score of the initial recommendation model according to the predicted representation;

[0021] Based on the preset loss function, calculate the recommendation loss between the predicted interest score and the true interest score in the training samples, and train the item recommendation model according to the recommendation loss.

[0022] In some embodiments, the calculation of each Rankformer layer is as follows:

[0023]

[0024] where Z (l) and Z (l-1)respectively represent the predicted representations of each layer output by the current layer l and the previous layer (l-1); τ represents the temperature coefficient of the Rankformer layer; represents the target weight of the current layer l, which is obtained by normalizing the weights and through normalization calculation:

[0025]

[0026] where u represents a user, represents the set of users; for each user the set of items that have been observed to interact with user u is d u represents the set of items the size of; m represents the size of the set of items for each item the set of users that have been observed to interact with item i is and represent the attention weights; and respectively represent the weights of user u for item i and item i for user u; j represents an item that has not interacted with user u.

[0027] In some of these embodiments, it further includes:

[0028] In the calculation of each of the Rankformer layers, linearized expansion calculation is performed on the user representation and the item representation.

[0029] In some of these embodiments, predicting the interest score of a user for a recommended item based on the user representation and the item representation output by the item recommendation model, and obtaining the recommended ranking result of the recommended item according to the interest score, includes:

[0030] Based on the item recommendation model, obtaining the predicted user representation and item representation;

[0031] Determining the interest score according to the user representation and the item representation;

[0032] Sorting the recommended items according to the interest score to obtain the recommended ranking result.

[0033] In a second aspect, a recommendation device based on a ranking target is provided in this embodiment, including:

[0034] A sample acquisition module for acquiring training samples; the training samples include historical interaction data of multiple sample users and corresponding items;

[0035] A model construction module, configured to construct an optimization objective of an initial recommendation model based on the AUC metric in a recommendation scenario;

[0036] A model training module, configured to train the initial recommendation model based on the optimization objective, the training samples, and a preset loss function to obtain an item recommendation model;

[0037] An item recommendation module, configured to predict an interest score of a user and a recommended item based on the user representation and the item representation output by the item recommendation model, and obtain a recommended ranking result of the recommended item according to the interest score.

[0038] In a third aspect, in this embodiment, a computer device is provided, including a memory, a processor, and a computer program stored on the memory and executable on the processor. When the processor executes the computer program, the recommendation method based on a ranking objective described in the first aspect above is implemented.

[0039] In a fourth aspect, in this embodiment, a storage medium is provided, on which a computer program is stored. When the program is executed by a processor, the recommendation method based on a ranking objective described in the first aspect above is implemented.

[0040] Compared with the related art, in the recommendation method, device, computer device, and storage medium based on a ranking objective provided in this embodiment, training samples are obtained; the training samples include historical interaction data of multiple sample users and corresponding items;

[0041] An optimization objective of an initial recommendation model is constructed based on the AUC metric in a recommendation scenario; the initial recommendation model is trained based on the optimization objective, the training samples, and a preset loss function to obtain an item recommendation model; an interest score of a user and a recommended item is predicted based on the user representation and the item representation output by the item recommendation model, and a recommended ranking result of the recommended item is obtained according to the interest score. In this embodiment, an optimization objective of an initial recommendation model can be constructed based on the AUC metric in a recommendation scenario, so that the optimization objective and the ranking objective in the recommendation scenario are more unified and adapted, thereby improving the ranking effect of the recommendation model.

[0042] Details of one or more embodiments of the present application are set forth in the following drawings and description to make other features, objects, and advantages of the present application more comprehensible. BRIEF DESCRIPTION OF THE DRAWINGS

[0043] The drawings described herein are used to provide a further understanding of the present application and constitute a part of the present application. The illustrative embodiments and descriptions thereof of the present application are used to explain the present application and do not constitute an improper limitation of the present application. In the drawings:

[0044] Figure 1 It is a hardware structure block diagram of a terminal for a recommendation method based on a ranking objective in an embodiment;

[0045] Figure 2 It is a flowchart of a recommendation method based on a ranking objective in an embodiment;

[0046] Figure 3 It is a schematic diagram of the encoding process of the Rankformer layer in an embodiment;

[0047] Figure 4 It is a flowchart of a recommendation method based on a ranking objective in an embodiment;

[0048] Figure 5 It is a structure block diagram of a recommendation device based on a ranking objective in an embodiment.

[0049] In the figure: 102, a processor; 104, a memory; 106, a transmission device; 108, an input / output device; 10, a sample acquisition module; 20, a model construction module; 30, a model training module; 40, an item recommendation module. Detailed implementation manners

[0050] To more clearly understand the purpose, technical solution, and advantages of the present application, the present application is described and illustrated below with reference to the accompanying drawings and embodiments.

[0051] Unless otherwise defined, technical or scientific terms used in this application shall have the ordinary meanings understood by those of ordinary skill in the technical field to which this application pertains. In this application, words such as "a", "an", "one kind", "the", "these", etc. do not indicate a limitation in quantity and can be singular or plural. The terms "include", "comprise", "have" and any variants thereof used in this application are intended to cover non-exclusive inclusion; for example, a process, method, system, product or device that includes a series of steps or modules (units) is not limited to the listed steps or modules (units), but may include unlisted steps or modules (units), or may include other steps or modules (units) inherent in these processes, methods, products or devices. The terms "connected", "coupled" and the like used in this application do not limit to physical or mechanical connections, but may include electrical connections, whether direct or indirect. The term "plurality" used in this application means two or more. "And / or" describes the association relationship of associated objects and indicates that there can be three relationships. For example, "A and / or B" can represent: A exists alone, A and B exist simultaneously, and B exists alone. Usually, the character " / " indicates that the objects associated before and after are in an "or" relationship. The terms "first", "second", "third", etc. used in this application only distinguish similar objects and do not represent a specific order for the objects.

[0052] The method embodiments provided in this embodiment can be executed on a terminal, a computer or a similar computing device. For example, running on a terminal, Figure 1 is a hardware structure block diagram of the terminal of the recommendation method based on sorting objectives in this embodiment. As Figure 1 shown, the terminal may include one or more ( Figure 1 only one is shown in the figure) processors 102 and a memory 104 for storing data. Among them, the processor 102 may include, but is not limited to, a processing device such as a microprocessor MCU or a programmable logic device FPGA. The above terminal may further include a transmission device 106 for communication functions and an input / output device 108. Those of ordinary skill in the art can understand that Figure 1 the structure shown in the figure is only schematic and does not limit the structure of the above terminal. For example, the terminal may further include more or fewer components than Figure 1 shown in the figure, or have a different configuration from Figure 1 shown in the figure.

[0053] The memory 104 can be used to store computer programs, for example, software programs and modules of application software, such as the computer program corresponding to the recommendation method based on sorting objectives in this embodiment. The processor 102 executes various functional applications and data processing by running the computer program stored in the memory 104, that is, implements the above method. The memory 104 may include a high-speed random access memory, and may also include a non-volatile memory, such as one or more magnetic storage devices, flash memories, or other non-volatile solid-state memories. In some instances, the memory 104 may further include a memory remotely disposed relative to the processor 102, and these remote memories can be connected to the terminal through a network. Examples of the above network include but are not limited to the Internet, intranet, local area network, mobile communication network, and combinations thereof.

[0054] The transmission device 106 is used to receive or send data via a network. The above network includes a wireless network provided by a communication provider of the terminal. In one instance, the transmission device 106 includes a network adapter (Network Interface Controller, abbreviated as NIC), which can be connected to other network devices through a base station and thus can communicate with the Internet. In one instance, the transmission device 106 can be a radio frequency (RF) module, which is used to communicate with the Internet wirelessly.

[0055] With the development of Internet technology, more and more users conduct various activities through network platforms such as e-commerce, streaming media, and social networks. The recommendation system aims to provide personalized recommendations to users and plays a core role in the network platform.

[0056] Existing recommendation models based on graph neural networks model users and items as nodes in a graph and model interactions as edges. The graph structure can make full use of the collaborative information in the recommendation system. However, graph neural networks have some defects:

[0057] (1) The multi-layer message passing mechanism of graph neural networks is prone to the problem of over-smoothing. The representations of nodes are likely to become similar or indistinguishable.

[0058] (2) The optimization objective of graph neural networks is essentially to make the features between nodes connected by edges as smooth as possible, that is, to promote adjacent nodes to have similar representations. This optimization objective is highly aligned with tasks such as node classification, but is not unified with the objectives of the recommendation scenario. However, the current optimization objective of graph neural networks is essentially to make the features between nodes connected by edges as smooth as possible, that is, to promote adjacent nodes to have similar representations. This optimization objective is highly aligned with tasks such as node classification, but is not unified with the objectives of the recommendation scenario.

[0059] The ultimate goal of the recommended scenario is to provide users with a sorted sequence of items. The optimization objectives of existing recommendation models do not match the goals of the recommended scenario. The graph neural network only optimizes the similarity of neighbors, which may not necessarily improve the sorting effect. Instead, it will reduce the gap between the interest scores of different items, leading to the so-called over-smoothing problem and resulting in poor performance of the recommendation model.

[0060] In this embodiment, a method is provided. Figure 2 It is a flowchart of the recommendation method based on the sorting objective in this embodiment, as Figure 2 shown. The method includes the following steps:

[0061] Step S210, obtain training samples; the training samples include historical interaction data of multiple sample users and corresponding items.

[0062] Specifically, in this embodiment, the historical interaction data of sample users and recommended items in the item recommendation scenario is obtained as training samples. The sample users can be users of Internet application platforms. For example, if the application platform is an e-commerce shopping platform, the sample users can be users who use the e-commerce shopping platform for shopping, and the recommended items are commodities; if the application platform is a social platform, the sample users can be users of the social platform, and the recommended items are the content published on the social platform.

[0063] The historical interaction data can be the usage data of sample users on the application platform. In terms of form, it is a data pair of users and recommended items, indicating that there has been an interaction between users and recommended items. For example, the historical interaction data of sample users on the e-commerce shopping platform and recommended items includes, but is not limited to, data such as clicks, views, collections, orders, and consultations; for example, the historical interaction data of sample users on the social platform and recommended items includes, but is not limited to, data such as clicks, views, collections, and comments. In addition, the training samples can also include basic information such as the age and gender of sample users, as well as information such as the attributes and labels of recommended items.

[0064] Use the historical interaction data collected in the real production environment to obtain training samples for collaborative filtering. Specifically, let the user set be with a size of n; the item set is with a size of m. For each user the set of items that have been observed to interact with user u is with a size of d u ; for each item the set of users that have been observed to interact with item i is

[0065] Step S220, based on the AUC metric in the recommended scenario, construct the optimization objective of the initial recommendation model.

[0066] Specifically, the AUC (Area under curve) metric in the recommendation scenario specifically refers to the area under the ROC (Receiver Operating Characteristic Curve). AUC is used to evaluate the model performance by calculating the area under the ROC curve. The abscissa of the ROC curve is the False Positive Rate (FPR), which represents the proportion of samples that are actually negative but are wrongly predicted as positive; the ordinate is the True Positive Rate (TPR), which represents the proportion of samples that are actually positive and are correctly predicted as positive. The value of AUC ranges from 0 to 1, and the larger the AUC value, the better the classification performance of the model.

[0067] Based on the AUC metric, an optimization objective for the initial recommendation model is constructed, that is, starting from the AUC metric for derivation, with maximizing the AUC area as the optimization objective. Among them, an initial recommendation model is pre-constructed. The initial recommendation model can be a recommendation model based on a graph neural network, etc. Further, due to the multi-layer message passing mechanism of the graph neural network, it is easy to cause the problem of over-smoothing, making the representations of nodes easy to become similar or indistinguishable. In this embodiment, a graph Transformer architecture can also be adopted, which can alleviate the overfitting problem of the traditional graph neural network to a certain extent.

[0068] Step S230, based on the optimization objective, training samples, and a preset loss function, train the initial recommendation model to obtain an item recommendation model.

[0069] Specifically, the initial recommendation model adopting the graph Transformer architecture has multiple Rankformer layers. Rankformer is the abbreviation of the recommendation method based on the ranking objective proposed in this embodiment, and the network structure is called the Rankformer layer. According to the optimization objective, the initial representations are encoded in each Rankformer layer during iteration, and finally the prediction representations of the initial recommendation model are obtained. Among them, the initial representations include user initial representations and item initial representations; the prediction representations include user prediction representations and item prediction representations.

[0070] Based on the user prediction representation and the item prediction representation, the predicted interest score of each item for the user can be obtained. Based on a preset loss function and training samples, the recommendation loss of the predicted interest score obtained each time is calculated, and the initial recommendation model is trained until the preset loss function converges or reaches the maximum number of training rounds, obtaining the item recommendation model. Among them, the preset loss function includes but is not limited to the BPR (Bayesian Personalized Ranking Loss Operator) loss function, the Softmax loss function (SL-Loss), etc.

[0071] Optionally, during the training process of the item recommendation model, the Adam optimizer is used to update the model parameters to adaptively learn the learning rate of each model parameter and accelerate the optimization process, including setting hyperparameters of the temperature coefficient of the Adam optimizer, such as the learning rate and the L2 regularization coefficient (weight decay).

[0072] Step S240: Predict the interest score of the user and the recommended item based on the user representation and the item representation output by the item recommendation model, so as to obtain the recommendation ranking result of the recommended item according to the interest score.

[0073] Specifically, the trained item recommendation model is used to output the predicted user representation and item representation to achieve personalized item recommendation for users. For each user, the final interest score of the user and the recommended item is predicted, and the recommended items are ranked according to the final interest score to obtain the recommendation ranking result.

[0074] Through the above steps, based on the AUC metric in the recommendation scenario, the optimization objective of the initial recommendation model can be deduced and constructed, and finally the item recommendation model is trained, making the item recommendation model more adaptable to the recommendation scenario and the objectives more unified, thereby improving the ranking effect of the recommendation model.

[0075] In some of the embodiments, constructing the optimization objective of the initial recommendation model based on the AUC metric in the above step S210 includes the following steps:

[0076] Based on the graph Transformer architecture, construct the initial recommendation model; in the initial recommendation model, the optimization objective is to maximize the AUC metric.

[0077] Specifically, use the graph Transformer architecture with a full-graph attention mechanism to construct the initial recommendation model, including designing the Rankformer layer and the preset loss function. Optionally, the preset loss function can be the BPR loss function, and the specific calculation is as follows:

[0078]

[0079] Among them, and respectively represent the interest scores of user u for item i and item j; j is an item randomly sampled that user u has not interacted with; σ(·) is the activation function. Taking the interest score of user u for item i as an example, the specific calculation is as follows:

[0080]

[0081] Among them, represents the user representation output by the last Rankformer layer L; represents the item representation output by the last Rankformer layer L. Similarly, the interest score of user u for item j can be obtained

[0082] The Graph Transformer architecture is a deep learning architecture that combines the advantages of the Graph Neural Network (GNN) and the Transformer model. In the recommendation scenario, it can be used to capture the complex relationships between users and items and generate personalized recommendation lists, and can alleviate the overfitting problem of traditional graph neural networks to a certain extent. The global graph attention mechanism is an extension of the self-attention mechanism in the Transformer model to graph-structured data. In the traditional Transformer, the self-attention mechanism mainly processes sequential data, and captures the dependencies between elements by calculating the attention scores between each element in the sequence and other elements. In the Graph Transformer, the global graph attention mechanism allows the model to compare and interact each node in the graph with all other nodes when processing graph-structured data, thereby capturing the global information of the graph.

[0083] In the initial recommendation model, each encoding through a Rankformer layer is equivalent to performing a gradient ascent on the AUC objective in the recommendation scenario ranking task. Therefore, in the recommendation scenario, the optimization objective is constructed to maximize the AUC metric (area).

[0084] In some of these embodiments, the optimization objective is:

[0085] argmax z E(Z; δ)

[0086]

[0087] Among them, u represents the user, represents the set of users; the set of triples Each element (u, i, j) in it indicates that user u has interacted with item i and has not interacted with item j; represents the set of items that have interacted with user u; and respectively represent the interest scores of predicting user u, item i, and item j; δ(·) represents the activation function; Z is the representation, which includes user representation and item representation; represents the expectation; is the regularization term, and λ is the regularization coefficient.

[0088] Specifically, E(Z; δ) is the content to be maximized in this optimization objective. The optimization objective indicates that among the interacted item i and the non-interacted item j, user u prefers item i. Therefore, in the optimization objective represents maximizing the interest score of item i relative to item j at user u, where δ(·) is a non-decreasing activation function.

[0089] Further transform the above formula to obtain the following formula:

[0090]

[0091] where u represents the user, represents the set of users; for each user the set of items that have been observed to interact with user u is d u represents the size of the set of interacted items m represents the size of the set of items ; z u represents the user representation; z i and z j represent the item representations.

[0092] Further perform a second-order Taylor expansion approximation on the activation function, then optimizing the above formula is equivalent to optimizing the following:

[0093]

[0094] where the attention weight α is an adjustable parameter.

[0095] In the way of gradient ascent, optimize to maximize the AUC metric. Calculate the gradient of the above formula:

[0096]

[0097] where the objective weight Ω is obtained by normalizing the weights and :

[0098]

[0099] Among them, and respectively represent the weights of user u for item i and item i for user u; and represent the attention weights.

[0100] Each Rankformer layer performs one gradient ascent on the above formula with a step size of ∈ to simulate maximizing the AUC, that is:

[0101] Z (l) =(1 - ∈τ)Z (l-1) +∈Ω (l) Z (l-1)

[0102] Among them, Z (l) and Z (l-1) respectively represent the predicted representations of each layer output by the current layer l and the previous layer (l - 1); τ represents the temperature coefficient of the Rankformer layer.

[0103] Finally, to maintain numerical stability, normalization is introduced, and the hyperparameters are combined to simplify the above formula, obtaining the following formula:

[0104]

[0105] Among them ||·||1 represents the L1 norm.

[0106] By using the graph Transformer architecture to construct the initial recommendation model in this embodiment, problems such as over-smoothing existing in traditional graph neural network recommendation methods can be avoided. Moreover, based on the AUC metric in the recommendation scenario, an optimization objective is constructed, and the encoding process of each Rankformer layer in the initial recommendation model is derived, so as to perform encoding on each layer with the optimization objective in subsequent steps, making the optimization objective of the model more adapted to the recommendation scenario.

[0107] In some of these embodiments, training the initial recommendation model to obtain an item recommendation model based on the optimization objective, training samples, and a preset loss function in step S230 includes the following steps:

[0108] Step S231, in the iterative Rankformer layer of the initial recommendation model, encode the initial user representation and the initial item representation with the optimization objective to obtain the predicted representation output by the initial recommendation model.

[0109] Specifically, a d-dimensional vector is randomly generated for each user and item as the initial representation Z (0) :

[0110]

[0111] Among them, represents the initial representation of user u; represents the initial representation of item i.

[0112] According to the derivation in the above embodiments, the calculation of each Rankformer layer is as follows:

[0113]

[0114] Among them, Z (l) and Z (l-1) respectively represent the predicted representation of each layer output by the current layer l and the previous layer (l - 1); τ represents the temperature coefficient of the Rankformer layer; represents the target weight of the current layer l, which is obtained by normalizing the weights and as follows:

[0115]

[0116] Among them, u represents the user, represents the set of users; for each user the set of items that have been observed to interact with user u is d u represents the set of items the size of; m represents the size of the set of items for each item the set of users that have been observed to interact with item i is and represent the attention weights; and respectively represent the weights of user u for item i and item i for user u; j represents an item that has not interacted with user u; the attention weight is calculated as follows:

[0117]

[0118] Among them, α is an adjustable parameter; ||·||2 represents the L2 norm. Similarly, the calculation method of the attention weight can be obtained.

[0119] Finally, the predicted representation Z (L) output by the initial recommendation model is obtained:

[0120]

[0121] Among them, Z (L)is the predicted representation of the output of the last Rankformer layer; represents the user u representation of the model output, indicating the user preferences predicted by the model; represents the item i representation of the model output, indicating the item features predicted by the model.

[0122] Figure 3 is a schematic diagram of the encoding process of the Rankformer layer in this embodiment, as Figure 3 shown. Taking the current layer l as an example, Z (0) is the initial representation input to the first Rankformer layer, and Z (L) is the predicted representation output by the last Rankformer layer. In each Rankformer layer, the predicted representation Z (l-1) output by the previous layer (l-1) is normalized, and further is used to calculate the attention weights The target weights Ω of the Rankformer layer are calculated according to the attention weights (l) , and are normalized to obtain Finally, according to the normalized target weights and the predicted representation Z (l-1) output by the previous layer (l-1), they are combined to obtain the predicted representation Z (l) of the current layer l, and finally the predicted representation Z (L) output by the last Rankformer layer is obtained.

[0123] Step S232: Determine the predicted interest score of the initial recommendation model according to the predicted representation.

[0124] Specifically, according to the above predicted representation, the predicted interest score is determined according to the user representation and item representation therein

[0125]

[0126] Among them, the predicted interest score represents the degree of interest of user u in item i.

[0127] Step S233: Calculate the recommendation loss between the predicted interest score and the true interest score in the training sample based on a preset loss function, so as to train the item recommendation model according to the recommendation loss.

[0128] Specifically, the BPR loss function is selected as the preset loss function, and the recommendation loss between the predicted interest score obtained each time and the true score in the historical interaction data in the training sample is calculated to train the initial recommendation model until the preset loss function converges or reaches the maximum number of training rounds to obtain the item recommendation model.

[0129] Optionally, during the training of the item recommendation model, the Adam optimizer is used to update the model parameters to adaptively learn the learning rate of each model parameter and accelerate the optimization process, including setting hyperparameters of the temperature coefficient of the Adam optimizer, such as the learning rate and the L2 regularization coefficient (weight decay).

[0130] In this embodiment, through the Rankformer layer in the initial recommendation model, encoding calculations are performed with the optimization objective, and the model is trained based on the preset loss function and the predicted interest scores of the initial recommendation model, and an item recommendation model adapted to the recommendation scenario can be obtained.

[0131] In some of these embodiments, the above method further includes: in the calculation of each Rankformer layer, linearized expansion calculations are performed on the user representation and the item representation.

[0132] Specifically, the above user representation and the item representation are linearly expanded, and the specific calculation is as follows:

[0133]

[0134] where C is the normalization coefficient:

[0135]

[0136]

[0137] where Ω + and Ω - can be written as:

[0138]

[0139] where The superscripts "+" and "-" respectively represent the positive and negative directions.

[0140] The above formula after linearized expansion can be calculated with a complexity of O(Nd 2 +Ed). Where N represents the total number of users and items, that is, N = n + m; E is the number of interacted sample pairs, that is

[0141] By linearly expanding the user representation and the item representation in this embodiment, the calculation efficiency can be improved, the complexity can be reduced in each layer of Rankformer encoding calculation, and the disadvantage in efficiency of the general graph Transformer method can be avoided.

[0142] In some of these embodiments, in step S240 above, based on the user representation and item representation output by the item recommendation model, the interest score between the user and the recommended item is predicted, and the recommended ranking result of the recommended item is obtained according to the interest score, including the following steps:

[0143] Step S241, based on the item recommendation model, obtain the predicted user representation and item representation.

[0144] Step S242, determine the interest score according to the user representation and item representation.

[0145] Step S243, rank the recommended items according to the interest score to obtain the recommended ranking result.

[0146] Specifically, when the user logs in or accesses the application platform, in response to the user's access request, the application platform, based on the item recommendation model, predicts the user representation and the item representation of the recommended items on the application platform, and calculates the interest score between the user representation and all item representations. The interest score can reflect the degree of interest of the user in each recommended item.

[0147] Rank the recommended items from high to low according to the interest score, return the recommended ranking result from all the recommended items, and display it to the user on the page of the application platform. Optionally, select a certain number of items from all the recommended items as the recommended ranking result to display to the user for personalized item recommendation. For example, if the application platform is an e-commerce shopping platform and the recommended items are commodities, the recommended ranking result finally displayed to the user is the commodities with a higher degree of interest of the user; for example, if the application platform is a social platform and the recommended items are the content published on the social platform, the recommended ranking result finally displayed to the user is the media content, tweets, etc. with a higher degree of interest of the user.

[0148] By applying the above item recommendation model to the personalized recommendation scenario in this embodiment, the accuracy and effect of personalized recommendation can be improved, more content that the user is interested in can be provided, and the user experience can be improved.

[0149] The following describes and illustrates this embodiment through preferred embodiments.

[0150] Figure 4 is the flowchart of the recommendation method based on the ranking target in this embodiment, as Figure 4 shown, the method includes the following steps:

[0151] Step S410, obtain training samples; the training samples include historical interaction data of multiple sample users and corresponding items.

[0152] Among them, use the historical interaction data collected in the real production environment to obtain the training samples for collaborative filtering. Specifically, let the user set be with size n; the item set is with size m. For each user the item set that has been observed to interact with user u is with size d u ; for each item the user set that has been observed to interact with item i is

[0153] Step S420, randomly generate a d-dimensional vector for each user and item as the initial representation.

[0154] Among them,

[0155]

[0156] ; represents the initial representation of user u; represents the initial representation of item i.

[0157] Step S430, in the Rankformer layer iteratively in the initial recommendation model, encode the user initial representation and the item initial representation with the optimization objective to obtain the predicted representation output by the initial recommendation model.

[0158] Among them, the initial recommendation model is constructed based on the graph Transformer architecture, with maximizing the AUC area as the optimization objective, and the calculation of each Rankformer layer is as follows:

[0159]

[0160] Among them, Z (l) and Z (l-1) respectively represent the predicted representation of each layer output by the current layer l and the previous layer (l-1); τ represents the temperature coefficient of the Rankformer layer; represents the target weight of the current layer l, which is obtained by normalizing the weights and as follows:

[0161]

[0162] Among them, u represents the user, represents the user set; for each user the item set that has been observed to interact with user u is d u represents the item set with size; m represents the item set with size; for each item The set of users who have been observed to interact with item i is and denote attention weights; and denote the weights of user u for item i and item i for user u, respectively; j denotes an item that user u has not interacted with; the attention weight is calculated as follows:

[0163]

[0164] where α is an adjustable parameter; ||·||2 denotes the L2 norm. Similarly, the calculation method of the attention weight can be obtained.

[0165] Finally, the predicted representation Z output by the initial recommendation model is obtained (L) :

[0166]

[0167] where Z (L) is the predicted representation of the output of the last Rankformer layer; denotes the user representation output by the model, representing the user preferences predicted by the model; denotes the item representation output by the model, representing the item features predicted by the model.

[0168] Step S440, according to the Bayesian personalized ranking loss function, training samples, and predicted representation, train the initial recommendation model to obtain an item recommendation model.

[0169] Among them, the Bayesian personalized ranking loss function is specifically calculated as follows:

[0170]

[0171] where and denote the interest scores of user u for item i and item j, respectively; j is an item randomly sampled that user u has not interacted with; σ(·) is the activation function. Taking the interest score of user u for item i as an example, the specific calculation is as follows:

[0172]

[0173] where denotes the user representation output by the last Rankformer layer L; denotes the item representation output by the last Rankformer layer L. Similarly, the interest score of user u for item j can be obtained

[0174] Step S450, in the calculation of each Rankformer layer, linearly expand and calculate the user representation and the item representation.

[0175] Among them, the user representation and the item representation are linearly expanded, and the specific calculation is as follows:

[0176]

[0177] Among them, normalize the weights, and C is the normalization coefficient:

[0178]

[0179] Among them, Ω + and Ω - can be written as:

[0180]

[0181] Among them,

[0182] Step S460, according to the user representation and the item representation output by the item recommendation model, predict the interest score of the user and the recommended item, so as to obtain the recommended ranking result of the recommended item according to the interest score.

[0183] On multiple open-source datasets (including the shopping platform display ad click-through rate prediction dataset Ali_Display_Ad_Click, Amazon-kindle, and Amazon-cds), test the recommendation method (Rankformer in Table 1) in this embodiment relative to the current graph recommendation models (including: LightGCN (Lightweight Graph Convolutional Neural Network), XSimGCL (Extremely Simple Graph Contrastive Learning Method), GFormer, MGFormer) and graph Transformer models (including: Nodeformer (a graph representation model architecture), DIFFormer (a diffusion process-inspired Transformer), SGFormer (a self-guided Transformer)) in terms of the improvement of the recommendation metric performance in ndcg@20 (Normalized Discounted Cumulative Gain) and recall@20 (Recall Rate). The test results are shown in Table 1 below.

[0184] Table 1:

[0185]

[0186] It can be seen from Table 1 that the recommendation metric performance of the recommendation method (Rankformer) in this embodiment has been greatly improved on multiple datasets.

[0187] Based on the AUC metric of the derivation and approximation recommendation scenario in this embodiment, a recommendation method based on a ranking objective (i.e., Rankformer) is proposed, achieving good recommendation performance. On the one hand, the ultimate goal of the recommendation scenario is to recommend a ranked item sequence for each user. Starting from the AUC metric in this embodiment, the derivation can highly align the item recommendation model with the ranking task of the recommendation scenario and solve problems such as over-smoothing existing in traditional graph neural network recommendation methods. On the other hand, through linearization expansion in model calculation and ingenious matrix operation design, the efficiency disadvantage of general graph Transformer methods is avoided, and a recommendation method of graph Transformer with a full-graph attention mechanism is realized.

[0188] It should be noted that the steps shown in the above process or the flowchart of the accompanying drawings can be executed in a computer system such as a set of computer-executable instructions. And although the logical order is shown in the flowchart, in some cases, the steps shown or described can be executed in a different order from here.

[0189] In this embodiment, a recommendation device based on a ranking objective is also provided. This device is used to implement the above embodiment and the preferred implementation manners, and those that have been described will not be repeated. The following terms such as "module", "unit", "sub-unit", etc. can be a combination of software and / or hardware that can achieve a predetermined function. Although the devices described in the following embodiments are preferably implemented in software, implementation in hardware, or a combination of software and hardware is also possible and contemplated.

[0190] Figure 5 is the structural block diagram of the recommendation device based on a ranking objective in this embodiment, as Figure 5 shown. The device includes:

[0191] A sample acquisition module 10, configured to acquire training samples; the training samples include historical interaction data of multiple sample users and corresponding items;

[0192] A model construction module 20, configured to construct an optimization objective of an initial recommendation model based on the AUC metric in the recommendation scenario;

[0193] A model training module 30, configured to train the initial recommendation model based on the optimization objective, the training samples, and a preset loss function to obtain an item recommendation model;

[0194] An item recommendation module 40, configured to predict the interest scores of users and recommended items according to the user representations and item representations output by the item recommendation model, so as to obtain a recommended ranking result of the recommended items according to the interest scores.

[0195] Through the device provided in this embodiment, it is possible to derive and construct the optimization objective of the initial recommendation model based on the AUC metric in the recommendation scenario, and finally train an item recommendation model, making the item recommendation model more adaptable to the recommendation scenario and more unified in objectives, thereby improving the ranking effect of the recommendation model.

[0196] In some of these embodiments, the model construction module 20 is further configured to:

[0197] Construct an initial recommendation model based on the graph Transformer architecture; in the initial recommendation model, the optimization objective is to maximize the AUC metric.

[0198] In some of these embodiments, the optimization objective in the model construction module 20 is:

[0199] argmax z E(Z; δ)

[0200]

[0201] where u represents a user, represents the set of users; each element (u, i, j) in the triple set means that user u has interacted with item i and has not interacted with item j; represents the set of items that user u has interacted with; and respectively represent the predicted interest scores of user u and item i, item j; δ(·) represents the activation function; Z is a representation, which includes user representation and item representation; represents the expectation; is the regularization term, and λ is the regularization coefficient.

[0202] In some of these embodiments, the model training module 30 is further configured to:

[0203] In the Rankformer layer iteratively in the initial recommendation model, encode the initial user representation and the initial item representation with the optimization objective to obtain the predicted representation output by the initial recommendation model;

[0204] Determine the predicted interest score of the initial recommendation model according to the predicted representation;

[0205] Based on a preset loss function, calculate the recommendation loss between the predicted interest score and the true interest score in the training sample, and train an item recommendation model according to the recommendation loss.

[0206] In some of these embodiments, the calculation of each Rankformer layer in the model training module 30 is as follows:

[0207]

[0208] Among them, Z (l) and Z (l-1) respectively represent the predicted representations of each layer output by the current layer l and the previous layer (l - 1); τ represents the temperature coefficient of the Rankformer layer; represents the target weight of the current layer l, which is obtained by normalizing the weights and as follows:

[0209]

[0210] Among them, u represents the user, represents the set of users; for each user the set of items that have been observed to interact with user u is d u represents the set of items the size of; m represents the size of the set of items ; for each item the set of users that have been observed to interact with item i is and represent the attention weights; and respectively represent the weights of user u for item i and item i for user u; j represents an item that has not interacted with user u.

[0211] In some of these embodiments, the model training module 30 is further configured to: in the calculation of each Rankformer layer, perform linear expansion calculation on the user representation and the item representation.

[0212] In some of these embodiments, the item recommendation module 40 is further configured to:

[0213] Based on the item recommendation model, obtain the predicted user representation and item representation;

[0214] Determine the interest score according to the user representation and the item representation;

[0215] Sort the recommended items according to the interest score to obtain the recommended sorting result.

[0216] It should be noted that the above-mentioned various modules can be functional modules or program modules, and can be implemented either by software or by hardware. For the modules implemented by hardware, the above-mentioned various modules can be located in the same processor; or the above-mentioned various modules can also be located in different processors in any combination form.

[0217] In this embodiment, a computer device is further provided, which includes a memory and a processor. A computer program is stored in the memory, and the processor is configured to run the computer program to execute the steps in any of the above method embodiments.

[0218] Optionally, the above computer device may further include a transmission device and an input / output device. Among them, the transmission device is connected to the above processor, and the input / output device is connected to the above processor.

[0219] It should be noted that for the specific examples in this embodiment, reference may be made to the examples described in the above embodiments and optional implementation manners, which will not be elaborated herein.

[0220] In addition, in combination with the recommendation method based on sorting objectives provided in the above embodiments, a storage medium may also be provided to implement it in this embodiment. A computer program is stored on the storage medium; when the computer program is executed by a processor, any one of the recommendation methods based on sorting objectives in the above embodiments is implemented.

[0221] It should be noted that the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data for analysis, stored data, displayed data, etc.) involved in this application are all information and data authorized by the user or fully authorized by all parties.

[0222] It should be understood that the specific embodiments described here are only used to explain this application, rather than to limit it. According to the embodiments provided in this application, all other embodiments obtained by those of ordinary skill in the art without creative efforts fall within the protection scope of this application.

[0223] Obviously, the drawings are only some examples or embodiments of this application. For those of ordinary skill in the art, this application can also be applied to other similar situations according to these drawings without creative efforts. In addition, it can be understood that although the work done during the development process here may be complex and time-consuming, for those of ordinary skill in the art, certain design, manufacturing, or production changes based on the technical content disclosed in this application are only conventional technical means and should not be regarded as insufficient disclosure of this application.

[0224] The term "embodiment" in this application means that the specific features, structures, or characteristics described in combination with the embodiment may be included in at least one embodiment of this application. The phrase appears in various positions in the specification does not necessarily mean the same embodiment, nor does it mean being independent or alternative to other embodiments and mutually exclusive. Those of ordinary skill in the art can clearly or implicitly understand that the embodiments described in this application can be combined with other embodiments without conflict.

[0225] The above-described embodiments merely represent several implementation manners of the present application. The description thereof is relatively specific and detailed, but it should not be construed as a limitation on the scope of patent protection. It should be noted that for those of ordinary skill in the art, without departing from the concept of the present application, several variations and improvements can still be made, and these all fall within the protection scope of the present application. Therefore, the protection scope of the present application shall be subject to the appended claims.

Claims

1. A recommendation method based on ranking objectives, characterized in that: include: Get training samples; The training samples include historical interaction data between multiple sample users and corresponding items; Based on the AUC indicator in the recommendation scenario, the optimization target of the initial recommendation model is constructed; Based on the optimization target, the training sample and the preset loss function, the initial recommendation model is trained to obtain an item recommendation model; wherein the initial recommendation model has multiple Rankformer layers, and in each Rankformer layer l, the prediction representation Z output by the previous layer (l-1) is (l-1) Perform normalization processing and further characterize the normalized prediction Calculating attention weights Calculate the target weight Ω of the Rankformer layer based on the attention weight (l) , and normalized to get the normalized target weight Finally, according to the normalized target weight and the predicted representation Z output by the previous layer (l-1) (l-1) , merge to get the prediction representation Z of the current layer l (l) , and finally get the prediction representation Z output by the last Rankformer layer (L) ; According to the user representation and the item representation output by the item recommendation model, the interest scores of the user and the recommended item are predicted, so as to obtain the recommendation ranking results of the recommended items according to the interest scores.

2. The recommendation method based on ranking target according to claim 1, characterized in that: The optimization goal of constructing the initial recommendation model based on the AUC indicator in the recommendation scenario includes: Based on the graph Transformer architecture, construct the initial recommendation model; In the initial recommendation model, maximizing the AUC index is the optimization goal.

3. The recommendation method based on ranking target according to claim 2, characterized in that: The optimization goal is: argmax z E(Z;δ) Among them, u represents the user, Represents a user set; a triple set Each element (u,i,j) in indicates that user u has interacted with item i but not item j. Represents the set of items that interact with user u; and denote the predicted interest scores of user u and item i and item j respectively; δ(·) denotes the activation function; Z is the representation, which includes user representation and item representation; express expectations; is the regularization term, and λ is the regularization coefficient.

4. The recommendation method based on ranking target according to claim 1, characterized in that: The step of training the initial recommendation model based on the optimization objective, the training sample, and the preset loss function to obtain an item recommendation model includes: In the iterative Rankformer layer in the initial recommendation model, the user initial representation and the item initial representation are encoded with the optimization objective to obtain the predicted representation output by the initial recommendation model; Determining a predicted interest score of the initial recommendation model according to the predicted representation; Based on the preset loss function, the recommendation loss between the predicted interest score and the real interest score in the training sample is calculated to obtain the item recommendation model through training according to the recommendation loss.

5. The recommendation method based on ranking target according to claim 4, characterized in that: The calculation of each Rankformer layer is as follows: Among them, Z (l) and Z (l-1) Represents the prediction representation of each layer output by the current layer l and the previous layer (l-1) respectively; τ represents the temperature coefficient of the Rankformer layer; Represents the target weight of the current layer l, which is composed of weight and After normalization calculation, we get: Among them, u represents the user, Represents a user set; for each user The set of items that have been observed to interact with user u is d u Represents a collection of items The size of m represents the set of items. size; for each item The set of users who have been observed to interact with item i is and represents the attention weight; and They represent the weight of user u to item i, and the weight of item i to user u respectively; j represents the item that user u has not interacted with.

6. The recommendation method based on ranking target according to claim 4, characterized in that: Also includes: In the calculation of each Rankformer layer, the user representation and the item representation are linearly expanded and calculated.

7. The recommendation method based on ranking target according to claim 1, characterized in that: The predicting the interest scores of the user and the recommended item according to the user representation and the item representation output by the item recommendation model, so as to obtain the recommendation ranking results of the recommended items according to the interest scores, includes: Based on the item recommendation model, a predicted user representation and item representation are obtained; Determining the interest score according to the user representation and the item representation; The recommended items are sorted according to the interest scores to obtain the recommendation sorting result.

8. A recommendation device based on a ranking target, characterized in that: include: A sample acquisition module is used to acquire training samples; The training samples include historical interaction data between multiple sample users and corresponding items; The model building module is used to build the optimization target of the initial recommendation model based on the AUC indicator in the recommendation scenario; A model training module is used to train the initial recommendation model to obtain an item recommendation model based on the optimization target, the training sample and the preset loss function; wherein the initial recommendation model has multiple Rankformer layers, and in each Rankformer layer l, the prediction representation Z output by the previous layer (l-1) is (l-1) Perform normalization processing and further characterize the normalized prediction Calculating attention weights Calculate the target weight Ω of the Rankformer layer based on the attention weight (l) , and normalized to get the normalized target weight Finally, according to the normalized target weight and the predicted representation Z output by the previous layer (l-1) (l-1) , merge to get the prediction representation Z of the current layer l (l) , and finally get the prediction representation Z output by the last Rankformer layer (L) ; The item recommendation module is used to predict the interest scores of the user and the recommended items according to the user representation and the item representation output by the item recommendation model, so as to obtain the recommendation ranking results of the recommended items according to the interest scores.

9. A computer device comprising a memory and a processor, characterized in that: A computer program is stored in the memory, and the processor is configured to run the computer program to execute the recommendation method based on ranking objectives according to any one of claims 1 to 7.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the steps of the recommendation method based on ranking objectives described in any one of claims 1 to 7 are implemented.

Images