Recommendation method based on object perception strategy network

Abstract

The invention relates to a recommendation method based on an object perception strategy network, and the method comprises the steps: constructing an object perception strategy network which comprises a strategy network, a feature optimization model, and a deep recommendation model; the strategy network serves as a reinforcement learning agent, dynamically adjusts the embedding size of a user and an item, and provides embedding for the feature optimization model; the feature optimization model optimizes embedded features through an object perception method, and uniformly inputs the size of the embedded features; then, connecting the two uniform embedding parts and feeding the embedding parts into a depth recommendation model to calculate a prediction result; model training is carried out on the constructed object perception strategy network, and in the training process, a deep recommendation model and a feature optimization model are updated along with back propagation; and deploying the trained object perception strategy network into a recommendation system for recommendation work. The method can improve the performance of the recommendation system.

Description

Technical Field

[0001] The present invention relates to the technical field of data recommendation, and in particular to a recommendation method based on an object-aware strategy network. Background Art

[0002] Deep learning technology has been applied to many recommendation systems due to its excellent performance in nonlinear transformation and representation learning. Deep recommendation systems usually consist of a representation layer and an inference layer. The former learns to map discrete user and item identifiers to real-valued embedding representations, and the latter takes this as input and calculates it with MLP (multi-layer perception) or FM (factorization machine) to obtain predictions. Many works improve the performance of learning models by capturing high-order feature interactions, applying attention mechanisms, or combining factorization machines with MLP, but studies have shown that the representation layer is also an important factor in improving model performance.

[0003] In recent years, research on the representation layer has focused on searching for embedding sizes for different users and items. ESPAN (Embedding Sizing Policy Network), NIS (Neural Input Search), and AutoEmb (an end-to-end framework based on AutoML) advocate that the embedding size should be changed dynamically based on the popularity of the user-item pair. Applying high-dimensional embeddings to users / items with low frequency will lead to overfitting due to over-parameterization. In addition, when low-dimensional embeddings are applied to users / items with high frequency, such a model cannot be trained effectively. However, the above models only improve the model in terms of embedding size without optimizing the features in the embedding, which will affect the accuracy of the model. Summary of the invention

[0004] The purpose of the present invention is to provide a recommendation method based on an object-aware strategy network, which can improve the performance of a recommendation system.

[0005] In order to achieve the above object, the technical solution adopted by the present invention is: a recommendation method based on an object-aware strategy network, comprising:

[0006] Constructing an object-aware policy network, which includes a policy network, a feature optimization model, and a deep recommendation model; the policy network acts as a reinforcement learning agent, dynamically adjusts the embedding size of users and items, and provides embeddings for the feature optimization model; the feature optimization model optimizes the features in the embeddings through an object-aware method and unifies the size of the input embeddings; then, the two unified embeddings are concatenated and fed into the deep recommendation model to calculate the prediction results;

[0007] The constructed object perception strategy network is trained. During the training process, the deep recommendation model and feature optimization model will be updated with back propagation.

[0008] Deploy the trained object-aware policy network into the recommendation system for recommendation work.

[0009] Furthermore, the implementation method of the feature optimization model is:

[0010] The feature optimization model optimizes features by capturing the interaction between users and items through an object-aware method; after being fed into the feature optimization model, the embedding e of user u (u) A fixed-size embedding with item i Cascade, embedding e of item i (i) With fixed-size embedding of user u Cascade as follows:

[0011]

[0012] Among them, e′ (u) for e (u) and The result of the cascade, e′ (i) for e (i) and The result of cascade, operator: indicates cascade;

[0013] Assume that the embeddings of users and items are (u) 、e (i) There are n candidate embeddings of size d1,d2,···,d n , where d1<d2<···<d n ; e′ (u) and e′ (i) Simplified as e′, it has n candidate embeddings of size d1′,d2′,···,d n ′, where d i ′=d i +D f , i=1,2,···,n,D f yes or Then, to capture the interaction between users and items, e′ is sent to the transformation map as follows:

[0014] e′2=W 1→2 e1′+b 1→2

[0015] e′3=W 2→3 e′2+b 2→3 ......

[0017]

[0018] Among them, e′ kThe dimension is d k embedding, k=1,2,···,n-1, The dimension is d n Embedding of W k-1→k and b k-1→k are the learnable weights and bias parameters respectively; in addition, the dimension is d′ n The embedding e′ is transformed as follows:

[0019]

[0020] Among them, W n and b n are the learnable weight and bias parameters, respectively; thus, the dimension is ensured to be d j Embedding e′ j , j = 1, 2, ..., n, will always be converted to dimension d n Embedding.

[0021] Furthermore, the implementation method of the deep recommendation model is:

[0022] The two embeddings are input to the deep recommendation model; the deep recommendation model uses two channels to model users and items, so the two embeddings are combined by concatenating the features of these two channels:

[0023]

[0024] in, and are the embeddings of users and items, respectively; in order to capture the interaction between the latent features of users and items, the embeddings of users and items are fed to the inference layer, which consists of an MLP with m layers:

[0025] α1=σ(W1α0+b1),

[0026] α2=σ(W2α1+b2),

[0027] ······

[0028] α l =σ(W l α 1-1 +b l )

[0029] in, and Represent the model weight and bias of the lth layer respectively; k l represents the number of neurons perceived in the lth layer; l represents the layer depth, σ represents the activation function; then, α l Will be used to calculate the prediction results through the activation function.

[0030] Compared with the prior art, the present invention has the following beneficial effects:

[0031] 1. Performance improvement: The object-aware policy network (OPN) constructed by this method can significantly improve the performance of the recommendation system in binary classification and multi-class classification tasks by optimizing the features in the embedding.

[0032] 2. Dynamic embedding adjustment: Different from traditional fixed-size embeddings, OPN is able to dynamically adjust the embedding size based on the popularity of users and items, which helps avoid overfitting and reduce the number of parameters.

[0033] 3. Object-aware optimization: OPN introduces an object-aware approach to optimize features by learning users’ preferences when interacting with different items, which enables the model to more accurately capture users’ interests and the characteristics of items.

[0034] 4. Less computational overhead: Although OPN increases some computational complexity, experimental results show that its increased time consumption is within an acceptable range, which means that it is efficient in practical applications.

[0035] 5. Parameter control: The number of parameters of OPN does not increase significantly compared to the baseline model ESPAN, indicating that it optimizes performance without significantly increasing the complexity of the model. BRIEF DESCRIPTION OF THE DRAWINGS

[0036] Figure 1 is an architecture diagram of an object-aware strategy network in an embodiment of the present invention;

[0037] Figure 2 is an architecture diagram of a feature optimization model in an embodiment of the present invention;

[0038] Figure 3 It is an architectural diagram of the deep recommendation model in an embodiment of the present invention. DETAILED DESCRIPTION

[0039] The present invention will be further described below in conjunction with the accompanying drawings and embodiments.

[0040] It should be noted that the following detailed descriptions are exemplary and are intended to provide further explanation of the present application. Unless otherwise specified, all technical and scientific terms used herein have the same meanings as those commonly understood by those skilled in the art to which the present application belongs.

[0041] It should be noted that the terms used herein are only for describing specific embodiments and are not intended to limit the exemplary embodiments according to the present application. As used herein, unless the context clearly indicates otherwise, the singular form is also intended to include the plural form. In addition, it should be understood that when the terms "comprise" and / or "include" are used in this specification, it indicates the presence of features, steps, operations, devices, components and / or combinations thereof.

[0042] This embodiment provides a recommendation method based on an object-aware policy network, comprising the following steps:

[0043] First, we build an object-aware policy network to optimize the features in the embedding by learning the preferences of users when they interact with different items, thereby improving the performance of the recommendation system. Figure 1 As shown in the figure, the object-aware policy network includes a policy network, a feature optimization model, and a deep recommendation model. The policy network acts as a reinforcement learning (RL) agent, dynamically adjusts the embedding size of users and items, and provides appropriate embeddings for the feature optimization model. The policy in the policy network defines how to adjust the embedding size of users and items. The feature optimization model optimizes the features in the embedding through an object-aware method and unifies the size of the input embedding. Then, the two unified embeddings are connected and fed into the deep recommendation model to calculate the prediction results.

[0044] On this basis, the constructed object perception policy network is trained. During the training process, the deep recommendation model and feature optimization model will be updated with back propagation.

[0045] Finally, the trained object-aware policy network is deployed into the recommendation system for recommendation work.

[0046] 1. Feature Optimization Model

[0047] The feature optimization model optimizes features by capturing the interaction between users and items through object-aware methods. Figure 2 As shown, after being fed into the feature optimization model, the embedding e of user u is (u) A fixed-size embedding with item i Cascade, embedding e of item i (i) With fixed-size embedding of user u Cascade as follows:

[0048]

[0049] Among them, e′ (u) for e (u) and The result of the cascade, e′ (i) for e (i) and The result of cascade. The operator: indicates cascade.

[0050] Assume that the embeddings of users and items are (u) 、e (i) There are n candidate embeddings of size d1,d2,···,d n , where d1<d2<···<d n For simplicity, e′ (u) and e′ (i) Simplified as e′, it has n candidate embeddings of size d1′,d2′,···,d n ′, where d i ′=d i +D f , i=1,2,···,n,D f yes or Then, to capture the interaction between users and items, e′ is sent to the transformation map as follows:

[0051] e′2=W 1→2 e1′+b 1→2

[0052] e′3=W 2→3 e′2+b 2→3 ......

[0054]

[0055] Among them, e′ k The dimension is d k embedding, k=1,2,···,n-1, The dimension is d n Embedding of W k-1→k and b k-1→k are the learnable weights and bias parameters respectively. Therefore, the dimension is d n The embedding e′ of ′ is transformed as follows:

[0056]

[0057] Among them, W n and b n are the learnable weights and bias parameters respectively. Therefore, we can ensure that the dimension is d j Embed j ′(j=1,2,…,n) will always be converted to dimension d n Embedding.

[0058] 2. Deep Recommendation Model

[0059] like Figure 3 As shown in Figure 2, the two embeddings are input into the deep recommendation model. Since the deep recommendation model uses two channels to model users and items, the two embeddings are combined by concatenating the features of the two channels:

[0060]

[0061] in, and are the embeddings of users and items, respectively. In order to capture the interaction between the latent features of users and items, the embeddings of users and items are fed to the inference layer, which consists of an MLP with m layers:

[0062] α1=σ(W1α0+b1),

[0063] α2=σ(W2α1+b2),

[0064] ······

[0065] α l =σ(W l α 1-1 +b l )

[0066] in, and Represent the model weight and bias of the lth layer respectively; k l represents the number of neurons perceived in the lth layer; l represents the layer depth, σ represents the activation function; then, α is used through the activation function l to calculate the prediction results.

[0067] The application process of this method is:

[0068] 1. Data input: data streams from different fields

[0069] 2. Data Preprocessing

[0070] 1) Data cleaning: removing missing values ​​and abnormal data

[0071] 2) Feature extraction: Extract useful features from the raw data, such as document type, field, and suitable population.

[0072] 3. Model training

[0073] 1) Use input data to train the network model

[0074] 2) Adjust model parameters, such as learning rate, batch size, training rounds, etc.

[0075] 3) Test and evaluate model performance and iterate training

[0076] 4. Model deployment: Deploy the trained model to specific application scenarios to assist in information extraction.

[0077] This method can be applied to:

[0078] 1. Personalized recommendation: By better understanding the interaction between users and items, we can recommend items that are more in line with their interests.

[0079] 2. Improve user satisfaction: Improve user satisfaction with the recommendation system by improving the relevance and accuracy of recommendations.

[0080] 3. E-commerce: In e-commerce platforms, help recommend products that users may be interested in, thereby increasing conversion rates and user engagement.

[0081] 4. Content recommendation platform: recommend content that users may be interested in in streaming services such as news, videos, and music.

[0082] 5. Advertising delivery: Applied to online advertising systems to improve the effectiveness of advertising delivery by more accurately predicting user reactions to advertising content.

[0083] Those skilled in the art will appreciate that the embodiments of the present application may be provided as methods, systems, or computer program products. Therefore, the present application may adopt the form of a complete hardware embodiment, a complete software embodiment, or an embodiment in combination with software and hardware. Moreover, the present application may adopt the form of a computer program product implemented in one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) that include computer-usable program code.

[0084] The present application is described with reference to the flowcharts and / or block diagrams of the methods, devices (systems), and computer program products according to the embodiments of the present application. It should be understood that each process and / or box in the flowchart and / or block diagram, as well as the combination of the processes and / or boxes in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to generate a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the processes in the flowchart and / or block diagram. Figure 1 A process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0085] These computer program instructions may also be stored in a computer-readable memory capable of directing a computer or other programmable data processing device to operate in a specific manner, so that the instructions stored in the computer-readable memory produce an article of manufacture comprising an instruction device, which implements the process Figure 1 A process or multiple processes and / or boxes Figure 1 A function specified in one or more boxes.

[0086] These computer program instructions can also be loaded onto a computer or other programmable data processing device so that a series of operating steps are executed on the computer or other programmable device to produce a computer-implemented process, thereby providing instructions for implementing the process. Figure 1 A process or multiple processes and / or boxes Figure 1 The steps for the functions specified in one or more boxes.

[0087] The above is only a preferred embodiment of the present invention, and does not limit the present invention in other forms. Any technician familiar with the profession may use the above disclosed technical content to change or modify it into an equivalent embodiment with equivalent changes. However, any simple modification, equivalent change and modification made to the above embodiment according to the technical essence of the present invention without departing from the technical solution of the present invention still belongs to the protection scope of the technical solution of the present invention.

Claims

1. A recommendation method based on object-aware policy network, characterized in that: include: Constructing an object-aware policy network, wherein the object-aware policy network includes a policy network, a feature optimization model, and a deep recommendation model; The policy network acts as a reinforcement learning agent, dynamically adjusts the embedding size of users and items, and provides embeddings to the feature optimization model; the feature optimization model optimizes the features in the embeddings through an object-aware method and unifies the size of the input embeddings; The two unified embeddings are then concatenated and fed into a deep recommendation model to compute the prediction results; The constructed object perception strategy network is trained. During the training process, the deep recommendation model and feature optimization model will be updated with back propagation. Deploy the trained object-aware policy network into the recommendation system for recommendation work.

2. The object-aware strategy network-based recommendation method according to claim 1, characterized in that: The implementation method of the feature optimization model is: The feature optimization model optimizes features by capturing the interactions between users and items through an object-aware approach; after being fed into the feature optimization model, the embedding e of user u (u) A fixed-size embedding with item i Cascade, embedding e of item i (i) With fixed-size embedding of user u Cascade as follows: Among them, e′ (u) for e (u) and The result of the cascade, e′ (i) for e (i) and The result of cascade, operator: indicates cascade; Assume that the embeddings of users and items are (u) 、e (i) There are n candidate embeddings of size d1,d2,···,d n , where d1<d2<···<d n ; e′ (u) and e′ (i) Simplified as e′, it has n candidate embeddings of size d1′,d2′,···,d n ′, where d i ′=d i +D f , i=1,2,···,n,D f yes or Then, to capture the interaction between users and items, e′ is sent to the transformation map as follows: e′2=W 1→2 e1′+b 1→2 e′3=W 2→3 e′2+b 2→3 ...... Among them, e′ k The dimension is d k embedding, k=1,2,···,n-1, The dimension is d n Embedding of W k-1→k and b k-1→k are the learnable weights and bias parameters respectively; in addition, the dimension is d′ n The embedding e′ is transformed as follows: Among them, W n and b n are the learnable weight and bias parameters, respectively; thus, the dimension is ensured to be d j Embedding e′ j , j = 1, 2, ..., n, will always be converted to dimension d n Embedding.

3. The object-aware strategy network-based recommendation method according to claim 1, characterized in that: The implementation method of the deep recommendation model is: The two embeddings are input to the deep recommendation model; the deep recommendation model uses two channels to model users and items, so the two embeddings are combined by concatenating the features of these two channels: in, and are the embeddings of users and items, respectively; in order to capture the interaction between the latent features of users and items, the embeddings of users and items are fed to the inference layer, which consists of an MLP with m layers: α1=σ(W1α0+b1), α2=σ(W2α1+b2), ······ a l =σ(W l a 1-1 +b l ) in, and Represent the model weight and bias of the lth layer respectively; k l represents the number of neurons perceived in the lth layer; l represents the layer depth, σ represents the activation function; then, α l Will be used to calculate the prediction results through the activation function.

Images