Product recommendation method, device, electronic device and storage medium

Abstract

The embodiment of the present invention discloses a product recommendation method, device, electronic device and storage medium. The product recommendation method specifically includes: obtaining user description data of the target user; inputting the user description data into a local offline product recommendation network trained by federated learning, and obtaining at least one product to be recommended corresponding to the target user; obtaining the current state data of the target user, and constructing query data corresponding to each product to be recommended according to the current state data and the user description data; inputting each query data into an online product recommendation network based on reinforcement learning, obtaining a product recommendation value corresponding to each product to be recommended, and recommending products to the target user according to each product recommendation value. The technical solution of the embodiment of the present invention can better construct user portraits and improve the accuracy of product recommendations.

Description

Technical Field

[0001] Embodiments of the present invention relate to the field of computer technology, and in particular, to a product recommendation method, device, electronic device, and storage medium. Background Art

[0002] With the development of machine learning, models are gradually integrated into people's lives. For example, models are used to recommend products of interest to users.

[0003] Existing product recommendation methods usually input user information and product information into a product recommendation network model to obtain product recommendation values ​​through the product recommendation network model, thereby recommending products to users based on the product recommendation values. However, the information data of different platforms are relatively independent, and the available information data of a single platform is relatively small. The user portrait constructed from this cannot accurately understand the user, which reduces the accuracy of product recommendations, resulting in the inability of existing product recommendation methods to accurately recommend products to users. Summary of the invention

[0004] The embodiments of the present invention provide a product recommendation method, device, electronic device and storage medium, which can better build user portraits and improve the accuracy of product recommendations.

[0005] According to one aspect of the present invention, a product recommendation method is provided, comprising:

[0006] Obtain user description data of target users;

[0007] Input the user description data into a local offline product recommendation network trained by federated learning, and obtain at least one product to be recommended corresponding to the target user;

[0008] Obtain the current status data of the target user, and construct query data corresponding to each product to be recommended based on the current status data and user description data;

[0009] Each query data is input into an online product recommendation network based on reinforcement learning to obtain the product recommendation value corresponding to each product to be recommended, and the product is recommended to the target user based on the product recommendation value.

[0010] According to another aspect of the present invention, there is provided a product recommendation device, comprising:

[0011] A user description data acquisition module is used to acquire user description data of target users;

[0012] A product acquisition module to be recommended, used to input user description data into a local offline product recommendation network trained by federated learning, and obtain at least one product to be recommended corresponding to a target user;

[0013] A query data building module is used to obtain the current status data of the target user and build query data corresponding to each product to be recommended based on the current status data and user description data;

[0014] The product recommendation value acquisition module is used to input each query data into the online product recommendation network based on reinforcement learning, obtain the product recommendation value corresponding to each product to be recommended, and recommend products to target users based on each product recommendation value.

[0015] According to another aspect of the present invention, there is provided an electronic device, the electronic device comprising:

[0016] at least one processor; and

[0017] a memory communicatively connected to the at least one processor; wherein,

[0018] The memory stores a computer program that can be executed by the at least one processor, and the computer program is executed by the at least one processor so that the at least one processor can execute the product recommendation method described in any embodiment of the present invention.

[0019] According to another aspect of the present invention, a computer-readable storage medium is provided, wherein the computer-readable storage medium stores computer instructions, and the computer instructions are used to enable a processor to implement the product recommendation method described in any embodiment of the present invention when executed.

[0020] The technical solution of the embodiment of the present invention obtains user description data of the target user, inputs the user description data into a local offline product recommendation network trained by federated learning, obtains at least one product to be recommended corresponding to the target user, obtains the current state data of the target user, and constructs query data corresponding to each product to be recommended based on the current state data and the user description data, so as to input each query data into an online product recommendation network based on reinforcement learning, obtain product recommendation values ​​corresponding to each product to be recommended, and thus recommend products to the target user based on each product recommendation value, thereby solving the problem that the existing product recommendation method cannot accurately recommend products to users, and can better construct user portraits and improve the accuracy of product recommendations.

[0021] It should be understood that the contents described in this section are not intended to identify the key or important features of the embodiments of the present invention, nor are they intended to limit the scope of the present invention. Other features of the present invention will become easily understood through the following description. BRIEF DESCRIPTION OF THE DRAWINGS

[0022] In order to more clearly illustrate the technical solutions in the embodiments of the present invention, the following briefly introduces the drawings required for use in the description of the embodiments. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without creative work.

[0023] Figure 1 is a flow chart of a product recommendation method provided in Embodiment 1 of the present invention;

[0024] Figure 2 is a flow chart of a product recommendation method provided by Embodiment 2 of the present invention;

[0025] Figure 3 is an example flow chart of a product recommendation method provided in Embodiment 3 of the present invention;

[0026] Figure 4 is a schematic diagram of a product recommendation device provided by a fourth embodiment of the present invention;

[0027] Figure 5 It is a structural schematic diagram of an electronic device for implementing the product recommendation method according to an embodiment of the present invention. DETAILED DESCRIPTION

[0028] In order to enable those skilled in the art to better understand the scheme of the present invention, the technical scheme in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work should fall within the scope of protection of the present invention.

[0029] It should be noted that the terms "including" and "having" and any variations thereof in the specification and claims of the present invention and the above-mentioned drawings are intended to cover non-exclusive inclusions. For example, a process, method, system, product or apparatus comprising a series of steps or units is not necessarily limited to those steps or units clearly listed, but may include other steps or units that are not clearly listed or inherent to these processes, methods, products or apparatus.

[0030] Embodiment 1

[0031] Figure 1This is a flowchart of a product recommendation method provided by the first embodiment of the present invention. This embodiment is applicable to the case of improving the accuracy of product recommendation. The method can be executed by a product recommendation device, which can be implemented by software and / or hardware, and can generally be directly integrated into an electronic device that executes the method. The electronic device can be a terminal device or a server device. The embodiment of the present invention does not limit the type of electronic device that executes the product recommendation method. Specifically, Figure 1 As shown, the product recommendation method may specifically include the following steps:

[0032] S110: Obtain user description data of the target user.

[0033] The target user may be any user. It is understandable that in the process of recommending products, any user may be the target user. The embodiment of the present invention determines the target user to recommend products matching the target user to different target users. The user description data may be data that can describe the characteristics of the target user, such as user ID (Identity document) data, user age data, or user place of origin data, etc., which is not limited in the embodiment of the present invention.

[0034] In the embodiment of the present invention, user description data of the target user is obtained, so as to input the user description data into the local offline product recommendation network trained by federated learning. It should be noted that the embodiment of the present invention does not limit the specific implementation method of obtaining the user description data of the target user, as long as the user description data of the target user can be obtained.

[0035] S120: Input the user description data into a local offline product recommendation network trained by federated learning, and obtain at least one product to be recommended corresponding to the target user.

[0036] The local offline product recommendation network may be a network that can perform offline product recommendations in a local node. Specifically, the local offline product recommendation network may be a network trained by federated learning. The product to be recommended may be a product that is screened by the local offline product recommendation network and can be recommended to the target user.

[0037] In an embodiment of the present invention, after obtaining the user description data of the target user, the user description data can be further input into a local offline product recommendation network trained by federated learning, and at least one product to be recommended corresponding to the target user can be obtained. It is understandable that each target user can correspond to multiple products to be recommended. The number of products to be recommended corresponding to different target users can be the same or different, and the embodiment of the present invention does not limit this.

[0038] S130: Acquire the current status data of the target user, and construct query data corresponding to each product to be recommended according to the current status data and the user description data.

[0039] Among them, the current state data may be the state data of the target user at the current moment, for example, it may be query state data, or it may be browsing state data, etc., and the embodiment of the present invention does not limit this. The query data corresponding to each product to be recommended may be data for querying the matching degree between each product to be recommended and the target user. It is understandable that the matching degree between each product to be recommended and the target user can be determined through the query data corresponding to each product to be recommended, so as to recommend the product to be recommended with the highest matching degree to the target user.

[0040] In an embodiment of the present invention, after inputting the user description data into the local offline product recommendation network trained by federated learning and obtaining at least one product to be recommended corresponding to the target user, the current state data of the target user can be further obtained, and query data corresponding to each product to be recommended can be constructed based on the current state data and the user description data. It can be understood that the number of query data matches the number of products to be recommended. It can also be understood that the query data matches the current state data of the target user, that is, the query data constructed by different state data is different.

[0041] S140: Input each query data into an online product recommendation network based on reinforcement learning, obtain a product recommendation value corresponding to each product to be recommended, and recommend products to target users based on each product recommendation value.

[0042] The online product recommendation network may be a network that recommends products to users while they are online. It is understandable that the online product recommendation network is a network built based on reinforcement learning. The product recommendation value may be a value that represents the degree of match between each product to be recommended and the target user.

[0043] In an embodiment of the present invention, after obtaining the current state data of the target user and constructing query data corresponding to each product to be recommended based on the current state data and the user description data, each query data can be further input into an online product recommendation network based on reinforcement learning to obtain a product recommendation value corresponding to each product to be recommended, so as to make product recommendations to the target user based on each product recommendation value.

[0044] The technical solution of this embodiment obtains user description data of the target user, inputs the user description data into a local offline product recommendation network trained by federated learning, obtains at least one product to be recommended corresponding to the target user, obtains the current state data of the target user, and constructs query data corresponding to each product to be recommended based on the current state data and the user description data, so as to input each query data into an online product recommendation network based on reinforcement learning, obtain product recommendation values ​​corresponding to each product to be recommended, and then recommend products to the target user based on each product recommendation value, thereby solving the problem that the existing product recommendation method cannot accurately recommend products to users, and can better construct user portraits and improve the accuracy of product recommendations.

[0045] Embodiment 2

[0046] Figure 2 This is a flowchart of a product recommendation method provided in Embodiment 2 of the present invention. This embodiment is a further refinement of the above-mentioned technical solutions, and provides multiple specific optional implementation methods for inputting user description data into a local offline product recommendation network trained by federated learning, and obtaining at least one product to be recommended corresponding to the target user, and inputting each query data into an online product recommendation network based on reinforcement learning, and obtaining product recommendation values ​​corresponding to each product to be recommended. The technical solution in this embodiment can be combined with each optional solution in one or more of the above-mentioned embodiments.

[0047] like Figure 2 As shown, the method may include the following steps:

[0048] S210: Obtain user description data of the target user.

[0049] S220: Input the user description data into a local offline product recommendation network trained by federated learning.

[0050] Optionally, before inputting the user description data into the local offline product recommendation network trained by federated learning, the method may also include: training the local offline product recommendation network according to the local user description data set and the product description data set to obtain local training parameters; sending the local training parameters to the federated learning server for joint training; wherein the federated learning server is used to perform joint training according to the local training parameters and third-party training parameters obtained by training at least one third-party node according to the third-party user description data set to obtain joint training parameters; and updating the parameters of the local offline product recommendation network according to the joint training parameters fed back by the federated learning server.

[0051] Among them, the local user description data set can be a collection of description data of all local users in the local node. The product description data set can be a collection of description data of all products in the local node. Exemplarily, the description data of the product may include product ID, product category, product version, product applicable population, product grade or product source, etc., which is not limited by the embodiment of the present invention. The local training parameters may be network parameters of the local offline product recommendation network obtained by training the local offline product recommendation network. The third-party node may be any node containing user description data other than the local node, for example, it may be an e-commerce platform, a bank APP (Application), or a payment platform, etc., which is not limited by the embodiment of the present invention. The third-party user description data set may be a collection of description data of all users in the third-party node.

[0052] The third-party training parameters may be network parameters obtained by training based on the third-party user description data set. The joint training parameters may be network parameters obtained by the federated learning server after joint training.

[0053] Specifically, before the local node inputs the user description data into the local offline product recommendation network trained by federated learning, it can also train the local offline product recommendation network according to the local user description data set and the product description data set to obtain local training parameters, and send the local training parameters to the federated learning server for joint training. At the same time, the third-party node trains the product recommendation network in the third-party node according to the third-party user description data set to obtain at least one third-party training parameter, and sends the third-party training parameter to the federated learning server for joint training. The federated learning server performs joint training according to the local training parameters and at least one third-party training parameter to obtain the joint training parameters, and feeds back the joint training parameters to the local node. The local node updates the parameters of the local offline product recommendation network according to the joint training parameters fed back by the federated learning server.

[0054] Optionally, the user description data may include user horizontal description data and user vertical description data. Specifically, the user horizontal description data is characterized by relatively large differences in data samples, but the characteristics of the data samples are mostly similar, such as the user data source of nodes such as bank nodes or financial third-party nodes. The user vertical description data is characterized by basically similar data samples, but the data characteristics are quite different, such as the user data source of nodes such as e-commerce platforms or video websites.

[0055] Optionally, the local offline product recommendation network may include a horizontal user model, a vertical user model and a product model. The horizontal user model may be used to perform feature analysis on the input horizontal user description data to output a local user horizontal feature vector. The vertical user model may be used to perform feature analysis on the input vertical user description data to output a local user vertical feature vector. The product model may be used to perform feature analysis on the input product description data to input a local product feature vector.

[0056] Optionally, the local user description data set may include a local user horizontal description data set and / or a local user vertical description data set. The third-party user description data set may include a third-party user horizontal description data set and / or a third-party user vertical description data set. The local training parameters may include at least one of local user horizontal training parameters, local user vertical training parameters, and local product training parameters. The third-party training parameters may include third-party user horizontal training parameters and / or third-party user vertical training parameters. The joint training parameters may include at least one of joint user horizontal training parameters, joint user vertical training parameters, and joint product training parameters.

[0057] Optionally, the federated learning server may perform joint training based on the local user horizontal training parameters and the at least one third-party user horizontal training parameters to obtain the joint user horizontal training parameters. The federated learning server may perform joint training based on the local user vertical training parameters and the at least one third-party user vertical training parameters to obtain the joint user vertical training parameters; the federated learning server may obtain the joint product training parameters based on the local product training parameters.

[0058] S230: Calculate a user feature vector matching the user description data through the local offline product recommendation network.

[0059] The user feature vector may be a vector representing user features. Optionally, the user feature vector may be obtained by combining the user's horizontal feature vector and the user's vertical feature vector.

[0060] In the embodiment of the present invention, after updating the parameters of the local offline product recommendation network according to the joint training parameters fed back by the federated learning server, the user feature vector matching the user description data can be further calculated through the local offline product recommendation network. Optionally, the user feature vector matching the user description data can be calculated through the user model of the local offline product recommendation network.

[0061] S240: Calculate the similarity between the product feature vector of each product and the user feature vector through the local offline product recommendation network.

[0062] The product feature vector may be a vector representing product features. It is understandable that the product feature vector may be calculated by a local offline product recommendation network based on the description data of each product.

[0063] In an embodiment of the present invention, after calculating the user feature vector matching the user description data through the local offline product recommendation network, the similarity between the product feature vector of each product and the user feature vector can be further calculated through the local offline product recommendation network. Optionally, calculating the similarity between the product feature vector of each product and the user feature vector through the local offline product recommendation network can be to calculate the cosine distance between the product feature vector of each product and the user feature vector through the local offline product recommendation network, and determine the similarity between the product feature vector of each product and the user feature vector through the cosine distance between the product feature vector of each product and the user feature vector.

[0064] Optionally, before calculating the similarity between the product feature vector of each product and the user feature vector through the local offline product recommendation network, the product feature vector of each product can be calculated through the local offline product recommendation network. It is understandable that in the case of no product update, the product feature vector only needs to be calculated once.

[0065] S250: Outputting the user feature vector and the product feature vector of at least one product to be recommended according to the similarity calculation result through the local offline product recommendation network.

[0066] The similarity calculation result may be a calculation result obtained based on the similarity between the product feature vector and the user feature vector of each product, for example, it may be a calculation result of similarity sorting in positive order, etc., and the embodiment of the present invention is not limited to this.

[0067] In an embodiment of the present invention, after calculating the similarity between the product feature vector and the user feature vector of each product through the local offline product recommendation network, the local offline product recommendation network can be further used to output the user feature vector and the product feature vector of at least one product to be recommended based on the similarity calculation result.

[0068] S260: Acquire current state data of the target user, and calculate a state feature vector matching the current state data.

[0069] The state feature vector may be a vector representing the current state feature of the target user.

[0070] In an embodiment of the present invention, after outputting a user feature vector and a product feature vector of at least one product to be recommended based on the similarity calculation result through a local offline product recommendation network, the current state data of the target user can be further obtained, and a state feature vector matching the current state data can be calculated.

[0071] S270: Combine the product feature vector of each product to be recommended with the user feature vector and the state feature vector to construct query data corresponding to each product to be recommended.

[0072] In an embodiment of the present invention, after obtaining the current state data of the target user and calculating the state feature vector matching the current state data, the product feature vector of each product to be recommended can be further combined with the user feature vector and the state feature vector to construct the query data corresponding to each product to be recommended. It can be understood that the query data of the product to be recommended can include the product feature vector, the user feature vector and the state feature vector of the product to be recommended.

[0073] S280: Input each query data into an online product recommendation network based on reinforcement learning, and obtain reward function data of a training network of the online product recommendation network.

[0074] Among them, the reward function data can be function data constructed according to the reward value of the user's transfer in different states. It is understandable that when the user transfers from the current state to another state, there will be a corresponding reward value. For example, assuming that the user is in the current state, he can browse or buy. For the purchase behavior, the reward value is the highest only when the purchase is successfully transferred to the state. For browsing behavior, the reward value will gradually decrease over time. For long-term non-browsing behavior, a negative reward value can be obtained. It is understandable that the user's transfer in different states can be achieved by the user generating actions (ie, behaviors). The actions generated by the user may include clicking, browsing, purchasing or viewing details. The user can only select one action at the same time. Optionally, when no product recommendation is made, the status can be updated regularly.

[0075] In an embodiment of the present invention, after the product feature vector of each product to be recommended is combined with the user feature vector and the state feature vector respectively to construct query data corresponding to each product to be recommended, each query data can be further input into an online product recommendation network based on reinforcement learning, and the reward function data of the training network of the online product recommendation network can be obtained.

[0076] Optionally, before obtaining the reward function data of the training network of the online product recommendation network, the method may further include: determining the reward function data of the training network of the online product recommendation network according to the user historical behavior data in the third-party node.

[0077] The user historical behavior data may be the behavior data of the user in the past period of time. It is understandable that the user historical behavior data may be the historical behavior data of all users.

[0078] Specifically, before obtaining the reward function data of the training network of the online product recommendation network, the reward function data of the training network of the online product recommendation network may be determined according to the user historical behavior data in the third-party node.

[0079] S290: Acquire, through the training network of the online product recommendation network, the recommended action data of the target user corresponding to the current state data of the target user.

[0080] The recommended action data may be action data generated by the target user in the current state after the recommended product matching the query data is recommended to the target user. It is understandable that the recommended action data may be different action data.

[0081] In an embodiment of the present invention, after inputting each query data into an online product recommendation network based on reinforcement learning and obtaining the reward function data of the training network of the online product recommendation network, the recommended action data of the target user corresponding to the current state data of the target user can be further obtained through the training network of the online product recommendation network.

[0082] S2100 , determining the update status data of the target user according to the recommendation action data through the training network of the online product recommendation network, and updating the reward function data to obtain updated reward function data.

[0083] The updated state data may be new state data generated when the user performs an action (or behavior) in the current state. The updated reward function data may be obtained by updating the reward function data after the user performs an action (or behavior).

[0084] In an embodiment of the present invention, after obtaining the recommended action data of the target user corresponding to the current state data of the target user through the training network of the online product recommendation network, the updated state data of the target user can be further determined according to the recommended action data through the training network of the online product recommendation network, and the reward function data can be updated according to the recommended action data to obtain the updated reward function data.

[0085] S2110 . Determine target value function data according to the recommendation action data, the updated state data, and the updated reward function data through a training network of the online product recommendation network.

[0086] The target value function data may be function data obtained by rewarding the user after taking an action in the current state.

[0087] In an embodiment of the present invention, after determining the update status data of the target user according to the recommendation action data and updating the reward function data through the training network of the online product recommendation network to obtain the updated reward function data, the target value function data can be further determined according to the recommendation action data, the update status data and the updated reward function data through the training network of the online product recommendation network.

[0088] S2120, determine whether the objective value function data meets the convergence condition; if so, execute S2130; otherwise, execute S290.

[0089] The convergence condition may be that the target value function data reaches a stable state. It is understandable that when the target value function data does not meet the convergence condition, the target value function data may change due to the difference in the recommended action data.

[0090] In an embodiment of the present invention, after determining the target value function data according to the recommended action data, the updated state data and the updated reward function data through the training network of the online product recommendation network, it can be further determined whether the target value function data meets the convergence condition.

[0091] If the target value function data meets the convergence condition, it means that the target value function data has reached a stable state, that is, the target value function data will not change due to different recommended action data, and the update network of the online product recommendation network can be updated according to the target value function data. If the target value function data does not meet the convergence condition, it means that the target value function data has not reached a stable state, that is, the target value function data will still change due to different recommended action data, and then it can return to execute the operation of obtaining the recommended action data of the target user corresponding to the current state data of the target user through the training network of the online product recommendation network.

[0092] S2130: Update an update network of an online product recommendation network according to the target value function data.

[0093] In an embodiment of the present invention, after determining that the target value function data satisfies the convergence condition, the update network of the online product recommendation network may be further updated according to the target value function data.

[0094] S2140: Obtain product recommendation values ​​corresponding to each product to be recommended according to the updated network of the updated online product recommendation network, and recommend products to the target user according to each product recommendation value.

[0095] In an embodiment of the present invention, after updating the update network of the online product recommendation network according to the target value function data, the product recommendation value corresponding to each product to be recommended can be further obtained according to the updated update network of the online product recommendation network, and the product recommendation can be made to the target user according to each product recommendation value.

[0096] The technical solution of this embodiment obtains the user description data of the target user and inputs the user description data into the local offline product recommendation network trained by federated learning, so as to calculate the user feature vector matching the user description data through the local offline product recommendation network, and calculate the similarity between the product feature vector of each product and the user feature vector, so as to output the user feature vector and the product feature vector of at least one product to be recommended according to the similarity calculation result. Then, the current state data of the target user is obtained, and the state feature vector matching the current state data is calculated, so as to combine the product feature vector of each product to be recommended with the user feature vector and the state feature vector, respectively, and construct the query data corresponding to each product to be recommended. Input each query data into the online product recommendation network based on reinforcement learning, and obtain the reward function data of the training network of the online product recommendation network. Through the training network of the online product recommendation network, obtain the recommended action data of the target user corresponding to the current state data of the target user, and determine the updated state data of the target user according to the recommended action data, and update the reward function data to obtain the updated reward function data, so as to determine the target value function data according to the recommended action data, the updated state data and the updated reward function data, so as to further judge whether the target value function data meets the convergence condition. When it is determined that the target value function data does not meet the convergence condition, return to execute the operation of obtaining the recommended action data of the target user corresponding to the current state data of the target user through the training network of the online product recommendation network. When it is determined that the target value function data meets the convergence condition, update the update network of the online product recommendation network according to the target value function data, obtain the product recommendation value corresponding to each product to be recommended according to the updated update network of the online product recommendation network, and recommend products to the target user according to each product recommendation value. The problem that the existing product recommendation method cannot accurately recommend products to users is solved, and user portraits can be better constructed to improve the accuracy of product recommendations.

[0097] Embodiment 3

[0098] The embodiment of the present invention takes the application scenario of banking service recommendation as an example to specifically illustrate that with the rapid development of banking services, the promotion of banking services has become increasingly important. The main way to promote banking services at present is to promote them when handling services at the front desk, and the services in the bank APP cannot be effectively promoted according to the actual situation of the user and the user's preferences. However, each user has more than one bank card, which leads to less behavior and available information of a user in a bank, and the value of the information is also lacking. Often, the user portrait constructed in this way can only understand one aspect of the user. Moreover, many transaction information and user preferences of users are not fully understood according to the bank's knowledge base, and need to be obtained from a third-party platform, but the user information of the third-party platform is independent and not shared. In addition, in the banking service recommendation scenario, the user stays for a very short time, and the requirements for online recommendations are high. Only by capturing the user's intention and psychology in real time can we better recommend services to users during the time period when the user is online. Therefore, banking service recommendation is of great significance. Figure 3 is an example flow chart of a product recommendation method provided in Embodiment 3 of the present invention. Figure 3 As shown, the following steps may be specifically included:

[0099] (1) Training a local offline product recommendation network in a local node. The local node may be a platform node that needs to recommend banking services. The local offline product recommendation network in the local node may include a horizontal user model, a vertical user model, and a product model. It is understood that the model parameters of the horizontal user model, the vertical user model, and the product model are independent of each other.

[0100] Specifically, a local user description dataset and a product description dataset in a local node are obtained, and the local user description dataset and the product description dataset are input into a local offline product recommendation network, and local user horizontal training parameters and local user horizontal feature vectors are obtained through a horizontal user model of the local offline product recommendation network, local user vertical training parameters and local user vertical feature vectors are obtained through a vertical user model of the local offline product recommendation network, and local product training parameters and local product feature vectors are obtained through a product model of the local offline product recommendation network.

[0101] Obtain a third-party user description dataset in a third-party node, and input the third-party user description dataset into the offline product recommendation network in the third-party node, and obtain third-party training parameters and third-party user feature vectors through the user model in the offline product recommendation network. It is understandable that the third-party user description dataset can be a third-party user horizontal description dataset or a third-party user vertical description dataset. When the third-party user description dataset is a third-party user horizontal description dataset, the user model in the third-party offline product recommendation network is a horizontal user model, the third-party training parameters are third-party user horizontal training parameters, and the third-party user feature vector is a third-party user horizontal feature vector. When the third-party user description dataset is a third-party user vertical description dataset, the user model in the third-party offline product recommendation network is a vertical user model, the third-party training parameters are third-party user vertical training parameters, and the third-party user feature vector is a third-party user vertical feature vector.

[0102] Upload each training parameter and each feature vector to the federated learning server in an encrypted manner. The joint training parameters of the horizontal user model of the offline product recommendation network in the federated learning server are obtained by integrating the local user horizontal training parameters and the third-party user horizontal training parameters. The joint training parameters of the vertical user model of the offline product recommendation network in the federated learning server are obtained by integrating the local user vertical training parameters and the third-party user vertical training parameters. The joint training parameters of the vertical product model of the offline product recommendation network in the federated learning server are the local product training parameters. The local user horizontal feature vector and the third-party user horizontal feature vector are input into the horizontal user model of the offline product recommendation network in the federated learning server to obtain the user horizontal feature vector. The local user vertical feature vector and the third-party user vertical feature vector are input into the vertical user model of the offline product recommendation network in the federated learning server to obtain the user vertical feature vector. The user horizontal feature vector and the user vertical feature vector are combined and added to the DNN network layer to obtain the user feature vector. The local product feature vector is input into the product model of the offline product recommendation network in the federated learning server to obtain the product feature vector. Calculate the cosine distance between the user feature vector and the product feature vector. It can be understood that the larger the cosine distance value is, the closer the distance between the product of the user's query, purchase, etc. is to the user.

[0103] During the training of the local offline product recommendation network, the loss function used is:

[0104] L hinge =y*(user*item)+(1-y)*max(margin-(user*item))

[0105] Among them, L hingerepresents the hinge loss function of the local offline product recommendation network; y represents whether the user purchased or clicked the product; user represents the user feature vector; item represents the product feature vector.

[0106] When the loss function reaches convergence, the training of the local offline product recommendation network is stopped. And according to the joint training parameters of the federated learning server, the parameters of the local offline product recommendation network are updated to complete the training of the local offline product recommendation network in the local node.

[0107] (2) Obtain user description data of the target user, input the user description data into a local offline product recommendation network trained by federated learning, and obtain the first N recommended products corresponding to the target user. Specifically, input the user description data into a local offline product recommendation network trained by federated learning; calculate the user feature vector that matches the user description data through the local offline product recommendation network; calculate the similarity between the product feature vector of each product and the user feature vector through the local offline product recommendation network; and output the user feature vector and the product feature vectors of the first N products to be recommended based on the similarity calculation results through the local offline product recommendation network. Optionally, outputting the user feature vector and the product feature vectors of the first N products to be recommended based on the similarity calculation results may include outputting the user feature vector and the product feature vectors of the first N products to be recommended based on the bank product operation rules and the similarity calculation results.

[0108] (3) Obtain the current status data of the target user, and construct query data corresponding to each product to be recommended based on the current status data and user description data.

[0109] (4) Inputting each query data into an online product recommendation network based on reinforcement learning, obtaining a product recommendation value corresponding to each product to be recommended, and recommending products to target users based on each product recommendation value.

[0110] Specifically, the training network of the online product recommendation network is used to determine the recommended action data obtained by the e-greedy algorithm (greedy algorithm) under the current state data of the target user; the target user performs the recommended action to obtain updated state data, updated reward function data and target value function data, and the updated state data, updated reward function data and target value function data are stored in the experience pool, and the experience pool is trained to update the training network of the online product recommendation network. When it is determined that the target value function data meets the convergence condition, the updated network of the online product recommendation network is updated according to the target value function data; according to the updated updated network of the online product recommendation network, the product recommendation value corresponding to each product to be recommended is obtained. And according to each product recommendation value, the product is recommended to the target user. In particular, when updating the training network of the online product recommendation network, the current network selection is added to reduce the update speed and reduce the deviation of the network model.

[0111] Optionally, when updating the training network of the online product recommendation network, the following optimization function can be used for optimization:

[0112]

[0113] in, Represents the target value function data at time t; R t+1 represents the reward function data at time t+1; γ represents the attenuation coefficient; S t+1 represents the state data at time t+1; a represents the recommended action data; θ t represents the model parameters of the training network of the online product recommendation network at time t; θ' t Represents the model parameters of the updated network of the online product recommendation network at time t.

[0114] The above technical solution, by collecting user information between other banking institutions and other third-party transaction platforms, including vertical and horizontal user information, makes the user portrait model more accurate and detailed, while protecting the data privacy of each data source and avoiding data leakage as much as possible; it can effectively collect information online, update the model in real time online, and realize online prediction functions, so that customers' actions and feedback are reflected in the recommended direction of banking business in real time, thereby improving the accuracy of product recommendations.

[0115] Embodiment 4

[0116] Figure 4 is a schematic diagram of a product recommendation device provided by Embodiment 4 of the present invention. Figure 4 As shown, the device includes: a user description data acquisition module 410, a to-be-recommended product acquisition module 420, a query data construction module 430 and a product recommendation value acquisition module 440, wherein:

[0117] A user description data acquisition module 410 is used to acquire user description data of a target user;

[0118] The to-be-recommended product acquisition module 420 is used to input the user description data into the local offline product recommendation network trained by federated learning, and acquire at least one to-be-recommended product corresponding to the target user;

[0119] The query data construction module 430 is used to obtain the current status data of the target user, and to construct query data corresponding to each product to be recommended according to the current status data and the user description data;

[0120] The product recommendation value acquisition module 440 is used to input each query data into the online product recommendation network based on reinforcement learning, obtain the product recommendation value corresponding to each product to be recommended, and recommend products to the target user according to each product recommendation value.

[0121] The technical solution of this embodiment obtains user description data of the target user, inputs the user description data into a local offline product recommendation network trained by federated learning, obtains at least one product to be recommended corresponding to the target user, obtains the current state data of the target user, and constructs query data corresponding to each product to be recommended based on the current state data and the user description data, so as to input each query data into an online product recommendation network based on reinforcement learning, obtain product recommendation values ​​corresponding to each product to be recommended, and then recommend products to the target user based on each product recommendation value, thereby solving the problem that the existing product recommendation method cannot accurately recommend products to users, and can better construct user portraits and improve the accuracy of product recommendations.

[0122] Optionally, the module 420 for obtaining products to be recommended can be specifically used to: train a local offline product recommendation network according to a local user description data set and a product description data set to obtain local training parameters; send the local training parameters to a federated learning server for joint training; wherein the federated learning server is used to perform joint training according to the local training parameters and third-party training parameters obtained by training at least one third-party node according to a third-party user description data set to obtain joint training parameters; and update the parameters of the local offline product recommendation network according to the joint training parameters fed back by the federated learning server.

[0123] Optionally, the module 420 for obtaining products to be recommended can also be specifically used to: input user description data into a local offline product recommendation network trained by federated learning; calculate a user feature vector matching the user description data through the local offline product recommendation network; calculate the similarity between the product feature vector and the user feature vector of each product through the local offline product recommendation network; and output the user feature vector and the product feature vector of at least one product to be recommended based on the similarity calculation result through the local offline product recommendation network.

[0124] Optionally, the query data construction module 430 can be specifically used to: calculate a state feature vector that matches the current state data; combine the product feature vector of each product to be recommended with the user feature vector and the state feature vector, respectively, to construct query data corresponding to each product to be recommended.

[0125] Optionally, the product recommendation value acquisition module 440 can be specifically used to: input each query data into an online product recommendation network based on reinforcement learning, and obtain the reward function data of the training network of the online product recommendation network; obtain the recommended action data of the target user corresponding to the current state data of the target user through the training network of the online product recommendation network; determine the updated state data of the target user according to the recommended action data through the training network of the online product recommendation network, and update the reward function data to obtain the updated reward function data; determine the target value function data according to the recommended action data, the updated state data and the updated reward function data through the training network of the online product recommendation network; when it is determined that the target value function data meets the convergence condition, update the update network of the online product recommendation network according to the target value function data; and obtain the product recommendation value corresponding to each product to be recommended according to the updated update network of the online product recommendation network.

[0126] Optionally, the product recommendation value acquisition module 440 may also be specifically used to determine the reward function data of the training network of the online product recommendation network according to the user historical behavior data in the third-party node.

[0127] Optionally, the product recommendation value acquisition module 440 can also be specifically used to: when it is determined that the target value function data does not meet the convergence conditions, return to execute the operation of obtaining the target user's recommended action data corresponding to the target user's current state data through the training network of the online product recommendation network until the target value function data meets the convergence conditions.

[0128] The product recommendation device provided in the embodiment of the present invention can execute the product recommendation method provided in any embodiment of the present invention, and has the corresponding functional modules and beneficial effects of the execution method.

[0129] Embodiment 5

[0130] Figure 5 A schematic diagram of the structure of an electronic device 10 that can be used to implement an embodiment of the present invention is shown. The electronic device is intended to represent various forms of digital computers, such as laptop computers, desktop computers, workstations, personal digital assistants, servers, blade servers, mainframe computers, and other suitable computers. The electronic device can also represent various forms of mobile devices, such as personal digital processing, cellular phones, smart phones, wearable devices (such as helmets, glasses, watches, etc.) and other similar computing devices. The components shown herein, their connections and relationships, and their functions are merely examples and are not intended to limit the implementation of the present invention described and / or required herein.

[0131] like Figure 5 As shown, the electronic device 10 includes at least one processor 11, and a memory connected to the at least one processor 11, such as a read-only memory (ROM) 12, a random access memory (RAM) 13, etc., wherein the memory stores a computer program that can be executed by at least one processor, and the processor 11 can perform various appropriate actions and processes according to the computer program stored in the read-only memory (ROM) 12 or the computer program loaded from the storage unit 18 to the random access memory (RAM) 13. In the RAM 13, various programs and data required for the operation of the electronic device 10 can also be stored. The processor 11, the ROM 12, and the RAM 13 are connected to each other through a bus 14. An input / output (I / O) interface 15 is also connected to the bus 14.

[0132] A number of components in the electronic device 10 are connected to the I / O interface 15, including: an input unit 16, such as a keyboard, a mouse, etc.; an output unit 17, such as various types of displays, speakers, etc.; a storage unit 18, such as a disk, an optical disk, etc.; and a communication unit 19, such as a network card, a modem, a wireless communication transceiver, etc. The communication unit 19 allows the electronic device 10 to exchange information / data with other devices through a computer network such as the Internet and / or various telecommunication networks.

[0133] The processor 11 may be a variety of general and / or special processing components with processing and computing capabilities. Some examples of the processor 11 include, but are not limited to, a central processing unit (CPU), a graphics processing unit (GPU), various special artificial intelligence (AI) computing chips, various processors running machine learning model algorithms, a digital signal processor (DSP), and any appropriate processor, controller, microcontroller, etc. The processor 11 executes the various methods and processes described above, such as a product recommendation method.

[0134] In some embodiments, the product recommendation method may be implemented as a computer program, which is tangibly contained in a computer-readable storage medium, such as a storage unit 18. In some embodiments, part or all of the computer program may be loaded and / or installed on the electronic device 10 via the ROM 12 and / or the communication unit 19. When the computer program is loaded into the RAM 13 and executed by the processor 11, one or more steps of the product recommendation method described above may be performed. Alternatively, in other embodiments, the processor 11 may be configured to perform the product recommendation method in any other appropriate manner (e.g., by means of firmware).

[0135] Various implementations of the systems and techniques described above herein can be implemented in digital electronic circuit systems, integrated circuit systems, field programmable gate arrays (FPGAs), application specific integrated circuits (ASICs), application specific standard products (ASSPs), systems on chips (SOCs), load programmable logic devices (CPLDs), computer hardware, firmware, software, and / or combinations thereof. These various implementations can include: being implemented in one or more computer programs that can be executed and / or interpreted on a programmable system including at least one programmable processor, which can be a special purpose or general purpose programmable processor that can receive data and instructions from a storage system, at least one input device, and at least one output device, and transmit data and instructions to the storage system, the at least one input device, and the at least one output device.

[0136] Computer programs for implementing the methods of the present invention may be written in any combination of one or more programming languages. These computer programs may be provided to a processor of a general-purpose computer, a special-purpose computer, or other programmable data processing device, so that when the computer program is executed by the processor, the functions / operations specified in the flow chart and / or block diagram are implemented. The computer program may be executed entirely on the machine, partially on the machine, partially on the machine and partially on a remote machine as a stand-alone software package, or entirely on a remote machine or server.

[0137] In the context of the present invention, a computer-readable storage medium may be a tangible medium that may contain or store a computer program for use by or in combination with an instruction execution system, device or equipment. A computer-readable storage medium may include, but is not limited to, electronic, magnetic, optical, electromagnetic, infrared, or semiconductor systems, devices or equipment, or any suitable combination of the foregoing. Alternatively, a computer-readable storage medium may be a machine-readable signal medium. A more specific example of a machine-readable storage medium may include an electrical connection based on one or more lines, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing.

[0138] To provide interaction with a user, the systems and techniques described herein may be implemented on an electronic device having: a display device (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor) for displaying information to the user; and a keyboard and a pointing device (e.g., a mouse or trackball) through which the user can provide input to the electronic device. Other types of devices may also be used to provide interaction with the user; for example, the feedback provided to the user may be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user may be received in any form (including acoustic input, voice input, or tactile input).

[0139] The systems and techniques described herein may be implemented in a computing system that includes backend components (e.g., as a data server), or a computing system that includes middleware components (e.g., an application server), or a computing system that includes frontend components (e.g., a user computer with a graphical user interface or a web browser through which a user can interact with implementations of the systems and techniques described herein), or a computing system that includes any combination of such backend components, middleware components, or frontend components. The components of the system may be interconnected by any form or medium of digital data communication (e.g., a communication network). Examples of communication networks include: a local area network (LAN), a wide area network (WAN), a blockchain network, and the Internet.

[0140] A computing system may include a client and a server. The client and the server are generally remote from each other and usually interact through a communication network. The client and server relationship is generated by computer programs running on the corresponding computers and having a client-server relationship with each other. The server may be a cloud server, also known as a cloud computing server or cloud host, which is a host product in the cloud computing service system to solve the defects of difficult management and weak business scalability in traditional physical hosts and VPS services.

[0141] It should be understood that the various forms of processes shown above can be used to reorder, add or delete steps. For example, the steps described in the present invention can be executed in parallel, sequentially or in different orders, as long as the desired results of the technical solution of the present invention can be achieved, and this document does not limit this.

[0142] The above specific implementations do not constitute a limitation on the protection scope of the present invention. It should be understood by those skilled in the art that various modifications, combinations, sub-combinations and substitutions can be made according to design requirements and other factors. Any modification, equivalent substitution and improvement made within the spirit and principle of the present invention should be included in the protection scope of the present invention.

Claims

1. A product recommendation method, characterized in that: include: Obtain user description data of target users; Inputting the user description data into a local offline product recommendation network trained by federated learning, and obtaining at least one product to be recommended corresponding to the target user; Acquire the current status data of the target user, and construct query data corresponding to each product to be recommended according to the current status data and the user description data; Input each query data into an online product recommendation network based on reinforcement learning, obtain a product recommendation value corresponding to each product to be recommended, and recommend products to the target user based on each product recommendation value; The step of inputting each query data into an online product recommendation network based on reinforcement learning to obtain a product recommendation value corresponding to each product to be recommended includes: Inputting each query data into the online product recommendation network based on reinforcement learning, and obtaining reward function data of a training network of the online product recommendation network; Acquiring, through a training network of the online product recommendation network, recommended action data of the target user corresponding to the current state data of the target user; Determining the update status data of the target user according to the recommendation action data through the training network of the online product recommendation network, and updating the reward function data to obtain updated reward function data; Determining target value function data according to the recommendation action data, the updated state data and the updated reward function data through a training network of the online product recommendation network; In the case where it is determined that the objective value function data satisfies a convergence condition, updating an update network of the online product recommendation network according to the objective value function data; According to the updated network of the online product recommendation network, the product recommendation value corresponding to each product to be recommended is obtained.

2. The method according to claim 1, characterized in that Before inputting the user description data into the local offline product recommendation network trained by federated learning, it also includes: Training the local offline product recommendation network according to the local user description dataset and the product description dataset to obtain local training parameters; Sending the local training parameters to a federated learning server for joint training; The federated learning server is used to perform joint training according to the local training parameters and third-party training parameters obtained by training at least one third-party node according to a third-party user description data set to obtain joint training parameters; The parameters of the local offline product recommendation network are updated according to the joint training parameters fed back by the federated learning server.

3. The method according to claim 1 or 2, characterized in that: Inputting the user description data into a local offline product recommendation network trained by federated learning, and obtaining at least one product to be recommended corresponding to the target user, including: Input user description data into the local offline product recommendation network trained by federated learning; Calculating a user feature vector matching the user description data through the local offline product recommendation network; Calculating the similarity between the product feature vector of each product and the user feature vector through the local offline product recommendation network; The user feature vector and the product feature vector of at least one product to be recommended are outputted according to the similarity calculation result through the local offline product recommendation network.

4. The method according to claim 3, characterized in that: According to the current status data and user description data, construct query data corresponding to each product to be recommended, including: Calculating a state feature vector matching the current state data; The product feature vector of each product to be recommended is combined with the user feature vector and the state feature vector to construct query data corresponding to each product to be recommended.

5. The method according to claim 1, characterized in that Before obtaining the reward function data of the training network of the online product recommendation network, the method further includes: The reward function data of the training network of the online product recommendation network is determined according to the user historical behavior data in the third-party node.

6. The method according to claim 1, characterized in that After determining the target value function data according to the recommended action data, the updated state data and the updated reward function data through the training network of the online product recommendation network, the further step further includes: When it is determined that the target value function data does not meet the convergence condition, return to execute the training network of the online product recommendation network to obtain the recommended action data of the target user corresponding to the current state data of the target user until the target value function data meets the convergence condition.

7. A product recommendation device, characterized in that: include: A user description data acquisition module is used to acquire user description data of target users; A product to be recommended acquisition module, used for inputting the user description data into a local offline product recommendation network trained by federated learning, and acquiring at least one product to be recommended corresponding to the target user; A query data construction module, used to obtain the current status data of the target user, and to construct query data corresponding to each product to be recommended according to the current status data and the user description data; A product recommendation value acquisition module is used to input each query data into an online product recommendation network based on reinforcement learning, obtain a product recommendation value corresponding to each product to be recommended, and recommend products to the target user according to each product recommendation value; Among them, the product recommendation value acquisition module is specifically used to input each query data into the online product recommendation network based on reinforcement learning, and obtain the reward function data of the training network of the online product recommendation network; obtain the recommended action data of the target user corresponding to the current state data of the target user through the training network of the online product recommendation network; determine the updated state data of the target user according to the recommended action data through the training network of the online product recommendation network, and update the reward function data to obtain the updated reward function data; determine the target value function data according to the recommended action data, the updated state data and the updated reward function data through the training network of the online product recommendation network; when it is determined that the target value function data meets the convergence condition, update the update network of the online product recommendation network according to the target value function data; obtain the product recommendation value corresponding to each product to be recommended according to the updated update network of the online product recommendation network.

8. An electronic device, characterized in that: The electronic device comprises: at least one processor; and a memory communicatively connected to the at least one processor; wherein, The memory stores a computer program executable by the at least one processor, and the computer program is executed by the at least one processor so that the at least one processor can execute the product recommendation method according to any one of claims 1 to 6.

9. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores computer instructions, and the computer instructions are used to enable a processor to implement the product recommendation method according to any one of claims 1 to 6 when executed.

Images