Interactive recommendation system and method based on item-side fairness enhancement

Abstract

The present invention relates to an interactive recommendation system and method based on item-side fairness enhancement, and belongs to the field of big data artificial intelligence. The system is composed of two layers of intelligent agents, high and low, connected in series, and any intelligent agent is a reinforcement learning network of an Actor-Critic framework. The method adopts a hierarchical learning method, and divides the guidance process into two stages: macro-learning and micro-learning: in the macro-learning stage, the high-level intelligent agent formulates fairness-oriented goals based on multi-step feedback, while considering fairness in terms of items and user satisfaction, to guide the micro-learning stage; in the micro-learning stage, the low-level intelligent agent converts the goal into an executable operation customized for a single user, balances the goal and user satisfaction, and gradually shifts user preferences to the desired goal. The method of the present invention enhances the fairness of items in the recommendation system while minimizing interference with user satisfaction.

Description

Technical Field

[0001] The present invention relates to an interactive recommendation system and method based on enhanced fairness on the item side, belongs to the field of big data artificial intelligence, and in particular to interactive recommendation based on enhanced fairness on the item side. Background Art

[0002] Interactive recommender systems (IRS) are an essential tool for modern online platforms, improving user engagement by providing personalized content and relevant item recommendations in real time. These systems effectively track and model evolving user preferences, significantly improving user experience in diverse areas ranging from e-commerce to live streaming platforms. However, despite their effectiveness, recommendation systems still face significant challenges related to fairness. Among them, the data used to train these models is often unbalanced, with recommendation datasets characterized by sparsity and long-tail distribution. As a result, a small number of popular items receive disproportionate attention, while a large number of less popular (i.e., long-tail) items suffer from unfair exposure, reducing their visibility to users.

[0003] To address the challenge of improving the fairness of long-tail item exposure, researchers have proposed various methods. For example, causal inference techniques are used to mitigate popularity bias, fairness-aware regularization constraints are incorporated during model training, or re-ranking strategies are used to adjust recommendation outputs. For dynamic environments, existing work typically uses reinforcement learning (RL) to improve long-tail item exposure by designing effective reward functions or incorporating fairness constraints. These methods follow the same strategy, which is to promote the exposure of long-tail items by directly mixing them into recommendations. From the user's perspective, we call this strategy a passive fairness enforcement strategy because users passively endure the mixing of long-tail items. However, this passive fairness enforcement strategy seriously undermines the consistency between recommended items and user preferences, thereby reducing user stickiness and user satisfaction. Summary of the Invention

[0004] The present invention aims to overcome the shortcomings of the aforementioned prior art by providing an interactive recommendation system and method based on item-side fairness enhancement. This approach aims to maintain user satisfaction while enhancing item-side fairness within the interactive recommendation environment. Formally, unlike existing passive fairness enforcement strategies, which directly mix long-tail items into recommendations to boost their visibility, thereby compromising user satisfaction, the present invention proposes a hierarchical reinforcement learning framework based on an active fairness guidance strategy. In this framework, high-level agents generate fairness-oriented goals, and low-level agents select recommended items based on these goals to ensure user stickiness and satisfaction, ultimately improving item-side fairness within the interactive recommendation system.

[0005] To achieve the above purpose, for any user u∈U and any item i∈I, where the set of users is U and the set of items is I, the user behavior is the n×m×k-dimensional rating matrix R that statistics the interaction between users and items, T=[T1,T2,…,T k ] is a vector composed of k historical moments. The present invention provides an interactive recommendation system based on item-side fairness enhancement, combined with Figure 1 , characterized in that it is composed of two layers of intelligent agents connected in series, the high-level intelligent agent is a reinforcement learning network of a high-level Actor-Critic framework; the reinforcement learning network of the high-level Actor-Critic framework is composed of a high-level Actor network and a high-level Critic network; the high-level Actor network is a multi-layer perception network 1 (MLP1); the high-level Critic network is also a multi-layer perception network 2 (MLP2); similarly, the low-level intelligent agent is a reinforcement learning network of a low-level Actor-Critic framework; the reinforcement learning network of the low-level Actor-Critic framework is composed of a low-level Actor network and a low-level Critic network; the low-level Actor network is a multi-layer perception network 3 (MLP3); the low-level Critic network is also a multi-layer perception network 4 (MLP4).

[0006] The reinforcement learning network of the high-level Actor-Critic framework consists of five elements: tuples of ; among them, is the state of the high-level agent at time t, including the user's historical interaction behavior; is the action of the high-level agent at time t, which is the fairness goal and aims to guide the direction of user preference transfer; r t h is the reward of the high-level agent at time t; For high-level agents in state After the action Migrate to state The probability of γ1 is the discount factor of the high-level agent; the historical interaction behaviors of all users are abstractly represented as the state space S h ; The set of all recommended items I is regarded as the action space A h .

[0007] The reinforcement learning network of the low-level Actor-Critic framework consists of five elements tuples of ; among them, is the state of the low-level agent at time t, including the user's historical interaction behavior and the high-level agent's actions is the action of the lower-level agent at time t, and is the action of the higher-level agent at time t. Under the guidance of r, the vector of recommended items by the lower-level intelligent agent; t l is the reward of the lower-level agent at time t; For high-level agents in state After the action Migrate to state The probability of γ2 is the discount factor of the low-level agent; the historical interaction behaviors of all users and the high-level agent actions at the corresponding moment Abstractly represented as state space S l ; The set of all recommended items I is regarded as the action space A l .

[0008] Preferably, the discount factor of the reinforcement learning network of the high-level Actor-Critic framework and the reinforcement learning network of the low-level Actor-Critic framework are the same.

[0009] Applied to an interactive recommendation system based on item-side fairness enhancement, the present invention also provides an interactive recommendation method based on item-side fairness enhancement, comprising the following steps:

[0010] S1: Record the historical interaction behavior data between users and projects to construct training sets and test sets;

[0011] S2: Build an interactive recommendation system based on item-side fairness enhancement;

[0012] S3: Generate high-level agent states based on historical interaction behavior data;

[0013] S4: Use the high-level Actor network to generate a Gaussian distribution of the high-level agent's actions, and generate the high-level agent's actions according to the distribution;

[0014] S5: Generate low-level agent states based on historical interaction behavior data and high-level agent actions;

[0015] S6: Use the low-level Actor network to generate sampling probabilities and generate actions for the low-level agents based on the probabilities;

[0016] S7: Use the high-level critic network to evaluate the actions of the high-level agent and optimize and update the parameters of the high-level agent;

[0017] S8: Use the low-level critic network to evaluate the actions of the low-level agent and optimize and update the parameters of the low-level agent;

[0018] S9: Use the training set and test set to train and test the interactive recommendation system based on item-side fairness enhancement;

[0019] S10: Use the trained item-side fairness-enhanced interactive recommendation system to recommend items to users.

[0020] Furthermore, the historical interaction behavior data between users and projects described in step S1, including users, projects, ratings and moments, constitute a set D = {(u, i, r, t)}, which is split into a training set and a test set according to the proportion, where u∈U, i∈I, r∈R, and t∈T.

[0021] Furthermore, the state of the high-level agent at time t described in step S3 is: Where N is the time window size of the user's historical interaction, which is a positive integer; e i is the item encoding vector of the user interaction at time i; r i is the user's rating of the interaction item at time i; For concatenation connection.

[0022] Furthermore, the step S4 is specifically as follows:

[0023] S401: Generate Gaussian distribution variance σ using high-level Actor networks t and mean Right now Among them, MLP1(·) is the high-level Actor network;

[0024] S402: Sample items according to the probability of Gaussian distribution, determine the guidance target, and then indicate the direction of user preference transfer as the action of the high-level intelligent agent

[0025] Preferably, according to the different item distributions targeted by the recommendation system, the probability distribution type of the items needs to be determined by analyzing the historical interaction behavior data, and the Gaussian distribution needs to be adjusted accordingly.

[0026] Furthermore, the state of the lower-level agent at time t described in step S5 is: in, N is the time window size of the user's historical interaction, which is a positive integer; e i is the item encoding vector of the user interaction at time i; r i is the user's rating of the interaction item at time i; For concatenation connection.

[0027] Furthermore, the sampling probability described in step S6 MLP3(·) is the low-level Actor network used to estimate The probability distribution of the item set I in state.

[0028] Furthermore, the step S7 is specifically as follows:

[0029] S701: The state of the high-level agent at the current moment and the actions of high-level agents According to probability Select the state of the high-level agent at the next moment

[0030] S702: Use the high-level Critic network to calculate the state value of the high-level agent at the current moment and the next moment and in, MLP2(·) is the high-level critic network;

[0031] S703: Calculate the advantage function of the high-level agent in, M is the number of steps to guide the target update, that is, how often the high-level agent updates its parameters; r i a represents the user rating of the recommended item at time i; f is a hyperparameter; For fair rewards; N i is the number of users who give a positive rating to the project at time i; |U| is the total number of users;

[0032] S704: By calculating the gradient, optimize and adjust the advantage function The value of is the largest to update the parameters of the high-level agent.

[0033] Taking into account the stability and speed of the recommendation system, it is preferred that the number of steps M for guiding the target update is an integer greater than 1.

[0034] Furthermore, the step S8 is specifically as follows:

[0035] S801: The current state of the lower-level agent and the actions of lower-level agents According to probability Select the state of the high-level agent at the next moment

[0036] S802: Use the low-level Critic network to calculate the current and next state values ​​of the low-level agent and in, MLP4(·) is the low-level critic network;

[0037] S803: Calculate the advantage function of the low-level agent Among them, r t l =r t a +λ g ·r t g , r t a represents the user rating of the recommended item at time t; g is a hyperparameter; is the guidance reward, that is, the similarity between the guidance target and the item encoding vector of the user interaction at the next moment.

[0038] S804: By calculating the gradient, optimize and adjust the advantage function The value of is the largest to update the parameters of the lower-level agent.

[0039] The present invention provides an interactive recommendation system and method based on enhanced item-level fairness. This system employs a hierarchical learning approach, dividing the guidance process into two phases: macro-learning and micro-learning. In the macro-learning phase, a high-level agent formulates fairness-oriented goals based on multi-step feedback, taking into account both item-level fairness and user satisfaction. This serves to guide the micro-learning phase. In the micro-learning phase, a low-level agent converts these goals into actionable actions tailored to individual users, balancing goals and user satisfaction and gradually shifting user preferences toward desired goals. This method enhances item-level fairness in the recommendation system while minimizing interference with user satisfaction. BRIEF DESCRIPTION OF THE DRAWINGS

[0040] In order to make the purpose and technical solution of the present invention more clear, the present invention provides the following drawings for illustration:

[0041] Figure 1 This is an architecture diagram of an interactive recommendation system based on item-side fairness enhancement according to Example 1 of the present invention;

[0042] Figure 2 This is a flow chart of an interactive recommendation system method based on item-side fairness enhancement according to embodiment 2 of the present invention;

[0043] Figure 3 Graph showing comparative experimental results of the method of the present invention and the prior art method in Example 2 of the present invention; wherein, HRL4PFG is the method of the present invention, SAC4IR is the method of reference [1], and DNAIR is the method of reference [2];

[0044] Figure 4This is a comparison chart of the results of a single-round update (M=1) and a multi-round update (M=4) of the method of the present invention in Example 2 of the present invention; wherein HRL4PFG- corresponds to a single-round update and HRL4PFG corresponds to a multi-round update; the vertical axis is the sum of the rewards of the high-level agent and the rewards of the low-level agent;

[0045] Figure 5 This is a schematic diagram of the structure of an electronic device in Example 3 of the present invention. DETAILED DESCRIPTION

[0046] In order to make the purpose and technical solution of the present invention more clear, the present invention is described in detail below with reference to the accompanying drawings and embodiments.

[0047] Example 1: This example is illustrated using two real-world recommendation system datasets: KuaiEnv and KuaiRand, as shown in Table 1. Items are divided into two groups according to their popularity (measured by the number of interactions), where the top 20% with the highest number of interactions are classified as the popular group, and the remaining 80% are classified as the long-tail group. The data in Table 1 are sorted and symbolically defined as follows: any user u∈U, any item i∈I, where the set of users is U, the set of items is I, and user behavior is the n×m×k-dimensional rating matrix R that counts the interaction between users and items, T=[T1,T2,…,T k ] is a vector consisting of k historical moments.

[0048] Table 1 KuaiEnv and KuaiRand datasets

[0049] Dataset name use Number of users Number of projects Number of ratings KuaiEnv training set 7,176 10,728 12,530,806 KuaiEnv Test set 1,411 3,327 4,676,570 KuaiRand training set 26,285 7,551 1,436,609 KuaiRand Test set 27,285 7,583 1,186,059

[0050] Since traditional recommendation datasets are too sparse or lack necessary information, in order to improve item-side fairness and maximize long-term user satisfaction in an interactive recommendation setting, this embodiment provides an "interactive recommendation system based on item-side fairness enhancement."

[0051] Combine Figure 1, characterized in that it is composed of two layers of intelligent agents connected in series, the high-level intelligent agent is a reinforcement learning network of a high-level Actor-Critic framework; the reinforcement learning network of the high-level Actor-Critic framework is composed of a high-level Actor network and a high-level Critic network; the high-level Actor network is a multi-layer perception network 1 (MLP1); the high-level Critic network is also a multi-layer perception network 2 (MLP2); similarly, the low-level intelligent agent is a reinforcement learning network of a low-level Actor-Critic framework; the reinforcement learning network of the low-level Actor-Critic framework is composed of a low-level Actor network and a low-level Critic network; the low-level Actor network is a multi-layer perception network 3 (MLP3); the low-level Critic network is also a multi-layer perception network 4 (MLP4).

[0052] The reinforcement learning network of the high-level Actor-Critic framework consists of five elements: tuples of ; among them, is the state of the high-level agent at time t, including the user's historical interaction behavior; is the action of the high-level agent at time t, which is the fairness goal and aims to guide the direction of user preference transfer; r t h is the reward of the high-level agent at time t; For high-level agents in state After the action Migrate to state The probability of γ1 is the discount factor of the high-level agent; the historical interaction behaviors of all users are abstractly represented as the state space S h ; The set of all recommended items I is regarded as the action space A h .

[0053] The reinforcement learning network of the low-level Actor-Critic framework consists of five elements tuples of ; among them, is the state of the low-level agent at time t, including the user's historical interaction behavior and the high-level agent's actions is the action of the lower-level agent at time t, and is the action of the higher-level agent at time t. The vector of recommended items by the low-level agent under r t l is the reward of the lower-level agent at time t; For high-level agents in state After the action Migrate to state The probability of γ2 is the discount factor of the low-level agent; the historical interaction behaviors of all users and the high-level agent actions at the corresponding moment Abstractly represented as state space S l ; The set of all recommended items I is regarded as the action space A l .

[0054] In order to maintain the consistency of the intelligent agent, γ1=γ2 is set in this embodiment.

[0055] By using actions generated by high-level agents as guiding goals, low-level agents are guided in the direction of shifting user preferences, thereby driving item recommendations. During the macro-learning phase, high-level agents formulate fairness-oriented goals based on multi-step feedback, taking into account both item fairness and user satisfaction. These goals guide the micro-learning phase. During the micro-learning phase, low-level agents translate these goals into actionable actions tailored to individual users, balancing goals and user satisfaction, and gradually shifting user preferences toward the desired goal.

[0056] Example 2: In response to the scenario in Example 1, this example provides a method for an interactive recommendation system based on item-side fairness enhancement, namely, "an interactive recommendation method based on item-side fairness enhancement, comprising the following steps:

[0057] Step 1: Enter the historical interaction behavior data between users and projects to construct training sets and test sets.

[0058] The historical interaction behavior data between users and projects includes users, projects, ratings and moments, and the set is D = {(u, i, r, t)}, which is split into training set and test set according to the proportion, where u∈U, i∈I, r∈R, and t∈T.

[0059] Step 2: Build an interactive recommendation system based on item-side fairness enhancement.

[0060] Step 3: Generate high-level agent state based on historical interaction behavior data.

[0061] The state of the high-level agent at time t is: Where N is the time window size of the user's historical interaction, which is a positive integer; e i is the item encoding vector of the user interaction at time i; r i is the user's rating of the interaction item at time i; It is connected by concatenation.

[0062] Step 4: Use the high-level Actor network to generate the Gaussian distribution of the high-level agent's actions, and generate the high-level agent's actions according to the distribution. Specifically:

[0063] S401: Generate Gaussian distribution variance σ using high-level Actor networks t and mean Right now Among them, MLP1(·) is the high-level Actor network;

[0064] S402: Sample items according to the probability of Gaussian distribution, determine the guidance target, and then indicate the direction of user preference transfer as the action of the high-level intelligent agent

[0065] It should be noted that for a recommendation system such as taking a bus, its items follow a Poisson distribution. Depending on the item distribution targeted by the recommendation system, the probability distribution type of the items needs to be determined by analyzing historical interaction behavior data, and the Gaussian distribution needs to be adjusted accordingly.

[0066] Step 5: Generate the state of the low-level agent based on the historical interaction behavior data and the actions of the high-level agent.

[0067] The state of the low-level agent at time t is: in, N is the time window size of the user's historical interaction, which is a positive integer; e i is the item encoding vector of the user interaction at time i; r i is the user's rating of the interaction item at time i; For concatenation connection.

[0068] Step 6: Use the low-level Actor network to generate sampling probabilities and generate actions for the low-level agent based on the probabilities.

[0069] The sampling probability MLP3(·) is the low-level Actor network used to estimate The probability distribution of the item set I in state.

[0070] Step 7: Use the high-level critic network to evaluate the actions of the high-level agent and optimize and update the parameters of the high-level agent. Specifically:

[0071] S701: The state of the high-level agent at the current moment and the actions of high-level agents According to probability Select the state of the high-level agent at the next moment

[0072] S702: Use the high-level Critic network to calculate the state value of the high-level agent at the current moment and the next moment and in, MLP2(·) is the high-level critic network;

[0073] S703: Calculate the advantage function of the high-level agent in, M is the number of steps to guide the target update, that is, how often the high-level agent updates its parameters; r i a represents the user rating of the recommended item at time i; f is a hyperparameter; For fair rewards; N i is the number of users who give a positive rating to the project at time i; |U| is the total number of users;

[0074] S704: By calculating the gradient, optimize and adjust the advantage function The value of is the largest to update the parameters of the high-level agent.

[0075] Step 8: Use the low-level critic network to evaluate the actions of the low-level agent and optimize and update the parameters of the low-level agent. Specifically:

[0076] S801: The current state of the lower-level agent and the actions of lower-level agents According to probability Select the state of the high-level agent at the next moment

[0077] S802: Use the low-level Critic network to calculate the current and next state values ​​of the low-level agent and in, MLP4(·) is the low-level critic network;

[0078] S803: Calculate the advantage function of the low-level agent Among them, r t l =r t a λ g ·r t g , r t a represents the user rating of the recommended item at time t; g is a hyperparameter; is the guidance reward, that is, the similarity between the guidance target and the item encoding vector of the user interaction at the next moment.

[0079] S804: By calculating the gradient, optimize and adjust the advantage function The value of is the largest to update the parameters of the lower-level agent.

[0080] Step 9: Use the training set and test set to train and test the interactive recommendation system based on item-side fairness enhancement.

[0081] Step 10: Use the trained item-side fairness-enhanced interactive recommendation system to recommend items to users.

[0082] In order to better demonstrate the beneficial effects of the method of the present invention, this example was compared with the existing methods (SAC4IR method [1] and DNAIR method [2]). The experimental results are as follows: Figure 3 The experiment also sets a limit on the number of interactions a user can perform, no more than 50 times. It also implements a fairness-aware exit mechanism: if the recommendation model continuously recommends W popular groups of products to the user, the interaction is terminated.

[0083] Among them, in the HRL4PFG method of the present invention, for the KuaiRec dataset, M=4, λ f =0.1,λ g =0.3; for the KuaiRand dataset, set M=4, λ f =0.05,λ g =0.1.

[0084] [1] Xiaoyu Shi, Quanliang Liu, Hong Xie, Yanan Bai, and Mingsheng Shang. 2024. Maximum Entropy Policy for Long-Term Fairness in Interactive Recommender Systems. IEEE Transactions on Services Computing (2024).

[0085] [2] Xiaoyu Shi, Quanliang Liu, Hong Xie, Di Wu, Bo Peng, MingSheng Shang, and Defu Lian. 2023. Relieving popularity bias in interactive recommendation: Adiversity-novelty-aware reinforcement learning approach. ACM Transactions on Information Systems 42, 2(2023), 1–30.

[0086] pass Figure 3 This demonstrates the robustness of our method, HRL4PFG, under different environmental settings. When the maximum number of consecutive popular group items that can be recommended, W, is changed within the exit mechanism to 2, 3, 4, 5, and 6, HRL4PFG consistently achieves higher cumulative rewards than other methods. The cumulative reward is the sum of the total rewards of the recommendation system, which corresponds to the sum of the rewards of the high-level agent and the rewards of the low-level agents in our method.

[0087] At the same time, this embodiment also conducts comparative experiments on different number of steps M for guiding target update, namely single-round update (M=0) and multi-round update (M=4) of high-level agent parameters. The experimental results are as follows: Figure 4 As shown in the figure, it can be seen that the rewards for multiple rounds of updates of high-level agents are better and the recommendation system performs better.

[0088] Example 3: For the scenario of Example 1, Figure 5 A schematic diagram of an electronic device (90) that can be used to implement an embodiment of the present invention is shown. The electronic device is intended to represent various forms of digital computers, such as laptop computers, desktop computers, workstations, personal digital assistants, servers, blade servers, mainframe computers, and other suitable computers.

[0089] Electronic devices may also refer to various forms of mobile devices, such as personal digital assistants, cellular phones, smart phones, wearable devices (such as helmets, glasses, watches, etc.) and other similar computing devices. The components shown herein, their connections and relationships, and their functions are merely examples and are not intended to limit the implementation of the invention described and / or claimed herein.

[0090] like Figure 5As shown, the electronic device (90) includes at least one processor (91) and a memory connected to the at least one processor (91), such as a read-only memory (ROM) (92), a random access memory (RAM) (93), etc., wherein the memory stores a computer program that can be executed by the at least one processor, and the processor (91) can perform various appropriate actions and processes according to the computer program stored in the read-only memory (ROM) (92) or the computer program loaded from the storage unit (98) to the random access memory (RAM) (93). Various programs and data required for the operation of the electronic device (90) can also be stored in the RAM 43. The processor (91), ROM 42 and RAM 43 are connected to each other via a bus (94). An input / output (I / O) interface (95) is also connected to the bus (94).

[0091] Multiple components in the electronic device (90) are connected to the I / O interface (95), including: an input unit (96), such as a keyboard, a mouse, etc.; an output unit (97), such as various types of displays, speakers, etc.; a storage unit (98), such as a magnetic disk, an optical disk, etc.; and a communication unit (99), such as a network card, a modem, a wireless communication transceiver, etc. The communication unit (99) allows the electronic device (90) to exchange information / data with other devices via a computer network such as the Internet and / or various telecommunication networks.

[0092] The processor (91) can be a variety of general and / or specialized processing components with processing and computing capabilities. Some examples of the processor (91) include, but are not limited to, a central processing unit (CPU), a graphics processing unit (GPU), various specialized artificial intelligence (AI) computing chips, various processors that run machine learning model algorithms, a digital signal processor (DSP), and any appropriate processor, controller, microcontroller, etc. The processor (91) executes the various methods and processes described above, such as an interactive recommendation system based on item-side fairness enhancement.

[0093] In some embodiments, the interactive recommendation system based on item-side fairness enhancement can be implemented as a computer program, which is tangibly contained in a computer-readable storage medium, such as a storage unit (98). In some embodiments, part or all of the computer program can be loaded and / or installed on the electronic device (90) via the ROM (92) and / or the communication unit (99). When the computer program is loaded into the RAM (93) and executed by the processor (91), one or more steps of the interactive recommendation system based on item-side fairness enhancement described above can be performed. Alternatively, in other embodiments, the processor (91) can be configured to execute the interactive recommendation system based on item-side fairness enhancement by any other appropriate means (for example, by means of firmware).

[0094] Various embodiments of the systems and techniques described herein can be implemented in digital electronic circuit systems, integrated circuit systems, field programmable gate arrays (FPGAs), application specific integrated circuits (ASICs), application specific standard products (ASSPs), system-on-chip systems (SOCs), programmable logic devices (CPLDs), computer hardware, firmware, software, and / or combinations thereof. These various embodiments can include being implemented in one or more computer programs that are executable and / or interpreted on a programmable system that includes at least one programmable processor, which can be a special purpose or general purpose programmable processor that can receive data and instructions from a storage system, at least one input device, and at least one output device, and transmit data and instructions to the storage system, the at least one input device, and the at least one output device.

[0095] Computer programs for implementing the methods of the present invention may be written in any combination of one or more programming languages. These computer programs may be provided to a processor of a general-purpose computer, a special-purpose computer, or other programmable data processing device, such that when the computer program is executed by the processor, the functions / operations specified in the flowcharts and / or block diagrams are implemented. The computer program may be executed entirely on the machine, partially on the machine, as a stand-alone software package, partially on the machine and partially on a remote machine, or entirely on a remote machine or server.

[0096] In the context of the present invention, computer-readable storage media can be tangible media that can contain or store a computer program for use with an instruction execution system, device or equipment or used in combination with an instruction execution system, device or equipment. Computer-readable storage media can include but are not limited to electronic, magnetic, optical, electromagnetic, infrared or semiconductor systems, devices or equipment, or any suitable combination of the foregoing. Alternatively, computer-readable storage media can be machine-readable signal media. More specific examples of machine-readable storage media can include electrical connections based on one or more lines, portable computer disks, hard disks, random access memories (RAM), read-only memories (ROM), erasable programmable read-only memories (EPROM or flash memory), optical fibers, portable compact disk read-only memories (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination of the foregoing.

[0097] To provide interaction with a user, the systems and techniques described herein can be implemented on an electronic device having: a display device (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor) for displaying information to the user; and a keyboard and pointing device (e.g., a mouse or trackball) through which the user can provide input to the electronic device. Other types of devices can also be used to provide interaction with the user; for example, the feedback provided to the user can be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user can be received in any form (including acoustic input, voice input, or tactile input).

[0098] The systems and techniques described herein can be implemented in a computing system that includes back-end components (e.g., as a data server), or a computing system that includes middleware components (e.g., an application server), or a computing system that includes front-end components (e.g., a user computer with a graphical user interface or web browser through which a user can interact with implementations of the systems and techniques described herein), or a computing system that includes any combination of such back-end components, middleware components, or front-end components. The components of the system can be interconnected by any form or medium of digital data communication (e.g., a communication network). Examples of communication networks include: a local area network (LAN), a wide area network (WAN), a blockchain network, and the Internet.

[0099] A computing system may include clients and servers. The clients and servers are typically remote from each other and typically interact via a communication network. This client-server relationship arises through computer programs running on the respective computers, creating a client-server relationship. The server may be a cloud server, also known as a cloud computing server or cloud host. This server is a hosting product within the cloud computing service ecosystem that addresses the management difficulties and limited scalability of traditional physical hosting and VPS services.

[0100] Finally, it should be noted that the above preferred embodiments are only used to illustrate the technical solutions of the present invention and are not limiting. Although the present invention has been described in detail through the above preferred embodiments, those skilled in the art should understand that various changes can be made in form and details without departing from the scope defined by the claims of the present invention.

Claims

1. An interactive recommendation system based on item-side fairness enhancement is proposed. For any user u∈U and any item i∈I, where the set of users is U and the set of items is I, the user behavior is the n×m×k-dimensional rating matrix R that statistics the interaction between users and items, T=[T1,T2,…,T k ] is a vector composed of k historical moments, characterized by: It is composed of two layers of intelligent agents connected in series: the high-level intelligent agent is a reinforcement learning network of a high-level Actor-Critic framework; the reinforcement learning network of the high-level Actor-Critic framework is composed of a high-level Actor network and a high-level Critic network connected in series; the high-level Actor network is a multi-layer perception network 1; the high-level Critic network is also a multi-layer perception network 2; similarly, the low-level intelligent agent is a reinforcement learning network of a low-level Actor-Critic framework; the reinforcement learning network of the low-level Actor-Critic framework is composed of a low-level Actor network and a low-level Critic network connected in series; the low-level Actor network is a multi-layer perception network 3; the low-level Critic network is also a multi-layer perception network 4; The reinforcement learning network of the high-level Actor-Critic framework consists of five elements: tuples of ; among them, is the state of the high-level agent at time t, including the user's historical interaction behavior; is the action of the high-level agent at time t, which is the fairness goal and aims to guide the direction of user preference transfer; is the reward of the high-level agent at time t; For high-level agents in state After the action Migrate to state The probability of γ1 is the discount factor of the high-level agent; the historical interaction behaviors of all users are abstractly represented as the state space S h ; The set of all recommended items I is regarded as the action space A h ; The reinforcement learning network of the low-level Actor-Critic framework consists of five elements tuples of ; among them, is the state of the low-level agent at time t, including the user's historical interaction behavior and the high-level agent's actions is the action of the lower-level agent at time t, and is the action of the higher-level agent at time t. The vector of recommended items by the lower-level agent under ; is the reward of the lower-level agent at time t; For high-level agents in state After the action Migrate to state The probability of γ2 is the discount factor of the low-level agent; the historical interaction behaviors of all users and the high-level agent actions at the corresponding moment Abstractly represented as state space S l ; The set of all recommended items I is regarded as the action space A l .

2. The interactive recommendation system based on item-side fairness enhancement according to claim 1, characterized in that: The discount factor of the reinforcement learning network of the high-level Actor-Critic framework and the reinforcement learning network of the low-level Actor-Critic framework are the same.

3. The interactive recommendation method based on item-side fairness enhancement according to any one of claims 1 to 2, characterized in that: The steps include: S1: Record the historical interaction behavior data between users and projects to construct training sets and test sets; S2: Build an interactive recommendation system based on item-side fairness enhancement; S3: Generate high-level agent states based on historical interaction behavior data; S4: Use the high-level Actor network to generate a Gaussian distribution of the high-level agent's actions, and generate the high-level agent's actions according to the distribution; S5: Generate low-level agent states based on historical interaction behavior data and high-level agent actions; S6: Use the low-level Actor network to generate sampling probabilities and generate actions for the low-level agents based on the probabilities; S7: Use the high-level critic network to evaluate the actions of the high-level agent and optimize and update the parameters of the high-level agent; S8: Use the low-level critic network to evaluate the actions of the low-level agent and optimize and update the parameters of the low-level agent; S9: Use the training set and test set to train and test the interactive recommendation system based on item-side fairness enhancement; S10: Use the trained item-side fairness-enhanced interactive recommendation system to recommend items to users; The historical interaction behavior data between users and items in step S1, including users, items, ratings, and moments, is divided into a training set and a test set according to the proportion, where u∈U, i∈I, r∈R, and t∈T; The state of the high-level agent at time t described in step S3 is: Where N is the time window size of the user's historical interaction, which is a positive integer; e i is the item encoding vector of the user interaction at time i; r i is the user's rating of the interaction item at time i; For concatenation connection; The step S4 is specifically as follows: S401: Generate Gaussian distribution variance σ using high-level Actor networks t and mean Right now Among them, MLP1(·) is the high-level Actor network; S402: Sample items according to the probability of Gaussian distribution, determine the guidance target, and then indicate the direction of user preference transfer as the action of the high-level intelligent agent The state of the lower-level agent at time t described in step S5 is: in, N is the time window size of the user's historical interaction, which is a positive integer; e i is the item encoding vector of the user interaction at time i; r i is the user's rating of the interaction item at time i; For concatenation connection; The sampling probability described in step S6 MLP3(·) is the low-level Actor network used to estimate The probability distribution of the item set I in state; The step S7 is specifically as follows: S701: The state of the high-level agent at the current moment and the actions of high-level agents According to probability Select the state of the high-level agent at the next moment S702: Use the high-level Critic network to calculate the state value of the high-level agent at the current moment and the next moment and in, MLP2(·) is the high-level critic network; S703: Calculate the advantage function of the high-level agent in, M is the number of steps to guide the target update, that is, how often the high-level agent updates its parameters; represents the user rating of the recommended item at time i; f is a hyperparameter; For fair rewards; N i is the number of users who give a positive rating to the project at time i; |U| is the total number of users; S704: By calculating the gradient, optimize and adjust the advantage function The value of is the largest to update the parameters of the high-level agent; The step S8 is specifically as follows: S801: The current state of the lower-level agent and the actions of lower-level agents According to probability Select the state of the high-level agent at the next moment S802: Use the low-level Critic network to calculate the current and next state values ​​of the low-level agent and in, MLP4(·) is the low-level critic network; S803: Calculate the advantage function of the low-level agent in, represents the user rating of the recommended item at time t; g is a hyperparameter; The guidance reward is the similarity between the guidance target and the item encoding vector of the user interaction at the next moment; S804: By calculating the gradient, optimize and adjust the advantage function The value of is the largest to update the parameters of the lower-level agent.

4. The interactive recommendation method based on item-side fairness enhancement according to claim 3, characterized in that: Depending on the distribution of items targeted by the recommendation system, it is necessary to analyze historical interaction behavior data to determine the probability distribution type of the items, and the Gaussian distribution needs to be adjusted accordingly.

5. The interactive recommendation method based on item-side fairness enhancement according to claim 3, characterized in that: The number of steps M for updating the guidance target is an integer greater than 1.

6. An electronic device for the interactive recommendation method based on item-side fairness enhancement according to any one of claims 3 to 5, comprising at least one processor; and a memory communicatively connected to the at least one processor; wherein: The memory stores a computer program executed by the at least one processor, and the computer program is executed by the at least one processor so as to enable the at least one processor to perform the above-mentioned interactive recommendation method based on item-side fairness enhancement.

7. A computer-readable storage medium for the interactive recommendation method based on item-side fairness enhancement according to any one of claims 3 to 5, characterized in that: The computer-readable storage medium stores computer instructions, and the computer instructions are used to enable a processor to implement the above-mentioned interactive recommendation method based on item-side fairness enhancement when executed.

Images