A method for reducing information cocoons in interactive recommendation

Abstract

The present invention relates to a method for reducing information cocoons in interactive recommendations, and belongs to the field of big data artificial intelligence. It is implemented by a system for reducing information cocoons in interactive recommendations, which is composed of a reinforcement learning model in which environmental interactions are replaced by a series of Transformer networks. The method comprises the following steps: S1: recording historical interaction behavior data; S2: building a system for reducing information cocoons in interactive recommendations; S3: calculating user preference rewards; S4: calculating the diversity rewards at the current moment; S5: fusing to obtain the rewards corresponding to the reinforcement learning model at the current moment; S6: inputting the rewards and actions at the current moment into the Transformer network to represent the user's status at the current moment; S7: training and testing the system for reducing information cocoons in interactive recommendations; S8: recommending items to users. The method of the present invention can accurately capture user preferences, solve the information cocoon problem, and maintain the accuracy of recommendations.

Description

Technical Field

[0001] The present invention relates to a method for reducing information cocoons in interactive recommendations, belongs to the field of big data artificial intelligence, and in particular to a method for reducing information cocoons in interactive recommendations. Background Art

[0002] Interactive recommender systems (IRS) have dramatically changed the way individuals access information. Through personalized recommendations, these systems transform the arduous process of information search into a convenient recommendation experience. They analyze user behavioral data to tailor recommendations based on user preferences, thereby increasing user engagement. However, this focus on leveraging historical behavior often comes at the expense of content diversity, leading to the emergence of information cocoons. In this scenario, users are exposed to an increasingly narrow range of information, which reinforces their current preferences and potentially limits their exposure to diverse content and perspectives.

[0003] The emergence of information cocoons can be primarily explained from two perspectives. From the user's perspective, users tend to engage with content that aligns with their existing interests, which limits their exposure to diverse viewpoints. From the perspective of recommendation systems, algorithms that rely on user behavior and preferences can reinforce existing preferences and potentially exclude alternative and diverse content. This further leads to overexposure to specific content, narrowing the range of content users are exposed to.

[0004] There are two main measures to solve this problem: (1) By guiding users to form diversified preferences, as in the invention patent - A deep reinforcement learning recommendation method integrating user preference prediction (application number: 202111519219.9). (2) By increasing the diversity of recommendation results, as in the invention patent - Diversity-aware interactive recommendation method based on deep learning (application number: 202210726897.0). However, most of these types of research focus on studying information cocoons under static or one-time recommendation settings, which limits their ability to fundamentally solve the cumulative effect of overexposure. This is because user preferences and product popularity continue to evolve during the interaction process of the recommendation system. Therefore, these methods cannot maintain long-term user satisfaction.

[0005] [1]Guo, H.; Tang, R.; Ye, Y.; Li, Z.; and He, X. 2017. DeepFM: a factorization-machine based neural network for CTR prediction. arXiv preprint arXiv:1703.04247.

[0006] [2]Vaswani,A.;Shazeer,N.;Parmar,N.;Uszkoreit,J.;Jones,L.;Gomez,AN;Kaiser, and Polosukhin,I.2017.Attention is all you need.Advances in neuralinformation processing systems,30.

[0007] [3] Gao, C.; Wang, S.; Li, S.; Chen, J.; He, X.; Lei, W.; Li, B.; Zhang, Y.;

[0008] [4] Swaminathan, A.; and Joachims, T. 2015. Counterfactual risk minimization: Learning from logged bandit feedback. In International Conference on Machine Learning, 814–823. PMLR. Summary of the Invention

[0009] The present invention aims to overcome the shortcomings of the above-mentioned existing technologies and provide a method for reducing information cocoons in interactive recommendation. This method aims to solve the information cocoon problem in interactive recommendation environments under dynamic settings. By formatting the recommendation process as a sequential decision-making process, the overexposure effect can be tracked and modeled. Furthermore, long-term user satisfaction is viewed as a continuous balancing act between the relevance and diversity of recommendations throughout the interaction period, achieving accurate and diverse recommendations despite the changing user preferences and item popularity.

[0010] In order to achieve the above purpose, for the set of m users U=[u1,u2,…,u m ], the set of n items I=[i1,i2,…,i n] scenario, user behavior is the n×m×k-dimensional rating matrix R that statistics the interaction between users and items, T=[T1,T2,…,T k ] is a vector composed of k historical moments. The present invention provides a method for reducing information cocoons in interactive recommendations, which is implemented by a system for reducing information cocoons in interactive recommendations.

[0011] Combine Figure 1 , characterized in that the information cocoon reduction system in the interactive recommendation is composed of a reinforcement learning model in which the environmental interaction is replaced by a series of Transformer networks; the reinforcement learning model consists of four elements (s t ,a t ,r t ,π(s t |a t )) of the tuple; among them, s t ∈S is the state corresponding to time t; a t ∈A is the action corresponding to time t; r t is the reward corresponding to time t, π(s t |a t ) is in state s t Next, make an action a t The probability of t is t, γ is the discount factor; the state of all users at all times is abstractly represented as the state space S; the set of all recommended items I is regarded as the action space A; the Transformer network uses the action a generated by the reinforcement learning model at time t t and reward r t Generate the state s corresponding to time t+1 t+1 Input into the reinforcement learning model to achieve state update, that is, for the prediction and understanding of behavioral intentions.

[0012] A method for reducing information cocoons in interactive recommendation, comprising the following steps:

[0013] S1: Record the historical interaction behavior data between users and projects to construct training sets and test sets;

[0014] S2: Build a system to reduce information cocoons in interactive recommendations;

[0015] S3: Calculate user personality factors and environmental factors based on historical interaction behavior data, integrate them, and obtain user preference rewards;

[0016] S4: Calculate the Euclidean distance between interactions based on historical interaction behavior data to obtain the diversity reward at the current moment;

[0017] S5: Fusion of the current user preference reward and the current diversity reward to obtain the current reward of the reinforcement learning model.

[0018] S6: Input the current reward and action into the Transformer network to represent the user's current state as the input of the reinforcement learning model;

[0019] S7: Use the training set and test set to train and test an information cocoon reduction system in interactive recommendation;

[0020] S8: Use a trained information cocoon reduction system in interactive recommendation to recommend items to users.

[0021] Furthermore, the historical interaction behavior data between users and projects described in step S1, including users, projects, ratings and moments, constitute a set D = {(u, i, r, t)}, which is split into a training set and a test set according to the proportion, where u∈U, i∈I, r∈R, and t∈T.

[0022] Furthermore, the step S3 is specifically as follows:

[0023] S301: Calculate the user's unique preference for the item {MF(u,i)|u∈U,i∈I} for each user and item;

[0024] S302: Calculate the environmental factors of the popularity of each project among all users {EF i |i∈I};

[0025] S303: Calculate the herd factor of each user that follows the changes in public interests {C u |u∈U};

[0026] S304: Using the herd factor integration to obtain the user preference reward for each user for each project

[0027] Furthermore, the calculation of the personality factor PF of user u for item i in step S3 adopts a matrix factorization model (MF), i.e., PF = MF(u, i). This is specifically implemented according to the method of the following reference [1].

[0028] Furthermore, the environmental factors described in step S3 are calculated as the average score EF of the recommended item i. i ,Right now Among them, (u*,i*,r*) is an element in the historical interaction behavior data set D, is the indicator function.

[0029] Furthermore, the user preference reward of user u for item i in step S3 is

[0030] Preferably, the matrix decomposition model is trained using user preference rewards, and the loss function used is: Where σ is the sigmoid function.

[0031] Furthermore, the diversity reward of user u for item i at the current time t in step S4 is: in, tl is the time before the current time; il is the item that user u interacted with at time tl; dist(i,j) represents the Euclidean distance between item i and item j; threshold is the threshold, which is a hyperparameter obtained through training;

[0032] Preferably, the Euclidean distance in step S4 can be replaced by cosine similarity.

[0033] Furthermore, the step S5 is specifically as follows: using Fusion of the current moment’s user preference reward and the current moment’s diversity reward yields the current moment’s reward r for the reinforcement learning model. t , where 0≤α≤1 is the hyperparameter to be learned.

[0034] Preferably, the step S6 is specifically as follows:

[0035] S601: Extract action a t The eigenvector e a ;

[0036] S602: Use a layer of neural network to calculate the current moment reward and feature vector e a The fusion of features g t =σ(W·concat(r t ,e a )+b), where the weight matrix W and bias b are to be learned, concat() is the concat connection operation, and σ is the sigmoid function;

[0037] S603: The eigenvector e a and feature g t Input into the Transformer network to calculate the user's current state in, Represents element-wise multiplication, and Transformer( ) represents the Transformer network mapping.

[0038] An electronic device comprising at least one processor; and a memory communicatively connected to the at least one processor; wherein,

[0039] The memory stores a computer program executed by the at least one processor, and the computer program is executed by the at least one processor so that the at least one processor can execute the above-mentioned method for reducing information cocoons in interactive recommendations.

[0040] Finally, the present invention also discloses a computer-readable storage medium, which stores computer instructions, and the computer instructions are used to enable a processor to implement the above-mentioned method for reducing information cocoons in interactive recommendations when executed.

[0041] The present invention provides a method for reducing information cocoons in interactive recommendations, implemented by a system for reducing information cocoons in interactive recommendations. This method improves the quality of recommended items by integrating user preference rewards and diversity rewards. Furthermore, the system utilizes a Transformer network to accurately express user states using rewards and actions, thereby more accurately capturing user preferences and resolving the information cocoon problem while maintaining recommendation accuracy. In particular, the diversity reward, combined with the slope characteristics of the sigmoid function, enhances the reward's sensitivity to item diversity. BRIEF DESCRIPTION OF THE DRAWINGS

[0042] In order to make the purpose and technical solution of the present invention more clear, the present invention provides the following drawings for illustration:

[0043] Figure 1 This is an architecture diagram of an information cocoon reduction system in an interactive recommendation of the present invention;

[0044] Figure 2 This is a flow chart of a method for eliminating information cocoons in interactive recommendation in Example 1 of the present invention; the arrows indicate the direction of data flow;

[0045] Figure 3 The figure shows the comparative experimental results of the method of the present invention and the existing method on the KuaiEnv and KuaiRand datasets in Example 1 of the present invention; wherein the horizontal axis is the number of iterations, IPS is the method of reference [4], CIRS is the method of reference [3], and DDIR is the method of the present invention;

[0046] Figure 4DDIR wo comformity refers to the case where the reward in the method of the present invention is only the diversity reward, DDIR wodiversity refers to the case where the reward in the method of the present invention is only the user preference reward, and DDIR refers to the case where the reward in the method of the present invention is fused reward.

[0047] Figure 5 The cumulative satisfaction comparison results on the KuaiEnv and KuaiRand datasets in Example 1 of the present invention are shown in Figure 1. CIRS is the method in reference [3], and DDIR is the method of the present invention.

[0048] Figure 6 This is a schematic diagram of the structure of an electronic device in Example 2 of the present invention. DETAILED DESCRIPTION

[0049] In order to make the purpose and technical solution of the present invention more clear, the present invention is described in detail below with reference to the accompanying drawings and embodiments.

[0050] Example 1: Since traditional recommendation datasets are too sparse or lack necessary information, in order to improve user experience, solve the information cocoon problem, and maintain the accuracy of recommendations, this example uses two real-world recommendation system datasets: KuaiEnv and KuaiRand, as shown in Table 1, as examples.

[0051] Table 1 KuaiEnv and KuaiRand datasets

[0052] Dataset name use Number of users Number of projects Number of ratings KuaiEnv training set 7,176 10,728 12,530,806 KuaiEnv Test set 1,411 3,327 4,676,570 KuaiRand training set 26,285 7,551 1,436,609 KuaiRand Test set 27,285 7,583 1,186,059

[0053] In order to make the environment reflect the user's reaction to the filter bubble, the boredom and exit mechanism set in reference [3] is introduced here: for the first N rounds of interaction, if the strategy recommends more than M items with the same category, the interactive recommendation system terminates.

[0054] This embodiment provides a method for reducing information cocoons in interactive recommendations, which is implemented by a system for reducing information cocoons in interactive recommendations. The method includes the following steps:

[0055] Step 1: Enter the historical interaction behavior data of users and projects into the dataset KuaiEnv or KuaiRand, and construct the training set and test set, as shown in Table 1.

[0056] The historical interaction behavior data between users and projects includes users, projects, ratings and moments, and the set is D = {(u, i, r, t)}, which is split into training set and test set according to the proportion, where u∈U, i∈I, r∈R, and t∈T.

[0057] Step 2: Build a system to reduce information cocoons in interactive recommendations.

[0058] like Figure 1 As shown, the information cocoon reduction system in the interactive recommendation is composed of replacing the environmental interaction in the reinforcement learning model with a series of Transformer networks; the reinforcement learning model consists of four elements (s t ,a t ,r t ,π(s t |a t )) of the tuple; among them, s t ∈S is the state corresponding to time t; a t ∈A is the action corresponding to time t; r t is the reward corresponding to time t, π(s t |a t ) is in state s t Next, make an action a t The probability of t is t, γ is the discount factor; the state of all users at all times is abstractly represented as the state space S; the set of all recommended items I is regarded as the action space A; the Transformer network uses the action a generated by the reinforcement learning model at time t t and reward r t Generate the state s corresponding to time t+1 t+1 Input into the reinforcement learning model to achieve state update, that is, for the prediction and understanding of behavioral intentions.

[0059] Step 3: Calculate user personality factors and environmental factors based on historical interaction behavior data, integrate them, and obtain user preference rewards. Specifically:

[0060] S301: Calculate the user's unique preference for the item {MF(u,i)|u∈U,i∈I} for each user and item;

[0061] S302: Calculate the environmental factors of the popularity of each project among all users {EF i |i∈I};

[0062] S303: Calculate the herd factor of each user that follows the public interest changes {C u |u∈U};

[0063] S304: Using the herd factor integration to obtain the user preference reward for each user for each project

[0064] The calculation of the personality factor PF of user u for item i adopts the matrix factorization model (MF), that is, PF = MF (u, i). It is specifically implemented according to the method of the following reference [1].

[0065] The environmental factor is calculated as the average score EF of the recommended item i i ,Right now Among them, (u*,i*,r*) is an element in the historical interaction behavior data set D, is the indicator function.

[0066] The user preference reward of user u for item i is

[0067] Preferably, the matrix decomposition model is trained using user preference rewards, and the loss function used is: Where σ is the sigmoid function.

[0068] Step 4: Calculate the Euclidean distance between interactions based on historical interaction behavior data to obtain the diversity reward at the current moment.

[0069] The diversity reward of user u for item i at the current time t is: in, tl is the time before the current time; il is the item that user u interacted with at time tl; dist(i,j) represents the Euclidean distance between item i and item j; threshold is the threshold, which is a hyperparameter obtained through training;

[0070] Preferably, the Euclidean distance in step 4 can be replaced by cosine similarity.

[0071] Step 5: Combine the current user preference reward and the current diversity reward to obtain the reward corresponding to the reinforcement learning model at the current moment.

[0072] Specifically: using Fusion of the current moment’s user preference reward and the current moment’s diversity reward yields the current moment’s reward r for the reinforcement learning model. t , where 0≤α≤1 is the hyperparameter to be learned. In this embodiment, α is set to 0.5.

[0073] It should be noted that when conducting an ablation experiment, step three or step four can be omitted, and the diversity reward or user preference reward at the current moment can be directly used as the reward corresponding to the current moment in step five.

[0074] Step 6: Input the current reward and action into the Transformer network to represent the user's current state as the input of the reinforcement learning model. Specifically:

[0075] S601: Extract action a t The eigenvector e a ;

[0076] S602: Use a layer of neural network to calculate the current moment reward and feature vector e a The fusion of features g t =σ(W·concat(r t ,e a )+b), where the weight matrix W and bias b are to be learned, concat() is the concat connection operation, and σ is the sigmoid function;

[0077] S603: The eigenvector e a and feature g t Input into the Transformer network to calculate the user's current state in, Represents element-wise multiplication, and Transformer( ) represents the Transformer network mapping.

[0078] For better representation, the state in interactive recommendation should contain as much important information as possible. In this embodiment, the Transformer in reference [2] is used to construct the state tracker.

[0079] Step 7: Use the training set and test set to train and test an information cocoon reduction system in interactive recommendation.

[0080] Step 8: Use a trained interactive recommendation information cocoon reduction system to recommend items to users.

[0081] In order to better demonstrate the beneficial effects of the method of the present invention, this embodiment conducts comparative experiments based on the KuaiEnv and KuaiRand datasets with existing methods, namely the IPS method of reference [4] and the CIRS method of reference [3]. For all methods, the basic model used is the DeepFM model with fixed parameters. The key parameters are the same parameters that are tuned in the KuaiEnv and KuaiRand datasets. The exit mechanism parameters are: M = 1, N = 1, and the maximum number of rounds is 100. The experimental results are shown in Figure 2. Figure 3 As shown in Figure 2, all results are average metrics of 100 interaction trajectories. It can be seen that the proposed method (DDIR method) outperforms all other methods on all datasets, except for the short-term indicator (i.e., single-round satisfaction) of KuaiEnv. This is because, compared with static methods, the DDIR method targets the long-term satisfaction of users.

[0082] At the same time, this embodiment also conducts ablation experiments on the KuaiEnv and KuaiRand datasets, that is, considering the impact of diversity rewards and user preference rewards on user satisfaction. The results are as follows: Figure 4 As shown in the figure, both rewards contribute significantly to improving the performance of the recommendation system. In the KuaiEnv dataset, the user preference reward contributes more significantly than the diversity reward. In contrast, on the KuaiRand dataset, the diversity reward contributes more significantly. Combining the two rewards can further improve the performance of the recommendation system.

[0083] Finally, for the exit mechanism, we conducted a study on customer satisfaction accumulation. This example also conducted a comparative experiment with the CIRS method in reference [3] on the KuaiEnv and KuaiRand datasets. The results are as follows: Figure 5 As shown, it can be seen that the customer cumulative satisfaction of the recommendation results of the method of the present invention (DDIR method) is always much higher than that of the CIRS method.

[0084] In summary, it can be seen that the method of the present invention can ensure the accuracy of recommendations and improve customer satisfaction in the long term.

[0085] Example 2: For the scenario of Example 1, Figure 6 A schematic diagram of an electronic device (90) that can be used to implement an embodiment of the present invention is shown. The electronic device is intended to represent various forms of digital computers, such as laptop computers, desktop computers, workstations, personal digital assistants, servers, blade servers, mainframe computers, and other suitable computers.

[0086] Electronic devices may also refer to various forms of mobile devices, such as personal digital assistants, cellular phones, smart phones, wearable devices (such as helmets, glasses, watches, etc.) and other similar computing devices. The components shown herein, their connections and relationships, and their functions are merely examples and are not intended to limit the implementation of the invention described and / or claimed herein.

[0087] like Figure 6 As shown, the electronic device (90) includes at least one processor (91) and a memory connected to the at least one processor (91), such as a read-only memory (ROM) (92), a random access memory (RAM) (93), etc., wherein the memory stores a computer program that can be executed by the at least one processor, and the processor (91) can perform various appropriate actions and processes according to the computer program stored in the read-only memory (ROM) (92) or the computer program loaded from the storage unit (98) to the random access memory (RAM) (93). Various programs and data required for the operation of the electronic device (90) can also be stored in the RAM 43. The processor (91), ROM 42 and RAM 43 are connected to each other via a bus (94). An input / output (I / O) interface (95) is also connected to the bus (94).

[0088] Multiple components in the electronic device (90) are connected to the I / O interface (95), including: an input unit (96), such as a keyboard, a mouse, etc.; an output unit (97), such as various types of displays, speakers, etc.; a storage unit (98), such as a magnetic disk, an optical disk, etc.; and a communication unit (99), such as a network card, a modem, a wireless communication transceiver, etc. The communication unit (99) allows the electronic device (90) to exchange information / data with other devices via a computer network such as the Internet and / or various telecommunication networks.

[0089] The processor (91) can be any general-purpose and / or specialized processing component with processing and computing capabilities. Some examples of the processor (91) include, but are not limited to, a central processing unit (CPU), a graphics processing unit (GPU), various specialized artificial intelligence (AI) computing chips, various processors that run machine learning model algorithms, a digital signal processor (DSP), and any appropriate processor, controller, microcontroller, etc. The processor (91) executes the various methods and processes described above, such as a method for reducing information cocoons in interactive recommendations.

[0090] In some embodiments, a method for reducing information cocoons in interactive recommendations can be implemented as a computer program, which is tangibly contained in a computer-readable storage medium, such as a storage unit (98). In some embodiments, part or all of the computer program can be loaded and / or installed on the electronic device (90) via the ROM (92) and / or the communication unit (99). When the computer program is loaded into the RAM (93) and executed by the processor (91), one or more steps of the method for reducing information cocoons in interactive recommendations described above can be performed. Alternatively, in other embodiments, the processor (91) can be configured to perform a method for reducing information cocoons in interactive recommendations by any other appropriate means (for example, by means of firmware).

[0091] Various embodiments of the systems and techniques described herein can be implemented in digital electronic circuit systems, integrated circuit systems, field programmable gate arrays (FPGAs), application specific integrated circuits (ASICs), application specific standard products (ASSPs), system-on-chip systems (SOCs), programmable logic devices (CPLDs), computer hardware, firmware, software, and / or combinations thereof. These various embodiments can include being implemented in one or more computer programs that are executable and / or interpreted on a programmable system that includes at least one programmable processor, which can be a special purpose or general purpose programmable processor that can receive data and instructions from a storage system, at least one input device, and at least one output device, and transmit data and instructions to the storage system, the at least one input device, and the at least one output device.

[0092] Computer programs for implementing the methods of the present invention may be written in any combination of one or more programming languages. These computer programs may be provided to a processor of a general-purpose computer, a special-purpose computer, or other programmable data processing device, such that when the computer program is executed by the processor, the functions / operations specified in the flowcharts and / or block diagrams are implemented. The computer program may be executed entirely on the machine, partially on the machine, as a stand-alone software package, partially on the machine and partially on a remote machine, or entirely on a remote machine or server.

[0093] In the context of the present invention, computer-readable storage media can be tangible media that can contain or store a computer program for use with an instruction execution system, device or equipment or used in combination with an instruction execution system, device or equipment. Computer-readable storage media can include but are not limited to electronic, magnetic, optical, electromagnetic, infrared or semiconductor systems, devices or equipment, or any suitable combination of the foregoing. Alternatively, computer-readable storage media can be machine-readable signal media. More specific examples of machine-readable storage media can include electrical connections based on one or more lines, portable computer disks, hard disks, random access memories (RAM), read-only memories (ROM), erasable programmable read-only memories (EPROM or flash memory), optical fibers, portable compact disk read-only memories (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination of the foregoing.

[0094] To provide interaction with a user, the systems and techniques described herein can be implemented on an electronic device having: a display device (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor) for displaying information to the user; and a keyboard and pointing device (e.g., a mouse or trackball) through which the user can provide input to the electronic device. Other types of devices can also be used to provide interaction with the user; for example, the feedback provided to the user can be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user can be received in any form (including acoustic input, voice input, or tactile input).

[0095] The systems and techniques described herein can be implemented in a computing system that includes back-end components (e.g., as a data server), or a computing system that includes middleware components (e.g., an application server), or a computing system that includes front-end components (e.g., a user computer with a graphical user interface or web browser through which a user can interact with implementations of the systems and techniques described herein), or a computing system that includes any combination of such back-end components, middleware components, or front-end components. The components of the system can be interconnected by any form or medium of digital data communication (e.g., a communication network). Examples of communication networks include: a local area network (LAN), a wide area network (WAN), a blockchain network, and the Internet.

[0096] A computing system may include clients and servers. The clients and servers are typically remote from each other and typically interact via a communication network. This client-server relationship arises through computer programs running on the respective computers, creating a client-server relationship. The server may be a cloud server, also known as a cloud computing server or cloud host. This server is a hosting product within the cloud computing service ecosystem that addresses the management difficulties and limited scalability of traditional physical hosting and VPS services.

[0097] Finally, it should be noted that the above preferred embodiments are only used to illustrate the technical solutions of the present invention and are not limiting. Although the present invention has been described in detail through the above preferred embodiments, those skilled in the art should understand that various changes can be made in form and details without departing from the scope defined by the claims of the present invention.

Claims

1. A method for eliminating information cocoons in interactive recommendation, characterized in that: The system is implemented by an information cocoon reduction system in an interactive recommendation, wherein the information cocoon reduction system in an interactive recommendation is composed of a reinforcement learning model in which the environment interaction is replaced by a series of Transformer networks; the reinforcement learning model is composed of four elements (s t ,a t ,r t ,π(s t |a t )) of the tuple; among them, s t ∈S is the state corresponding to time t; a t ∈A is the action corresponding to time t; r t is the reward corresponding to time t, π(s t |a t ) is in state s t Next, make an action a t The probability of t is t, γ is the discount factor; the state of all users at all times is abstractly represented as the state space S; the set of all recommended items I is regarded as the action space A; the Transformer network uses the action a generated by the reinforcement learning model at time t t and reward r t Generate the state s corresponding to time t+1 t+1 Input into the reinforcement learning model to achieve state update, that is, for prediction and understanding of behavioral intentions; The method comprises the following steps: S1: Record the historical interaction behavior data between users and projects to construct training sets and test sets; S2: Build a system to reduce information cocoons in interactive recommendations; S3: Calculate user personality factors and environmental factors based on historical interaction behavior data, integrate them, and obtain user preference rewards; S4: Calculate the Euclidean distance between interactions based on historical interaction behavior data to obtain the diversity reward at the current moment; S5: Fusion of the current user preference reward and the current diversity reward to obtain the current reward of the reinforcement learning model. S6: Input the current reward and action into the Transformer network to represent the user's current state as the input of the reinforcement learning model; S7: Use the training set and test set to train and test an information cocoon reduction system in interactive recommendation; S8: Use a trained interactive recommendation information cocoon reduction system to recommend items to users; The historical interaction behavior data between users and items in step S1, including users, items, ratings, and moments, is divided into a training set and a test set according to the proportion, where u∈U, i∈I, r∈R, and t∈T; The step S3 is specifically as follows: S301: Calculate the user's unique preference for the item for each user and item, namely, the personality factor ∈MF(u,i)|u∈U,i∈I}; S302: Calculate the environmental factors of the popularity of each project among all users {EF i |i∈I}; S303: Calculate the herd factor of each user that follows the public interest changes {C u |u∈U}; S304: Using the herd factor integration to obtain the user preference reward for each user for each project The calculation of the personality factor PF of user u for item i in step S3 adopts the matrix factorization model (MF), that is, PF = MF (u, i); The calculation of the environmental factors in step S3 is the average score EF of the recommended item i i ,Right now Among them, (u*,i*,r*) is an element in the historical interaction behavior data set D, is the indicator function; The user preference reward of user u for item i in step S3 is The diversity reward of user u for item i at the current time t in step S4 is: in, tl is the time before the current time; il is the item that user u interacted with at time tl; dist(i,j) represents the Euclidean distance between item i and item j; threshold is the threshold, which is a hyperparameter obtained through training; 2. The method for eliminating information cocoons in interactive recommendation according to claim 1, characterized in that: The matrix decomposition model is trained using user preference rewards, and the loss function used is: Where σ is the sigmoid function.

3. The method for eliminating information cocoons in interactive recommendation according to claim 1, characterized in that: The step S6 is specifically as follows: S601: Extract action a t The eigenvector e a ; S602: Use a layer of neural network to calculate the current moment reward and feature vector e a The fusion of features g t =σ(W·concat(r t ,e a )+b), where the weight matrix W and bias b are to be learned, concat() is the concat operation, and σ is the sigmoid function; S603: The eigenvector e a and feature g t Input into the Transformer network to calculate the user's current state in, Represents element-wise multiplication, and Transformer( ) represents the Transformer network mapping.

4. The method for eliminating information cocoons in interactive recommendation according to claim 1, characterized in that: The Euclidean distance in step S4 may be replaced by cosine similarity.

5. The method for eliminating information cocoons in interactive recommendation according to claim 1, characterized in that: The reward r corresponding to the current moment of the reinforcement learning model is obtained by integrating the user preference reward and the diversity reward at the current moment in step S5. t , which is Among them, 0≤α≤1 is the hyperparameter to be learned.

6. An electronic device according to any one of claims 1 to 5, comprising at least one processor; and a memory communicatively connected to the at least one processor; wherein: The memory stores a computer program executed by the at least one processor, and the computer program is executed by the at least one processor so that the at least one processor can execute the above-mentioned method for reducing information cocoons in interactive recommendations.

7. A computer-readable storage medium according to any one of claims 1 to 5, characterized in that: The computer-readable storage medium stores computer instructions, and the computer instructions are used to enable the processor to implement the above-mentioned method for reducing information cocoons in interactive recommendations when executed.

Images