Information cocoon house reduction method in interactive recommendation

Abstract

The invention relates to an information cocoon house reduction method in interactive recommendation, and belongs to the field of big data artificial intelligence. The method is realized by an information cocoon house reduction system in interactive recommendation, and the system is formed by connecting a Transform network in series instead of environment interaction in a reinforcement learning model. The method comprises the following steps: S1, inputting historical interaction behavior data; s2, establishing an information cocoon house reduction system in interactive recommendation; s3, calculating a user preference reward; s4, calculating a diversity reward at the current moment; s5, fusing to obtain a reward corresponding to the current moment of the reinforcement learning model; s6, rewards and actions at the current moment are input into a Transform network, and the state of the user at the current moment is represented; s7, training and testing an information cocoon house reduction system in interactive recommendation; and S8, performing project recommendation on the user. According to the method, the user preference can be accurately captured, the information cocoon house problem is solved, and meanwhile the recommendation accuracy is kept.

Description

Technical Field

[0001] The present invention relates to a method for reducing information cocoons in interactive recommendations, belongs to the field of big data artificial intelligence, and in particular to a method for reducing information cocoons in interactive recommendations. Background Art

[0002] Interactive recommender systems (IRS) have dramatically changed the way individuals access information. Through personalized recommendations, these systems transform the difficult process of information search into a convenient recommendation experience. They improve user engagement by analyzing users' behavioral data and tailoring recommendations to their preferences. However, this focus on leveraging historical behavior often comes at the expense of content diversity, leading to the emergence of information cocoons. In this case, users are exposed to an increasingly narrow range of information that reinforces their current preferences, potentially limiting their exposure to diverse content and perspectives.

[0003] The reasons for the emergence of information cocoons can be viewed from two main aspects. From the user's perspective, users tend to engage with content that is consistent with their existing interests, which limits their exposure to different viewpoints. From the perspective of recommendation systems, recommendation algorithms that rely on user behavior and preferences may reinforce existing preferences and potentially exclude alternative and diverse content. This further leads to overexposure to specific content, thereby narrowing the range of content that users are exposed to.

[0004] There are two main measures to solve this problem: (i) By guiding users to form diversified preferences, as in the invention patent - a deep reinforcement learning recommendation method integrating user preference prediction (application number: 202111519219.9). (ii) By increasing the diversity of recommendation results, as in the invention patent - a diversity-aware interactive recommendation method based on deep learning (application number: 202210726897.0). However, most of these types of studies focus on studying information cocoons under static or one-time recommendation settings, which limits their ability to fundamentally solve the cumulative effect of overexposure. This is because user preferences and product popularity continue to evolve during the interaction process of the recommendation system. Therefore, these methods cannot maintain long-term user satisfaction.

[0005] [1]Guo, H.; Tang, R.; Ye, Y.; Li, Z.; and He, X. 2017. DeepFM: a factorization-machine based neural network for CTR prediction. arXiv preprint arXiv:1703.04247.

[0006] [2]Vaswani,A.;Shazeer,N.;Parmar,N.;Uszkoreit,J.;Jones,L.;Gomez,AN;Kaiser, and Polosukhin,I.2017.Attention is all you need.Advances in neuralinformation processing systems,30.

[0007] [3] Gao, C.; Wang, S.; Li, S.; Chen, J.; He, X.; Lei, W.; Li, B.; Zhang, Y.;

[0008] [4] Swaminathan, A.; and Joachims, T. 2015. Counterfactual risk minimization: Learning from logged bandit feedback. In International Conference on Machine Learning, 814–823. PMLR. Summary of the invention

[0009] The purpose of the present invention is to overcome the shortcomings of the above-mentioned prior art and provide a method for reducing information cocoons in interactive recommendation, which is intended to solve the problem of information cocoons in interactive recommendation environments under dynamic settings. By formatting the recommendation process as a sequential decision process, the overexposure effect can be tracked and modeled; in addition, long-term user satisfaction is regarded as a continuous balancing act between the relevance and diversity of recommendations during the entire interaction period, so as to achieve accuracy and diversity of recommendations in the process of changing user preferences and item popularity.

[0010] To achieve the above purpose, for the set of m users U = [u1,u2,…,u m ], the set of n items I = [i1,i2,…,i n] scenario, user behavior is the n×m×k-dimensional rating matrix R that counts the interaction between users and items, T=[T1,T2,…,T k ] is a vector composed of k historical moments. The present invention provides a method for reducing information cocoons in interactive recommendations, which is implemented by a system for reducing information cocoons in interactive recommendations.

[0011] Combination Figure 1 , characterized in that the information cocoon reduction system in the interactive recommendation is composed of replacing the environmental interaction in the reinforcement learning model with a series of Transformer networks; the reinforcement learning model consists of four elements (s t ,a t ,r t ,π(s t |a t )) tuple; among them, s t ∈S is the state corresponding to time t; a t ∈A is the action corresponding to time t; r t is the reward corresponding to time t, π(s t |a t ) is in state s t Next action a t The probability of t is t, γ is the discount factor; the state of all users at all times is abstractly represented as the state space S; the set of all recommended items I is regarded as the action space A; the Transformer network uses the action a generated by the reinforcement learning model at time t t and reward r t Generate the state s corresponding to time t+1 t+1 Input into the reinforcement learning model to update the state, which is used to predict and understand behavioral intentions.

[0012] A method for reducing information cocoons in interactive recommendation, comprising the following steps:

[0013] S1: Input historical interaction behavior data between users and projects and construct training and test sets;

[0014] S2: Build a system to reduce information cocoons in interactive recommendations;

[0015] S3: Calculate user personality factors and environmental factors based on historical interaction behavior data, and integrate them to obtain user preference rewards;

[0016] S4: Calculate the Euclidean distance between interaction behaviors based on historical interaction behavior data to obtain the diversity reward at the current moment;

[0017] S5: Fusion the current user preference reward and the current diversity reward to obtain the reward corresponding to the reinforcement learning model at the current moment;

[0018] S6: Input the current reward and action into the Transformer network to represent the user's current state as the input of the reinforcement learning model;

[0019] S7: Use the training set and test set to train and test an information cocoon reduction system in interactive recommendation;

[0020] S8: Use a trained information cocoon reduction system in interactive recommendation to recommend items to users.

[0021] Furthermore, the historical interaction behavior data between users and projects described in step S1, including users, projects, ratings and moments, constitute a set D = {(u, i, r, t)}, which is split into a training set and a test set in proportion, where u∈U, i∈I, r∈R, t∈T.

[0022] Further, the step S3 is specifically as follows:

[0023] S301: Calculate the personality factor {MF(u,i)|u∈U,i∈I} of the user's unique preference for the item for each user and item;

[0024] S302: Calculate the environmental factors of the popularity of each project among all users {EF i |i∈I};

[0025] S303: Calculate the herd factor {C u |u∈U};

[0026] S304: Using the herd factor integration to obtain the user preference reward for each user for each project

[0027] Furthermore, the calculation of the personality factor PF of user u for item i in step S3 adopts a matrix factorization model (MF for short), that is, PF = MF (u, i), which is specifically implemented according to the method of the following reference [1].

[0028] Furthermore, the environmental factor described in step S3 is calculated as the average score EF of the recommended item i. i ,Right now Among them, (u*,i*,r*) is an element in the historical interaction behavior data set D, is the indicator function.

[0029] Further, the user preference reward of user u for item i in step S3 is

[0030] Preferably, the matrix decomposition model is trained using user preference rewards, and the loss function used is: Among them, σ is the sigmoid function.

[0031] Furthermore, the diversity reward of user u for item i at the current time t described in step S4 is: in, tl is the time before the current time; il is the item interacted by user u at time tl; dist(i,j) represents the Euclidean distance between item i and item j; threshold is the threshold value, which is a hyperparameter obtained through training;

[0032] Preferably, the Euclidean distance in step S4 can be replaced by cosine similarity.

[0033] Further, the step S5 is specifically as follows: using Fusion of the current user preference reward and the current diversity reward to obtain the current reward r of the reinforcement learning model t , where 0≤α≤1 is the hyperparameter to be learned.

[0034] Preferably, the step S6 is specifically as follows:

[0035] S601: Extract action a t The eigenvector e a ;

[0036] S602: Use a layer of neural network to calculate the current moment's reward and feature vector e a The fusion of features g t =σ(W·concat(r t ,e a )+b), where the weight matrix W and bias b are to be learned, concat() is the concat connection operation, and σ is the sigmoid function;

[0037] S603: The feature vector e a and feature g t Input into the Transformer network to calculate the user's current state in, represents element-by-element multiplication, and Transformer() represents the Transformer network mapping.

[0038] An electronic device comprises at least one processor; and a memory connected to the at least one processor in communication; wherein:

[0039] The memory stores a computer program executed by the at least one processor, and the computer program is executed by the at least one processor so that the at least one processor can execute the above-mentioned method for reducing information cocoons in interactive recommendations.

[0040] Finally, the present invention also discloses a computer-readable storage medium, which stores computer instructions, and the computer instructions are used to enable a processor to implement the above-mentioned method for reducing information cocoons in interactive recommendations when executed.

[0041] The beneficial effects of the present invention are: providing a method for reducing information cocoons in interactive recommendations, which is implemented by an information cocoon reduction system in interactive recommendations, improving the quality of recommended items by integrating user preference rewards and diversity rewards, and using a Transformer network to accurately express user status using rewards and actions, thereby more accurately capturing user preferences, solving the information cocoon problem, and maintaining the accuracy of recommendations. In particular, the diversity reward combined with the slope characteristics of the sigmoid function improves the sensitivity of the reward to item diversity. BRIEF DESCRIPTION OF THE DRAWINGS

[0042] In order to make the purpose and technical solution of the present invention more clear, the present invention provides the following drawings for explanation:

[0043] Figure 1 This is an architecture diagram of an information cocoon reduction system in an interactive recommendation of the present invention;

[0044] Figure 2 This is a flow chart of a method for eliminating information cocoons in interactive recommendation in Example 1 of the present invention; wherein the arrows indicate the direction of data flow;

[0045] Figure 3 : This is a graph showing the comparative experimental results of the method of the present invention and the existing method on the KuaiEnv and KuaiRand data sets in Example 1 of the present invention; wherein the horizontal axis is the number of iterations, IPS is the method of reference [4], CIRS is the method of reference [3], and DDIR is the method of the present invention;

[0046] Figure 4is the ablation experiment result of the method of the present invention on the KuaiEnv and KuaiRand data sets in Example 1 of the present invention; wherein, DDIR wo comformity is the case where the reward in the method of the present invention is only the diversity reward, DDIR wodiversity is the case where the reward in the method of the present invention is only the user preference reward, and DDIR is the case where the method of the present invention integrates the reward;

[0047] Figure 5 The cumulative satisfaction comparison results on the KuaiEnv and KuaiRand datasets in Example 1 of the present invention are shown in FIG. 1 , where CIRS is the method of reference [3] and DDIR is the method of the present invention.

[0048] Figure 6 It is a schematic diagram of the structure of an electronic device in Embodiment 2 of the present invention. DETAILED DESCRIPTION

[0049] In order to make the purpose and technical solution of the present invention more clearly understood, the present invention is described in detail below with reference to the accompanying drawings and embodiments.

[0050] Example 1: Since traditional recommendation datasets are too sparse or lack necessary information, in order to improve user experience, solve the information cocoon problem, and maintain the accuracy of recommendations, this example uses two real-world recommendation system datasets: KuaiEnv and KuaiRand as shown in Table 1 as examples.

[0051] Table 1 KuaiEnv and KuaiRand datasets

[0052] Dataset name use Number of users Number of projects Number of ratings KuaiEnv Training set 7,176 10,728 12,530,806 KuaiEnv Test Set 1,411 3,327 4,676,570 KuaiRand Training set 26,285 7,551 1,436,609 KuaiRand Test Set 27,285 7,583 1,186,059

[0053] In order to make the environment reflect the user's reaction to the filter bubble, the boredom and exit mechanism set in reference [3] is introduced here: for the first N rounds of interaction, if the strategy recommends more than M items with the same category, the interactive recommendation system terminates.

[0054] This embodiment provides a method for reducing information cocoons in interactive recommendations, which is implemented by a system for reducing information cocoons in interactive recommendations. The method includes the following steps:

[0055] Step 1: Enter the historical interaction behavior data set of users and projects, KuaiEnv or KuaiRand, and construct the training set and test set, as shown in Table 1.

[0056] The historical interaction behavior data between users and projects includes users, projects, ratings and moments, and the set is D = {(u, i, r, t)}, which is split into a training set and a test set according to the proportion, where u∈U, i∈I, r∈R, and t∈T.

[0057] Step 2: Build a system to reduce information cocoons in interactive recommendations.

[0058] like Figure 1 As shown, the information cocoon reduction system in the interactive recommendation is composed of replacing the environmental interaction in the reinforcement learning model with a series of Transformer networks; the reinforcement learning model consists of four elements (s t ,a t ,r t ,π(s t |a t )) tuple; among them, s t ∈S is the state corresponding to time t; a t ∈A is the action corresponding to time t; r t is the reward corresponding to time t, π(s t |a t ) is in state s t Next action a t The probability of t is t, γ is the discount factor; the state of all users at all times is abstractly represented as the state space S; the set of all recommended items I is regarded as the action space A; the Transformer network uses the action a generated by the reinforcement learning model at time t t and reward r t Generate the state s corresponding to time t+1 t+1 Input into the reinforcement learning model to update the state, which is used to predict and understand behavioral intentions.

[0059] Step 3: Calculate user personality factors and environmental factors based on historical interaction behavior data, and integrate them to obtain user preference rewards. Specifically:

[0060] S301: Calculate the personality factor {MF(u,i)|u∈U,i∈I} of the user's unique preference for the item for each user and item;

[0061] S302: Calculate the environmental factors of the popularity of each project among all users {EF i |i∈I};

[0062] S303: Calculate the herd factor {C u |u∈U};

[0063] S304: Using the herd factor integration to obtain the user preference reward for each user for each project

[0064] The calculation of the personality factor PF of user u for item i adopts a matrix factorization model (MF for short), that is, PF = MF (u, i). It is specifically implemented according to the method of the following reference [1].

[0065] The environmental factor is calculated as the average score EF of the recommended item i i ,Right now Among them, (u*,i*,r*) is an element in the historical interaction behavior data set D, is the indicator function.

[0066] The user preference reward of user u for item i is

[0067] Preferably, the matrix decomposition model is trained using user preference rewards, and the loss function used is: Among them, σ is the sigmoid function.

[0068] Step 4: Calculate the Euclidean distance between interaction behaviors based on historical interaction behavior data to obtain the diversity reward at the current moment.

[0069] The diversity reward of user u at the current time t for item i is: in, tl is the time before the current time; il is the item interacted by user u at time tl; dist(i,j) represents the Euclidean distance between item i and item j; threshold is the threshold value, which is a hyperparameter obtained through training;

[0070] Preferably, the Euclidean distance in step 4 can be replaced by cosine similarity.

[0071] Step 5: Combine the current user preference reward and the current diversity reward to obtain the reward corresponding to the reinforcement learning model at the current moment.

[0072] Specifically: using Fusion of the current user preference reward and the current diversity reward to obtain the current reward r of the reinforcement learning model t , where 0≤α≤1 is the hyperparameter to be learned. In this embodiment, α is 0.5.

[0073] It should be noted that when conducting an ablation experiment, step three or step four can be omitted, and the diversity reward or user preference reward at the current moment can be directly used as the reward corresponding to the current moment in step five.

[0074] Step 6: Input the current reward and action into the Transformer network to represent the user's current state as the input of the reinforcement learning model. Specifically:

[0075] S601: Extract action a t The eigenvector e a ;

[0076] S602: Use a layer of neural network to calculate the current moment's reward and feature vector e a The fusion of features g t =σ(W·concat(r t ,e a )+b), where the weight matrix W and bias b are to be learned, concat() is the concat connection operation, and σ is the sigmoid function;

[0077] S603: The feature vector e a and feature g t Input into the Transformer network to calculate the user's current state in, represents element-by-element multiplication, and Transformer() represents the Transformer network mapping.

[0078] For better representation, the state in interactive recommendation should contain as much important information as possible. In this embodiment, the Transformer in reference [2] is used to construct a state tracker.

[0079] Step 7: Use the training set and test set to train and test an information cocoon reduction system in interactive recommendation.

[0080] Step 8: Use a trained interactive recommendation information cocoon reduction system to recommend items to users.

[0081] In order to better demonstrate the beneficial effects of the method of the present invention, this embodiment conducts comparative experiments based on the KuaiEnv and KuaiRand datasets with existing methods, namely the IPS method in reference [4] and the CIRS method in reference [3]. For all methods, the basic model used is the DeepFM model with fixed parameters. The key parameters are the same parameters tuned in the KuaiEnv and KuaiRand datasets. The exit mechanism parameters selected are: M = 1, N = 1, and the maximum number of rounds is 100. The experimental results are shown in Figure 2. Figure 3 As shown in Figure 1, all results are average metrics of 100 interaction trajectories. It can be seen that the method of the present invention (DDIR method) outperforms all other methods on all datasets, except for the short-term indicator (i.e., single-round satisfaction) of KuaiEnv, because compared with static methods, the target of DDIR method is the long-term satisfaction of users.

[0082] At the same time, this embodiment also conducts ablation experiments on the KuaiEnv and KuaiRand datasets, that is, considering the impact of diversity rewards and user preference rewards on user satisfaction. The results are as follows: Figure 4 As shown in the figure, it can be seen that both rewards contribute to the improvement of the recommendation system performance. In the KuaiEnv dataset, the user preference reward contributes more than the diversity reward. In contrast, on the KuaiRand dataset, the diversity reward contributes more. Combining the two rewards can better improve the performance of the recommendation system.

[0083] Finally, for the exit mechanism, we conducted a study on customer satisfaction accumulation. This example also conducted a comparative experiment with the CIRS method in reference [3] on the KuaiEnv and KuaiRand datasets. The results are as follows: Figure 5 As shown, it can be seen that the customer cumulative satisfaction of the recommendation results of the method of the present invention (DDIR method) is always much higher than that of the CIRS method.

[0084] In summary, it can be seen that the method of the present invention can ensure the accuracy of recommendations and improve customer satisfaction in the long term.

[0085] Example 2: For the scenario of Example 1, Figure 6 A schematic diagram of an electronic device (90) that can be used to implement an embodiment of the present invention is shown. The electronic device is intended to represent various forms of digital computers, such as laptop computers, desktop computers, workstations, personal digital assistants, servers, blade servers, mainframe computers, and other suitable computers.

[0086] Electronic devices may also represent various forms of mobile devices, such as personal digital assistants, cellular phones, smart phones, wearable devices (such as helmets, glasses, watches, etc.) and other similar computing devices. The components shown herein, their connections and relationships, and their functions are only examples and are not intended to limit the implementation of the invention described and / or claimed herein.

[0087] like Figure 6 As shown, the electronic device (90) includes at least one processor (91), and a memory connected to the at least one processor (91) in communication, such as a read-only memory (ROM) (92), a random access memory (RAM) (93), etc., wherein the memory stores a computer program that can be executed by the at least one processor, and the processor (91) can perform various appropriate actions and processes according to the computer program stored in the read-only memory (ROM) (92) or the computer program loaded from the storage unit (98) to the random access memory (RAM) (93). In RAM 43, various programs and data required for the operation of the electronic device (90) can also be stored. The processor (91), ROM 42 and RAM 43 are connected to each other through a bus (94). An input / output (I / O) interface (95) is also connected to the bus (94).

[0088] A number of components in the electronic device (90) are connected to the I / O interface (95), including: an input unit (96), such as a keyboard, a mouse, etc.; an output unit (97), such as various types of displays, speakers, etc.; a storage unit (98), such as a disk, an optical disk, etc.; and a communication unit (99), such as a network card, a modem, a wireless communication transceiver, etc. The communication unit (99) allows the electronic device (90) to exchange information / data with other devices through a computer network such as the Internet and / or various telecommunication networks.

[0089] The processor (91) may be any general and / or special processing component with processing and computing capabilities. Some examples of the processor (91) include, but are not limited to, a central processing unit (CPU), a graphics processing unit (GPU), various special artificial intelligence (AI) computing chips, various processors running machine learning model algorithms, a digital signal processor (DSP), and any appropriate processor, controller, microcontroller, etc. The processor (91) executes the various methods and processes described above, such as a method for reducing information cocoons in interactive recommendations.

[0090] In some embodiments, a method for reducing information cocoons in interactive recommendations can be implemented as a computer program, which is tangibly contained in a computer-readable storage medium, such as a storage unit (98). In some embodiments, part or all of the computer program can be loaded and / or installed on an electronic device (90) via a ROM (92) and / or a communication unit (99). When the computer program is loaded into a RAM (93) and executed by a processor (91), one or more steps of a method for reducing information cocoons in interactive recommendations described above can be performed. Alternatively, in other embodiments, the processor (91) can be configured to execute a method for reducing information cocoons in interactive recommendations by any other appropriate means (for example, by means of firmware).

[0091] Various implementations of the systems and techniques described above herein can be implemented in digital electronic circuit systems, integrated circuit systems, field programmable gate arrays (FPGAs), application specific integrated circuits (ASICs), application specific standard products (ASSPs), systems on chips (SOCs), load programmable logic devices (CPLDs), computer hardware, firmware, software, and / or combinations thereof. These various implementations can include: being implemented in one or more computer programs that can be executed and / or interpreted on a programmable system including at least one programmable processor, which can be a special purpose or general purpose programmable processor that can receive data and instructions from a storage system, at least one input device, and at least one output device, and transmit data and instructions to the storage system, the at least one input device, and the at least one output device.

[0092] Computer programs for implementing the methods of the present invention may be written in any combination of one or more programming languages. These computer programs may be provided to a processor of a general-purpose computer, a special-purpose computer, or other programmable data processing device, so that when the computer program is executed by the processor, the functions / operations specified in the flow chart and / or block diagram are implemented. The computer program may be executed entirely on the machine, partially on the machine, partially on the machine and partially on a remote machine as a stand-alone software package, or entirely on a remote machine or server.

[0093] In the context of the present invention, a computer-readable storage medium may be a tangible medium that may contain or store a computer program for use by or in conjunction with an instruction execution system, device, or equipment. A computer-readable storage medium may include, but is not limited to, electronic, magnetic, optical, electromagnetic, infrared, or semiconductor systems, devices, or equipment, or any suitable combination of the foregoing. Alternatively, a computer-readable storage medium may be a machine-readable signal medium. A more specific example of a machine-readable storage medium may include an electrical connection based on one or more lines, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing.

[0094] To provide interaction with a user, the systems and techniques described herein may be implemented on an electronic device having: a display device (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor) for displaying information to the user; and a keyboard and a pointing device (e.g., a mouse or trackball) through which the user can provide input to the electronic device. Other types of devices may also be used to provide interaction with the user; for example, the feedback provided to the user may be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user may be received in any form (including acoustic input, voice input, or tactile input).

[0095] The systems and techniques described herein may be implemented in a computing system that includes backend components (e.g., as a data server), or a computing system that includes middleware components (e.g., an application server), or a computing system that includes frontend components (e.g., a user computer with a graphical user interface or a web browser through which a user can interact with implementations of the systems and techniques described herein), or a computing system that includes any combination of such backend components, middleware components, or frontend components. The components of the system may be interconnected by any form or medium of digital data communication (e.g., a communication network). Examples of communication networks include: a local area network (LAN), a wide area network (WAN), a blockchain network, and the Internet.

[0096] A computing system may include a client and a server. The client and the server are generally remote from each other and usually interact through a communication network. The client and server relationship is generated by computer programs running on the corresponding computers and having a client-server relationship with each other. The server may be a cloud server, also known as a cloud computing server or cloud host, which is a host product in the cloud computing service system to solve the defects of difficult management and weak business scalability in traditional physical hosts and VPS services.

[0097] Finally, it should be noted that the above preferred embodiments are only used to illustrate the technical solutions of the present invention rather than to limit it. Although the present invention has been described in detail through the above preferred embodiments, those skilled in the art should understand that various changes can be made in form and details without departing from the scope defined by the claims of the present invention.

Claims

1. A method for eliminating information cocoons in interactive recommendation, characterized in that: The invention is realized by a system for eliminating information cocoons in interactive recommendation, wherein the system for eliminating information cocoons in interactive recommendation is composed of replacing the environment interaction in the reinforcement learning model with a series of Transformer networks; the reinforcement learning model is composed of four elements (s t ,a t ,r t ,π(s t |a t )) tuple; among them, s t ∈S is the state corresponding to time t; a t ∈A is the action corresponding to time t; r t is the reward corresponding to time t, π(s t |a t ) is in state s t Next action a t The probability of t is t, γ is the discount factor; the states of all users at all times are abstractly represented as the state space S; the set of all recommended items I is regarded as the action space A; the Transformer network uses the action a generated by the reinforcement learning model at time t t and reward r t Generate the state s corresponding to time t+1 t+1 Input into the reinforcement learning model to update the state, that is, to predict and understand behavioral intentions; The method comprises the following steps: S1: Input historical interaction behavior data between users and projects and construct training and test sets; S2: Build a system to reduce information cocoons in interactive recommendations; S3: Calculate user personality factors and environmental factors based on historical interaction behavior data, and integrate them to obtain user preference rewards; S4: Calculate the Euclidean distance between interaction behaviors based on historical interaction behavior data to obtain the diversity reward at the current moment; S5: Fusion the current user preference reward and the current diversity reward to obtain the reward corresponding to the reinforcement learning model at the current moment; S6: Input the current reward and action into the Transformer network to represent the user's current state as the input of the reinforcement learning model; S7: Use the training set and test set to train and test an information cocoon reduction system in interactive recommendation; S8: Use a trained information cocoon reduction system in interactive recommendation to recommend items to users; The historical interaction behavior data between users and items in step S1 includes users, items, ratings and moments, and the set D = {(u, i, r, t)} is split into a training set and a test set according to the proportion, where u∈U, i∈I, r∈R, t∈T; The step S3 is specifically as follows: S301: Calculate the user's unique preference for the item for each user and item ∈MF(u,i)|u∈U,i∈I}; S302: Calculate the environmental factors of the popularity of each project among all users {EF i |i∈I}; S303: Calculate the herd factor {C u |u∈U}; S304: Using the herd factor integration to obtain the user preference reward for each user for each project The calculation of the personality factor PF of user u for item i in step S3 adopts a matrix factorization model (MF for short), that is, PF = MF (u, i); The calculation of the environmental factors in step S3 is the average score EF of the recommended item i i ,Right now Among them, (u*,i*,r*) is an element in the historical interaction behavior data set D, is the indicator function; The user preference reward of user u for item i in step S3 is The diversity reward of user u for item i at the current time t in step S4 is: in, tl is the time before the current time; il is the item interacted by user u at time tl; dist(i,j) represents the Euclidean distance between item i and item j; threshold is the threshold value, which is a hyperparameter obtained through training; 2. According to claim 1, the method for eliminating information cocoons in interactive recommendation is characterized in that: The matrix decomposition model is trained using user preference rewards, and the loss function used is: Among them, σ is the sigmoid function.

3. The method for eliminating information cocoons in interactive recommendation according to claim 1, characterized in that: The step S6 is specifically as follows: S601: Extract action a t The eigenvector e a ; S602: Use a layer of neural network to calculate the current moment's reward and feature vector e a The fusion of features g t =σ(W·concat(r t ,e a )+b), where the weight matrix W and bias b are to be learned, concat() is the concat operation, and σ is the sigmoid function; S603: The feature vector e a and feature g t Input into the Transformer network to calculate the user's current state in, represents element-by-element multiplication, and Transformer() represents the Transformer network mapping.

4. The method for eliminating information cocoons in interactive recommendation according to claim 1, characterized in that: The Euclidean distance in step S4 may be replaced by cosine similarity.

5. The method for eliminating information cocoons in interactive recommendation according to claim 1, characterized in that: Step S5 combines the current user preference reward and the current diversity reward to obtain the reward r corresponding to the reinforcement learning model at the current moment. t , that is Among them, 0≤α≤1 is the hyperparameter to be learned.

6. An electronic device applied to any one of claims 1 to 5, comprising at least one processor; and a memory in communication connection with the at least one processor; wherein: The memory stores a computer program executed by the at least one processor, and the computer program is executed by the at least one processor so that the at least one processor can execute the above-mentioned method for reducing information cocoons in interactive recommendations.

7. A computer-readable storage medium as claimed in any one of claims 1 to 5, characterized in that: The computer-readable storage medium stores computer instructions, and the computer instructions are used to enable the processor to implement the above-mentioned method for reducing information cocoons in interactive recommendations when executed.

Images