A text recommendation method and device in a scene oriented to user demand dynamic change

Abstract

The application relates to a text recommendation method and device in a scene dynamically changed for user demand. The method comprises the following steps: constructing a user interest pool according to user clicks on a text; updating user features in the user interest pool by using a pre-constructed user feature updating method; constructing an initial text recommendation model according to a reinforcement learning algorithm; preprocessing the initial text recommendation model to obtain a text recommendation model; optimizing the text recommendation model by using the updated user interest pool, constructing a training data set and a test data set according to news data in a MIND data set, training the optimized text recommendation model by using the training data set, testing the trained text recommendation model according to the test data set, and performing text recommendation by using the tested text recommendation model. The method can capture dynamic changes in user interest.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of data processing, in particular to a text recommendation method and device in a dynamic change scene for user demand, computer equipment and storage medium. BACKGROUND

[0002] The current network has become an important channel for people to obtain information, and using the Internet to obtain knowledge, publish information and communicate has become part of people's daily life. With the development of Internet technology, Internet data grows by PB level every day, and people's ability to process information is limited. In the face of massive data, people need to spend more time and effort to filter information they need. On the other hand, the Internet contains a variety of data forms such as text, images, sound and video, and text is the most widely distributed and information-rich carrier. Therefore, we study a method for user demand to help users filter valuable information from massive Internet information.

[0003] However, the current recommendation system effectively alleviates the above-mentioned problems of users by automatically recommending valuable information from massive data, and its application value is also increasingly valued in the big data era. The core of the recommendation system is the recommendation algorithm. Traditional recommendation technology is mainly divided into two categories, namely collaborative filtering-based recommendation and content-based recommendation. The main idea of collaborative filtering is to recommend through the similarity between users or the similarity between items. This way is simple in principle and easy to implement, and the overall recommendation effect is also very good, but it has a serious cold start and data sparsity problem. Content-based recommendation is mainly based on item-related information, user-related information and user behavior to build a recommendation model. This way is intuitive and easy to understand, and the cold start problem is not as difficult to solve as the collaborative filtering method. But its recommendation is completely based on user historical behavior data, the recommendation range is narrow, it cannot provide users with diverse results, the recommendation accuracy is not as good as the collaborative filtering method, and the user's preferences change quickly in real scenarios. Traditional recommendation algorithms and deep learning recommendation algorithms are limited to the data set in the training phase, and can only learn the interests of users displayed in the training set. The above cannot capture the dynamic changes of user interests. SUMMARY

[0004] Therefore, it is necessary to provide a text recommendation method, device, computer equipment and storage medium in a dynamic change scene for user demand, which can capture the dynamic changes of user interests.

[0005] A text recommendation method in a dynamic change scene for user demand, the method comprising:

[0006] acquiring user clicked texts and candidate texts; the user clicked texts include historical clicked texts of the user and all clicked interactive clicked texts of the user in the process of interaction with the agent;

[0007] constructing a user interest pool according to the user clicked texts; updating user features in the user interest pool by using a pre-constructed user feature updating method to obtain an updated user interest pool;

[0008] constructing an initial text recommendation model according to a reinforcement learning algorithm; pre-processing the initial text recommendation model to obtain a text recommendation model;

[0009] optimizing the text recommendation model by using the updated user interest pool to obtain an optimized text recommendation model;

[0010] constructing a training data set and a test data set according to news data in a MIND data set, training the optimized text recommendation model by using the training data set to obtain a trained text recommendation model;

[0011] testing the trained text recommendation model according to the test data set to obtain a tested text recommendation model;

[0012] performing text recommendation by using the tested text recommendation model.

[0013] In one of the embodiments, the user interest pool includes a user interest pool of user features and a user interest pool of candidate texts; the user interest pool of current features is formed by adding two texts randomly selected from the user interest pool to ten texts representing the user features and removing two texts randomly from the ten texts representing the user features; the user interest pool of candidate texts is formed by randomly selecting texts from the user interest pool as candidate texts.

[0014] In one of the embodiments, the user features are ten texts randomly selected from the historical clicked texts of the user, the categories, sub-categories and titles of the ten texts are extracted, and initial features are obtained by vectorizing the categories, sub-categories and titles by using a TF-IDF method; the user features in the user interest pool are updated by using a pre-constructed user feature updating method to obtain an updated user interest pool, including:

[0015] In the process of interaction between the agent and the user, when the user actually clicks a candidate text, one text is randomly removed from the ten texts representing the current features of the user, and the text currently clicked by the user is added to the texts representing the user features to obtain new ten texts;

[0016] The categories, sub-categories and title of the new ten texts are vectorized according to the TF-IDF method to form new user features, and an updated user interest pool is obtained.

[0017] In one of the embodiments, the initial text recommendation model comprises an environment, an agent, a state, an action and a reward; the initial text recommendation model is preprocessed to obtain a text recommendation model, comprising:

[0018] The user and the candidate text are taken as the environment, the reinforcement learning algorithm is taken as the agent, the user features and the features of the candidate text constitute the state of the environment, whether the agent recommends the candidate text to the user is taken as the action, and the feedback of the user to the candidate text is taken as the reward; the feedback comprises that the user behavior is consistent with the prediction of the agent to obtain a reward, and the user behavior is inconsistent with the prediction of the agent to obtain a punishment.

[0019] In one of the embodiments, the updated user interest pool is used to optimize the text recommendation model to obtain an optimized text recommendation model, comprising:

[0020] According to the user features and the features of the candidate text in the updated user interest pool, the agent and the user are interacted, the behavior of the user clicking or ignoring the candidate text is fed back to the agent, the recommendation strategy of the agent itself is adjusted according to the obtained reward and punishment to optimize the text recommendation model, and an optimized text recommendation model is obtained.

[0021] In one of the embodiments, the training data set comprises historical behavior data of the user and news report related information; the optimized text recommendation model is trained by using the training data set to obtain a trained text recommendation model, comprising:

[0022] The agent in the optimized text recommendation model reads the historical behavior data, stores the historical clicked news of the user into the updated user interest pool, randomly extracts news from the updated user interest pool, vectorizes the news according to the TF-IDF method, and obtains user features;

[0023] The agent reads the impression log of the user from the historical behavior data, and extracts the title, category, sub-category and other information of the news corresponding to the news ID from the news report related information according to the impression log to construct candidate news features; the news report related information comprises a candidate news list.

[0024] The initial features of the user and the candidate news features are input into the agent in the optimized text recommendation model, the agent predicts whether the user is interested in the candidate news, and the prediction result of the agent is compared with the actual behavior label of the user to the news; when the prediction result is consistent with the actual label, the agent is rewarded, otherwise, the agent is punished.

[0025] When the actual label is of interest to the user, the candidate news is added to the user interest pool, and the user features are updated as the user features at the next time according to the user feature updating method; when the actual label is not of interest to the user, the user features remain unchanged, that is, the user features at the next time are consistent with the user features at the current time;

[0026] The user features at the next time and the features of the next candidate news are input into the agent, and it is regarded as the end of a round of training until all the news in the candidate news list are circulated; in the circulation process, it is determined whether the preset period parameter step is reached each time, and if yes, the user features are updated or the candidate news is replaced; after the judgment process, the user features and the features of the candidate news are input into the agent to start the next round of training until the preset convergence condition is reached, and the training is completed.

[0027] A text recommendation device for user demand dynamic transformation scenarios, the device comprising:

[0028] A user interest pool construction module is configured to obtain user clicked texts and candidate texts; the user clicked texts include historical clicked texts of the user and all clicked interactive clicked texts in the interaction process of the user and the agent; a user interest pool is constructed according to the user clicked texts; and a user feature updating method is used to update the user features in the user interest pool to obtain an updated user interest pool.

[0029] A text recommendation model construction and optimization module is configured to construct an initial text recommendation model according to a reinforcement learning algorithm; the initial text recommendation model is preprocessed to obtain a text recommendation model; and the updated user interest pool is used to optimize the text recommendation model to obtain an optimized text recommendation model.

[0030] A model training and text recommendation module is configured to construct a training data set and a test data set according to news data in a MIND data set, train the optimized text recommendation model by using the training data set to obtain a trained text recommendation model, test the trained text recommendation model according to the test data set to obtain a tested text recommendation model, and recommend texts by using the tested text recommendation model.

[0031] A computer device comprises a memory and a processor, the memory stores a computer program, and the processor implements the following steps when executing the computer program:

[0032] User clicked texts and candidate texts are obtained; the user clicked texts include historical clicked texts of the user and all clicked interactive clicked texts in the interaction process of the user and the agent;

[0033] construct a user interest pool according to the user clicked texts; update the user features in the user interest pool by using a pre-constructed user feature updating method to obtain an updated user interest pool;

[0034] construct an initial text recommendation model according to a reinforcement learning algorithm; pre-process the initial text recommendation model to obtain a text recommendation model;

[0035] optimize the text recommendation model by using the updated user interest pool to obtain an optimized text recommendation model;

[0036] construct a training data set and a test data set according to news data in a MIND data set, train the optimized text recommendation model by using the training data set to obtain a trained text recommendation model;

[0037] test the trained text recommendation model according to the test data set to obtain a tested text recommendation model;

[0038] use the tested text recommendation model to perform text recommendation.

[0039] A computer readable storage medium has a computer program stored thereon, and the computer program is executed by a processor to implement the following steps:

[0040] obtain user clicked texts and candidate texts; the user clicked texts include historical clicked texts of a user and all clicked interactive texts in a process in which the user interacts with an intelligent agent;

[0041] construct a user interest pool according to the user clicked texts; update the user features in the user interest pool by using a pre-constructed user feature updating method to obtain an updated user interest pool;

[0042] construct an initial text recommendation model according to a reinforcement learning algorithm; pre-process the initial text recommendation model to obtain a text recommendation model;

[0043] optimize the text recommendation model by using the updated user interest pool to obtain an optimized text recommendation model;

[0044] construct a training data set and a test data set according to news data in a MIND data set, train the optimized text recommendation model by using the training data set to obtain a trained text recommendation model;

[0045] test the trained text recommendation model according to the test data set to obtain a tested text recommendation model;

[0046] use the tested text recommendation model to perform text recommendation.

[0047] The aforementioned text recommendation methods, devices, computer equipment, and storage media addressing dynamically changing user needs. This application utilizes reinforcement learning algorithms to construct a text recommendation model. The interaction mechanism between the agent and the environment in reinforcement learning algorithms can be used to simulate the interaction process between the recommendation model and the user, capturing dynamic changes in users in real time. Compared to traditional static methods, this approach is more consistent with real-world recommendation scenarios. By continuously interacting with users and receiving feedback, the reinforcement learning algorithm enables the agent to continuously learn and adjust its recommendation strategy, thereby capturing dynamic changes in user interests. This overcomes the limitations of traditional recommendation algorithms and deep learning algorithms in scenarios with dynamically changing user interests. Furthermore, a user interest pool is designed to discover past or potential user interest types, diversifying the agent's recommendation types and preventing the reinforcement learning agent from falling into extreme recommendation situations. This approach not only captures dynamically changing user interests but also improves the accuracy of user recommendations. Attached Figure Description

[0048] Figure 1 This is a flowchart illustrating a text recommendation method for dynamically changing user needs in one embodiment.

[0049] Figure 2 This is a schematic diagram of the initial text recommendation model in one embodiment;

[0050] Figure 3 This is a schematic diagram illustrating the updating of the user interest pool for user features and the user interest pool for candidate texts in one embodiment.

[0051] Figure 4 This is a schematic diagram of user feature updates in another embodiment;

[0052] Figure 5 This is a structural block diagram of a text recommendation device in a dynamically changing user demand scenario, as shown in one embodiment.

[0053] Figure 6 This is an internal structural diagram of a computer device in one embodiment. Detailed Implementation

[0054] To make the objectives, technical solutions, and advantages of this application clearer, the following detailed description is provided in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative and not intended to limit the scope of this application.

[0055] In one embodiment, such as Figure 1 As shown, a text recommendation method for dynamically changing scenarios based on user needs is provided, including the following steps:

[0056] In step 102, the user clicks on the text and the candidate text; the user clicks on the text include the user's historical clicks on the text and all the interactive clicks on the text in the process of the user interacting with the intelligent agent; the user interest pool is constructed according to the user clicks on the text; the user features in the user interest pool are updated by using the pre-constructed user feature updating method, and the updated user interest pool is obtained.

[0057] The user interest pool includes the user interest pool of the user features and the user interest pool of the candidate text, and the dynamic change of the current user can be recognized by continuously updating the user interest pool, thereby improving the accuracy of the text recommendation.

[0058] In step 104, an initial text recommendation model is constructed according to a reinforcement learning algorithm; the text recommendation model is obtained by preprocessing the initial text recommendation model; the text recommendation model is optimized by using the updated user interest pool, and the optimized text recommendation model is obtained.

[0059] The interaction mechanism of the intelligent agent and the environment in the reinforcement learning algorithm can be used to simulate the interaction process between the recommendation model and the user, and the dynamic change of the user can be captured in real time, which is more consistent with the real recommendation scene than the traditional static method. The feedback is obtained by continuously interacting with the user by using the reinforcement learning algorithm, the intelligent agent continuously learns and adjusts its recommendation strategy, thereby capturing the dynamic change of the user interest, and solving the limitations of the traditional recommendation algorithm and the deep learning algorithm in the dynamic change of the user interest. The interaction process of the model to the user recommendation text is defined as a reinforcement learning problem, and the text recommendation model is constructed by defining the five elements of reinforcement learning, as follows: environment: the user and the candidate text are observed by the intelligent agent; intelligent agent: the intelligent agent is responsible for the reinforcement learning algorithm; state: the state of the environment is composed of the user features and the candidate text features; action: whether the intelligent agent recommends the candidate text to the user; reward and punishment: the feedback of the user to the candidate text, the user behavior is consistent with the prediction of the intelligent agent, and the reward is obtained, otherwise the punishment is obtained.

[0060] However, the mechanism of maximizing the reward of the reinforcement learning makes it pay too much attention to the current reward and ignore the long-term reward in the learning process, which may lead the intelligent agent to fall into the extreme recommendation of the short-term attention content of the user, and ignore the long-term attention of the user. For example, during the World Cup, the user is interested in the text related to the World Cup, the intelligent agent learns the interest of the user, and always recommends such content to the user. When the event is over, the user's interest has shifted, but the intelligent agent has strengthened its cognition of "the user is interested in the football related text" because the user has always clicked on its recommended content, so that the intelligent agent fails to capture the change of the user's interest in time. And always recommending the same type of content will make the user feel tired.

[0061] There are also such scenarios in real life. For some reason, the user does not click on some text that he is interested in recently. In fact, the user is interested in it, and when the intelligent agent recommends it to him, the user will still click on it. Similarly, taking the user's attention to the World Cup as an example, the user's attention to it will decrease after a tournament, but when the next tournament comes, the user's attention will rise again.

[0062] The above two scenarios are that the intelligent agent only focuses on the user's current interest type and ignores the user's past or future interest type, which makes the intelligent agent's recommendation to the user become single, and eventually falls into an extreme recommendation situation. The present application designs a user interest pool to explore the user's past or potential interest type, optimizes the text recommendation model using the user interest pool, makes the intelligent agent's recommendation type more diversified, prevents the reinforcement learning intelligent agent from falling into the above extreme recommendation situation, captures the user's dynamic changes in interest, and improves the accuracy of user recommendation.

[0063] Step 106, constructing a training data set and a test data set according to the news data in the MIND data set, training the optimized text recommendation model using the training data set to obtain a trained text recommendation model; testing the trained text recommendation model according to the test data set to obtain a tested text recommendation model; and using the tested text recommendation model for text recommendation.

[0064] The optimized text recommendation model is trained and tested using the existing data set, so that the various parameters of the model are optimized, and the recommendation accuracy is greatly improved when the tested text recommendation model is used for text recommendation.

[0065] In the above text recommendation method for dynamic change of user demand, the reinforcement learning algorithm is used to construct the text recommendation model. The interaction mechanism between the intelligent agent and the environment in the reinforcement learning algorithm can be used to simulate the interaction process between the recommendation model and the user, and can capture the dynamic changes of the user in real time. Compared with the traditional static method, it is more in line with the real recommendation scene. The present application uses the reinforcement learning algorithm to obtain feedback by continuously interacting with the user, prompting the intelligent agent to continuously learn and adjust its recommendation strategy, thereby realizing the capture of the dynamic changes of the user's interest, solving the limitations of traditional recommendation algorithms and deep learning algorithms in the user interest dynamic change scene, and designing a user interest pool to explore the user's past or potential interest type, making the intelligent agent's recommendation type more diversified, preventing the reinforcement learning intelligent agent from falling into the above extreme recommendation situation, capturing the user's dynamic changes in interest, and improving the accuracy of user recommendation.

[0066] In one of the embodiments, the user interest pool includes a user interest pool of user features and a user interest pool of candidate texts; the user interest pool of current features is that two texts are randomly extracted from the user interest pool at regular intervals, and two texts are randomly removed from the ten texts representing the current user features, the two texts extracted from the user interest pool are added to the texts representing the user features to form a new user interest pool; the user interest pool of candidate texts is that a text is randomly extracted from the user interest pool as a candidate text at regular intervals, and the user interest pool is constructed according to the candidate text.

[0067] In a specific embodiment, the present application designs a user interest pool M to store all the texts clicked by the user, including the historical clicks of the user and all the texts clicked by the user during the interaction with the agent. These stored texts will be randomly extracted at regular intervals for the agent to learn the long-term and short-term interests of the user, so as to avoid the agent from falling into some extreme recommendation situation. A period parameter step is set for randomly extracting texts from the user interest pool M. For the texts stored in the user interest pool M, the method designs two utilization methods, as shown in the following table, and names them as user interest pool M-S and user interest pool M-N, which are as follows: Figure 3

[0068] The user interest pool M-S of user features: through the way of updating the current features s of the user, that is, randomly extracting 2 texts from the user interest pool M at regular intervals, and randomly removing 2 texts from the 10 texts representing the current user features, adding the 2 texts extracted from the user interest pool M to the texts representing the user features to form a new user feature s'.

[0069] The user interest pool M-N of candidate texts: through the way of replacing the candidate text n, that is, randomly extracting a text from the user interest pool M as a candidate text n, so that the agent learns the text features that the user is interested in again, and deepens the impression.

[0070] The way of the user interest pool M-S directly changes the current features of the user, forces the agent to realize that the user's interest has changed, and adjusts the recommendation strategy to recommend the text type that meets the current features of the user. The way of the user interest pool M-N makes the agent learn the texts that the user is interested in again, arouses the memory of the agent to the historical interest of the user, and the learning again also enables the agent to deeply explore the potential interest of the user, so as to adjust the recommendation strategy to recommend diversified text types.

[0071] ​In one of the embodiments, the user feature is ten texts randomly selected from the user's historical click texts, the category, subcategory and title of the ten texts are extracted, and the initial feature is obtained by vectorizing the category, subcategory and title using the TF-IDF method; the user feature in the user interest pool is updated using the pre-constructed user feature updating method to obtain an updated user interest pool, including:

[0072] In the interaction process between the agent and the user, when the user actually clicks on a certain candidate text, one text is randomly removed from the ten texts representing the current user feature, and the text currently clicked by the user is added to the texts representing the user feature to obtain new ten texts;

[0073] The category, subcategory and title of the new ten texts are vectorized according to the TF-IDF method to form new user features, and an updated user interest pool is obtained.

[0074] In a specific embodiment, the candidate text feature is formed by vectorizing the category, subcategory and title of the current candidate text in the same way.

[0075] The user feature reflects the type of text that the user is currently interested in, so in the interaction process between the agent and the user, the user's feature will change with the text he clicks, as shown in Figure 4 The updating method of the user feature s is designed as follows: in the interaction process between the agent and the user, when the user actually clicks on a certain candidate text, one text is randomly removed from the ten texts representing the current user feature, and the text currently clicked by the user is added to the texts representing the user feature, and the category, subcategory and title of the ten texts are vectorized in the same way to form new user features s', and an updated user interest pool is constructed according to the new user features and the candidate text features. By continuously updating the user interest pool, the agent can recognize the dynamic changes of the current user and thus improve the accuracy of text recommendation.

[0076] In one of the embodiments, the initial text recommendation model includes an environment, an agent, a state, an action and a reward; the initial text recommendation model is preprocessed to obtain a text recommendation model, including:

[0077] The user and the candidate text are taken as the environment, the reinforcement learning algorithm is taken as the agent, the user feature and the feature of the candidate text constitute the state of the environment, whether the agent recommends the candidate text to the user is taken as the action, and the feedback of the user to the candidate text is taken as the reward; the feedback includes that the user behavior is consistent with the prediction of the agent to obtain a reward, and the user behavior is inconsistent with the prediction of the agent to obtain a punishment.

[0078] In a specific embodiment, as shown in Figure 2As shown, the agent observes the current features s of the user in the environment and the candidate text features n, s and n together as the input of the agent. The output is the prediction of the agent that whether the user is interested in the candidate text. When the agent predicts that the user is interested in the candidate text, the agent will recommend the candidate text to the user. The behavior of the user clicking (indicating interest) or ignoring (indicating disinterest) the candidate text will be fed back to the agent. If the prediction of the agent is consistent with the actual behavior of the user, the agent will be rewarded, otherwise, the agent will be punished. The reward and punishment obtained will promote the agent to adjust its own recommendation strategy, and finally achieve the effect that the recommended text meets the user's preference.

[0079] In one of the embodiments, the text recommendation model is optimized by using the updated user interest pool, and an optimized text recommendation model is obtained, including:

[0080] According to the user features in the updated user interest pool and the features of the candidate text, the agent is input into the text recommendation model, the agent interacts with the user, the behavior of the user clicking or ignoring the candidate text is fed back to the agent, and the recommendation strategy of the agent is adjusted according to the reward and punishment obtained to optimize the text recommendation model, and an optimized text recommendation model is obtained.

[0081] In one of the embodiments, the training data set includes historical behavior data of the user and news report related information; the optimized text recommendation model is trained by using the training data set, and a trained text recommendation model is obtained, including:

[0082] The agent in the optimized text recommendation model reads the historical behavior data, stores the historical clicked news of the user into the updated user interest pool, randomly extracts news from the updated user interest pool, and vectorizes the news according to the TF-IDF method to obtain user features;

[0083] The agent reads the impression log of the user from the historical behavior data, and according to the impression log, sequentially extracts the title, category, subcategory and other information of the news corresponding to the news ID from the news report related information to construct candidate news features; the news report related information includes a candidate news list;

[0084] The initial features of the user and the candidate news features are input into the agent in the optimized text recommendation model, the agent predicts whether the user is interested in the candidate news, and the prediction result of the agent is compared with the actual behavior label of the user on the news. When the prediction result is consistent with the actual label, the agent is rewarded, otherwise, the agent is punished;

[0085] When the actual label is of interest to the user, the candidate news is added to the user interest pool, and the user features are updated as the user features at the next time according to the user feature updating method; when the actual label is not of interest to the user, the user features remain unchanged, that is, the user features at the next time are consistent with the user features at the current time;

[0086] The user features at the next time and the features of the next candidate news are input into the agent, and it is considered that a round of training is completed until the news in the candidate news list is completely circulated. In the circulation process, it is first determined whether the pre-set cycle parameter step is reached each time, and if yes, the user features are updated or the candidate news is replaced. After the judgment process, the user features and the features of the candidate news are input into the agent to start the next round of training until the pre-set convergence condition is reached, and the training is completed.

[0087] In specific embodiments, a data set is first obtained, and the data set is a Microsoft open source news data set MIND. The MIND collects the click behavior records of users on Microsoft News within six weeks and performs anonymization processing. The behavior.tsv file in the data set is the user click history and impression log, and each row represents a user, and there are five columns, and the format is [num, uID, Time, ClickHist, ImpLog]. Among them, ClickHist is the click history of the user, and saves the news IDs clicked by the user, and the content form is [nID1 nID2……]. ImpLog is the interaction behavior of the user with the news, and the content form is [nID1-label1 nID2-label2……], and label is 0 or 1, indicating whether the user has clicked the news. This method only uses the [uID, ClickHist, ImpLog] three columns of information. The news.tsv file is the information related to the news report, and each row represents the related information of a news, and there are eight columns, and the format is [nID, category, subcategory, title, abstract, url, title entities, abstract entites]. The url is the link of the whole news, and the title entities and the abstract entites are the entities contained in the news title and the entities contained in the news abstract, respectively. This application only uses the [nID, category, subcategory, title] four columns of information.

[0088] In order to verify whether the model can capture the dynamic interest of the user, the data set to be tested should be able to simulate the process of user interest change, and the MIND data set cannot meet the requirements of the experiment. Therefore, based on the news data in the MIND data set, a new data set is constructed, and the interest change trend of the user is designed in the data set. The data set contains 1500 news, and the positive and negative sample ratio is 1:4, of which 1000

[0089] news are used for training, and 500 are used for testing.

[0090] First, the prepared data set is used to train the optimized text recommendation model. The specific steps are as follows:

[0091] 1.1 The agent reads the user's historical behavior data, that is, the data in the ClickHist column 0 of the behavior.tsv file, and stores the user's historical click news in the user interest pool M.

[0092] 1.2 Randomly extract news from the user's historical behavior data, and construct the user's initial feature s according to the method of step 3.

[0093] 1.3 The agent reads the user's impression log, that is, the data in the ImpLog column of the behavior.tsv file.

[0094] 1.4 According to the data in the ImpLog column, the title, category, subcategory and other information of the news corresponding to the news ID are extracted from the news.tsv file in turn to construct the candidate news feature n.

[0095] 1.5 The user's initial feature s and the candidate news feature n are input into the agent.

[0096] 1.6 The agent outputs whether it predicts that the user is interested in the candidate news or not, 1 means the user is interested, and 0 means the user is not interested. Compare the agent's prediction with the user's actual behavior label, when the prediction result is consistent with the actual label, give the agent a reward, otherwise give a punishment, the agent will adjust its learning and recommendation strategy according to these rewards and punishments. When the actual label is 1, the candidate news is added to the user interest pool, and the user feature is updated as the user feature s' of the next moment according to the method of step 3. When the actual label is 0, the user feature remains unchanged, that is, the user feature s' of the next moment is consistent with the user feature s of the current moment.

[0097] 1.7 Input the next time user feature s' and the next candidate news feature n into the agent, repeat the step of 5.65 until all the news in the candidate news list are circulated, then it is considered that a round of training is completed. In the process of circulation, each time first judge whether the cycle parameter step has been reached, if yes, then update the user feature s or replace the candidate news n according to the method described in the second step. After the judgment process, input the user feature s' and the candidate news feature n into the agent.

[0098]

[0099] 1.8 Start the next round of training until convergence.

[0100] Secondly, the model trained in the last step can be used for testing. The testing process is also to simulate the actual process of recommending text to users and interacting with users, and the specific steps are as follows:

[0101] 2.1 Input the user feature s at the end of training as the current user feature, and input the new candidate news according to the way of the third step to form the candidate news feature n into the trained agent.

[0102] 2.2 The agent outputs 0 or 1. When the agent outputs 0, it means that the agent predicts that the user is not interested in the candidate news, and the agent does not recommend the news to the user; when the agent outputs 1, it means that the agent predicts that the user is interested in the candidate news, and the agent recommends the news to the user.

[0103] 2.3 The user gives feedback on the candidate news recommended by the agent. If the user clicks on the news, the agent is rewarded, and the news is added to the user interest pool M, and the user feature s is updated as the next time user feature s' according to the way of the third step; if the user ignores the news, the agent is punished, and the next time user feature s' is unchanged.

[0104] 2.4 Input the next time user feature s' and the next candidate news feature n into the agent, repeat the steps of 2.2 and 2.3 until the test data is completed (corresponding to the actual application is to always circulate 2.2, 2.3 and 2.4). Similarly, each time in the process of circulation is also to first judge whether the cycle parameter step has been reached, if yes, update the user feature s or replace the candidate news n. After the judgment process, input the user feature s' and the candidate news feature n into the agent.

[0105] It should be understood that, although Figure 1 the steps in the flowchart of the method are displayed in sequence according to the arrows, these steps are not necessarily executed in the order indicated by the arrows. Unless otherwise specified in this article, the execution of these steps has no strict order restriction, and these steps can be executed in other order. Moreover, Figure 1 ​At least one of the steps in the method can comprise a plurality of sub-steps or a plurality of stages, which are not necessarily executed at the same time, but can be executed at different times, and the order of execution of the sub-steps or stages is not necessarily sequential, but can be executed alternately or alternately with at least one of the other steps or sub-steps or stages of the other steps.

[0106] In one embodiment, as shown in Figure 5 In one embodiment, as shown in

[0107] The user interest pool construction module 502 is configured to obtain user clicked texts and candidate texts; the user clicked texts include historical clicked texts of the user and all clicked interactive texts in the process of interaction between the user and the intelligent agent; a user interest pool is constructed according to the user clicked texts; and the user features in the user interest pool are updated by using a pre-constructed user feature updating method to obtain an updated user interest pool.

[0108] The text recommendation model construction and optimization module 504 is configured to construct an initial text recommendation model according to a reinforcement learning algorithm; to pre-process the initial text recommendation model to obtain a text recommendation model; and to optimize the text recommendation model by using the updated user interest pool to obtain an optimized text recommendation model.

[0109] The model training and text recommendation module 506 is configured to construct a training data set and a test data set according to news data in a MIND data set, to train the optimized text recommendation model by using the training data set to obtain a trained text recommendation model, to test the trained text recommendation model according to the test data set to obtain a tested text recommendation model, and to recommend texts by using the tested text recommendation model.

[0110] The specific limitations of the text recommendation device for the dynamic transformation scenario of user demand can be referred to the limitations of the text recommendation method for the dynamic transformation scenario of user demand in the foregoing, which will not be repeated here. Each module in the above text recommendation device for the dynamic transformation scenario of user demand can be realized by software, hardware and their combinations. The above modules can be embedded in or independent of the processor in the computer device in hardware form, or can be stored in the memory in the computer device in software form, so as to call and execute the operations corresponding to the above modules by the processor.

[0111] In one embodiment, a computer device is provided, which can be a terminal, and the internal structure diagram thereof can be as shown in Figure 6As shown in the figure. The computer device includes a processor, a memory, a network interface, a display screen and an input device connected through a system bus. Among them, the processor of the computer device is used to provide computing and control capabilities. The memory of the computer device includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system and a computer program. The internal memory provides an environment for the operating system and the computer program in the non-volatile storage medium to run. The network interface of the computer device is used to communicate with the external terminal through the network connection. The computer program is executed by the processor to implement a text recommendation method in a scene dynamically changed for user demand. The display screen of the computer device can be a liquid crystal display screen or an electronic ink display screen, and the input device of the computer device can be a touch layer overlaid on the display screen, or a key, trackball or touchpad arranged on the shell of the computer device, or an external keyboard, touchpad or mouse, etc.

[0112] Those skilled in the art can understand that, Figure 6 The skilled in the art can understand that,

[0113] Those skilled in the art can understand that all or part of the processes in the above-mentioned embodiment methods can be completed by a computer program instructing related hardware. The computer program can be stored in a non-volatile computer readable storage medium. When the computer program is executed, it can include the processes of the above-mentioned embodiments. Any reference to memory, storage, database or other medium used in the embodiments provided by the present application can include non-volatile and / or volatile memory. Non-volatile memory can include read-only memory (ROM), programmable ROM (PROM), electrically programmable ROM (EPROM), electrically erasable programmable ROM (EEPROM) or flash memory. Volatile memory can include random access memory (RAM) or external cache memory. As an illustration but not limitation, RAM is available in various forms, such as static RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), double data rate SDRAM (DDR SDRAM), enhanced SDRAM (ESDRAM), synchronous link (Synchlink) DRAM (SLDRAM), memory bus (Rambus) direct RAM (RDRAM), direct memory bus dynamic RAM (DRDRAM) and memory bus dynamic RAM (RDRAM) and the like.

[0114] Any combination of the technical features in the above embodiments can be made, and for the sake of brevity, not all possible combinations are described above, however, as long as the combination of the technical features does not exist in contradiction, it shall be considered within the scope of the present disclosure.

[0115] The above embodiments only express several implementation manners of the present application, and the description is relatively specific and detailed, but it shall not be understood as a limitation on the patent scope of the present application. It shall be pointed out that, for ordinary skilled persons in the art, several modifications and improvements can be made without departing from the concept of the present application, and these shall be within the protection scope of the present application. Therefore, the patent protection scope of the present application shall be subject to the appended claims.

Claims

1. A text recommendation method in a dynamic scene change for user demand, characterized in that, The method comprises: acquiring user click text and candidate text; the user click text comprises historical click text of a user and all clicked interactive click text of the user in the process of interaction with an intelligent agent; constructing a user interest pool according to the user click text; updating user features in the user interest pool by using a pre-constructed user feature updating method to obtain an updated user interest pool; constructing an initial text recommendation model according to a reinforcement learning algorithm; preprocessing the initial text recommendation model to obtain a text recommendation model; optimizing the text recommendation model by using the updated user interest pool to obtain an optimized text recommendation model; constructing a training data set and a test data set according to news data in a MIND data set; training the optimized text recommendation model by using the training data set to obtain a trained text recommendation model; testing the trained text recommendation model according to the test data set to obtain a tested text recommendation model; using the tested text recommendation model for text recommendation; the user interest pool comprises a user interest pool of user features and a user interest pool of candidate text; the user features are initial features obtained by randomly selecting ten texts from the historical click text of the user, extracting the categories, subcategories and titles of the ten texts, and vectorizing the categories, subcategories and titles by using a TF-IDF method; the user features in the user interest pool are updated by using a pre-constructed user feature updating method to obtain an updated user interest pool, which comprises: in the process of interaction between the intelligent agent and the user, when the user actually clicks on a certain candidate text, one text is randomly removed from the ten texts representing the current features of the user, and the text currently clicked by the user is added to the texts representing the features of the user to obtain new ten texts; new user features are formed by vectorizing the categories, subcategories and titles of the new ten texts according to the TF-IDF method to obtain an updated user interest pool; the user interest pool of the user features is formed by periodically randomly extracting two texts from the user interest pool, and randomly removing two texts from the ten texts representing the current features of the user, and adding the two extracted texts from the user interest pool to the texts representing the features of the user to form a new user interest pool; the user interest pool of the candidate text is formed by periodically randomly extracting texts from the user interest pool as candidate texts.

2. The method of claim 1, wherein, the initial text recommendation model comprises an environment, an intelligent agent, a state, an action and a reward and punishment; the initial text recommendation model is preprocessed to obtain a text recommendation model, which comprises: the user and the candidate text are taken as the environment, the reinforcement learning algorithm is taken as the intelligent agent, the features of the user and the candidate text constitute the state of the environment, whether the intelligent agent recommends the candidate text to the user is taken as the action, and the feedback of the user to the candidate text is taken as the reward and punishment; the feedback comprises that the user behavior is consistent with the prediction of the intelligent agent to obtain a reward, and the user behavior is inconsistent with the prediction of the intelligent agent to obtain a punishment.

3. The method of claim 1, wherein, Optimizing the text recommendation model by using the updated user interest pool to obtain an optimized text recommendation model, comprising: According to the user characteristics in the updated user interest pool and the characteristics of the candidate text, inputting into the text recommendation model, the agent interacts with the user, the user's click or ignore behavior of the candidate text is fed back to the agent, and the obtained reward and punishment are used to adjust the recommendation strategy of the agent itself to optimize the text recommendation model, and an optimized text recommendation model is obtained.

4. The method of claim 3, wherein, The training data set includes historical behavior data of users and news report related information; Training the optimized text recommendation model by using the training data set to obtain a trained text recommendation model, comprising: Using the agent in the optimized text recommendation model to read the historical behavior data, storing the user's historical click news in the updated user interest pool, randomly extracting news from the updated user interest pool, and vectorizing the news according to the TF-IDF method to obtain user characteristics; Using the agent to read the user's impression log from the historical behavior data, and extracting the news information corresponding to the news ID from the news report related information according to the impression log to construct candidate news characteristics; the news report related information includes a candidate news list; the news information includes title, category, and subcategory; Inputting the user initial characteristics and the candidate news characteristics into the agent in the optimized text recommendation model, the agent predicting whether the user is interested in the candidate news, comparing the agent prediction result with the actual behavior label of the user on the news, and rewarding the agent when the prediction result is consistent with the actual label, otherwise, punishing the agent; When the actual label is that the user is interested, the candidate news is added to the user interest pool, and the user characteristics are updated as the user characteristics at the next moment according to the user characteristic updating method; when the actual label is that the user is not interested, the user characteristics remain unchanged, that is, the user characteristics at the next moment are consistent with the user characteristics at the current moment; Inputting the user characteristics at the next moment and the characteristics of the next candidate news into the agent until the news in the candidate news list is completely circulated, which is regarded as the end of a round of training; in the circulation process, it is first determined whether the pre-set cycle parameter step is reached every time, and if yes, the user characteristics are updated or the candidate news is replaced; after the judgment process, the user characteristics and the candidate news characteristics are input into the agent to start the next round of training until the pre-set convergence condition is reached, and the training is completed.

5. A text recommendation device in a dynamic scene change oriented to user needs, characterized in that, The device comprises: The user interest pool construction module is configured to obtain user clicked texts and candidate texts; the user clicked texts include historical clicked texts of a user and all clicked interactive clicked texts of the user in the process of interaction with an intelligent agent; a user interest pool is constructed according to the user clicked texts; user features in the user interest pool are updated by using a pre-constructed user feature updating method to obtain an updated user interest pool; the user interest pool includes a user interest pool of user features and a user interest pool of candidate texts; the user features are initial features obtained by randomly selecting ten texts from the historical clicked texts of the user, extracting categories, subcategories and titles of the ten texts, and vectorizing the categories, subcategories and titles by using a TF-IDF method; the user features in the user interest pool are updated by using the pre-constructed user feature updating method to obtain the updated user interest pool, including: in the process of interaction between the intelligent agent and the user, when the user actually clicks a certain candidate text, one text is randomly removed from the ten texts representing the current features of the user, and the text currently clicked by the user is added to the texts representing the features of the user to obtain new ten texts; new user features are formed by vectorizing the categories, subcategories and titles of the new ten texts according to the TF-IDF method to obtain the updated user interest pool; the user interest pool of the user features is obtained by periodically randomly extracting two texts from the user interest pool, randomly removing two texts from the ten texts representing the current features of the user, and adding the two extracted texts from the user interest pool to the texts representing the features of the user to form a new user interest pool; the user interest pool of the candidate texts is obtained by periodically randomly extracting texts from the user interest pool as candidate texts to construct a user interest pool according to the candidate texts; The text recommendation model construction and optimization module is configured to construct an initial text recommendation model according to a reinforcement learning algorithm; the initial text recommendation model is preprocessed to obtain a text recommendation model; the updated user interest pool is used to optimize the text recommendation model to obtain an optimized text recommendation model; The model training and text recommendation module is configured to construct a training data set and a test data set according to news data in a MIND data set, train the optimized text recommendation model by using the training data set to obtain a trained text recommendation model, test the trained text recommendation model according to the test data set to obtain a tested text recommendation model, and recommend texts by using the tested text recommendation model. 6.A computer device, comprising a memory and a processor, wherein the memory stores a computer program, and the computer device is configured to perform the method according to any one of claims 1-5 when the computer program is executed by the processor. The processor executes the computer program to implement the steps of the method in any one of claims 1 to 4.

7. A computer-readable storage medium having stored thereon a computer program, characterized in that The computer program is executed by the processor to implement the steps of the method in any one of claims 1 to 4.

Images