Content recommendation model training method, content recommendation method and device

Abstract

The present disclosure relates to a training method, content recommendation method and device for a content recommendation model, the method comprising: obtaining first sample data; the first sample data comprising interaction information between a sample object and sample content; based on the first sample data, obtaining target content recommended to a target object by the content recommendation model to be trained; simulating feedback information of the target object on the target content through a trained prediction model; the prediction model is used to predict the initial feedback information of the target object on the target content, and adjust the predicted initial feedback information based on the recommendation frequency information corresponding to the target content, to obtain feedback information of the target object on the target content; adjusting the content recommendation model to be trained based on the feedback information, to obtain a target content recommendation model. This method can solve the problem of information cocoons while ensuring the accuracy of recommendation results.

Description

Technical Field

[0001] The present disclosure relates to the field of computer technology, and in particular to a content recommendation model training method, a content recommendation method, a device, an electronic device, a storage medium, and a program product. Background Art

[0002] Information cocoon refers to the phenomenon that people's information fields are habitually guided by their own preferences, thus confining their lives in a "cocoon" like a silkworm cocoon. In the field of recommendation systems, the information cocoon problem refers to the fact that the recommendation model can make recommendations based on the user's various preferences in the initial stage, but with further parameter updates and strategy iterations of the recommendation model, the recommendation results are gradually dominated by the user's mainstream preferences, while ignoring the user's other preferences.

[0003] The current solutions to the information cocoon problem in recommendation systems include actively helping users improve their perception of diverse viewpoints, or improving the model to improve the diversity and fairness of recommendation results. However, these methods are static recommendation strategies, and they all sacrifice the accuracy of recommendations in exchange for the goal of avoiding information cocoons. Therefore, the current methods for breaking the information cocoon are difficult to take into account both the accuracy and diversity of recommendation results. Summary of the invention

[0004] The present disclosure provides a content recommendation model training method, content recommendation method, method, device, electronic device, storage medium and program product, so as to at least solve the problem that the method of breaking the information cocoon in the related art is difficult to take into account both the accuracy and diversity of the recommendation results.

[0005] The technical solution of the present disclosure is as follows:

[0006] According to a first aspect of an embodiment of the present disclosure, a method for training a content recommendation model is provided, comprising:

[0007] Acquire first sample data; the first sample data includes interaction information between a sample object and sample content;

[0008] Based on the first sample data, obtaining target content recommended by the content recommendation model to be trained to a target object; the target object is any object among the sample objects, and the target content belongs to the sample content in the first sample data;

[0009] The trained prediction model is used to simulate the feedback information of the target object on the target content; the prediction model is used to predict the initial feedback information of the target object on the target content, and adjust the predicted initial feedback information based on the recommendation frequency information corresponding to the target content to obtain the feedback information of the target object on the target content; the recommendation frequency information represents the number of times the associated content is recommended to the target object, and the associated content is the content that matches the target content in at least one attribute;

[0010] The content recommendation model to be trained is adjusted based on the feedback information to obtain a target content recommendation model.

[0011] In an exemplary embodiment, the prediction model is obtained by the following training method:

[0012] Acquire second sample data; the second sample data includes interaction information between the sample object and the sample content;

[0013] Based on multiple sample contents corresponding to the same sample object in the second sample data, obtaining sample recommendation frequency information corresponding to each sample content of the same sample object; the sample recommendation frequency information represents the number of associated sample contents whose timestamps are before each sample content, and the timestamps represent the time when each sample content is sampled to train the prediction model;

[0014] Based on the sample recommendation frequency information and the interaction information between the sample object and the sample content in the second sample data, the prediction model to be trained is trained to obtain the trained prediction model.

[0015] In an exemplary embodiment, the acquiring, based on a plurality of sample contents corresponding to the same sample object in the second sample data, sample recommendation frequency information corresponding to each sample content of the same sample object includes:

[0016] Determine, based on difference information between the multiple sample contents corresponding to the same sample object, associated sample contents whose timestamps are before the target sample contents; the target sample contents belong to the multiple sample contents corresponding to the same sample object;

[0017] Acquire a time difference between the timestamps corresponding to the target sample content and the associated sample content;

[0018] Based on the difference information and the time difference, the sample recommendation frequency information corresponding to the target sample content is determined.

[0019] In an exemplary embodiment, the training of the prediction model to be trained based on the sample recommendation frequency information and the interaction information between the sample object and the sample content in the second sample data to obtain the trained prediction model includes:

[0020] Based on the interactive information in the second sample data, positive sample content and negative sample content corresponding to each sample object are sampled from the second sample data; the positive sample content represents the content for which the sample object has made positive feedback, and the negative sample content represents the content for which the sample object has made negative feedback;

[0021] For each sample object, obtaining first feedback information of the sample object on the positive sample content and second feedback information of the sample object on the negative sample content according to the sample recommendation frequency information of the positive sample content and the sample recommendation frequency information of the negative sample content;

[0022] Based on the loss value between the first feedback information and the second feedback information, the model parameters of the prediction model to be trained are adjusted until a preset number of training times is reached or the loss value converges, thereby obtaining the trained prediction model.

[0023] In an exemplary embodiment, the adjusting the predicted initial feedback information based on the recommendation frequency information corresponding to the target content to obtain the feedback information of the target object on the target content includes:

[0024] Based on the recommendation frequency information, determining adjustment amplitude information for the initial feedback information; the adjustment amplitude information is positively correlated with the recommendation frequency information;

[0025] The initial feedback information is adjusted according to the adjustment amplitude information to obtain feedback information that changes inversely with the recommendation frequency information as feedback information of the target object to the target content.

[0026] In an exemplary embodiment, the content recommendation model to be trained determines the target content to be recommended to the target object in the following manner:

[0027] Acquire state information associated with the target object; the state information is obtained based on object feature information of the target object, historical recommended content recommended for the target object, weight information of the historical recommended content, and feedback information of the target object for the historical recommended content;

[0028] The target content recommended to the target object is determined by using the content recommendation model to be trained and the state information.

[0029] In an exemplary embodiment, adjusting the content recommendation model to be trained based on the feedback information to obtain a target content recommendation model includes:

[0030] Based on the feedback information, adjusting the model parameters of the content recommendation model to be trained to obtain a new content recommendation model;

[0031] and, updating state information associated with the target object according to the target content and feedback information corresponding to the target content;

[0032] Through the new content recommendation model and the updated status information associated with the target object, the new target content recommended to the target object is determined again, and feedback information for the new target content is obtained through the trained prediction model until the content recommendation accuracy for the target object reaches a threshold, thereby obtaining a target content recommendation model.

[0033] In an exemplary embodiment, after obtaining the target content recommendation model, the method further includes:

[0034] According to the first sample data, a performance parameter of the target content recommendation model for content recommendation of the sample object is obtained; the performance parameter is obtained when the recommendation frequency information of the target content recommendation model within a preset time period reaches a threshold;

[0035] Based on the performance parameters, the performance of the target content recommendation model is evaluated.

[0036] According to a second aspect of an embodiment of the present disclosure, a content recommendation method is provided, including:

[0037] Obtain multiple contents to be recommended and objects to be recommended;

[0038] The target content recommendation model is used to select target recommended content from the multiple contents to be recommended and push it to the object to be recommended; the target content recommendation model is trained by the training method of the above-mentioned content recommendation model.

[0039] According to a third aspect of an embodiment of the present disclosure, a training device for a content recommendation model is provided, comprising:

[0040] A first acquisition unit is configured to acquire first sample data; the first sample data includes interaction information between a sample object and sample content;

[0041] A recommendation unit is configured to obtain target content recommended to a target object by a content recommendation model to be trained based on the first sample data; the target object is any object among the sample objects, and the target content belongs to the sample content in the first sample data;

[0042] a simulation unit configured to execute the prediction model completed through training, and simulate the feedback information of the target object on the target content; the prediction model is used to predict the initial feedback information of the target object on the target content, and adjust the predicted initial feedback information based on the recommendation frequency information corresponding to the target content, so as to obtain the feedback information of the target object on the target content; the recommendation frequency information represents the number of times the associated content is recommended for the target object, and the associated content is the content that matches the target content in at least one attribute;

[0043] The adjustment unit is configured to adjust the content recommendation model to be trained based on the feedback information to obtain a target content recommendation model.

[0044] In an exemplary embodiment, the apparatus further comprises:

[0045] A second acquisition unit is configured to acquire second sample data; the second sample data includes interaction information between the sample object and the sample content;

[0046] a frequency determination unit configured to execute, based on a plurality of sample contents corresponding to the same sample object in the second sample data, obtaining sample recommendation frequency information corresponding to each sample content of the same sample object; the sample recommendation frequency information represents the number of associated sample contents whose timestamps are before each sample content, and the timestamps represent the time when each sample content is sampled to train the prediction model;

[0047] The training unit is configured to train the prediction model to be trained based on the sample recommendation frequency information and the interaction information between the sample object and the sample content in the second sample data to obtain the trained prediction model.

[0048] In an exemplary embodiment, the frequency determination unit is further configured to determine, based on difference information between multiple sample contents corresponding to the same sample object, associated sample content whose timestamp is before the target sample content; the target sample content belongs to the multiple sample contents corresponding to the same sample object; obtain the time difference between the timestamps corresponding to the target sample content and the associated sample content; and determine the sample recommendation frequency information corresponding to the target sample content based on the difference information and the time difference.

[0049] In an exemplary embodiment, the training unit is configured to execute sampling of positive sample content and negative sample content corresponding to each sample object from the second sample data based on the interactive information in the second sample data; the positive sample content represents the content for which the sample object has made positive feedback, and the negative sample content represents the content for which the sample object has made negative feedback; for each sample object, first feedback information of the sample object on the positive sample content and second feedback information on the negative sample content are obtained according to the sample recommendation frequency information of the positive sample content and the sample recommendation frequency information of the negative sample content; based on the loss value between the first feedback information and the second feedback information, the model parameters of the prediction model to be trained are adjusted until a preset number of training times is reached or the loss value converges, thereby obtaining the trained prediction model.

[0050] In an exemplary embodiment, the training unit is further configured to determine adjustment amplitude information for the initial feedback information based on the recommendation frequency information; the adjustment amplitude information is positively correlated with the recommendation frequency information; the initial feedback information is adjusted according to the adjustment amplitude information to obtain feedback information that changes in the opposite direction to the recommendation frequency information as feedback information of the target object to the target content.

[0051] In an exemplary embodiment, the recommendation unit is further configured to execute acquisition of status information associated with the target object; the status information is obtained based on object feature information of the target object, historical recommended content recommended for the target object, weight information of the historical recommended content, and feedback information of the target object for the historical recommended content; and the target content recommended to the target object is determined through the content recommendation model to be trained and the status information.

[0052] In an exemplary embodiment, the adjustment unit is further configured to adjust the model parameters of the content recommendation model to be trained based on the feedback information to obtain a new content recommendation model; and, based on the target content and the feedback information corresponding to the target content, update the status information associated with the target object; through the new content recommendation model and the updated status information associated with the target object, again determine the new target content recommended to the target object, and obtain feedback information for the new target content through the trained prediction model, until the content recommendation accuracy for the target object reaches a threshold, and the target content recommendation model is obtained.

[0053] In an exemplary embodiment, the device also includes an evaluation unit configured to obtain performance parameters of the target content recommendation model for content recommendation to the sample object based on the first sample data; the performance parameters are obtained when the recommendation frequency information of the target content recommendation model within a preset time period reaches a threshold; based on the performance parameters, the performance of the target content recommendation model is evaluated.

[0054] According to a fourth aspect of an embodiment of the present disclosure, there is provided a content recommendation device, including:

[0055] An acquisition unit is configured to acquire a plurality of contents to be recommended and objects to be recommended;

[0056] The recommendation unit is configured to execute a target content recommendation model to select target recommended content from the multiple contents to be recommended and push it to the object to be recommended; the target content recommendation model is trained by the training method of the above-mentioned content recommendation model.

[0057] According to a fifth aspect of an embodiment of the present disclosure, there is provided an electronic device, including:

[0058] processor;

[0059] a memory for storing instructions executable by the processor;

[0060] The processor is configured to execute the instructions to implement the method as described in any one of the above items.

[0061] According to a sixth aspect of an embodiment of the present disclosure, a computer-readable storage medium is provided. When instructions in the computer-readable storage medium are executed by a processor of an electronic device, the electronic device is enabled to execute any of the methods described above.

[0062] According to a seventh aspect of an embodiment of the present disclosure, a computer program product is provided, wherein the computer program product includes instructions, and when the instructions are executed by a processor of an electronic device, the electronic device is enabled to execute any of the methods described above.

[0063] The technical solution provided by the embodiments of the present disclosure brings at least the following beneficial effects:

[0064] After obtaining the first sample data, the target content recommended to the target object is first determined by the content recommendation model to be trained, and then the feedback information of the target object on the target content is simulated by the trained prediction model. Finally, the content recommendation model to be trained is adjusted based on the feedback information to obtain the target content recommendation model. Since the feedback information of the target object on the target content simulated by the prediction model is the feedback information obtained by taking into account the recommendation frequency information, and the recommendation frequency information can characterize the degree to which the target content is trapped in the information cocoon, the content recommendation model to be trained is adjusted based on the feedback information, so that the obtained target content recommendation model can learn the ability to jump out of the information cocoon, thereby solving the problem of the information cocoon on the basis of ensuring the accuracy of the recommendation results.

[0065] It is to be understood that the foregoing general description and the following detailed description are exemplary and explanatory only and are not restrictive of the present disclosure. BRIEF DESCRIPTION OF THE DRAWINGS

[0066] The drawings herein are incorporated into and constitute a part of the specification, illustrate embodiments consistent with the present disclosure, and together with the description are used to explain the principles of the present disclosure, and do not constitute improper limitations on the present disclosure.

[0067] Figure 1 (a) is a schematic diagram of a recommendation framework based on reinforcement learning according to an exemplary embodiment.

[0068] Figure 1 (b) is a schematic diagram of the information cocoon problem according to an exemplary embodiment.

[0069] Figure 1 (c) is a brief flowchart of a method for training a content recommendation model according to an exemplary embodiment.

[0070] Figure 2 The present invention is a flowchart of a method for training a content recommendation model according to an exemplary embodiment.

[0071] Figure 3 The figure is a schematic diagram showing the relationship between the first sample data and the second sample data according to an exemplary embodiment.

[0072] Figure 4 The figure is a flowchart of a content recommendation method according to an exemplary embodiment.

[0073] Figure 5 It is a flowchart of a training method for a content recommendation model according to another exemplary embodiment.

[0074] Figure 6It is a schematic diagram of an overall model framework combining a causal inference model based on counterfactual reasoning and an offline reinforcement learning framework according to an exemplary embodiment.

[0075] Figure 7 (a) is a causal structure diagram commonly used in a recommendation system model according to an exemplary embodiment.

[0076] Figure 7 (b) is a causal structure diagram after combining the repeated recommendation effect according to an exemplary embodiment.

[0077] Figure 7 (c) is a causal structure diagram adopted by the interactive recommendation strategy according to an exemplary embodiment.

[0078] Figure 8 The figure is a structural block diagram of a training device for a content recommendation model according to an exemplary embodiment.

[0079] Fig. 9 The figure is a structural block diagram of a content recommendation device according to an exemplary embodiment.

[0080] Fig.10 It is a block diagram of an electronic device according to an exemplary embodiment. DETAILED DESCRIPTION

[0081] In order to enable ordinary persons in the art to better understand the technical solutions of the present disclosure, the technical solutions in the embodiments of the present disclosure will be clearly and completely described below in conjunction with the accompanying drawings.

[0082] It should be noted that the embodiments described in the following exemplary embodiments do not represent all embodiments consistent with the present disclosure. Instead, they are merely examples of devices and methods consistent with some aspects of the present disclosure as detailed in the appended claims.

[0083] It should be noted that the terms "first", "second", etc. in the specification and claims of the present disclosure and the above-mentioned drawings are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequence. It should be understood that the data used in this way can be interchangeable where appropriate, so that the embodiments of the present disclosure described herein can be implemented in an order other than those illustrated or described herein. It should also be noted that the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data for display, data for analysis, etc.) involved in the present disclosure are all information and data authorized by the user or fully authorized by all parties.

[0084] Interactive recommendation means that the system can obtain real-time feedback information from users (such as likes, reposts, and viewing time) during the interaction with users. By making full use of this feedback information, an online algorithm strategy can be learned to dynamically adjust the recommendation scheme, and it will replace the static recommendation algorithm under manually specified rules. The interactive recommendation algorithm is a mainstream algorithm in the application scenario of content recommendation. For example, reinforcement learning is a technical branch that supports this interactive recommendation. Its workflow is as follows: Figure 1 As shown in (a), the technical principle is to enable an intelligent agent to learn how to make automatic decisions in different situations in the interaction with users, so as to pursue specific optimal long-term benefits. However, reinforcement learning is usually difficult to put online. The fundamental reason is that a large amount of interaction data is required for training, and most application scenarios cannot accept the model training process that requires real users to participate in the interaction. For example, the platform cannot directly obtain the degree of a user's preference for all billions of content.

[0085] In current content recommendation systems, the results of recommendation systems usually come from the integrated results of multiple static recommendation models. For each static recommendation model, the principle is to use the content that users have interacted with to train the model in order to estimate the user's preference for content that they have not interacted with. This can be regarded as a special case of an interactive recommendation system, that is, the strategy module is static and monotonically unchanged, and a unified online parameter update is performed after a specified time period.

[0086] However, almost all current recommendation strategies face problems such as “the more you push, the narrower the scope” and information filter bubble. Figure 1 As shown in (b), users have multiple preferences. The recommendation system can capture more than one preference for recommendation in the initial stage, such as the sports and food videos in the figure. However, with the further parameter update and strategy iteration of the recommendation system, the recommendation results will gradually be dominated by the user's mainstream preferences, such as sports food in the figure, while ignoring other user preferences. This gradually monotonous recommendation result will make users feel tired, and thus distrust and boredom with the recommendation system. In the current recommendation system, this phenomenon is widely present, so the current recommendation strategy is a compromise and not the best.

[0087] In view of the above problems, this paper proposes a training method for a content recommendation model for solving the information cocoon problem existing in current recommendation strategies. Figure 1(c) This method introduces reinforcement learning as a policy model into the production environment, and uses causal reasoning technology based on counterfactual models to explicitly model the repeated recommendation effect in user preferences, thereby effectively avoiding the "narrower and narrower" problem while automatically iterating and updating the decision-making process. The basic idea is to learn a prediction model that can estimate user preferences based on historical interaction data based on the counterfactual model in causal reasoning, and then use the prediction model to generate feedback information to plan and train the content recommendation model based on the reinforcement learning recommendation strategy (RL policy). Finally, the learned content recommendation model is put online to improve the adaptive ability of the current content recommendation system. In order to effectively evaluate the impact of information cocoons, this method will be verified in the interactive recommendation system, and this method is also applicable to ordinary static recommendation strategies.

[0088] refer to Figure 2 , is a flowchart of a training method for a content recommendation model according to an exemplary embodiment. This embodiment uses the method applied to a terminal as an example. It can be understood that the method can also be applied to a server, and can also be applied to a system including a terminal and a server, and is implemented through the interaction between the terminal and the server. Among them, the terminal can be but is not limited to various personal computers, laptops, smart phones, tablets, Internet of Things devices and portable wearable devices. The Internet of Things devices can be smart speakers, smart TVs, smart air conditioners, smart car-mounted devices, etc. Portable wearable devices can be smart watches, smart bracelets, head-mounted devices, etc. The server can be implemented as an independent server or a server cluster composed of multiple servers. In this embodiment, the method includes the following steps:

[0089] In step S210, first sample data is obtained; the first sample data includes interaction information between a sample object and sample content.

[0090] Among them, interactive information can be understood as information that can characterize the sample subject's preference for the sample content. For example, interactive information can be viewing time, click-through rate, whether to view comments or post comments, whether to like, whether to collect, etc.

[0091] The first sample data may include interaction information between each sample object and each sample content.

[0092] In a specific implementation, multiple sample objects and multiple sample contents may be determined, and interaction information of all sample objects to all sample contents may be obtained as first sample data, so as to improve the accuracy of subsequent testing and evaluation of the content recommendation model.

[0093] In step S220, based on the first sample data, the target content recommended to the target object by the content recommendation model to be trained is obtained; the target object is any object in the sample objects, and the target content belongs to the sample content in the first sample data.

[0094] Among them, the content recommendation model can be a model obtained by an interactive recommendation strategy based on reinforcement learning (RL Agent), for example, it can be a model obtained by a strategy based on a PPO algorithm (Proximal Policy Optimization) or a model obtained by a DDPG algorithm (Deep Deterministic Policy Gradient) or the like.

[0095] Among them, reinforcement learning is a field in machine learning that emphasizes how to act based on the environment to maximize expected benefits. Reinforcement learning does not require labeled input-output pairs, nor does it need to accurately correct non-optimal solutions. Its focus is on finding a balance between exploration (of unknown areas) and utilization (of existing knowledge), and the "exploration-utilization" exchange in reinforcement learning.

[0096] In a specific implementation, the content recommendation model to be trained can gradually learn the preference information of the target object during the training process, and select the target content recommended to the target object from the sample content included in the first sample data based on the learned preference information of the target object. More specifically, during the training process, the historical recommended content recommended to the target object and the target object's feedback information on the historical recommended content can also be recorded. Based on the historical recommended content and the target object's feedback information on the historical recommended content, the target content recommended to the target object is determined by the content recommendation model.

[0097] In step S230, the feedback information of the target object on the target content is simulated through the trained prediction model; the prediction model is used to predict the initial feedback information of the target object on the target content, and adjust the predicted initial feedback information based on the recommendation frequency information corresponding to the target content to obtain the feedback information of the target object on the target content; the recommendation frequency information represents the number of times the related content is recommended to the target object, and the related content is the content that matches the target content in at least one attribute.

[0098] Among them, the prediction model can be a causal inference model based on counterfactual reasoning.

[0099] The feedback information may be in the form of a score of the target object's preference for the target content, or may be two feedback results of preference and non-preference. The specific form may be determined based on actual needs, and the present disclosure does not impose any specific limitation on this.

[0100] Among them, the recommendation frequency information can be understood as the number of recommendations per unit time, and the recommendation frequency information can reflect the degree to which the recommended target content is trapped in the information cocoon.

[0101] The attributes may include the creator or publisher of the content, the category of the content, the tag of the content, etc.

[0102] In a specific implementation, the prediction model may include two units: a prediction unit and an adjustment unit. The prediction unit is used to predict the initial feedback information of the target object on the target content, and the adjustment unit is used to adjust the initial feedback information obtained by the prediction unit based on the recommendation frequency information of the target content. Since the recommendation frequency information can reflect the degree to which the target content is trapped in an information cocoon, after adjusting the initial feedback information through the recommendation frequency information, the prediction model can solve the information cocoon problem and simulate the real feedback information of the target object on the target content.

[0103] In step S240, the content recommendation model to be trained is adjusted based on the feedback information to obtain a target content recommendation model.

[0104] In this step, the training process of the content recommendation model is carried out through interaction with the prediction model. In this interaction process, the prediction model acts as a user simulator, providing the content recommendation model with timely and accurate reward signals as feedback information. Since this reward signal does not exist in reality, it can be called a counterfactual reward. Through this feedback information, the content recommendation model can be adjusted to prevent repeated recommendation behaviors of recommending the same or similar content too many times in a short period of time, so that the content recommendation model can learn to automatically avoid repeated recommendations and jump out of the information cocoon.

[0105] In a specific implementation, after obtaining the feedback information of the target object on the target content through the prediction model simulation, the model parameters of the content recommendation model to be trained can be adjusted based on the feedback information to obtain a new content recommendation model, and the new target content is recommended to the target object again through the new content recommendation model, and feedback information on the new target content is obtained through the prediction model until the accuracy of the content recommendation for the target object reaches a threshold, and the first content recommendation model after interactive training for the target object is obtained. Then, for the next target object, the first content recommendation model is trained through the interaction between the prediction model and the first content recommendation model to obtain the second content recommendation model, and so on. When each sample object has participated in the training of the content recommendation model to be trained, the target content recommendation model is obtained.

[0106] In the training method of the above content recommendation model, after obtaining the first sample data, the target content recommended to the target object is first determined by the content recommendation model to be trained, and then the feedback information of the target object on the target content is simulated by the trained prediction model, and finally the content recommendation model to be trained is adjusted based on the feedback information to obtain the target content recommendation model. Since the feedback information of the target object on the target content simulated by the prediction model is the feedback information obtained by taking into account the recommendation frequency information, and the recommendation frequency information can characterize the degree to which the target content is trapped in the information cocoon, the content recommendation model to be trained is adjusted based on the feedback information, so that the obtained target content recommendation model can learn the ability to jump out of the information cocoon, thereby solving the problem of the information cocoon on the basis of ensuring the accuracy of the recommendation results.

[0107] In an exemplary embodiment, the prediction model is trained in the following manner:

[0108] Step S310, obtaining second sample data; the second sample data includes interaction information between the sample object and the sample content;

[0109] Step S320, based on multiple sample contents corresponding to the same sample object in the second sample data, obtain sample recommendation frequency information corresponding to each sample content of the same sample object; the sample recommendation frequency information represents the number of associated sample contents with a timestamp before each sample content, and the timestamp represents the time when each sample content is sampled to train the prediction model;

[0110] Step S330: Based on the sample recommendation frequency information and the interaction information between the sample object and the sample content in the second sample data, the prediction model to be trained is trained to obtain a trained prediction model.

[0111] The second sample data is data used to train the prediction model.

[0112] In the present disclosure, since the first sample data is used for subsequent testing and evaluation of the content recommendation model, all sample objects in the first sample data have at least one interaction record with all content, that is, the first sample data is full exposure data. The second sample data is used to train the prediction model, and therefore, it may include the interaction information of each sample object with respect to at least part of the sample content, that is, the second sample data may not be full exposure data. For example, Figure 3, is a schematic diagram of the relationship between the first sample data and the second sample data, in which the horizontal axis represents the sample content, the vertical axis represents the sample object, and each color block can represent the interactive information between a sample object and a sample content, wherein a deep color block represents interactive data, and a colorless color block represents that no interactive data has been collected. Correspondingly, area 31 can represent the first sample data, and area 32 can represent the second sample data. If the interactive information density is used to represent the ratio of interactive information with interactive value in the first sample data or the second sample data to the total sample data, then the interactive information density of the first sample data is greater than or equal to the interactive information density of the second sample data.

[0113] In a specific implementation, in order to enable the prediction model to simulate the real feedback information of the target object on the target content, when training the prediction model, it is also necessary to consider the impact of monotonous and repeated recommendation results on the user experience. Therefore, when training the prediction model, for the same sample object, based on multiple sample contents obtained from the second sample data that have interactive information with the same sample object, as multiple sample contents corresponding to the same sample object, the recommendation frequency information of each sample content in the multiple sample contents is calculated to reflect the degree to which each sample content is trapped in the information cocoon. Further, based on the recommendation frequency information, the initial feedback information of the same sample object on each sample content is adjusted to obtain the real feedback information of the same sample object on each sample content, and the prediction model to be trained is iteratively trained based on the feedback information to obtain a trained prediction model.

[0114] In this embodiment, when training the prediction model, the impact of monotonous and repetitive recommendation results on user experience is taken into consideration. Therefore, recommendation frequency information is introduced to determine the feedback information of each sample object on the sample content, so that the prediction model trained based on the recommendation frequency information can correctly characterize the repeated recommendation effect in the recommendation results, thereby detecting and correcting it in the embryonic stage of the information cocoon.

[0115] In an exemplary embodiment, the above step S320 can be implemented by the following steps:

[0116] Step S320a, based on the difference information between the multiple sample contents corresponding to the same sample object, determining the associated sample contents whose timestamps are before the target sample contents; the target sample contents belong to the multiple sample contents corresponding to the same sample object;

[0117] Step S320b, obtaining the time difference between the timestamps corresponding to the target sample content and the associated sample content;

[0118] Step S320c, based on the difference information and the time difference, determine the sample recommendation frequency information corresponding to the target sample content.

[0119] The difference information may be determined according to preset attributes. For example, the difference information may be the feature distance between representation vectors of two sample contents, or the difference information between creators or publishers of two sample contents, or the difference information between categories or tags of two sample contents.

[0120] The associated sample content refers to content that matches the target sample content in at least one attribute.

[0121] The timestamp represents the time when each sample content is sampled to train the prediction model.

[0122] In a specific implementation, taking the target sample content among multiple sample contents corresponding to the same sample object as an example, the corresponding sample recommendation frequency information is determined in the following manner: first, sample content with a timestamp before the target sample content is determined from the multiple sample contents, recorded as candidate sample content, and the difference information between the target sample content and each candidate sample content is obtained. Based on the difference information, content matching the target sample content in at least one attribute is determined from the candidate sample content as the associated sample content of the target sample content. The time difference between the timestamp of the target sample content and the timestamp of the associated sample content is further obtained, and based on the time difference between the target sample content and the associated sample content, the sample recommendation frequency information corresponding to the target sample content is determined.

[0123] For example, taking feature distance as difference information, for multiple sample contents corresponding to the same sample object, the method for determining the sample recommendation frequency information corresponding to any one of the target sample contents can be: obtaining the feature distance between the target sample content and other sample contents (i.e., sample contents other than the target sample content in the multiple sample contents), and based on the feature distance, determining from the other sample contents a sample content with a timestamp before the target sample content and matching the target sample content in at least one attribute as the associated sample content of the target sample content, and then determining the sample recommendation frequency information corresponding to the target sample content based on the feature distance and time difference between the target sample content and the associated sample content.

[0124] Furthermore, in order to improve the accuracy of the obtained sample recommendation frequency information, in an exemplary embodiment, step S320c may further include: obtaining the sensitivity of the same sample object to the repeated recommendation effect (which can be understood as too many times of recommending the same or similar content in a short period of time), and the classic degree information or tolerance degree information of the target sample content; based on the sensitivity of the same sample object to the repeated recommendation effect, the classic degree information or tolerance degree information of the target sample content, the difference information between the target sample content and the associated sample content, and the time difference between the target sample content and the associated sample content, determine the sample recommendation frequency information corresponding to the target sample content.

[0125] More specifically, for the recommendation frequency information (denoted as e t ) can be determined by the following relationship:

[0126]

[0127] Among them, u represents the sample object, i represents the target sample content, α u Indicates the sensitivity of sample object u to the repeated recommendation effect, β i Indicates the classic degree or tolerance degree of the target sample content i, i l represents the associated sample content whose timestamp is before the target sample content i, t l Indicates the associated sample content i l The timestamp of target sample content i is represented by t. is a set that records all sample contents recommended by the prediction model and sample object u so far in this round of interactive recommendation, such as any triple in the set Represents the content i of the lth object pushed to the sample object u l , whose timestamp is t l τ is the temperature coefficient, which is a hyperparameter that needs to be manually set and adjusted. dist(i,i l ) represents the distance between the target sample content i and the representation vector of the associated sample content. Recommendation frequency information e t The intuitive meaning of the definition of (u,i) can be understood as: if in this round of interaction, the system l smaller) recommended other content related to the target sample content (distance function dist(i,i l ) is smaller), then the recommendation frequency information e corresponding to the target sample content t (u,i) will be larger.

[0128] In this embodiment, the sample recommendation frequency information corresponding to the target sample content is determined by the difference information and time difference between the target sample content and the associated sample content, so that the obtained recommendation frequency information can characterize the degree to which the target sample content is repeatedly recommended, thereby achieving an accurate characterization of the repeated recommendation effect in the recommendation results.

[0129] In an exemplary embodiment, the above step S330 can be implemented by the following steps:

[0130] Step S330a, based on the interactive information in the second sample data, sample the positive sample content and negative sample content corresponding to each sample object from the second sample data; the positive sample content represents the content for which the sample object has made positive feedback, and the negative sample content represents the content for which the sample object has made negative feedback;

[0131] Step S330b, for each sample object, obtaining first feedback information of the sample object on the positive sample content and second feedback information on the negative sample content according to the sample recommendation frequency information of the positive sample content and the sample recommendation frequency information of the negative sample content;

[0132] Step S330c, based on the loss value between the first feedback information and the second feedback information, adjust the model parameters of the prediction model to be trained until the preset number of training times is reached or the loss value converges, thereby obtaining a trained prediction model.

[0133] In a specific implementation, positive sample content can be understood as content preferred by the sample object, which can be sample content with interactive data with the sample object in the second sample data, while sample content without interactive data with the sample object in the second sample data can be determined as content not preferred by the sample object, as negative sample content. Figure 3 In the area 31 representing the second sample data, the sample content corresponding to the dark color block can be used as the positive sample content of the sample object corresponding to the color block, and the sample content corresponding to the color block without interactive data can be used as the negative sample content of the sample object corresponding to the color block. Therefore, based on the interactive information in the second sample data, multiple positive sample contents and multiple negative sample contents corresponding to each sample object can be sampled from the second sample data, wherein the negative sample contents appear together with the positive sample contents.

[0134] For each sample object, after obtaining multiple positive sample contents and multiple negative sample contents corresponding to the sample object, for each pair of positive sample contents and negative sample contents, the sample recommendation frequency information corresponding to each of the positive sample contents and the negative sample contents can be determined based on the method described in steps S320a to S320c, and then the initial feedback information of the positive sample contents obtained through the prediction model is adjusted according to the sample recommendation frequency information corresponding to the positive sample contents to obtain the first feedback information of the positive sample contents, and the initial feedback information of the negative sample contents obtained through the prediction model is adjusted according to the sample recommendation frequency information corresponding to the negative sample contents to obtain the second feedback information of the negative sample contents. The difference between the first feedback information and the second feedback information is obtained as the loss value, and the model parameters of the prediction model to be trained are adjusted based on the loss value until the preset number of training times is reached or the loss value converges, and a trained prediction model is obtained.

[0135] More specifically, for example, the classic MSE (mean-square error) loss function or the BPR (Bayesian Personalized Ranking) loss function can be used as the loss function of the prediction model to guide the training. Taking the BPR loss function as an example, the relationship of the BPR loss function is:

[0136]

[0137] in, is an activation function used to map data from the range of (-∞,∞) to the range of (0,1). In the above formula, i represents the positive sample content and j represents the negative sample content. represents the first feedback information of sample object u to positive sample content i, Representing the second feedback information of the sample object u to the sampled negative sample content j, the process of optimizing the above BPR loss function is the learning process of traversing the known interaction samples in the 10729 sample contents that have interacted with all 7176 sample object pairs in the above second sample data.

[0138] In this embodiment, the prediction model is trained by the loss value between the first feedback information of the positive sample content and the second feedback information of the negative sample content to widen the prediction score between the positive sample content and the negative sample content, so that the score of the feedback information of the positive sample content is as high as possible, and the score of the feedback information of the negative sample content is as low as possible, so that the prediction model can learn the relative preference order of the sample object for different sample contents, improve the prediction effect of the prediction model, and avoid the traditional method of only using the loss value between the actual interaction data of a single sample content and the prediction value for training, resulting in the prediction model having a small difference in feedback information for all sample contents and difficulty in accurately distinguishing the preferences of the sample objects.

[0139] In an exemplary embodiment, in the above step S230, the prediction model adjusts the predicted initial feedback information based on the recommendation frequency information corresponding to the target content to obtain the feedback information of the target object on the target content, which can be achieved in the following way: based on the recommendation frequency information, determine the adjustment amplitude information for the initial feedback information; the adjustment amplitude information is positively correlated with the recommendation frequency information; according to the adjustment amplitude information, adjust the initial feedback information to obtain feedback information that changes in the opposite direction to the recommendation frequency information as the feedback information of the target object on the target content.

[0140] In a specific implementation, the purpose of adjusting the initial feedback information predicted by the prediction model through the recommendation frequency information is to make the target object give feedback information less than the initial feedback information for the target content under the overly high recommendation frequency information, that is, the higher the value of the recommendation frequency information, the smaller the score reflected by the adjusted feedback information. Therefore, based on the characteristic that the recommendation frequency information and the feedback information change inversely, the initial feedback information can be adjusted, and when adjusting the initial feedback information, the adjustment amplitude information of the initial feedback information can also be set, and the adjustment amplitude information is positively correlated with the recommendation frequency information, that is, the higher the value of the recommendation frequency information, the greater the amplitude of the reduction of the initial feedback information.

[0141] More specifically, the influence of recommendation frequency information on the feedback information of the target object can be defined by the following relational expression:

[0142]

[0143] Among them, y ui It can represent the initial feedback information of target object u to target content i, e t (u,i) can represent the recommendation frequency information corresponding to the target content i. It can represent feedback information after adjustment of recommendation frequency information.

[0144] In this embodiment, the initial feedback information is adjusted by adjusting the adjustment amplitude information that is proportional to the recommendation frequency information, and feedback information that changes in the opposite direction to the recommendation frequency information is obtained as the feedback information of the target object to the target content. When the recommendation frequency information is too high, the target object will give feedback information that is smaller than the initial feedback information to the target content, thereby achieving correction in the embryonic stage of the information cocoon and solving the information cocoon problem.

[0145] In an exemplary embodiment, in the above step S220, the content recommendation model to be trained may determine the target content recommended to the target object in the following manner:

[0146] Step S220a, obtaining state information associated with the target object; the state information is obtained based on object feature information of the target object, historical recommended content recommended for the target object, weight information of the historical recommended content, and feedback information of the target object on the historical recommended content;

[0147] Step S220b: Determine the target content to be recommended to the target object through the content recommendation model to be trained and the state information.

[0148] Among them, the object feature information can be understood as a representation vector that characterizes the preference of the target object.

[0149] Among them, the weight information can be used to represent the recommendation probability of the content.

[0150] In a specific implementation, when the content recommendation model to be trained determines the target content recommended to the target object from the first sample data, in addition to the model itself, it can also refer to the object feature information representing the target object's preference, the historical recommended content recommended for the target object, the weight information of the historical recommended content, and the target object's feedback information on the historical recommended content, etc., which may affect the recommendation result and are associated with the target object. The content recommendation model to be trained determines the target content recommended to the target object in combination with the state information associated with the target object.

[0151] In this embodiment, by combining the state information associated with the target object to determine the target content recommended to the target object by the content recommendation model to be trained, the accuracy of the recommended target content can be improved.

[0152] In an exemplary embodiment, in the above step S240, adjusting the content recommendation model to be trained based on the feedback information to obtain the target content recommendation model includes:

[0153] Step S240a, adjusting the model parameters of the content recommendation model to be trained based on the feedback information to obtain a new content recommendation model;

[0154] Step S240b, updating the state information associated with the target object according to the target content and the feedback information corresponding to the target content;

[0155] Step S240c, through the new content recommendation model and the updated status information associated with the target object, determine the new target content recommended to the target object again, and obtain feedback information for the new target content through the trained prediction model until the content recommendation accuracy for the target object reaches the threshold, and obtain the target content recommendation model.

[0156] In a specific implementation, during the training process of the content recommendation model to be trained, as the content recommendation model interacts with the target object, the content recommendation model will gradually learn the preferences of the target object. In this process, the state information associated with the target object will also change accordingly. Therefore, after recommending the target content to the target object once, on the one hand, the model parameters of the content recommendation model to be trained can be adjusted based on the feedback information of the target object on the target content simulated by the prediction model to obtain a new content recommendation model, so that the new content recommendation model can improve the accuracy of content recommendation for the target object compared with the content recommendation model to be trained. On the other hand, the state information associated with the target object can be updated in time according to the target content recommended each time and the feedback information of the target object on the target content. Through the new content recommendation model and the updated state information associated with the target object, the new target content recommended to the target object is determined again and the feedback information of the target object on the new target content is obtained through the trained prediction model until the accuracy of content recommendation for the target object reaches a threshold, and the target content recommendation model is obtained to improve the adaptability of the recommendation results of the content recommendation model to the recommended object.

[0157] More specifically, for the training of the content recommendation model, the objective function of the training can be to maximize the following expected function in the PPO algorithm:

[0158]

[0159] The optimization goal is to obtain the content recommendation model θ that maximizes the cumulative benefit, that is, in the current state information s t Recommended target content t The probability π θ (a t |s t ), where ∈ is a hyperparameter used to control the maximum single-step update amplitude of the content recommendation model θ, by using the function clip (x,a,b) to truncate the value of the variable x to the interval [a,b]. old Represents the content recommendation model that generates the interaction data, that is, the version of the current content recommendation model before the update. is an advantage function representing the cumulative return, which is realized as:

[0160]

[0161] in, The definition is It represents the time series residual of the value function V, while γ is a hyperparameter representing the discount factor, and λ∈[0,1] is a hyperparameter that weighs the deviation and variance. The value function V is defined as follows:

[0162]

[0163] The key to optimizing the target content recommendation model is to let the pre-trained prediction model answer each question raised by the content recommendation model to be trained, that is, to give the feedback information of the target objects among the 1411 sample objects in the first sample data to the target content among the 3327 sample contents, that is, the satisfaction score r t . Due to r t It is not given by the real user but by the prediction model, so this score is also defined as the counterfactual reward score.

[0164] In this embodiment, during the training process of the content recommendation model, feedback information is given by the prediction model. After the training is completed, the knowledge and information in the prediction model will be transferred to the target content recommendation model, so that the target content recommendation model can make automatic reasoning decisions in the interaction with real objects. As a result, the trained target content recommendation model will be able to grasp the preferences of all objects, and the target content recommendation model can intelligently avoid repeated recommendation behaviors, thereby solving the information cocoon problem in the recommendation system from the root.

[0165] In order to evaluate whether the trained target content recommendation model can characterize and eliminate the information cocoon problem in the recommendation system, in an exemplary embodiment, after obtaining the target content recommendation model in step S240, the method further includes:

[0166] Step S250, obtaining, based on the first sample data, a performance parameter of the target content recommendation model for content recommendation of the sample object; the performance parameter is obtained when the recommendation frequency information of the target content recommendation model within a preset time period reaches a threshold;

[0167] Step S260: Evaluate the performance of the target content recommendation model based on the performance parameters.

[0168] The performance parameter may be the interaction time between the target content recommendation model and a sample object for content recommendation.

[0169] In the specific implementation, in order to evaluate whether the trained target content recommendation model can characterize and eliminate the information cocoon problem in the recommendation system, this embodiment is further improved based on the first sample data, and a mechanism is proposed to make the real object feel bored and exit the interaction under repeated recommendations. More specifically, random sampling is performed from the 1411 sample objects collected in the first sample data, and the sampled target objects interact with the trained target content recommendation model. After the target content recommendation model returns the recommendation result, that is, after the target content is recommended from the 3327 sample contents in the first sample data, the real interactive data of the target object on the target content in the first sample data is queried, and returned to the target content recommendation model as a supervision signal, and the process is executed repeatedly until the condition for the termination of the interaction is met: that is, if the currently recommended target content has an associated sample content exceeding the threshold with the historical recommended content recommended in the recent time period, the interaction is exited, and the interaction duration of this interaction process is obtained as a performance parameter to evaluate the performance of the target content recommendation model. Under this evaluation method, the target content recommendation model must ensure that each round of recommendations has a high accuracy rate, and that the recommended content cannot be repeated in consecutive recommendations, thereby testing the automatic balancing ability of the target content recommendation model. The ultimate evaluation goal is to allow the target content recommendation model to achieve the maximum cumulative benefit in interactive recommendations, that is, the sum of the scores of the target object's feedback information on each recommended content is the highest.

[0170] In this embodiment, the target content recommendation model is evaluated by obtaining performance parameters when the recommendation frequency information of the target content recommendation model reaches a threshold within a preset time period. The evaluation method provided in this embodiment can be intuitively compared horizontally with the currently commonly used recommendation strategies and models, and the performance results of different methods under the information cocoon problem can be intuitively and efficiently obtained.

[0171] refer to Figure 4 , is a flow chart of a content recommendation method proposed by an exemplary embodiment. In this embodiment, the method includes the following steps:

[0172] Step S410, obtaining a plurality of contents to be recommended and objects to be recommended;

[0173] Step S420 , selecting target recommended content from multiple contents to be recommended through a target content recommendation model, and pushing the target recommended content to the object to be recommended; the target content recommendation model is trained by the method of the above embodiment.

[0174] In a specific implementation, after obtaining multiple contents to be recommended and objects to be recommended, the target content recommendation model trained by the method of the above embodiment can be used to process each content to be recommended and the object to be recommended to obtain the interaction prediction information of the object to be recommended for each content to be recommended. Based on the interaction prediction information, the target recommended content with the highest score of the interaction prediction information is selected from each content to be recommended and pushed to the object to be recommended.

[0175] In this embodiment, the target content recommendation model trained by the above method can accurately characterize the repeated recommendation effect of the recommendation results, thereby overcoming the problem of information cocoon. Therefore, the target recommendation content pushed to the object to be recommended can avoid making the object to be recommended tired and is more in line with the preferences of the object to be recommended.

[0176] In an exemplary embodiment, in order to facilitate those skilled in the art to understand the embodiment of the present disclosure, the following will take the recommended content as a video as an example, combined with the attached Figure 5-7 The method is explained with a specific example. Figure 5 This is the overall flow chart of the training method for the content recommendation model. The specific steps are as follows:

[0177] (1) Step S510, collecting first sample data and second sample data.

[0178] In order to measure the performance of the content recommendation model and the information cocoon effect, that is, the impact of monotonous and repetitive recommendation results on user experience, we can obtain the true preferences of all users for all videos in a set, that is, know all the values ​​in a "user-video" matrix as the first sample data. In addition, in order to train the prediction model, we can also obtain sample data other than the full exposure matrix as the second sample data.

[0179] For example, refer to Figure 3 , collected 1411 users and 3327 representative videos as the first sample data of full exposure, and further collected interaction data consisting of 7176 users and 10729 videos as the second sample data. Since the second sample data is used to train the prediction model, full exposure is not required.

[0180] (2) Step S520, constructing a model framework that combines a causal inference model based on counterfactual reasoning and an offline reinforcement learning framework.

[0181] refer to Figure 6, which is a schematic diagram of the overall model framework that combines the causal inference model based on counterfactual reasoning and the offline reinforcement learning framework. It includes four modules: a prediction model (also called a causal user model, Causal UserModel), a state tracker based on the Transformer model (State Tracker), an interactive recommendation strategy based on reinforcement learning (RL Agent, i.e., content recommendation model), and a real evaluation environment (RealEnvironment). The functions of each module are introduced as follows:

[0182] The prediction model further includes a prediction unit and an adjustment unit. The prediction unit is used to predict the initial feedback information of the sample object on the sample content. The adjustment unit is used to adjust the repeated recommendation results in the dynamic interactive recommendation, that is, to give a negative reward signal.

[0183] The state tracker is based on the Transformer model. The state tracker can automatically extract the user preference from the representation vector e u And a series of representation vectors of historically recommended videos Automatically extract the most relevant information for the current recommended video. Figure 6 The representation vector of the state tracker in will undergo a transformation process, and the user vector e u After a simple feedforward neural network (FFN), a new vector expression e′ is obtained. u , and the processing of the video representation vector is defined as the operator: The ⊙ symbol represents the element-wise multiplication, g t is a gating vector, which automatically controls the weight of the embedding of the corresponding recommended video according to the prediction model or the feedback information given by the real user. Its specific definition is as follows:

[0184]

[0185] Among them, W and b are parameters that need to be learned during the training process, and Concat(a,b) is a vector concatenation symbol, which functions to concatenate vectors a and b.

[0186] Among them, a t represents the action taken by the interactive recommendation strategy at time t. An action represents the recommendation of a video i. Therefore, the representation vector e of an action a is a And the video recommended by this action i The standard vector of is equivalent, that is, e a =e i . represents the interaction state at time t, which should contain as much interaction information as possible, including the representation vector e representing the user's preference information u , and the video information recommended to the user during the entire interaction trajectory (i.e. from time 1 to time t-1) is expressed as: The role of the state tracker is to extract the truly useful parts from this information. t Indicates that at time t the system makes a recommended action a t After that, we get the feedback information given by the user, which is a scalar score representing the degree of satisfaction. Maximizing the cumulative reward of the entire trajectory sequence is the goal of the entire interactive strategy optimization. Among them, the reward signal can be the satisfaction score given by the real online user, or the counterfactual reward given by the trained prediction model.

[0187] (3) Training Model

[0188] Step S530, the first stage of model training is to train the prediction model through the second sample data, so that the trained prediction model can grasp the preferences of 7176 users for 10729 videos.

[0189] During the training process of the prediction model, the prediction model will follow Figure 7 The causal structure diagram shown in the figure is as follows. In the figure, the U node represents the user's preference, the I node represents the characteristics of the video, the R node represents the feedback information given by the user, the Y node represents the user's real preference, and the E node represents the user's real preference. t It represents the frequency of the current recommendation, indicating the degree of being trapped in the information cocoon. is a random variable E t Intuitively, if a video or a certain type of video is recommended repeatedly, the recommendation frequency information E t will be too large, and the user will feel bored, and then give a negative feedback signal R relative to the real preference Y. The shaded nodes in the figure represent hidden variables, which cannot be directly observed through historical data.

[0190] Figure 7 The first diagram in Figure 7 The causal diagram in (a) is a common assumption in current recommendation system models, that is, user feedback information is only determined by user preferences and product characteristics. Based on the traditional assumption, this paper explicitly models the recommendation frequency information E t , and get Figure 7 The causal structure diagram shown in (b) assumes that the user's final feedback signal is determined by two paths:

[0191] ①(U,I)→Y→R: This path describes the impact of the user’s real interest on the final feedback information. It can be implemented through the traditional DeepFM (Deep Factorization Machines, a model that includes neural networks and factorization machines) recommendation model, or it can be implemented by other recommendation models.

[0192] ②I→E t →R: This path describes the impact of recommendation frequency information on the user’s final feedback information.

[0193] Step S540, the second stage is the training of the interactive recommendation strategy based on reinforcement learning.

[0194] In the second stage of training, the reward signal of the interactive recommendation strategy based on reinforcement learning is given by the prediction model obtained in the first stage of training. At this time, the recommendation frequency information of the video can no longer be obtained from the original collected historical data, but is obtained by counterfactual reasoning, such as Figure 7 As shown in (c), the original recommendation frequency information E needs to be cut off t The impact of user feedback Y on this path is replaced by the new real-time inference Replace the original E t .

[0195] During the training process of this stage, the interactive information in the sample data is not used. Instead, the prediction model trained in the first stage is used to conduct a large number of efficient interactions with the interactive recommendation strategy. In this process, the prediction model plays the role of a user simulator. Through the counterfactual reasoning module, the prediction model can adjust the repeated recommendation behavior of the interactive recommendation strategy, thereby allowing the interactive recommendation strategy module to learn the ability to automatically avoid repeated recommendations and jump out of the information cocoon.

[0196] The method provided by the present disclosure is dedicated to explicitly modeling and evaluating information cocoons in an interactive recommendation environment. Specifically, by adopting causal reasoning, a more detailed modeling of user preferences in interactive recommendations is performed in an offline reinforcement learning framework, and the impact of the user's true preferences and the repeated recommendation effect on the user's final experience is distinguished, thereby avoiding and solving the information cocoon problem from the root, and the method disclosed in the present disclosure can also be extended to more common traditional static models.

[0197] After experimental verification, the method provided by the present disclosure has achieved good results in solving the information cocoon problem in the recommendation system. The specific conclusions are as follows: (1) The prediction model based on counterfactual causal inference can correctly characterize the repeated recommendation effect in the recommendation results, thereby detecting and correcting the information cocoon in its infancy. (2) The interactive recommendation strategy based on reinforcement learning can well grasp the counterfactual reward signal given by the prediction model, so as to make adaptive plan adjustments in real-time interaction with real users. The experimental results show that no matter how the environment changes, the method proposed by the present disclosure can work well, and the effect is far better than the comparison algorithm.

[0198] It should be understood that, although the various steps in the flowcharts involved in the above-mentioned embodiments are displayed in sequence according to the indication of the arrows, these steps are not necessarily executed in sequence according to the order indicated by the arrows. Unless there is a clear explanation in this article, the execution of these steps does not have a strict order restriction, and these steps can be executed in other orders. Moreover, at least a part of the steps in the flowcharts involved in the above-mentioned embodiments can include multiple steps or multiple stages, and these steps or stages are not necessarily executed at the same time, but can be executed at different times, and the execution order of these steps or stages is not necessarily to be carried out in sequence, but can be executed in turn or alternately with other steps or at least a part of the steps or stages in other steps.

[0199] It can be understood that the same / similar parts between the various embodiments of the above method in this specification can refer to each other, and each embodiment focuses on the differences from other embodiments. For related points, please refer to the description of other method embodiments.

[0200] Based on the same inventive concept, the embodiments of the present disclosure also provide a content recommendation model training device for implementing the above-mentioned content recommendation model training method, and a content recommendation device for implementing the above-mentioned content recommendation method.

[0201] Figure 8 is a structural block diagram of a training device for a content recommendation model according to an exemplary embodiment. Figure 8 The device includes: a first acquisition unit 810, a recommendation unit 820, a simulation unit 830 and an adjustment unit 840, wherein:

[0202] The first acquisition unit 810 is configured to acquire first sample data; the first sample data includes interaction information between the sample object and the sample content;

[0203] The recommendation unit 820 is configured to obtain target content recommended to the target object by the content recommendation model to be trained based on the first sample data; the target object is any object in the sample objects, and the target content belongs to the sample content in the first sample data;

[0204] The simulation unit 830 is configured to execute the prediction model completed by training, and simulate the feedback information of the target object on the target content; the prediction model is used to predict the initial feedback information of the target object on the target content, and adjust the predicted initial feedback information based on the recommendation frequency information corresponding to the target content, so as to obtain the feedback information of the target object on the target content; the recommendation frequency information represents the number of times the associated content is recommended for the target object, and the associated content is the content that matches the target content in at least one attribute;

[0205] The adjustment unit 840 is configured to adjust the content recommendation model to be trained based on the feedback information to obtain a target content recommendation model.

[0206] In an exemplary embodiment, the above device further includes:

[0207] A second acquisition unit is configured to acquire second sample data; the second sample data includes interaction information between the sample object and the sample content;

[0208] A frequency determination unit is configured to execute, based on a plurality of sample contents corresponding to the same sample object in the second sample data, obtaining sample recommendation frequency information corresponding to each sample content of the same sample object; the sample recommendation frequency information represents the number of associated sample contents whose timestamps are before each sample content, and the timestamp represents the time when each sample content is sampled to train the prediction model;

[0209] The training unit is configured to train the prediction model to be trained based on the sample recommendation frequency information and the interaction information between the sample object and the sample content in the second sample data to obtain a trained prediction model.

[0210] In an exemplary embodiment, the frequency determination unit is further configured to execute, based on difference information between multiple sample contents corresponding to the same sample object, determining associated sample contents whose timestamps are before the target sample contents; the target sample contents belong to multiple sample contents corresponding to the same sample object; obtaining the time difference between the timestamps corresponding to the target sample contents and the associated sample contents; and determining, based on the difference information and the time difference, sample recommendation frequency information corresponding to the target sample contents.

[0211] In an exemplary embodiment, the training unit is configured to execute sampling of positive sample content and negative sample content corresponding to each sample object from the second sample data based on the interactive information in the second sample data; the positive sample content represents the content for which the sample object has made positive feedback, and the negative sample content represents the content for which the sample object has made negative feedback; for each sample object, first feedback information of the sample object on the positive sample content and second feedback information on the negative sample content are obtained based on the sample recommendation frequency information of the positive sample content and the sample recommendation frequency information of the negative sample content; based on the loss value between the first feedback information and the second feedback information, the model parameters of the prediction model to be trained are adjusted until a preset number of training times is reached or the loss value converges, thereby obtaining a trained prediction model.

[0212] In an exemplary embodiment, the training unit is further configured to determine adjustment amplitude information for initial feedback information based on recommendation frequency information; the adjustment amplitude information is positively correlated with the recommendation frequency information; the initial feedback information is adjusted according to the adjustment amplitude information to obtain feedback information that changes in the opposite direction to the recommendation frequency information as feedback information of the target object on the target content.

[0213] In an exemplary embodiment, the recommendation unit 820 is further configured to execute the acquisition of state information associated with the target object; the state information is obtained based on object feature information of the target object, historical recommended content recommended for the target object, weight information of the historical recommended content, and feedback information of the target object for the historical recommended content; and the target content recommended to the target object is determined through the content recommendation model to be trained and the state information.

[0214] In an exemplary embodiment, the adjustment unit 840 is further configured to adjust the model parameters of the content recommendation model to be trained based on the feedback information to obtain a new content recommendation model; and, based on the target content and the feedback information corresponding to the target content, update the status information associated with the target object; through the new content recommendation model and the updated status information associated with the target object, again determine the new target content recommended to the target object, and obtain feedback information for the new target content through the trained prediction model, until the content recommendation accuracy for the target object reaches a threshold, and the target content recommendation model is obtained.

[0215] In an exemplary embodiment, the above-mentioned device also includes an evaluation unit, which is configured to execute, based on the first sample data, obtaining performance parameters of the target content recommendation model for content recommendation to the sample object; the performance parameters are obtained when the recommendation frequency information of the target content recommendation model within a preset time period reaches a threshold; based on the performance parameters, the performance of the target content recommendation model is evaluated.

[0216] Fig. 9is a structural block diagram of a content recommendation device according to an exemplary embodiment. Fig. 9 The device includes: an acquisition unit 910 and a recommendation unit 920, wherein:

[0217] An acquisition unit 910 is configured to acquire a plurality of contents to be recommended and objects to be recommended;

[0218] The recommendation unit 920 is configured to execute a target content recommendation model to select target recommended content from multiple to-be-recommended contents and push it to the to-be-recommended object; the target content recommendation model is trained by the method of any one of claims 1 to 8.

[0219] Regarding the device in the above embodiment, the specific manner in which each module performs operations has been described in detail in the embodiment of the method, and will not be elaborated here.

[0220] Fig.10 1 is a block diagram of an electronic device 1000 for implementing a training method for a content recommendation model or a content recommendation method according to an exemplary embodiment. For example, the electronic device 1000 may be a mobile phone, a computer, a digital broadcast terminal, a messaging device, a game console, a tablet device, a medical device, a fitness device, a personal digital assistant, etc.

[0221] Reference Fig.10 , the electronic device 1000 may include one or more of the following components: a processing component 1002 , a memory 1004 , a power component 1006 , a multimedia component 1008 , an audio component 1010 , an input / output (I / O) interface 1012 , a sensor component 1014 , and a communication component 1016 .

[0222] The processing component 1002 generally controls the overall operation of the electronic device 1000, such as operations associated with display, phone calls, data communications, camera operations, and recording operations. The processing component 1002 may include one or more processors 1020 to execute instructions to complete all or part of the steps of the above-mentioned method. In addition, the processing component 1002 may include one or more modules to facilitate the interaction between the processing component 1002 and other components. For example, the processing component 1002 may include a multimedia module to facilitate the interaction between the multimedia component 1008 and the processing component 1002.

[0223] The memory 1004 is configured to store various types of data to support operations on the electronic device 1000. Examples of such data include instructions for any application or method operating on the electronic device 1000, contact data, phone book data, messages, pictures, videos, etc. The memory 1004 may be implemented by any type of volatile or non-volatile storage device or a combination thereof, such as a static random access memory (SRAM), an electrically erasable programmable read-only memory (EEPROM), an erasable programmable read-only memory (EPROM), a programmable read-only memory (PROM), a read-only memory (ROM), a magnetic memory, a flash memory, a magnetic disk, an optical disk, or a graphene memory.

[0224] The power supply component 1006 provides power to various components of the electronic device 1000. The power supply component 1006 may include a power management system, one or more power supplies, and other components associated with generating, managing, and distributing power for the electronic device 1000.

[0225] The multimedia component 1008 includes a screen that provides an output interface between the electronic device 1000 and the user. In some embodiments, the screen may include a liquid crystal display (LCD) and a touch panel (TP). If the screen includes a touch panel, the screen may be implemented as a touch screen to receive input signals from the user. The touch panel includes one or more touch sensors to sense touch, slide, and gestures on the touch panel. The touch sensor may not only sense the boundaries of the touch or slide action, but also detect the duration and pressure associated with the touch or slide operation. In some embodiments, the multimedia component 1008 includes a front camera and / or a rear camera. When the electronic device 1000 is in an operating mode, such as a shooting mode or a video mode, the front camera and / or the rear camera may receive external multimedia data. Each front camera and rear camera may be a fixed optical lens system or have a focal length and optical zoom capability.

[0226] The audio component 1010 is configured to output and / or input audio signals. For example, the audio component 1010 includes a microphone (MIC), and when the electronic device 1000 is in an operating mode, such as a call mode, a recording mode, and a speech recognition mode, the microphone is configured to receive an external audio signal. The received audio signal can be further stored in the memory 1004 or sent via the communication component 1016. In some embodiments, the audio component 1010 also includes a speaker for outputting an audio signal.

[0227] I / O interface 1012 provides an interface between processing component 1002 and peripheral interface modules, such as keyboards, click wheels, buttons, etc. These buttons may include but are not limited to: home button, volume button, start button, and lock button.

[0228] The sensor assembly 1014 includes one or more sensors for providing various aspects of status assessment for the electronic device 1000. For example, the sensor assembly 1014 can detect the open / closed state of the electronic device 1000, the relative positioning of components, such as the display and keypad of the electronic device 1000, and the sensor assembly 1014 can also detect changes in the position of the electronic device 1000 or components of the electronic device 1000, the presence or absence of contact between the user and the electronic device 1000, the orientation or acceleration / deceleration of the device 1000, and the temperature change of the electronic device 1000. The sensor assembly 1014 may include a proximity sensor configured to detect the presence of nearby objects without any physical contact. The sensor assembly 1014 may also include an optical sensor, such as a CMOS or CCD image sensor, for use in imaging applications. In some embodiments, the sensor assembly 1014 may also include an acceleration sensor, a gyroscope sensor, a magnetic sensor, a pressure sensor, or a temperature sensor.

[0229] The communication component 1016 is configured to facilitate wired or wireless communication between the electronic device 1000 and other devices. The electronic device 1000 can access a wireless network based on a communication standard, such as WiFi, a carrier network (such as 2G, 3G, 4G or 5G), or a combination thereof. In an exemplary embodiment, the communication component 1016 receives a broadcast signal or broadcast-related information from an external broadcast management system via a broadcast channel. In an exemplary embodiment, the communication component 1016 also includes a near field communication (NFC) module to facilitate short-range communication. For example, the NFC module can be implemented based on radio frequency identification (RFID) technology, infrared data association (IrDA) technology, ultra-wideband (UWB) technology, Bluetooth (BT) technology and other technologies.

[0230] In an exemplary embodiment, the electronic device 1000 may be implemented by one or more application-specific integrated circuits (ASICs), digital signal processors (DSPs), digital signal processing devices (DSPDs), programmable logic devices (PLDs), field programmable gate arrays (FPGAs), controllers, microcontrollers, microprocessors, or other electronic components to perform the above methods.

[0231] In an exemplary embodiment, a computer-readable storage medium including instructions is also provided, such as a memory 1004 including instructions, and the instructions can be executed by the processor 1020 of the electronic device 1000 to complete the above method. For example, the computer-readable storage medium can be a ROM, a random access memory (RAM), a CD-ROM, a magnetic tape, a floppy disk, an optical data storage device, etc.

[0232] In an exemplary embodiment, a computer program product is further provided. The computer program product includes instructions. The instructions can be executed by the processor 1020 of the electronic device 1000 to complete the above method.

[0233] It should be noted that the above-mentioned devices, electronic devices, computer-readable storage media, computer program products, etc. may also include other implementation methods according to the description of the method embodiments. The specific implementation methods can refer to the description of the relevant method embodiments, which will not be described one by one here.

[0234] Those skilled in the art will readily appreciate other embodiments of the present disclosure after considering the specification and practicing the invention disclosed herein. The present disclosure is intended to cover any variations, uses or adaptations of the present disclosure that follow the general principles of the present disclosure and include common knowledge or customary techniques in the art that are not disclosed in the present disclosure. The description and examples are to be considered exemplary only, and the true scope and spirit of the present disclosure are indicated by the claims.

[0235] It should be understood that the present disclosure is not limited to the exact structures that have been described above and shown in the drawings, and that various modifications and changes may be made without departing from the scope thereof. The scope of the present disclosure is limited only by the appended claims.

Claims

1. A method for training a content recommendation model, characterized in that: include: Obtaining first sample data; The first sample data includes interaction information between the sample object and the sample content; Based on the first sample data, obtaining target content recommended by the content recommendation model to be trained to the target object; The target object is any object among the sample objects, and the target content belongs to the sample content in the first sample data; The trained prediction model is used to simulate the feedback information of the target object on the target content; the prediction model is used to predict the initial feedback information of the target object on the target content, and based on the recommendation frequency information corresponding to the target content and the condition that the feedback information and the recommendation frequency information change inversely, the predicted initial feedback information is adjusted to obtain the feedback information of the target object on the target content; the recommendation frequency information represents the number of times the associated content is recommended to the target object, and the associated content is the content that matches the target content in at least one attribute; The content recommendation model to be trained is adjusted based on the feedback information to obtain a target content recommendation model.

2. The method according to claim 1, characterized in that The prediction model is obtained by the following training method: Acquire second sample data; the second sample data includes interaction information between the sample object and the sample content; Based on a plurality of sample contents corresponding to the same sample object in the second sample data, obtaining sample recommendation frequency information corresponding to each sample content of the same sample object; The sample recommendation frequency information represents the number of associated sample contents whose timestamps are before each sample content, and the timestamps represent the time when each sample content is sampled to train the prediction model; Based on the sample recommendation frequency information and the interaction information between the sample object and the sample content in the second sample data, the prediction model to be trained is trained to obtain the trained prediction model.

3. The method according to claim 2, characterized in that The acquiring, based on a plurality of sample contents corresponding to a same sample object in the second sample data, sample recommendation frequency information corresponding to each sample content of the same sample object includes: Determine, based on difference information between the multiple sample contents corresponding to the same sample object, associated sample contents whose timestamps are before the target sample contents; the target sample contents belong to the multiple sample contents corresponding to the same sample object; Acquire a time difference between the timestamps corresponding to the target sample content and the associated sample content; Based on the difference information and the time difference, the sample recommendation frequency information corresponding to the target sample content is determined.

4. The method according to claim 2, characterized in that: The training of the prediction model to be trained based on the sample recommendation frequency information and the interaction information between the sample object and the sample content in the second sample data to obtain the trained prediction model includes: Based on the interactive information in the second sample data, positive sample content and negative sample content corresponding to each sample object are sampled from the second sample data; the positive sample content represents the content for which the sample object has made positive feedback, and the negative sample content represents the content for which the sample object has made negative feedback; For each sample object, obtaining first feedback information of the sample object on the positive sample content and second feedback information of the sample object on the negative sample content according to the sample recommendation frequency information of the positive sample content and the sample recommendation frequency information of the negative sample content; Based on the loss value between the first feedback information and the second feedback information, the model parameters of the prediction model to be trained are adjusted until a preset number of training times is reached or the loss value converges, thereby obtaining the trained prediction model.

5. The method according to claim 1, characterized in that The step of adjusting the predicted initial feedback information based on the recommendation frequency information corresponding to the target content and the condition that the feedback information and the recommendation frequency information change inversely to obtain the feedback information of the target object on the target content includes: Based on the recommendation frequency information, determining adjustment amplitude information for the initial feedback information; the adjustment amplitude information is positively correlated with the recommendation frequency information; The initial feedback information is adjusted according to the adjustment amplitude information to obtain feedback information that changes inversely with the recommendation frequency information as feedback information of the target object to the target content.

6. The method according to claim 1, characterized in that The content recommendation model to be trained determines the target content to be recommended to the target object in the following manner: Acquire state information associated with the target object; the state information is obtained based on object feature information of the target object, historical recommended content recommended for the target object, weight information of the historical recommended content, and feedback information of the target object for the historical recommended content; The target content recommended to the target object is determined by using the content recommendation model to be trained and the state information.

7. The method according to claim 6, characterized in that The adjusting the content recommendation model to be trained based on the feedback information to obtain a target content recommendation model includes: Based on the feedback information, adjusting the model parameters of the content recommendation model to be trained to obtain a new content recommendation model; and, updating state information associated with the target object according to the target content and feedback information corresponding to the target content; Through the new content recommendation model and the updated status information associated with the target object, the new target content recommended to the target object is determined again, and feedback information for the new target content is obtained through the trained prediction model until the content recommendation accuracy for the target object reaches a threshold, thereby obtaining a target content recommendation model.

8. The method according to claim 1, characterized in that After obtaining the target content recommendation model, it also includes: According to the first sample data, a performance parameter of the target content recommendation model for content recommendation of the sample object is obtained; the performance parameter is obtained when the recommendation frequency information of the target content recommendation model within a preset time period reaches a threshold; Based on the performance parameters, the performance of the target content recommendation model is evaluated.

9. A content recommendation method, characterized in that: include: Obtain multiple contents to be recommended and objects to be recommended; The target content recommendation model is used to select target recommended content from the multiple contents to be recommended and push it to the object to be recommended; the target content recommendation model is trained by the method described in any one of claims 1 to 8.

10. A training device for a content recommendation model, characterized in that: include: A first acquisition unit is configured to acquire first sample data; The first sample data includes interaction information between the sample object and the sample content; A recommendation unit is configured to obtain target content recommended to a target object by a content recommendation model to be trained based on the first sample data; The target object is any object among the sample objects, and the target content belongs to the sample content in the first sample data; a simulation unit configured to execute the prediction model completed by training, and simulate the feedback information of the target object on the target content; the prediction model is used to predict the initial feedback information of the target object on the target content, and adjust the predicted initial feedback information based on the recommendation frequency information corresponding to the target content and the condition that the feedback information and the recommendation frequency information change inversely, so as to obtain the feedback information of the target object on the target content; the recommendation frequency information represents the number of times the associated content is recommended for the target object, and the associated content is the content that matches the target content in at least one attribute; The adjustment unit is configured to adjust the content recommendation model to be trained based on the feedback information to obtain a target content recommendation model.

11. The device according to claim 10, characterized in that The device also includes: A second acquisition unit is configured to acquire second sample data; the second sample data includes interaction information between the sample object and the sample content; a frequency determination unit configured to execute, based on a plurality of sample contents corresponding to the same sample object in the second sample data, obtaining sample recommendation frequency information corresponding to each sample content of the same sample object; the sample recommendation frequency information represents the number of associated sample contents whose timestamps are before each sample content, and the timestamps represent the time when each sample content is sampled to train the prediction model; The training unit is configured to train the prediction model to be trained based on the sample recommendation frequency information and the interaction information between the sample object and the sample content in the second sample data to obtain the trained prediction model.

12. The device according to claim 11, characterized in that The frequency determination unit is further configured to determine the associated sample content whose timestamp is before the target sample content based on the difference information between the multiple sample contents corresponding to the same sample object; The target sample content belongs to the multiple sample contents corresponding to the same sample object; Acquire a time difference between the timestamps corresponding to the target sample content and the associated sample content; Based on the difference information and the time difference, the sample recommendation frequency information corresponding to the target sample content is determined.

13. The device according to claim 11, characterized in that The training unit is configured to perform sampling of positive sample content and negative sample content corresponding to each sample object from the second sample data based on the interaction information in the second sample data; the positive sample content represents the content for which the sample object has made positive feedback, and the negative sample content represents the content for which the sample object has made negative feedback; For each sample object, first feedback information of the sample object on the positive sample content and second feedback information on the negative sample content are obtained according to the sample recommendation frequency information of the positive sample content and the sample recommendation frequency information of the negative sample content respectively; based on the loss value between the first feedback information and the second feedback information, the model parameters of the prediction model to be trained are adjusted until the preset number of training times is reached or the loss value converges, thereby obtaining the trained prediction model.

14. The device according to claim 10, characterized in that The simulation unit is further configured to determine adjustment amplitude information for the initial feedback information based on the recommendation frequency information; the adjustment amplitude information is positively correlated with the recommendation frequency information; the initial feedback information is adjusted according to the adjustment amplitude information to obtain feedback information that changes in the opposite direction to the recommendation frequency information as feedback information of the target object to the target content.

15. The device according to claim 10, characterized in that The recommendation unit is further configured to execute acquisition of status information associated with the target object; the status information is obtained based on object feature information of the target object, historical recommended content recommended for the target object, weight information of the historical recommended content, and feedback information of the target object for the historical recommended content; and determine the target content recommended to the target object through the content recommendation model to be trained and the status information.

16. The device according to claim 15, characterized in that The adjustment unit is further configured to adjust the model parameters of the content recommendation model to be trained based on the feedback information to obtain a new content recommendation model; and update the status information associated with the target object according to the target content and the feedback information corresponding to the target content; determine the new target content recommended to the target object again through the new content recommendation model and the updated status information associated with the target object, and obtain feedback information for the new target content through the trained prediction model, until the content recommendation accuracy for the target object reaches a threshold, and the target content recommendation model is obtained.

17. The device according to claim 10, characterized in that The device also includes an evaluation unit configured to execute, based on the first sample data, obtaining performance parameters of the target content recommendation model for content recommendation of the sample object; the performance parameters are obtained when the recommendation frequency information of the target content recommendation model within a preset time period reaches a threshold; based on the performance parameters, the performance of the target content recommendation model is evaluated.

18. A content recommendation device, characterized in that: include: An acquisition unit is configured to acquire a plurality of contents to be recommended and objects to be recommended; The recommendation unit is configured to execute a target content recommendation model to select target recommended content from the multiple contents to be recommended and push it to the object to be recommended; the target content recommendation model is trained by the method described in any one of claims 1 to 8.

19. An electronic device, characterized in that: include: processor; a memory for storing instructions executable by the processor; The processor is configured to execute the instructions to implement the method as claimed in any one of claims 1 to 9.

20. A computer-readable storage medium, characterized in that: When the instructions in the computer-readable storage medium are executed by a processor of an electronic device, the electronic device is enabled to perform the method as claimed in any one of claims 1 to 9.

21. A computer program product, comprising instructions, characterized in that: When the instructions are executed by a processor of an electronic device, the electronic device is enabled to perform the method as claimed in any one of claims 1 to 9.

Images