Content processing method, device, storage medium and electronic device

Abstract

The disclosed embodiments provide a content processing method, device, storage medium and electronic device, which filter the content related to the user's explicit negative feedback behavior in the filtering link in the recommendation calculation process of the recommendation system; and suppress the ranking weight of the content related to at least the user's implicit negative feedback behavior in the detailed sorting link. Thus, in the calculation process of the recommendation system, the content related to the user's explicit and implicit negative feedback behavior is filtered and suppressed in a targeted manner, effectively reducing the parts of the recommended content that the user does not like or is not interested in, effectively ensuring the accuracy of the recommendation system, and effectively improving the user experience.

Description

Technical Field

[0001] The embodiments of the present disclosure relate to the field of data processing technology. More specifically, the embodiments of the present disclosure relate to a content processing method, device, storage medium and electronic device. Background Art

[0002] This section is intended to provide a background or context to the embodiments of the disclosure that are recited in the claims, and no description herein is admitted to be prior art by inclusion in this section.

[0003] Recommendation algorithms have been widely used in various user services on the Internet. However, some current recommendation algorithms have inaccurate filtering methods for content that users may not like, resulting in a large amount of content recommended to users that still contains content that users are not interested in. Summary of the invention

[0004] In the recommendation algorithm, the following methods are generally used to eliminate content that users may not like in the recommended content. One method is to block all content related to the above tags and not recommend it to the user based on the user's obvious dislike behavior, such as based on the tags that the user clicks to block the content. However, since these content tags may not be comprehensive, it is impossible to limit the same type of content from being recommended to the user; the other method is to express the user's preference for content through scoring, but the scoring may be affected by other factors that may be related to the user's preference. When other factors are inconsistent with the user's preference, it is easy to cause the user to dislike the content but the score is not low and it is recommended, which is also inaccurate. It can be seen that the above methods may cause the final recommended content to still contain content that the user does not like, resulting in a poor user experience.

[0005] To this end, an improved solution is urgently needed to accurately filter content that users dislike in the recommendation algorithm to improve the user experience.

[0006] In this context, embodiments of the present disclosure provide a content processing method, an apparatus, a storage medium, and an electronic device.

[0007] According to a first aspect of the present disclosure, a content processing method is provided, which is applied to a recommendation system. The recommendation calculation process of the recommendation system includes: a rough sorting stage, a filtering stage and a fine sorting stage of the content; the method includes:

[0008] In the filtering step, content related to the user's explicit negative feedback behavior is filtered;

[0009] In the detailed ranking stage, the ranking weight of the content at least related to the implicit negative feedback behavior of the user is suppressed.

[0010] According to a second aspect of the present disclosure, a content processing device is provided, which is applied to a recommendation system. The calculation process of the recommendation system includes: a rough sorting stage, a filtering stage and a detailed sorting stage of the content; the device includes:

[0011] An explicit negative feedback processing module, used to filter content related to the user's explicit negative feedback behavior in the filtering stage;

[0012] The implicit negative feedback processing module is used to suppress the ranking weight of the content at least related to the implicit negative feedback behavior of the user in the detailed ranking stage.

[0013] According to a third aspect of the present disclosure, there is provided a storage medium on which a computer program is stored. When the computer program is executed by a processor, the content processing method as described in any embodiment of the first aspect is implemented.

[0014] According to a fourth aspect of the present disclosure, there is provided an electronic device, including:

[0015] Processor; and

[0016] A memory, configured to store executable instructions of the processor;

[0017] The processor is configured to perform, via executing the executable instructions:

[0018] A content processing method as described in any embodiment of the first aspect.

[0019] According to the disclosed embodiment, the content related to the user's explicit negative feedback behavior is filtered in the filtering link in the recommendation calculation process of the recommendation system; and the ranking weight of the content related to at least the user's implicit negative feedback behavior is suppressed in the detailed sorting link. Thus, in the calculation process of the recommendation system, the content related to the user's explicit and implicit negative feedback behavior is filtered and suppressed in a targeted manner, effectively reducing the parts of the recommended content that the user does not like or is not interested in, effectively ensuring the accuracy of the recommendation system, and effectively improving the user experience. BRIEF DESCRIPTION OF THE DRAWINGS

[0020] The above and other objects, features and advantages of the exemplary embodiments of the present disclosure will become readily understood by reading the detailed description below with reference to the accompanying drawings. In the accompanying drawings, several embodiments of the present disclosure are shown in an exemplary and non-limiting manner, in which:

[0021] Figure 1 A schematic diagram showing a recommendation calculation process of a recommendation system in an example of the present disclosure is shown.

[0022] Figure 2 A flow chart of a content processing method in an embodiment of the present disclosure is shown.

[0023] Figure 3 A schematic diagram of the execution flow of step S201 of the content processing method in an embodiment of the present disclosure is shown.

[0024] Figure 4 A schematic diagram showing the principle of a user negative feedback model in an embodiment of the present disclosure is shown.

[0025] Figure 5 A schematic diagram of the execution flow of step S202 of the content processing method in an embodiment of the present disclosure is shown.

[0026] Figure 6 A flowchart of a method for training a machine learning model in one embodiment of the present disclosure is shown.

[0027] Figure 7 A block diagram of a content processing device in an embodiment of the present disclosure is shown.

[0028] Figure 8 A schematic diagram showing a storage medium in an embodiment of the present disclosure; and

[0029] Fig. 9 A block diagram of an electronic device in an embodiment of the present disclosure is shown.

[0030] In the drawings, the same or corresponding reference numerals represent the same or corresponding parts. DETAILED DESCRIPTION

[0031] The principles and spirit of the present disclosure will be described below with reference to several exemplary embodiments. It should be understood that these embodiments are provided only to enable those skilled in the art to better understand and implement the present disclosure, and are not intended to limit the scope of the present disclosure in any way. On the contrary, these embodiments are provided to make the present disclosure more thorough and complete, and to fully convey the scope of the present disclosure to those skilled in the art.

[0032] Those skilled in the art will appreciate that the embodiments of the present disclosure may be implemented as a system, device, apparatus, method or computer program product. Therefore, the present disclosure may be specifically implemented in the following forms, namely: complete hardware, complete software (including firmware, resident software, microcode, etc.), or a combination of hardware and software.

[0033] According to an embodiment of the present disclosure, a content processing method, an apparatus, a storage medium, and an electronic device are provided.

[0034] In this document, any number of elements in the drawings is used for illustration rather than limitation, and any naming is used only for distinction and does not have any limiting meaning.

[0035] The principle and spirit of the present disclosure are explained in detail below with reference to several representative embodiments of the present disclosure. SUMMARY OF THE INVENTION

[0037] With the widespread application of the Internet and mobile Internet, users can obtain various contents through the Internet. The content refers to the general term for all information distributed and readable on the Internet, which has various types of presentation forms, such as articles, videos, music, etc.

[0038] Since users have different interests and preferences in content, different content needs to be distributed to different users through computer algorithms to meet the different personalized needs of each user. This is called personalized recommendation. The personalized recommendation is usually performed through a recommendation system.

[0039] like Figure 1 The recommendation calculation process of the recommendation system for personalized recommendation shown includes: content recall 101, rough sorting 102, filtering 103, and fine sorting. Exemplarily, the fine sorting is shown to include fine sorting 104 and re-sorting 105. In other embodiments, there may be no re-sorting 105, and this example is not limited thereto.

[0040] Among them, content recall can also be generally referred to as tag recall, for example, based on the matching degree between the user's interest tag and the content tag, the content that the user may be interested in is determined. Furthermore, the amount of data of the recalled content is huge and complex. The recalled content is sorted, scored and filtered according to the user's interest, content quality, etc., and the content with the lowest ranking is removed to finally obtain recommended content and recommend it to the user.

[0041] The rough sorting link 102 can roughly sort and score the recalled content, sort them according to the scores, and take out the parts with higher rough sorting scores for subsequent links. Among them, the characteristics of the content based on the rough sorting scores can be obtained by statistics or abstracted. Taking graphic articles as an example, the characteristic dimensions of the scoring reference include text relevance, number of views, etc. The number of views can be obtained by statistics, and the text relevance is calculated by text matching. The characteristics and number of characteristics of the rough sorting scores are closely related to the content. Different types of content may have completely different characteristics and number of characteristics.

[0042] The filtering step 103 may be used to filter low-quality parts of the rough sorting results obtained after the rough sorting step 102, for example, filtering out content with tags related to bias (vulgarity, violence, disgusting, etc.), low page views, or old content.

[0043] The refined ranking step 104 is used to score the filtered results, sort them according to the refined ranking scores, and filter out the contents with lower scores to form refined ranking results.

[0044] The re-ranking step 105 can be used to re-rank at least part of the content in the refined ranking results (such as the part with higher scores), and finally obtain recommended content to recommend to the user.

[0045] It is understandable that the steps of recall, rough sorting, filtering, fine sorting, and re-ranking in the recommendation calculation process can be changed. For example, filtering can be added after recall, rough sorting, and fine sorting. For example, filtering can be omitted when the content quality is good, and re-ranking can be omitted when the recommendation accuracy requirement is not high. Figure 1 This is just a representative example.

[0046] In the recommendation calculation process, the following methods are used to remove content that users may not be interested in, low-quality content, etc.

[0047] One of the methods is to directly block content with the content tag "not interested" based on the user's selection of the content tag. For example, if the user clicks on the "pet" type of content that he does not like, the recommendation system will not recommend "pet" related content to the user. However, although this method is simple, it has limitations. For example, if you choose not to like the tag of a certain content source (such as an author or media), the author's content will not be recommended to the user in the future. But if the user actually does not like the type of content, the same type of content from other authors may still be pushed to the user. In addition, this filtering method is powerless for content that the user has skipped and did not choose to browse, or for content that the user is not interested in.

[0048] Another method is to express user preferences by scoring content tags, such as coarse ranking and fine ranking. However, when the recommendation system makes recommendations, it is often affected by popular behaviors, resulting in a large amount of content that users may not be interested in but may be popular in the scoring and sorting results, which fails to achieve the purpose of suppressing content that users are not interested in. Moreover, relying solely on scoring cannot artificially and controllably suppress the recommendation of a certain type of content in order to adjust the feedback sensitivity of the recommendation system. The higher the feedback sensitivity, the more timely the adjustment of the recommended content in response to user feedback behavior. For example, if a user browses a certain type of content, the next recommendation will give more information about this type of content.

[0049] Based on the above methods, the final recommended content may contain a large amount of content that users are not interested in or dislike, affecting the user experience.

[0050] In view of this, the present disclosure further strengthens the analysis of explicit and implicit negative feedback behaviors of users on content in the recommendation system, and performs corresponding filtering and ranking score suppression on content that is not suitable for recommendation, thereby effectively improving the user experience.

[0051] Here, the definition of the "negative feedback behavior" is first explained. "Negative feedback behavior" is the opposite of "positive feedback behavior". "Positive feedback behavior" expresses the user's interest in, preference for, and liking of content, such as browsing, clicking, etc. Correspondingly, "negative feedback behavior" refers to the behavior of not expressing interest, preference, or negativity towards the content. "Negative feedback behavior" can be further divided into "explicit negative feedback behavior" and "implicit negative feedback behavior". "Explicit negative feedback behavior" refers to the user's active negative feedback behavior, such as clicking the Dislike button on the recommended content, clicking the Reduce Recommendation button for content of a certain type of tag, etc. "Implicit negative feedback behavior" refers to the user's lack of positive active reading behavior after receiving the content recommendation display, such as quickly swiping across the displayed content without clicking to start reading.

[0052] After introducing the basic principles of the present disclosure, various non-limiting embodiments of the present disclosure are described in detail below.

[0053] It should be particularly noted that the content processing method, content processing device, etc. involved in the embodiments of the present disclosure collect user feedback on the content only with the user's permission and consent (such as electronic agreement, confirmation of authorization, etc.) to improve the user experience while ensuring the security of user data.

[0054] Exemplary Method Embodiments

[0055] refer to Figure 2 FIG. 1 is a flow chart showing a content processing method in an embodiment of the present disclosure. The content processing method can be applied to a recommendation system and can be applied to a recommendation calculation process of the recommendation system. Figure 2 In the example introduced Figure 1 The recommended calculation process is used as a reference.

[0056] The content processing method comprises:

[0057] Step S201: In the filtering phase, content related to the user's explicit negative feedback behavior is filtered.

[0058] Step S201 is executed in the filtering link 103 to strengthen the filtering effect of the filtering link 103. In addition, since the amount of content data flowing through the recommendation calculation process is reduced step by step, filtering the content that is obviously disliked by users in advance as far upstream as possible can also effectively reduce the amount of data that needs to be processed in subsequent links.

[0059] In some embodiments, step S201 may be applied in the filtering stage 103 to filter the content. In a possible implementation example, the operation logic of step S201 may implement an explicit negative feedback processing module that can be used in the filtering stage, which may be implemented based on software, hardware, or a combination of software and hardware.

[0060] Step S202: In the detailed sorting stage, suppressing the sorting weight of the content at least related to the implicit negative feedback behavior of the user.

[0061] In some embodiments, the detailed sorting stage may be a re-ranking stage or a fine sorting stage, that is, step S202 may be applied in the re-ranking stage or the fine sorting stage, so as to suppress the sorting weight of at least the content targeted by the implicit negative feedback behavior when the amount of content-related data in the re-ranking stage or the fine sorting stage is relatively small. The content with the suppressed score will be positioned relatively backward in the sorting results and is more likely to be filtered out, thereby efficiently and accurately reducing the part of the recommended content that the user is not interested in.

[0062] It should be noted that Figure 2 In the example, step S202 is applied to the re-arrangement stage to improve the processing efficiency. However, it is understandable that step S202 can be applied to the fine arrangement stage, and can be selected according to the processing efficiency, accuracy, etc., and is not limited to the above example. In a possible implementation example, the operation logic of step S202 can be implemented as an implicit negative feedback processing module that can be used in the fine arrangement stage or the re-arrangement stage, and the implicit negative feedback processing module can be implemented based on software, hardware, or a combination of software and hardware.

[0063] The implementation of each step in the above content processing method is described below through multiple embodiments.

[0064] like Figure 3 As shown, a schematic diagram of the process of executing step S201 in an exemplary embodiment of the present disclosure is shown.

[0065] refer to Figure 3 , the process includes steps S301 to S303.

[0066] Step S301: determining and storing negative content attributes among the content attributes targeted by the explicit negative feedback behavior.

[0067] Step S302: matching the negative content attribute with the content attribute of the matched content to obtain a matching result;

[0068] Step S303: filtering the matched contents whose content attributes satisfy the preset matching conditions in the matching results.

[0069] On the one hand, step S301 is described.

[0070] In some embodiments, the content attributes may include content tags. The content attributes may be obtained based on direct records or abstractions of the content. Taking graphic articles or videos as an example, the content attributes may exemplarily include: keywords, points of interest, content sources, categories, etc. For example, if the keyword "football" is recorded in an article, "football" may be one of the content attributes of the article; if a video is about lobster cooking, the user's point of interest in the video, "seafood", may be one of the content attributes of the video; if the author of an article is A, A may be one of the content attributes of the article; if the publisher of a video is self-media B, B may be one of the content attributes of the video; if a short video is about the interaction between a puppy and its owner, "pet" may be one of the content attributes of the video.

[0071] In some embodiments, the negative content attribute refers to the content attribute to which the user's explicit negative feedback behavior is directed. For example, when a user browses an article, he or she selects "dislike" to express his or her dislike for the article, and one or more of the multiple content attributes corresponding to the article may be determined as negative content attributes.

[0072] In some embodiments, the degree of dislike of the content attribute by the user can be analyzed through the explicit negative feedback behavior of the user. If it reaches a predetermined degree, the content attribute can be considered as a negative content attribute. In a possible implementation example, the negative content attribute can be screened based on the condition that the frequency of the explicit negative feedback behavior received in the retrospective period reaches a preset threshold. The retrospective period can be a time period from the current moment to a historical moment; the historical moment can be preset, or when judging whether a content attribute is a negative content attribute according to conditions, the earliest moment when the content attribute receives explicit negative feedback. In a possible example, with the target user as the target of content recommendation, the various explicit negative feedback behaviors made by the target user on the content in the retrospective period are collected, and the corresponding content attributes related to the content are counted. For example, user X browses content A, content B, content C, and content D during the retrospective period. User X gives explicit negative feedback to content A, such as disliking evaluation C1, and gives negative feedback to content C, such as disliking evaluation C2. Content attributes of content A and content C both include content attribute Y. Then, it can be counted that the number of times content attribute Y receives explicit negative feedback is 2. If the length of the retrospective period is 300 seconds, the frequency of user X's explicit negative feedback on content attribute Y is 1 / 150 times / second. By calculating whether the frequency reaches the preset threshold, reaching it means that the user dislikes a certain content attribute enough, and this content attribute can be confirmed as a negative content attribute.

[0073] Whether a content attribute is a negative content attribute is determined based on whether the statistical result of explicit negative feedback behavior (such as the above frequency) reaches a preset threshold. This is much more reliable than judging based on a single explicit negative feedback behavior, and helps to determine the real reason why users do not like a certain content.

[0074] Specifically, for example, if a user gives explicit negative feedback of "dislike" to a certain content, but does not specify the reason for disliking the content, that is, which content attribute of the content is targeted, each content attribute in the content is considered to have received an explicit negative feedback behavior. Similarly, by accumulating the number of explicit negative feedbacks for each content attribute in the retrospective period to obtain the frequency, when the frequency reaches the preset threshold, it means that the user may have repeatedly expressed his dislike for a certain content attribute, and it can be confirmed that the user does not like this content attribute, that is, the reason for the user's explicit negative feedback behavior is clarified.

[0075] Of course, the frequency calculation method may not be limited to the above example. In a possible embodiment, for a content attribute, the following formula (1) may be used as a condition to determine whether the content attribute is a negative content attribute.

[0076]

[0077] Wherein, n represents the number of times the content involved in the content attribute receives explicit negative feedback from a user during the lookback period, Δt represents the lookback period; β is the frequency smoothing parameter; μ is the weight coefficient; and b is the bias term.

[0078] In order to improve the accuracy of the obtained negative content attributes, in some embodiments, the conditions for determining the negative content attributes can be made more stringent. Optionally, the negative content attributes can be screened based on the condition that the content has received explicit negative feedback and has not received positive feedback during the retrospective period. That is, among the n times in formula (1), only the number of times the content has received explicit negative feedback and has not received positive feedback is counted, because once a content attribute-related content receives positive feedback after receiving explicit negative feedback, for example, after the user clicks "dislike" on a content A with content attribute M, and then browses a content B with this content attribute M, it may be explained that the content attribute is not actually disliked by the user, but may be associated with other content attributes in the same content that are disliked by the user, resulting in a misjudgment.

[0079] In some possible situations, the explicit negative feedback behaviors made by users may not be accurate and credible. For example, the explicit negative feedback behaviors caused by user misoperation (even if there is a clear reason, it may be caused by misoperation) may not conform to the user's actual interest preferences. If these unreliable explicit negative feedback behaviors are excluded, the negative feedback attributes obtained will be more accurate, which is conducive to avoiding the adverse consequences that the content related to the content attributes that the user may be interested in is blocked due to the user's misoperation.

[0080] Accordingly, in some embodiments, the content processing method may further include at least one of the following:

[0081] 1) Identify credible explicit negative feedback behaviors.

[0082] In some embodiments, credible explicit negative feedback behavior can be determined based on reference information formed by user selection. For example, the reference information is used to prove that the user clearly dislikes a certain content, such as the reason. If the user clearly expresses the reason for disliking, such as "dislike the content source", "dislike the author", "dislike the pet", etc., it can be initially considered as credible explicit negative feedback. The data of these reasons can be formed by the user's selection through the displayed options, or it can be formed by semantic recognition based on the text or voice input by the user.

[0083] 2) Correct or eliminate unreliable explicit negative feedback behaviors.

[0084] In some embodiments, in order to verify whether the user's current explicit negative feedback behavior is correct, it can be compared with the historical explicit negative feedback data of the content targeted by the explicit negative feedback behavior. Optionally, the first reason can be obtained based on the historical negative feedback data, and compared with the second reason of the current explicit negative feedback behavior to see if they are consistent; if they are consistent, it means that the current explicit negative feedback behavior is credible; otherwise, the current explicit negative feedback behavior is not credible.

[0085] In some embodiments, the types of the first reason and the second reason exemplarily include at least one of the following: content bias; dislike of content attributes related to content details; dislike of content source. Among them, content bias refers to content that is not positive, such as vulgar, bloody, violent, disgusting, etc.; dislike of content attributes related to content details refers to, for example, the keywords, points of interest, categories, etc. of the content that the user does not like; dislike of content source refers to, for example, the author, media, etc. of the content that the user does not like. It can be understood that the type setting of the above reasons can be changed according to needs and different content types. For example, under the type of dislike of content attributes related to content details, sub-types such as dislike of keywords, dislike of points of interest, and dislike of categories can also be set; or types such as dislike of keywords, dislike of points of interest, and dislike of categories can be directly used to replace dislike of content attributes related to content details.

[0086] In some embodiments, the method of inferring and calculating the first reason may be to calculate the ratio of the number of historical explicit negative feedback behaviors corresponding to at least one reason in the historical explicit negative feedback behavior data to the total number of historical explicit negative feedback behaviors, and obtain the comparison result of the ratio and the preset ratio threshold value for inferring the first reason. For example, calculate the number of historical explicit negative feedback behaviors that a certain content has received a preference from a user group, and the ratio of the total number of historical explicit negative feedback behaviors received by the content, for example, 1 / 5; determine whether this ratio reaches a preset ratio threshold, for example, 1 / 6. Since 1 / 5>1 / 6, it indicates a higher possibility that the first reason is a preference. The preset ratio threshold may be a preset one, or it may be one of the ratios corresponding to various content-related reasons as the preset ratio threshold, such as a minimum value, an intermediate value, a maximum value, etc.

[0087] Based on the above, examples are given to illustrate the types of unreliable explicit negative feedback behaviors.

[0088] For example, if the number of times a certain article has been historically disliked by a user group for bias (vulgar, bloody, violent, disgusting, etc.) accounts for a relatively low proportion of the total number of historical dislikes of the article (the proportion is lower than a certain preset proportion threshold), and the current feedback of the article by a certain user is biased content, the low proportion of historical biased feedback indicates that the article may not be biased content, but this user has reported it as biased content, and the two are inconsistent, indicating that the user's feedback is not credible.

[0089] For another example, if the number of times an article has received historical feedback from a disliked content source accounts for a relatively low proportion of the total number of historical dislikes for the article and is currently fed back as a disliked content source by a user, the two do not match, indicating that the user's current feedback is not credible.

[0090] For another example, if the number of feedbacks of historical dislikes of specific content items (including specific categories, keywords, points of interest, etc.) for an article accounts for a relatively high proportion of the total number of historical dislikes for the article, and a user currently feedbacks it as a source dislike, here, according to the high proportion of dislikes of specific content items in the total number of historical dislikes, it means that the more likely reason for the dislike of the content is that the specific content details are disliked, while the current user's dislike of the content is a dislike of the content source, the two are inconsistent, indicating that the current dislike of the content source is not credible.

[0091] The current explicit negative feedback behavior of users who are judged to be untrustworthy can be excluded and not used as the basis for analyzing user preferences. Alternatively, it can be modified and continued to be used. For example, according to the analysis of the historical explicit negative feedback data of an article, it is found that 90% of the dislikes are content bias, and the current explicit negative feedback behavior of the content indicates that the user does not like the author of the content, then the current explicit negative feedback behavior can be considered to be modified to "content bias".

[0092] The matching principle in step S302 is explained. If the number of negative content attributes that match the content is greater among the various content attributes of the content, that is, the more negative content attributes the content has, it means that the user dislikes the content more. In a possible example, a preset matching condition can be set according to the number of matching hits, for example, the preset matching condition includes: the number of matching content attributes in the matched content reaches a preset threshold, etc. Thus, the matched content that meets the preset matching condition can be filtered.

[0093] In a possible example, the content attributes may include keywords, points of interest, content sources, categories, biases, etc., and may exist in text form. Accordingly, the matching between the negative content attributes and the content attributes of the matched content can be achieved by text similarity matching. In a possible example, since the content attributes may include types such as keywords, points of interest, content sources, categories, biases, etc., the preset thresholds may be set comprehensively in the preset matching conditions according to various types of content attributes. For example, content with 4 matching hits in points of interest and 2 matching hits in categories is filtered; or content with 5 matching hits in keywords, 4 matching hits in points of interest, and 2 matching hits in categories is filtered.

[0094] For each user as a different individual, a set of negative content attributes will be stored respectively, and the set of negative content attributes can exist in the form of a negative content attribute library. As the number of users browsing content increases, the negative content attributes in the corresponding negative content attribute library will increase, and the amount of data occupied will increase accordingly. In addition, there may be situations where the user's preferences change. For example, you may change your opinion of an author you originally disliked, or become interested in a point of interest you originally disliked, etc. Moreover, as time goes by, the older the negative content attributes, the more likely it is that the user will change their opinion. Therefore, in some embodiments, rules can be set to update the stored negative content attributes, such as eliminating negative content attributes that the user has changed their opinion of.

[0095] In some embodiments, the elimination of negative content attributes can be triggered according to the attribute occupied capacity and / or attribute survival time of the negative content attributes. The attribute occupied capacity refers to the storage space occupied by the negative content attributes; the attribute survival time refers to the time that the stored negative content attributes have existed, etc. In a possible example, the elimination process of the stored negative content attributes can be triggered by the attribute occupied capacity and / or attribute survival time reaching a preset threshold as a trigger condition.

[0096] In some embodiments, in the elimination process, the importance of each negative content attribute can be calculated to eliminate negative content attributes according to their importance, for example, eliminating negative content attributes whose importance is lower than a threshold, or eliminating one or more negative content attributes with the lowest importance. Optionally, the importance can be calculated based on the survival time and matching score of the negative content attribute. Among them, the matching score is positively correlated with the number of times the negative content attribute is matched, that is, the more times it is matched, the higher the matching score of the negative content attribute. The importance is negatively correlated with the survival time and positively correlated with the matching score, which means that the shorter the survival time of the negative content attribute and the more times it is matched, the higher the importance of the negative content attribute.

[0097] In an exemplary embodiment, the calculation method of the importance can be designed as shown in the following formula (2):

[0098]

[0099] Among them, Δt i is the survival time of the ith element (i.e., the stored negative content attribute), which can be the time period from the update time to the current time. η is the time parameter; α is the weight coefficient, s i is the matching score of the i-th element.

[0100] In some embodiments, each of the negative content attributes can be classified into multiple types, and each type can correspond to a reason for an explicit negative feedback behavior. For example, the type of reason may include: dislike content source, dislike category, dislike keyword, etc., and the corresponding negative content attribute can be associated with the type of the corresponding reason according to the relevance. For example, under the type of dislike content source, negative content attributes such as dislike author A and dislike media B are included; under the classification of dislike category, negative content attributes such as dislike makeup and pets are included; under the type of dislike keyword, negative content attributes such as immortal and sword fairy are included.

[0101] Each reason may be configured with an attribute capacity threshold and / or attribute survival time for updating the negative content attributes of the respective classification. That is, the updates (such as additions and eliminations) of the negative content attributes under the classification of each reason are independent of each other. For example, if the capacity, length or number of the negative content attributes stored corresponding to "dislike content source" reaches a preset threshold, the elimination process of the negative content attributes under "dislike content source" may be triggered, but the elimination process of the negative content attributes under the classification of other reasons such as "dislike category" may not be triggered.

[0102] In some embodiments, the user's feedback behavior can be collected to find changes in the user's interests, so as to update the negative content attributes accordingly. For example, if the user becomes uninterested in the content that was originally interesting to him, then the content attribute may become a negative content attribute and be added to the storage; or if the user becomes fond of the content that was originally disliked, then the negative content attribute related to the content will change and needs to be removed from the set of stored negative content attributes.

[0103] In a possible example, by collecting user feedback on content, if it is found that the user has made positive feedback on the content and the feedback is directed to a negative content attribute, the negative content attribute can be removed. The positive feedback behavior directed to the negative content attribute can be directly directed or inferred. For example, a user's positive feedback behavior directly indicates "liking pets", and "pets" was previously a negative content attribute that the user did not like, then "pets" can be removed from the stored negative content attribute set; or, user behavior analysis can be performed with reference to the principle of formula (1). A user's positive feedback behavior is "liking" a certain content. Based on the user's positive feedback behavior over a period of time, behavior analysis can be performed, and the number or frequency of positive feedback behaviors directed to a certain negative content attribute can be counted. When the preset threshold is reached, it means that the user has changed his or her opinion on this negative content attribute, and it can be removed from the stored negative content attribute set.

[0104] By updating the stored negative content attributes, such as eliminating excessive (corresponding to the attribute capacity threshold) and / or old (corresponding to the attribute survival time) negative content attributes, such as adding negative content attributes that users do not like and removing negative content attributes that have changed, it is possible to more accurately adapt to user preferences and filter content more accurately.

[0105] For example Figure 4 As shown, in some embodiments, the matching calculation and the update of the negative content attribute set 401 in the above embodiment can be realized by constructing a user negative feedback model 400. The user negative feedback model 400 can access the stored negative content attribute set. By inputting each content attribute of the rough sorting result into the user negative feedback model 400; the user negative feedback model 400 matches each content attribute 403 of each matched content 402 with each stored negative content attribute, and when the preset matching condition is met, the corresponding content is filtered.

[0106] Optionally, the user negative feedback model 400 can also update the stored set of negative content attributes 401. On the one hand, the user negative feedback model 400 can also trigger the elimination process of negative content attributes based on trigger conditions related to attribute occupied capacity and / or attribute survival time, which is schematically shown in the figure as content attribute 401a being eliminated. Further optionally, the elimination process of the attributed negative content attributes can be triggered according to the trigger conditions corresponding to each reason. On the other hand, the user negative feedback model 400 analyzes the user's real-time feedback behavior to add negative content attributes that the user does not like, which is schematically shown in the figure as the addition of content attribute 401b, and remove the negative content attributes that have been improved, which is schematically shown in the figure as the removal of content attribute 401c.

[0107] refer to Figure 5 As shown, a schematic diagram of the process of executing step S202 in an embodiment of the present disclosure is shown. Figure 5 The process includes:

[0108] Step S501: Collecting user interaction behavior data on content.

[0109] In some embodiments, the interactive behavior data may be recorded data of a user browsing content, which may include the user's feedback behavior on the content, and the feedback behavior may include at least the user's implicit negative feedback behavior, that is, for example, the behavior of skipping and not browsing the content. The feedback behavior may also include the user's positive feedback behavior, explicit negative feedback behavior, etc., and the preferences are analyzed by collecting various types of interactive behaviors of the user, so that the sorting weight of the content can be adjusted more accurately according to the preferences. The label of the corresponding content may also be determined, such as the category, keywords, content source, bias, etc. of the content. In addition, optionally, the interactive behavior data may also include content attributes of the content interacted by the user, such as content labels, including the category, keywords, content source, bias, etc. of the content.

[0110] Taking graphic articles, videos and other content as examples, the ID of the content that has been recommended and displayed to users in the recent period, the content attributes of the corresponding content, and the user's interactive behavior data on the displayed content can be collected. From the interactive behavior data, the user's feedback behavior on the displayed content can be obtained, for example, real-time feedback behavior on the recommended information content. Exemplarily, the feedback behavior can include positive feedback behavior, such as clicking to read, like, share, etc. Exemplarily, the feedback behavior can include explicit negative feedback behavior, such as clicking to dislike the article, clicking to dislike the author, and clicking to dislike the keyword tag on the article. Optionally, some characteristic information of the user's interactive behavior can be obtained through the interactive behavior data, such as the display exposure duration of the recommended information content, the user's reading duration, etc., which can be used to assist in, for example, judging the positive and negative degree of the content expressed by the user's feedback behavior.

[0111] Step S502: Calculate the preference weight coefficient of the targeted content attribute according to the collected interaction behavior data.

[0112] The preference weight coefficient represents the user's current preference probability for each content attribute. The content attributes vary with the recommended content. For example, the content attributes of information may include: content source, category, keyword, bias (such as whether it is vulgar, violent, scary, etc.), whether it is an article, whether it is a short video, etc.

[0113] The preference weight coefficient reflects the user's preference for content attributes, including the influence of the user's implicit negative feedback behavior. The preference weight coefficient can act on the ranking score of the content attributes in the re-ranking stage or the refined ranking stage, so that the ranking weight of the content attributes that are at least affected by the user's implicit negative feedback behavior in the re-ranking stage or the refined ranking stage is reduced, and the corresponding content will be placed at the back of the ranking, or even filtered out, and will not appear in the final recommended content.

[0114] In some embodiments, the user's preference value for the content attribute targeted by the feedback behavior may be calculated based on the feedback behavior, and then the preference value may be mapped to a preset value range to obtain the preference weight coefficient.

[0115] In some embodiments, the preference value may be calculated by a preset formula. The preset formula calculates the sum of two items, one of which is a preference item representing the user preference and the other is a tentative item used to represent the user preference.

[0116] The preference item may be related to the user's positive feedback behavior and negative feedback behavior on the content related to the content attributes. The more frequent the positive feedback behavior, the stronger the user's preference, and the more frequent the negative feedback behavior, the weaker the user's preference. In addition, the user's preference for content may also change over time, and the older the feedback behavior, the less reliable it may be. In a possible implementation example, the value of the preference item is positively correlated to the positive feedback index, and the positive feedback index includes the sum of the behavior preference weights of each of the user's interactive behaviors under the time decay effect at the time of their occurrence, and the value of the behavior preference weight is positively correlated with the positive degree of the interactive behavior and negatively correlated with the negative degree, and the time decay effect weakens in the positive direction along the time axis.

[0117] The probe item may be related to the exposure of the content to the user. The more times the content is exposed to the user without being browsed, the less the user prefers it. In a possible implementation example, the exposure situation can be represented by an exposure index, and the value of the probe item is negatively correlated to the exposure index. The exposure index includes the sum of the exposures of the content related to the content attribute under the time decay effect at the exposure moment; the time decay effect decreases positively along the time axis.

[0118] For example, the following formula (3) provides an example of a preset formula for calculating a preference value. Formula (3) is only an implementation method and is not a limitation.

[0119]

[0120] Among them, the item on the left of the plus sign is the preference item, and the item on the right of the plus sign is the trial item. p represents the positive feedback index, which is obtained by weighting the user's feedback behavior on the content on the time axis. Each weighted item is multiplied by the time decay coefficient obtained by the time difference between the time when the behavior occurs and the current time, and the weighted sum is finally obtained to obtain the positive feedback index. Positive feedback behavior can take positive values, and can be assigned different values ​​according to the degree of positivity. Negative feedback behavior can take negative values, and can be assigned different values ​​according to the degree of negativity. For example, a user's reading behavior of content B related to content attribute A at moment 1, 1 hour away from the current time, has a weight of 1, and the time decay coefficient of moment 1 is 0.1; the user's dislike behavior of content C related to content attribute A at moment 2, 30 minutes away from the current time, has a weight of -10, and the time decay coefficient of moment 2 is 0.2; the user's sharing behavior of content D related to content attribute A at moment 3, 15 minutes away from the current time, has a weight of 2, and the time decay coefficient of moment 3 is 0.4. The smaller the value of the time decay coefficient is, the greater the decay effect. In this example, it can be calculated that p=1*0.1+(-10)*0.2+2*0.4=-1.1.

[0121] e represents the exposure index, which is the sum of the time decay coefficients obtained at the exposure time of the corresponding content attributes. For example, the content is exposed at time 1, time 2, and time 3, 1 hour, 30 minutes, and 15 minutes away from the current time, respectively, and the time decay coefficients are 0.1, 0.2, and 0.4, respectively, and the sum is 0.7. It should be noted that the time decay coefficients used in the calculation of p and e can be the same or different.

[0122] α and β are biases in the preference items, and their values ​​may be different for different content attributes; γ and λ are the reconciliation coefficients between preference and trial, such as γ = 0.4, λ = 500, etc.

[0123] In some embodiments, in addition to calculating the preference value by a preset formula, the preference value of the content attribute can also be predicted by a trained machine learning model. In a possible example, the machine learning model can use a supervised classification method to classify the corresponding content attributes to predict the user's preference probability on different content attributes as the preference value. Exemplarily, the machine learning models that can be used include gradient boosting decision tree (GBDT), deep neural network, etc.

[0124] like Figure 6 As shown, a flow chart showing a method for training a machine learning model in an embodiment of the present disclosure is shown. The machine learning model can be used to predict preference values ​​for content attributes. The training method includes:

[0125] Step S601: Acquire training samples, where the training samples include user features and label information.

[0126] In some embodiments, the user characteristics include: user characteristics extracted from the user's network activities, and user attribute preference characteristics related to the content attribute extracted from the user interaction data of the content to which the content attribute belongs. Exemplarily, the user characteristics may include but are not limited to at least one of the following: whether the user is active, the length of the most recent session, the number of sessions on the day, the user's recent click rate, the user's recent negative feedback rate, etc. Exemplarily, the user attribute preference characteristics may include but are not limited to at least one of the following: the number of exposures, the number of clicks, the number of negative feedback behaviors, the browsing time, etc. of the content related to a certain content attribute.

[0127] The label information is used to indicate that the content attribute that receives the positive feedback of the user is a positive sample, and the content attribute that indicates that the user has no feedback or receives the negative feedback of the user is a negative sample. That is, the label information includes two types, first information indicating a positive sample and second information indicating a negative sample, such as 0 and 1.

[0128] Step S602: Use the training samples to train the machine learning model, so that the machine learning model is used to predict the probability value of the content attribute belonging to the positive sample according to the input user characteristics, as the preference value.

[0129] The machine learning model can be configured with a loss function, which measures the output error of the machine learning model and adjusts the parameters of the machine learning model until preset training conditions are met. For example, the output error is reduced to below a preset threshold or the number of training times reaches a preset number, indicating that the training is completed. The model can be applied to prediction scenarios to predict the user's preference values ​​for content attributes in each category based on the current user feature input.

[0130] In some embodiments, before calculating the preference weight coefficient according to the preference value, the preset screening rules can be used to screen out the content attributes that do not need to suppress the ranking score. Among them, the preset screening rules include at least one of the following: screening out content attributes whose preference values ​​are greater than the preset threshold; screening out content attributes whose exposure times are lower than the preset threshold; screening out content attributes whose interval from the latest exposure time to the current moment exceeds the preset threshold. Specifically, the preference value is greater than the preset threshold, which means that the user prefers the content attribute more, and there is no need to suppress its ranking score, so it is screened out. The exposure times are lower than the preset threshold, which means that the exposure times are small. The content attributes with small exposure times will not be recommended and do not need to suppress the ranking score, and can be screened out. The interval time between the latest exposure time and the current moment exceeds the preset threshold, which means that the old content attribute will not be recommended and does not need to suppress the ranking score, and can also be screened out. It can be understood that different content recommendation occasions may have different screening rules and parameter settings. Here, some examples of screening rules are given for information browsing scenarios such as graphic articles, videos, etc., but are not limited to this.

[0131] In some embodiments, the preference value may be mapped to a desired preset value range to obtain a preference weight coefficient. For example, the mapping method may be a linear mapping method. The preference weight coefficient may be obtained according to the preference value mapping by using the following example formula (4).

[0132]

[0133] Among them, θ is the preference weight coefficient, ɑ is the maximum value of the preference weight coefficient, and b is the minimum value of the preference weight coefficient; ε is the parameter value for setting the starting threshold of θ; P is the preference value; ɑ, b, ε can change corresponding to different content attributes. For example, corresponding to one content attribute, ɑ=1, b=0.4, ε=0.05, etc.

[0134] It should be noted that, in possible examples, preference weight coefficients may also be directly obtained by analyzing user interaction behavior data on content without generating preference values. For example, the probability value predicted by the machine learning model is directly used as the preference weight coefficient.

[0135] Back to Figure 5 , step S503: adjusting the sorting weight of the content attribute in the detailed sorting stage according to the preference weight coefficient.

[0136] In some embodiments, the larger the preference weight coefficient of the content attribute, the greater the preference degree, and the lower the inhibitory effect on the ranking weight should be, that is, the preference weight coefficient of the content attribute and its inhibitory effect on the ranking weight should be negatively correlated. Therefore, the calculation method for the adjustment can be, for example, the linear weighted algorithm of the following formula (5):

[0137]

[0138] Among them, s is the refined ranking score before adjustment, is the adjusted refined ranking score, θ is the preference weight coefficient, σ is the retention score bias corresponding to the content attribute, and different σ values ​​can be set for different content attributes.

[0139] In summary, the disclosed embodiment filters the content related to the user's explicit negative feedback behavior in the filtering link in the recommendation calculation process of the recommendation system; and suppresses the ranking weight of the content related to at least the user's implicit negative feedback behavior in the detailed sorting link. Thus, in the calculation process of the recommendation system, the content related to the user's explicit and implicit negative feedback behavior is filtered and suppressed in a targeted manner, effectively reducing the parts of the recommended content that the user does not like or is not interested in, effectively ensuring the accuracy of the recommendation system, and effectively improving the user experience.

[0140] The disclosed embodiment introduces content filtering based on explicit negative feedback behavior in the filtering link of the recommendation system, which is used to solve the experience problems directly related to the explicit negative feedback behavior of users, and can timely block the content attributes corresponding to the content that users do not like, so that users can have direct interactive control over the recommendation system. According to experiments, this explicit negative feedback processing experiment is used in a personalized news recommendation product, and the comparative effect test shows that the user dislike rate is reduced by about 28%. In addition, the ranking score suppression based on at least implicit negative feedback behavior is introduced in the rearrangement link of the recommendation system, which can controllably reduce the recommendation amount of content that users are not currently interested in, controllably set the feedback sensitivity for different content, and enhance the user's experience of using recommendations. According to experiments, the implicit negative feedback processing experiment is used in a personalized news recommendation product, and the comparative effect test shows that the user dislike rate is reduced by about 10%, and the average stay time per person is also steadily improved. By combining the two approaches, the user's dislike rate for recommended content can be further reduced.

[0141] Exemplary Devices

[0142] After introducing the content processing method of the exemplary embodiment of the present disclosure, Figure 7 A content processing device according to an exemplary embodiment of the present disclosure is described.

[0143] refer to Figure 7As shown, the content processing device 700 of the exemplary embodiment of the present disclosure may include: an explicit negative feedback processing module 701, which is used to filter the content related to the user's explicit negative feedback behavior in the filtering stage; an implicit negative feedback processing module 702, which is used to suppress the ranking weight of at least the content related to the user's implicit negative feedback behavior in the detailed sorting stage. Thus, in the calculation process of the recommendation system, the content related to the user's explicit and implicit negative feedback behavior is filtered and suppressed in a targeted manner, effectively reducing the parts of the recommended content that the user does not like or is not interested in, effectively ensuring the accuracy of the recommendation system, and effectively improving the user experience.

[0144] In some embodiments, the explicit negative feedback processing module 701 includes: a user negative feedback behavior processing module, used to determine and store negative content attributes among the content attributes targeted by the explicit negative feedback behavior; a matching filtering module, used to match the negative content attributes with the content attributes of the matched content to obtain a matching result; and filtering the matched content to which the content attributes of the matching result satisfy a preset matching condition belong.

[0145] In some embodiments, the user negative feedback behavior processing module includes: a user behavior analysis module, which is used to filter the negative content attributes based on the condition that the frequency of explicit negative feedback behaviors received during the retrospective period reaches a preset threshold.

[0146] In some embodiments, the user negative feedback behavior processing module is further used to filter the negative content attributes based on the condition that the content has been subjected to explicit negative feedback behavior and has not been subjected to positive feedback behavior during the retrospective period.

[0147] In some embodiments, the content processing device 700 further includes at least one of the following modules:

[0148] A behavior trust determination module, used to determine trustworthy explicit negative feedback behaviors;

[0149] The untrustworthy behavior processing module is used to correct or eliminate untrustworthy explicit negative feedback behaviors.

[0150] In some embodiments, the behavior trustworthy determination module is used to determine a trustworthy explicit negative feedback behavior based on reference information formed by user selection.

[0151] In some embodiments, the untrusted behavior processing module includes:

[0152] An untrustworthy behavior determination module is used to determine untrustworthy explicit negative feedback behavior based on historical explicit negative feedback behavior data of the content targeted by the explicit negative feedback behavior, wherein the untrustworthiness refers to that the first reason inferred based on the historical explicit negative feedback behavior data is inconsistent with the second reason of the explicit negative feedback behavior;

[0153] The untrusted behavior processing execution module is used to modify the explicit negative feedback behavior until the second reason is consistent with the first reason, or to abandon the explicit negative feedback behavior.

[0154] In some embodiments, the behavior judgment module is used to calculate the ratio of the number of historical explicit negative feedback behaviors corresponding to at least one cause in the historical explicit negative feedback behavior data to the total number of historical explicit negative feedback behaviors, and obtain a comparison result between the ratio and a preset ratio threshold for inferring the first cause.

[0155] In some embodiments, the types of the first reason and the second reason include at least one of the following: content preference; dislike for content attributes related to content details; dislike for content source.

[0156] In some embodiments, the content processing device 700 further includes:

[0157] The attribute elimination module is used to trigger the elimination process of the stored negative content attributes in response to the attribute occupied capacity and / or the attribute survival time reaching a preset threshold, including: calculating the importance according to the survival time and matching score of the negative content attribute; wherein the importance is negatively correlated with the survival time and positively correlated with the matching score; the matching score is positively correlated with the number of times the negative content attribute is matched and hit; and eliminating the negative content attributes according to the importance.

[0158] In some embodiments, each of the negative content attributes is classified into multiple types; each type corresponds to a cause of explicit negative feedback behavior; each classification is correspondingly configured with an attribute capacity threshold and / or attribute survival time for updating the negative content attributes of the respective classification.

[0159] In some embodiments, the content processing device 700 includes:

[0160] The attribute adjustment module is used to remove the negative content attributes that are subject to positive feedback behavior from the storage space.

[0161] In some embodiments, the preset matching condition includes: the number of content attributes of the matching hits of the matched content reaches a preset threshold.

[0162] In some embodiments, the implicit negative feedback processing module 702 includes: a user behavior collection module, which is used to collect user interaction behavior data on content; wherein the interaction behavior includes feedback behavior, and the feedback behavior at least includes implicit negative feedback behavior; an attribute preference calculation module, which is used to calculate the preference weight coefficient of the targeted content attribute based on the collected interaction behavior data; a score adjustment module, which is used to adjust the ranking weight of the content attribute in the detailed sorting link according to the preference weight coefficient; wherein the value of the preference weight coefficient of the content attribute is affected by the corresponding implicit negative feedback behavior and is reduced to suppress the ranking weight of the content attribute in the detailed sorting link, so as to suppress the appearance of related content in the generated recommended content.

[0163] In some embodiments, the attribute preference calculation module includes: an attribute preference weight coefficient calculation module, used to calculate the user's preference value for the content attribute targeted by the interactive behavior; and a weight mapping module, used to map the preference value to a preset value range to obtain the preference weight coefficient.

[0164] In some embodiments, the attribute preference weight coefficient calculation module includes: a formula calculation module, used to calculate the preference value through a preset formula, and the preset formula is expressed as the sum of a preference item for characterizing user preferences and a trial item for characterizing trial user preferences; wherein the value of the preference item is positively correlated with a positive feedback index, and the positive feedback index includes the sum of the behavior preference weights of each of the interactive behaviors of the user under the time decay effect at the time of their occurrence, and the value of the behavior preference weight is positively correlated with the positive degree of the interactive behavior and negatively correlated with the negative degree; the value of the trial item is negatively correlated with an exposure index, and the exposure index includes the sum of each exposure of the relevant content of the content attribute under the time decay effect at its exposure time; the time decay effect weakens positively along the time axis.

[0165] In some embodiments, the attribute preference calculation module includes: a model calculation module, which is used to predict the user's preference value or preference weight coefficient for the content attribute according to user characteristics through a trained machine learning model.

[0166] In some embodiments, the content processing device 700 includes: a model training module, which is used to train the machine learning model, including:

[0167] A sample acquisition module is used to acquire training samples, wherein the training samples include: user features and label information; wherein the user features include: user features extracted from the user's network activities, and user attribute preference features related to the content attributes extracted from the user interaction data of the content to which the content attributes belong; the label information is used to indicate that the content attributes that receive positive feedback from the user are positive samples, and that the content attributes that receive no feedback from the user or receive negative feedback from the user are negative samples;

[0168] A training execution module is used to train the machine learning model using the training samples so that the machine learning model is used to predict the probability value of the content attribute belonging to the positive sample according to the input user characteristics as a preference value or a preference weight coefficient.

[0169] In some embodiments, the content processing device 700 also includes: an attribute filtering module, which is used to execute, before the weight mapping module calculates the preference weight coefficient based on the preference weight coefficient of the content attribute, the following operations: using preset screening rules to screen out content attributes; wherein the preset screening rules include at least one of the following: screening out content attributes whose preference weight coefficients or preference weight coefficients are greater than a preset threshold; screening out content attributes whose exposure times are lower than a preset threshold; screening out content attributes whose interval between the most recent exposure time and the current time exceeds a preset threshold.

[0170] Since the functional modules of the content processing device 700 of the embodiment of the present disclosure are the same as the corresponding steps of the content processing method in the above embodiment, they will not be described in detail here.

[0171] Exemplary Storage Media

[0172] After introducing the content processing method and content processing device according to the exemplary embodiment of the present disclosure, Figure 8 A storage medium according to an exemplary embodiment of the present disclosure is described.

[0173] refer to Figure 8 As shown, a storage medium 800 according to an embodiment of the present disclosure is described, which may contain program code and can be run on a device, such as a personal computer, to implement the execution of each step and sub-step in the content processing method of the present disclosure. In this document, a readable storage medium can be any tangible medium containing or storing a program that can be used by or in combination with an instruction execution system, device or device.

[0174] The program code may be in any combination of one or more readable media. The readable medium may be a readable signal medium or a readable storage medium. The readable storage medium may be, for example, but not limited to, a system, device or component of electricity, magnetism, light, electromagnetic, infrared, or semiconductor, or any combination thereof. More specific examples (non-exhaustive list) of readable storage media include: an electrical connection with one or more wires, a portable disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination thereof.

[0175] Computer readable signal media may include data signals propagated in baseband or as part of a carrier wave, in which readable program code is carried. Such propagated data signals may take a variety of forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination of the above. Readable signal media may also be any readable medium other than a readable storage medium, which may send, propagate, or transmit a program for use by or in conjunction with an instruction execution system, apparatus, or device.

[0176] The program code embodied on the readable medium may be transmitted using any appropriate medium, including but not limited to wireless, wired, optical cable, RF, etc., or any suitable combination of the foregoing.

[0177] Program code for performing the operations of the present disclosure may be written in any combination of one or more programming languages, including object-oriented programming languages ​​such as Java, C++, etc., and conventional procedural programming languages ​​such as "C" or similar programming languages. The program code may be executed entirely on the user computing device, partially on the user computing device, partially on a remote computing device, or entirely on a remote computing device or server. In the case of a remote computing device, the remote computing device may be connected to the user computing device through any type of network, including a local area network (LAN) or a wide area network (WAN), or may be connected to an external computing device (e.g., using an Internet service provider to connect through the Internet).

[0178] Exemplary Electronic Devices

[0179] After introducing the storage medium of the exemplary embodiment of the present disclosure, next, reference is made to Fig. 9 An electronic device according to an exemplary embodiment of the present disclosure is described.

[0180] Fig. 9The electronic device 900 shown is merely an example and should not bring any limitation to the functions and scope of use of the embodiments of the present disclosure.

[0181] like Fig. 9 As shown, the electronic device 900 is in the form of a general computing device. The components of the electronic device 900 may include but are not limited to: at least one processing unit 910, at least one storage unit 920, and a bus 930 connecting different system components (including the storage unit 920 and the processing unit 910).

[0182] The storage unit stores program codes, which can be executed by the processing unit 810, so that the processing unit 910 performs the steps and sub-steps of the content processing method described in the above embodiment of the present disclosure. For example, the processing unit 910 can perform the following steps: Figure 2 , Figure 3 , Figure 5 The steps shown in etc.

[0183] In some embodiments, the storage unit 920 may include a volatile storage unit, such as a random access storage unit (RAM) 9201 and / or a cache storage unit 9202 , and may further include a read-only storage unit (ROM) 9203 .

[0184] In some embodiments, the storage unit 920 may also include a program / utility 9204 having a set (at least one) of program modules 9205, such program modules 9205 including but not limited to: an operating system, one or more application programs, other program modules, and program data, each of which or some combination may include an implementation of a network environment.

[0185] In some embodiments, bus 930 may include a data bus, an address bus, and a control bus.

[0186] In some embodiments, the electronic device 900 may also communicate with one or more external devices 1000 (e.g., keyboards, pointing devices, Bluetooth devices, etc.), and such communication may be performed through an input / output (I / O) interface 950. Optionally, the electronic device 900 also includes an explicit unit 940, which is connected to the input / output (I / O) interface 950 for explicit display. In addition, the electronic device 900 may also communicate with one or more networks (e.g., local area networks (LANs), wide area networks (WANs) and / or public networks, such as the Internet) through a network adapter 960. As shown, the network adapter 960 communicates with other modules of the electronic device 900 through a bus 930. It should be understood that, although not shown in the figure, other hardware and / or software modules may be used in conjunction with the electronic device 900, including but not limited to: microcode, device drivers, redundant processing units, external disk drive arrays, RAID systems, tape drives, and data backup storage systems, etc.

[0187] It should be noted that, although the recognition model generation device and several modules or submodules of the recognition device are mentioned in the above detailed description, this division is merely exemplary and not mandatory. In fact, according to an embodiment of the present disclosure, the features and functions of two or more units / modules described above can be concretized in one unit / module. Conversely, the features and functions of one unit / module described above can be further divided into multiple units / modules to be concretized.

[0188] In addition, although the operations of the disclosed method are described in a specific order in the drawings, this does not require or imply that the operations must be performed in this specific order, or that all the operations shown must be performed to achieve the desired results. Additionally or alternatively, some steps may be omitted, multiple steps may be combined into one step, and / or one step may be decomposed into multiple steps.

[0189] Although the spirit and principle of the present disclosure have been described with reference to several specific embodiments, it should be understood that the present disclosure is not limited to the disclosed specific embodiments, and the division of various aspects does not mean that the features in these aspects cannot be combined to benefit, and such division is only for the convenience of expression. The present disclosure is intended to cover various modifications and equivalent arrangements included in the spirit and scope of the attached claims.

Claims

1. A content processing method is applied to a recommendation system, wherein the recommendation calculation process of the recommendation system includes: The rough sorting, filtering and detailed sorting of content; The method comprises: In the filtering step, content related to the user's explicit negative feedback behavior is filtered; In the detailed sorting stage, suppressing the sorting weight of the content at least related to the implicit negative feedback behavior of the user; Wherein, suppressing the ranking weight of the content at least related to the implicit negative feedback behavior of the user includes: Collecting user interaction behavior data on the content; wherein the interaction behavior includes feedback behavior, and the feedback behavior at least includes implicit negative feedback behavior; Calculating a preference weight coefficient of the targeted content attribute based on the collected interactive behavior data; According to the preference weight coefficient, the ranking weight of the content attribute in the detailed ranking link is adjusted; wherein the value of the preference weight coefficient of the content attribute is reduced under the influence of the corresponding implicit negative feedback behavior to suppress the ranking weight of the content attribute in the detailed ranking link, so as to suppress the appearance of related content in the generated recommended content; The step of calculating the preference weight coefficient of the content attribute according to the collected interactive behavior data includes: The user's preference value for the content attribute targeted by the interactive behavior is calculated by a preset formula; the preference value is mapped to a preset value range to obtain a preference weight coefficient; the preset formula is expressed as the sum of a preference item for characterizing the user's preference and a probe item for characterizing the user's preference for probing; wherein the value of the preference item is positively correlated with a positive feedback index, the positive feedback index includes the sum of the behavioral preference weights of each of the user's interactive behaviors under the time decay effect at the time of their occurrence, the value of the behavioral preference weight is positively correlated with the positive degree of the interactive behavior and negatively correlated with the negative degree; the value of the probe item is negatively correlated with an exposure index, the exposure index includes the sum of each exposure of the relevant content of the content attribute under the time decay effect at the exposure time; the time decay effect weakens in the positive direction along the time axis; or Predicting the user's preference value or preference weight coefficient for the content attribute based on user characteristics through a trained machine learning model; wherein the training of the machine learning model includes: obtaining training samples, and the training samples include: user characteristics and label information; wherein the user characteristics include: user characteristics extracted based on the user's network activities, and user attribute preference characteristics related to the content attribute extracted based on user interaction data of the content to which the content attribute belongs; the label information is used to indicate that the content attribute that has received positive feedback from the user is a positive sample, and the content attribute that has no feedback from the user or has received negative feedback from the user is a negative sample; the machine learning model is trained using the training samples, so that the machine learning model is used to predict the probability value of the content attribute belonging to the positive sample based on the input user characteristics, as the preference value or preference weight coefficient.

2. The content processing method according to claim 1, characterized in that: The filtering of content related to the user's explicit negative feedback behavior includes: Determining and storing negative content attributes among the content attributes targeted by the explicit negative feedback behavior; Matching the negative content attribute with the content attribute of the matched content to obtain a matching result; The matched contents whose content attributes satisfy the preset matching conditions in the matching results are filtered.

3. The content processing method according to claim 2, characterized in that: The step of determining a negative content attribute from among the content attributes targeted by the explicit negative feedback behavior includes: The negative content attributes are filtered based on the condition that the frequency of the explicit negative feedback behavior received during the retrospective period reaches a preset threshold.

4. The content processing method according to claim 3, characterized in that: The determining of negative content attributes among the content attributes targeted by the explicit negative feedback behavior further includes: The negative content attribute is filtered based on the condition that the content receives explicit negative feedback and does not receive positive feedback during the lookback period.

5. The content processing method according to claim 1 or 3, characterized in that: Also includes at least one of the following: 1) Determine credible explicit negative feedback behaviors; 2) Correct or eliminate unreliable explicit negative feedback behaviors.

6. The content processing method according to claim 5, characterized in that: The determining of a reliable explicit negative feedback behavior includes: Determine credible explicit negative feedback behavior based on reference information formed by user selection.

7. The content processing method according to claim 5, characterized in that: The correction or elimination of unreliable explicit negative feedback behaviors includes: Determining untrustworthy explicit negative feedback behavior based on historical explicit negative feedback behavior data of the content targeted by the explicit negative feedback behavior, wherein untrustworthy means that the first reason inferred based on the historical explicit negative feedback behavior data is inconsistent with the second reason of the explicit negative feedback behavior; The explicit negative feedback behavior is modified to be consistent with the historical explicit negative feedback behavior data, or the explicit negative feedback behavior is eliminated.

8. The content processing method according to claim 7, characterized in that: The inferring the first cause according to the historical explicit negative feedback behavior data includes: The ratio of the number of historical explicit negative feedback behaviors corresponding to at least one cause in the historical explicit negative feedback behavior data to the total number of historical explicit negative feedback behaviors is calculated, and a comparison result between the ratio and a preset ratio threshold is obtained for inferring the first cause.

9. The content processing method according to claim 7, characterized in that: The types of the first reason and the second reason include at least one of the following: content preference; dislike for content attributes related to content details; dislike for content source.

10. The content processing method according to claim 2, characterized in that: Also includes: In response to the attribute occupied capacity and / or the attribute survival time reaching a preset threshold, triggering execution of a process for eliminating the stored negative content attributes, including: The importance is calculated according to the survival time and the matching score of the negative content attribute; wherein the importance is negatively correlated with the survival time and positively correlated with the matching score; and the matching score is positively correlated with the number of times the negative content attribute is matched; Negative content attributes are eliminated according to the importance.

11. The content processing method according to claim 10, characterized in that: Each of the negative content attributes is classified into multiple types; each type corresponds to a cause of explicit negative feedback behavior; each category is correspondingly configured with an attribute capacity threshold and / or attribute survival time for updating the negative content attributes of the respective category.

12. The content processing method according to claim 2, characterized in that: Also includes: Negative content attributes that are subject to positive feedback behavior are removed from the stored space.

13. The content processing method according to claim 2, characterized in that: The preset matching condition includes: the number of the matching hit content attributes of the matched content reaches a preset threshold.

14. The content processing method according to claim 1, characterized in that: Before mapping the preference value to a preset value range to obtain the preference weight coefficient, the method further includes: Use preset filtering rules to filter out content attributes; Among them, the preset screening rules include at least one of the following: screening out content attributes whose preference weight coefficients or preference weight coefficients are greater than a preset threshold; screening out content attributes whose exposure times are lower than a preset threshold; screening out content attributes whose interval between the most recent exposure time and the current time exceeds a preset threshold.

15. A content processing device, characterized in that: Applied to a recommendation system, the calculation process of the recommendation system includes: a rough sorting link, a filtering link and a detailed sorting link of the content; the device includes: An explicit negative feedback processing module, used to filter the content related to the user's explicit negative feedback behavior in the filtering stage; An implicit negative feedback processing module, used for suppressing the ranking weight of the content at least related to the implicit negative feedback behavior of the user in the detailed ranking stage; The implicit negative feedback processing module comprises: A user behavior collection module, used to collect user interaction behavior data on content; wherein the interaction behavior includes feedback behavior, and the feedback behavior at least includes implicit negative feedback behavior; An attribute preference calculation module, used to calculate a preference weight coefficient of a targeted content attribute based on the collected interaction behavior data; A score adjustment module, used to adjust the ranking weight of the content attribute in the detailed ranking link according to the preference weight coefficient; wherein the value of the preference weight coefficient of the content attribute is reduced under the influence of the corresponding implicit negative feedback behavior to suppress the ranking weight of the content attribute in the detailed ranking link, so as to suppress the appearance of related content in the generated recommended content; Wherein, the attribute preference calculation module is configured as follows: The user's preference value for the content attribute targeted by the interactive behavior is calculated by a preset formula; the preference value is mapped to a preset value range to obtain a preference weight coefficient; the preset formula is expressed as the sum of a preference item for characterizing the user's preference and a probe item for characterizing the user's preference for probing; wherein the value of the preference item is positively correlated with a positive feedback index, the positive feedback index includes the sum of the behavioral preference weights of each of the user's interactive behaviors under the time decay effect at the time of their occurrence, the value of the behavioral preference weight is positively correlated with the positive degree of the interactive behavior and negatively correlated with the negative degree; the value of the probe item is negatively correlated with an exposure index, the exposure index includes the sum of each exposure of the relevant content of the content attribute under the time decay effect at the exposure time; the time decay effect weakens in the positive direction along the time axis; or Predicting the user's preference value or preference weight coefficient for the content attribute based on user characteristics through a trained machine learning model; wherein the training of the machine learning model includes: obtaining training samples, and the training samples include: user characteristics and label information; wherein the user characteristics include: user characteristics extracted based on the user's network activities, and user attribute preference characteristics related to the content attribute extracted based on user interaction data of the content to which the content attribute belongs; the label information is used to indicate that the content attribute that has received positive feedback from the user is a positive sample, and the content attribute that has no feedback from the user or has received negative feedback from the user is a negative sample; the machine learning model is trained using the training samples, so that the machine learning model is used to predict the probability value of the content attribute belonging to the positive sample based on the input user characteristics, as the preference value or preference weight coefficient.

16. The content processing device according to claim 15, characterized in that: The explicit negative feedback processing module comprises: A user negative feedback behavior processing module, used to determine and store negative content attributes among the content attributes targeted by the explicit negative feedback behavior; The matching and filtering module is used to match the negative content attribute with the content attribute of the matched content to obtain a matching result; and filter the matched content to which the content attribute of the matching result meets a preset matching condition belongs.

17. The content processing device according to claim 16, characterized in that: The user negative feedback behavior processing module includes: The user behavior analysis module is used to filter the negative content attributes based on the condition that the frequency of the explicit negative feedback behavior received during the retrospective period reaches a preset threshold.

18. The content processing device according to claim 17, characterized in that: The user negative feedback behavior processing module is further used to filter the negative content attributes based on the condition that the content has been subjected to explicit negative feedback behavior and has not been subjected to positive feedback behavior during the retrospective period.

19. The content processing device according to claim 15 or 17, characterized in that: Also includes at least one of the following modules: A behavior trust determination module, used to determine trustworthy explicit negative feedback behaviors; The untrustworthy behavior processing module is used to correct or eliminate untrustworthy explicit negative feedback behaviors.

20. The content processing device according to claim 19, characterized in that: The behavior trustworthy determination module is used to determine a trustworthy explicit negative feedback behavior according to reference information formed by user selection.

21. The content processing device according to claim 16, characterized in that: The untrusted behavior processing module includes: An untrustworthy behavior determination module is used to determine untrustworthy explicit negative feedback behavior based on historical explicit negative feedback behavior data of the content targeted by the explicit negative feedback behavior, wherein the untrustworthiness refers to that the first reason inferred based on the historical explicit negative feedback behavior data is inconsistent with the second reason of the explicit negative feedback behavior; The untrusted behavior processing execution module is used to modify the explicit negative feedback behavior until the second reason is consistent with the first reason, or to abandon the explicit negative feedback behavior.

22. The content processing device according to claim 21, characterized in that: The behavior judgment module is used to calculate the ratio of the number of historical explicit negative feedback behaviors corresponding to at least one cause in the historical explicit negative feedback behavior data to the total number of historical explicit negative feedback behaviors, and obtain a comparison result between the ratio and a preset ratio threshold value for inferring the first cause.

23. The content processing device according to claim 21, characterized in that: The types of the first reason and the second reason include at least one of the following: content preference; dislike for content attributes related to content details; dislike for content source.

24. The content processing device according to claim 23, characterized in that: Also includes: The attribute elimination module is used to trigger the elimination process of the stored negative content attributes in response to the attribute occupied capacity and / or the attribute survival time reaching a preset threshold, including: The importance is calculated according to the survival time and the matching score of the negative content attribute; wherein the importance is negatively correlated with the survival time and positively correlated with the matching score; and the matching score is positively correlated with the number of times the negative content attribute is matched; Negative content attributes are eliminated according to the importance.

25. The content processing device according to claim 24, characterized in that: Each of the negative content attributes is classified into multiple types; each type corresponds to a cause of explicit negative feedback behavior; each category is correspondingly configured with an attribute capacity threshold and / or attribute survival time for updating the negative content attributes of the respective category.

26. The content processing device according to claim 16, characterized in that: include: The attribute adjustment module is used to remove the negative content attributes that are subject to positive feedback behavior from the storage space.

27. The content processing device according to claim 16, characterized in that: The preset matching condition includes: the number of the matching hit content attributes of the matched content reaches a preset threshold.

28. The content processing device according to claim 15, characterized in that: Also includes: The attribute filtering module is used to perform the following operations before the weight mapping module calculates the preference weight coefficient according to the preference weight coefficient of the content attribute: using preset screening rules to screen out content attributes; wherein the preset screening rules include at least one of the following: screening out content attributes whose preference weight coefficient or preference weight coefficient is greater than a preset threshold; screening out content attributes whose exposure times are lower than a preset threshold; screening out content attributes whose interval between the most recent exposure time and the current time exceeds a preset threshold.

29. A storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, it realizes: The content processing method according to any one of claims 1 to 14.

30. An electronic device, characterized in that: include: processor; as well as A memory, configured to store executable instructions of the processor; The processor is configured to perform, via executing the executable instructions: The content processing method according to any one of claims 1 to 14.

Images