Diversity-aware model training method, recommendation ranking method and device

Abstract

The disclosure provides a diversity perception model training method and device, and a recommendation ranking method and device. It relates to the technical field of artificial intelligence, and in particular to the technical fields of big data processing, deep learning, intelligent recommendation, etc. The specific implementation scheme is as follows: a plurality of samples and a plurality of sample corresponding perception labels are obtained; the user features, resource features and diversity perception features corresponding to the plurality of samples are input into a to-be-trained model to obtain the perception values of the users to the resources corresponding to the plurality of samples; and the to-be-trained model is trained according to the perception values and the perception labels corresponding to the plurality of samples to obtain a diversity perception model. According to the scheme of the disclosure, the diversity perception model can predict the diversity perception of different users to resources, thereby improving the accuracy of recommendations.

Description

Technical Field

[0001] This disclosure relates to the field of artificial intelligence technology, specifically to the fields of big data processing, deep learning, and intelligent recommendation. Background Technology

[0002] Recommendation systems have become the core of many e-commerce and multimedia platforms, providing personalized recommendations that offer users a diverse range of resources. However, different users perceive resource diversity differently. For example, some users enjoy a variety of resources and explore multiple categories, while others only like different resources within a specific category. Furthermore, user preferences can change depending on the individual, the time, and the content. Therefore, how to measure recommendation diversity and how to improve recommendation accuracy based on this diversity are two key challenges currently facing recommendation systems. Summary of the Invention

[0003] This disclosure provides a method for training a diversity perception model, a method for recommending and ranking, and an apparatus.

[0004] According to a first aspect of this disclosure, a method for training a diversity-aware model is provided, comprising:

[0005] Obtain multiple samples and their corresponding perceptual labels;

[0006] The user features, resource features, and diversity perception features corresponding to multiple samples are input into the model to be trained to obtain the user's perception value of resources corresponding to multiple samples.

[0007] The model to be trained is obtained by training the model based on the perceptual values ​​and perceptual labels corresponding to multiple samples.

[0008] According to a second aspect of this disclosure, a recommendation ranking method is provided, comprising:

[0009] The user characteristics, diversity perception characteristics, and resource characteristics of the candidate resources of the target user are input into the diversity perception model to obtain the target user's perception value of the candidate resources output by the diversity perception model.

[0010] Based on the target user's perceived value of the candidate resources, a recommended ranking list for the target user is generated;

[0011] The diversity perception model is obtained by training using the method provided in the first aspect.

[0012] According to a third aspect of this disclosure, a diversity perception model training apparatus is provided, comprising:

[0013] The first acquisition module is used to acquire multiple samples and the perceptual labels corresponding to the multiple samples;

[0014] The first input module is used to input the user features, resource features and diversity perception features corresponding to multiple samples into the model to be trained, so as to obtain the user's perception value of the resource corresponding to multiple samples.

[0015] The training module is used to train the model to be trained based on the perceptual values ​​and perceptual labels corresponding to multiple samples, so as to obtain a diversity perceptual model.

[0016] According to a fourth aspect of this disclosure, a recommendation sorting apparatus is provided, comprising:

[0017] The second input module is used to input the user characteristics, diversity perception characteristics and resource characteristics of the target user and the candidate resources into the diversity perception model to obtain the target user's perception value of the candidate resources output by the diversity perception model.

[0018] The generation module is used to generate a recommended ranking list for the target user based on the target user's perceived value of the candidate resources;

[0019] The diversity perception model is obtained by training using the method provided in the first aspect.

[0020] According to a fifth aspect of this disclosure, an electronic device is provided, comprising:

[0021] At least one processor;

[0022] Memory that is communicatively connected to at least one processor;

[0023] The memory stores instructions that can be executed by at least one processor, which enables the at least one processor to execute the diversity-aware model training method provided in the first aspect and / or the recommendation ranking method provided in the second aspect.

[0024] According to a sixth aspect of this disclosure, a non-transitory computer-readable storage medium is provided storing computer instructions, wherein the computer instructions are used to cause a computer to execute the diversity-aware model training method provided in the first aspect and / or the recommendation ranking method provided in the second aspect.

[0025] According to a seventh aspect of this disclosure, a computer program product is provided, comprising a computer program that, when executed by a processor, implements the diversity-aware model training method provided in the first aspect and / or the recommendation ranking method provided in the second aspect.

[0026] According to the technical solution disclosed herein, the diversity perception model can predict the diversity perception of resources by different users, thereby improving the accuracy of recommendations.

[0027] The above overview is for illustrative purposes only and is not intended to be limiting in any way. In addition to the illustrative aspects, embodiments, and features described above, further aspects, embodiments, and features of this application will become readily apparent from the accompanying drawings and the following detailed description. Attached Figure Description

[0028] In the accompanying drawings, unless otherwise specified, the same reference numerals throughout the various drawings denote the same or similar parts or elements. These drawings are not necessarily drawn to scale. It should be understood that these drawings depict only some embodiments disclosed in this application and should not be construed as limiting the scope of this application.

[0029] Figure 1 This is a flowchart illustrating a diversity perception model training method according to an embodiment of the present disclosure;

[0030] Figure 2 This is a schematic diagram of the architecture of a diversity perception model according to an embodiment of the present disclosure;

[0031] Figure 3 This is a schematic diagram illustrating the acquisition of a sensing tag according to an embodiment of the present disclosure;

[0032] Figure 4 This is a schematic diagram illustrating the determination of diversity perception features according to an embodiment of this disclosure;

[0033] Figure 5 This is a flowchart illustrating the recommended sorting method according to an embodiment of the present disclosure;

[0034] Figure 6 This is a schematic diagram illustrating the generation of a recommended ranking list for a target user according to an embodiment of this disclosure;

[0035] Figure 7 This is a schematic diagram illustrating the online evolutionary learning process of the fusion value model according to an embodiment of this disclosure;

[0036] Figure 8 This is a schematic diagram of the structure of a diversity perception model training device according to an embodiment of the present disclosure;

[0037] Figure 9 This is a schematic diagram of the structure of a recommendation sorting device according to an embodiment of the present disclosure;

[0038] Figure 10 This is a schematic diagram of a scenario where a diversity perception model is trained according to an embodiment of this disclosure;

[0039] Figure 11This is a schematic diagram of a scenario based on the recommended sorting according to an embodiment of this disclosure;

[0040] Figure 12 This is a schematic diagram of the structure of an electronic device used to implement the diversity perception model training method and / or recommendation ranking method of the embodiments of this disclosure. Detailed Implementation

[0041] The exemplary embodiments of this disclosure are described below with reference to the accompanying drawings, including various details of the embodiments to aid understanding, and should be considered merely exemplary. Therefore, those skilled in the art will recognize that various changes and modifications can be made to the embodiments described herein without departing from the scope of this disclosure. Similarly, for clarity and brevity, descriptions of well-known functions and structures are omitted in the following description.

[0042] The terms "first," "second," and "third," etc., used in the embodiments, claims, and accompanying drawings of this disclosure are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. Furthermore, the terms "comprising" and "having," and any variations thereof, are intended to cover non-exclusive inclusion, such as including a series of steps or units. A method, system, product, or apparatus is not necessarily limited to those steps or units explicitly listed, but may include other steps or units not explicitly listed or inherent to these processes, methods, products, or apparatuses.

[0043] In related technologies, solutions for diverse experiences often assume that the more diverse the better, and the richer the user experience, the better. However, this overlooks the differences in how different users perceive diverse experiences. Furthermore, from the perspective of user or content ecosystem diversity, it sacrifices many users' needs for immersive and continuous browsing. In addition, user preferences may dynamically change depending on the scenario.

[0044] In related technologies, methods for measuring recommendation diversity include:

[0045] (1) Number of categories displayed / distributed per person: Directly count the number of categories displayed or distributed per person;

[0046] (2) Maximum Marginal Relevance (MMR) method: This method can maintain relevance while reducing the redundancy of ranking results. In recommendation scenarios, it recommends relevant content to users while ensuring the diversity of recommendation results.

[0047] (3) Determinantal Point Process (DPP): By estimating the maximum a posteriori probability, the subset with the greatest relevance and diversity in the candidate resource set is found and recommended to the user.

[0048] (4) Information entropy: The information entropy of the category distribution in a set is used to represent the diversity of the set. The larger the entropy, the better the diversity of resources in the set.

[0049] (5) The diversity evaluation index (Intra-List Similarity, ILS) is obtained from formula (1):

[0050]

[0051] Where R is the set of products recommended to the user, k is the number of products, and Sim is the similarity between two objects i and j. The less similar the objects in the recommendation list are, the smaller the ILS(R) value is, and the better the diversity of the recommendation results.

[0052] Existing technologies for measuring recommendation diversity either rely on explicit labeling-based information entropy, ILS, and the average number of categories displayed / distributed per user, or implicit labeling-based DPP and MMR. However, a method that combines explicit and implicit methods to measure recommendation diversity is lacking. Therefore, it is impossible to comprehensively characterize and measure the diversity of users' perception of resources.

[0053] In order to at least partially solve one or more of the above-mentioned problems and other potential problems, this disclosure proposes a diversity perception model training method that enables the diversity perception model to predict the diversity perception of resources by different users, thereby improving the accuracy of recommendations.

[0054] This disclosure provides a method for training a diversity-aware model. Figure 1 This is a flowchart illustrating a diversity perception model training method according to an embodiment of the present disclosure. This method can be applied to a diversity perception model training device, which is located in an electronic device. The electronic device includes, but is not limited to, fixed devices and / or mobile devices. For example, fixed devices include, but are not limited to, servers, which can be cloud servers or ordinary servers. Mobile devices include, but are not limited to, mobile phones, tablets, and vehicle-mounted terminals. In some possible implementations, the diversity perception model training method can also be implemented by a processor calling computer-readable instructions stored in memory. Figure 1 As shown, the training method for this diversity perception model includes:

[0055] S101: Obtain multiple samples and their corresponding perceptual labels;

[0056] S102: Input the user features, resource features and diversity perception features corresponding to multiple samples into the model to be trained to obtain the user's perception value of resources corresponding to multiple samples.

[0057] S103: Train the model to be trained based on the perceptual values ​​and perceptual labels corresponding to multiple samples to obtain a diversity perceptual model.

[0058] In this embodiment of the disclosure, user features are used to characterize a user's personalized features. User features can be obtained by analyzing user logs stored in a log service system. For example, user features may include a user's activity level over a period of time. Another example is that user features may include the number of historical clicks / views. Yet another example is that user features may include the time when a user requests access to a resource. The above are merely illustrative examples and are not intended to limit all possible content of user features; they are simply not exhaustive.

[0059] In this embodiment of the disclosure, resource features are used to represent the characteristics of a resource. For example, resource features may include the type of resource, which may include video, live stream, text and image types, etc. Another example is that resource features may include the historical number of clicks / impressions of the resource among all users. Yet another example is that resource features may include the distribution of the resource's category in the agile evaluation sample, where the resource category may include entertainment, sports, social, and financial categories, etc. The above are merely illustrative examples and are not intended to limit all possible content of resource features; they are simply not exhaustive.

[0060] In this embodiment of the disclosure, diversity perception features are used to represent users' perception of the diversity of recommendations. Recommendation evaluation metrics may include Longest Continuous No-Click Num (LCN), impression percentage, click percentage, and Click-Through-Rate (CTR), etc. Various statistical values ​​may include minimum, maximum, average, variance, and distribution, etc.

[0061] In this embodiment, user data can be extracted by analyzing user logs, and user characteristics can be obtained based on the user data. Resource characteristics can be obtained by analyzing resource data. Diversity perception characteristics can be determined by using various statistical values ​​of different recommendation evaluation indicators for target characteristics within different preset time periods. Here, the target characteristics can be divided into explicit features and implicit features. Implicit features may include resource title similarity and Graph Collaborative Filtering (GCF) similarity; explicit features may include resource classification features at different levels of refinement, such as general categories, primary categories, secondary categories, and points of interest; wherein the level of refinement of general categories, primary categories, secondary categories, and points of interest gradually increases.

[0062] In this embodiment of the disclosure, the user's perceived value of resources can be represented by a value between 0 and 1. When the perceived value is close to 1, it means that the user perceives too much of the resources. When the perceived value is close to 0, it means that the user perceives too little of the resources. When the perceived value is close to a certain intermediate value between 0 and 1, it means that the user perceives the resources appropriately.

[0063] In this embodiment of the disclosure, feedback results from multiple survey questionnaires corresponding to different samples are obtained. These questionnaires inquire about users' perceived experience of resource diversity. Based on the feedback results from these multiple samples, perception tags corresponding to each sample are obtained. These perception tags can include three categories: too many, too few, and appropriate, used to characterize the user's perception of resource diversity.

[0064] Figure 2 A schematic diagram of the diversity perception model is shown, such as... Figure 2 As shown, the diversity perception model can be a distributed gradient boosting library (XGBoost) model. By inputting the user features, resource features, and diversity perception features of the samples into the tree-structured XGBoost model, the predicted values ​​of user perception of resources are obtained.

[0065] In this embodiment of the disclosure, the diversity perception model can be an XGBoost model, a Support Vector Machine (SVM) model, or a Logistic Regression (LR) model. To train a more accurate diversity perception model, it can be fitted using an XGBoost model with relatively good interpretability to predict the user's perceived value of resources.

[0066] The model to be trained is used to output the user's perceived value of a resource based on the user features, resource features, and diversity perception features corresponding to multiple samples included in the training data. Therefore, it can be understood that in this embodiment, the model to be trained may include at least one feature extraction model for extracting at least user features, resource features, and diversity perception features; and at least one prediction model for predicting the user's perceived value of the resource corresponding to each of the multiple samples. The diversity perception model is obtained by training the model to be trained using the training data; therefore, it has the same model structure as the model to be trained, the difference being that its model parameters are updated after training.

[0067] The technical solution of this disclosure introduces diversity perception features and combines them with user features and resource features, making the input factors considered during model training richer. Since diversity perception features cover explicit and implicit features, by combining explicit and implicit features to measure diversity perception, diversity can be comprehensively characterized and measured, enabling the diversity perception model to predict the diversity perception of different users for resources, thereby improving the accuracy of recommendations.

[0068] In some embodiments, S101 includes:

[0069] S101a: Obtain the feedback results of the questionnaires corresponding to multiple samples. The questionnaires are used to inquire about users' perception of resource diversity.

[0070] S101b: Based on the questionnaire feedback results corresponding to multiple samples, obtain the perception labels corresponding to multiple samples respectively.

[0071] In this embodiment of the disclosure, the perception label may include "too many recommendations," "too few recommendations," and "appropriate recommendations." This perception label is used to represent a user's perception of resource diversity. The perception label can also be used to provide data support for resource classification or resource value assessment.

[0072] Figure 3 A schematic diagram illustrating the acquisition of sensory tags is shown, such as... Figure 3 As shown, the system receives the survey feedback results returned by each user, analyzes the survey feedback results returned by each user, and obtains the perception labels corresponding to each user.

[0073] In this embodiment, an agile assessment questionnaire can be used to inquire about users' perceived experience of resource diversity. Based on the questionnaire feedback results corresponding to multiple samples, perception labels corresponding to each sample are obtained. The questionnaire feedback results include: too many recommendations, too few recommendations, and appropriate recommendations. Through the questionnaire survey, users' true perceived experience of resource diversity can be obtained. Furthermore, the classification using the three labels "too many recommendations," "appropriate recommendations," and "too few recommendations" concisely summarizes the three states of users' true perceived experience of resource diversity, making it easy for users to understand and providing good differentiation.

[0074] In this way, by using questionnaires, we can obtain real data on users' perception of resource diversity, ensuring the authenticity and accuracy of the sample, which helps to improve the accuracy of the diversity perception model, and thus helps to improve the accuracy of recommendations.

[0075] In some embodiments, the questionnaire in S101a includes at least the following: whether the number of recommended resources similar to the current topic is appropriate.

[0076] In this embodiment of the disclosure, an agile evaluation questionnaire is used to inquire about users' perceived experience of resource diversity. For example, the specific question text could be: "Question: How many content recommendations are similar to the current topic? Options: Too many recommendations, Appropriate recommendations, Too few recommendations." Based on the above text, questionnaire feedback results corresponding to multiple samples are obtained. It should be noted that the question text must include at least "topic similarity" and "number of recommendations." The above is merely an illustrative example and does not constitute a limitation on all possible forms or content of the question text; it is simply not exhaustive.

[0077] Because different users may have different understandings of the question copy (users' own perceptions), some may think the question is coarse-grained, while others may think it is fine-grained. As a result, the survey results on the question copy include both coarse-grained and fine-grained similarities, as well as a combination of explicit and implicit similarities, thus comprehensively expressing users' perceptions of diversity.

[0078] In this way, by conducting a questionnaire survey, we can obtain diversity perception tags to determine whether the number of recommendations for resources similar to the current topic is appropriate. This helps to improve the accuracy of the diversity perception model, thereby improving the accuracy of recommendations.

[0079] In some embodiments, the diversity-aware model training method may further include:

[0080] S104: Determine multiple target features from multiple samples;

[0081] S105: Obtain the statistical values ​​of various recommended evaluation indicators for each target feature within different preset time periods;

[0082] S106: Based on the statistical values ​​of various recommendation evaluation indicators within different preset time periods corresponding to each target feature, determine multiple candidate diversity perception features corresponding to multiple samples.

[0083] S107: Based on multiple candidate diversity perception features corresponding to multiple samples, determine the diversity perception features corresponding to multiple samples respectively.

[0084] In this embodiment of the disclosure, the target feature can be divided into explicit features and implicit features. The implicit features may include resource title similarity and GCF similarity; the explicit features may include resource classification features at different levels of refinement; for example, generic categories, primary categories, secondary categories, and points of interest; wherein the level of refinement of generic categories, primary categories, secondary categories, and points of interest gradually increases.

[0085] In some implementations, the classification features of resources at different levels of refinement may include general categories, primary categories, secondary categories, and points of interest. Here, general categories, primary categories, and secondary categories refer to the types of resources. A general category is a larger type level of the resource; for example, entertainment, film and television, music, and sports. A primary category is a type level smaller than the general category; for example, football, volleyball, and basketball under the sports category. A secondary category is a type level smaller than the primary category; for example, football matches, football stars, and football players under the primary category of football. The level of refinement for general categories, primary categories, secondary categories, and points of interest gradually increases.

[0086] In some implementations, different preset time periods can be understood as multiple windows. These preset time periods can be one day, one week, or even 9 PM to 11 PM every night within a month. The preset time periods can be automatically adjusted according to model training needs, or they can be manually adjusted as required.

[0087] In some implementations, the recommended evaluation metrics may include continuous impressions without clicks, impression percentage, click percentage, and click pass rate.

[0088] In some implementations, various statistical values ​​may include minimum, maximum, average, variance, and distribution. These statistical values ​​are used to represent resource usage.

[0089] In some implementations, various statistical values ​​of evaluation indicators for each explicit and implicit feature can be recommended at different preset time periods, and the diversity perception feature can be calculated using the Cartesian product. Figure 4 A schematic diagram illustrating the determination of diversity perception characteristics is shown, such as... Figure 4 As shown, the specific determination method includes: determining multiple target features of multiple samples; obtaining various statistical values ​​of different recommendation evaluation indicators within different preset time periods corresponding to each target feature; determining multiple candidate diversity perception features corresponding to multiple samples based on the various statistical values ​​of different recommendation evaluation indicators within different preset time periods corresponding to each target feature; and determining the diversity perception features corresponding to multiple samples based on the multiple candidate diversity perception features corresponding to multiple samples.

[0090] Table 1 shows the diversity perception features corresponding to Scenario 1 and Scenario 2, including:

[0091]

[0092] Table 1

[0093] In some implementations, as shown in Table 1, this diversity can include: generics, primary categories, secondary categories, points of interest, title similarity, and GCF similarity. "Multi-window" refers to different preset time periods, such as the last 3 days or the last week. "Multi-dimensional" refers to recommendation evaluation indicators such as continuous display without clicks, display percentage, click percentage, and click-through rate. Statistical values ​​can include: minimum, maximum, average, variance, and distribution. By combining explicit and implicit diversity, the perceptual diversity of users in different scenarios can be comprehensively depicted.

[0094] In this way, based on multiple target features and various statistical values ​​of evaluation indicators recommended at different preset time periods, rich diversity perception features can be obtained through Cartesian product, which can provide data support for the training of diversity perception models and improve the accuracy of diversity perception models.

[0095] In this embodiment of the disclosure, the multiple target features include resource classification features at different levels of refinement, and the multiple target features also include at least one of resource title similarity and GCF vector similarity.

[0096] Thus, multiple target features, including resource classification features at different levels of refinement, as well as at least one of resource title similarity and GCF vector similarity, enable the diversity perception features obtained through Cartesian product to include both coarse and fine-grained similarity, as well as a combination of explicit and implicit similarity. This allows for a comprehensive expression of the user's perception of diversity, enriches the number of diversity perception features, enhances the input diversity of the diversity perception model, helps improve the accuracy of the diversity perception model, and further improves the accuracy of recommendations.

[0097] In some implementations, S107 includes:

[0098] S107a: Determine the feature importance of multiple candidate diversity-perceived features corresponding to multiple samples;

[0099] S107b: Based on the feature importance of multiple candidate diversity perception features corresponding to multiple samples, select a target number of candidate diversity perception features from the multiple candidate diversity perception features corresponding to multiple samples, and use them as the diversity perception features corresponding to multiple samples.

[0100] In this embodiment of the disclosure, if the number of candidate diversity-aware features corresponding to multiple samples is large, these candidate diversity-aware features will have a large number of sparse features. If the features are too sparse, it will affect the training effect of the model to be trained. Therefore, sparse features can be pruned and optimized by permutation importance to improve the usability of sample data.

[0101] In some implementations, feature importance is calculated using a permutation importance method. This method randomly shuffles the variables in the samples to disrupt the original relationship between sample variable X and target Y. If shuffling one variable significantly increases the loss function of the model on the validation set, then that variable is considered important. If shuffling a variable has no effect on the model's loss function on the validation set, or even decreases it, then that variable is considered unimportant or even harmful to the model, thus filtering the sample data.

[0102] Thus, by determining the feature importance of multiple candidate diversity perception features corresponding to multiple samples, feature optimization of the candidate diversity perception features can be performed, reducing sparse features in the samples and improving the accuracy of the diversity perception model.

[0103] Table 2 shows the user perception values ​​of resources obtained by inputting user features, resource features, and diversity perception features corresponding to multiple samples into the diversity perception model, as shown in Table 2:

[0104]

[0105] Table 2

[0106] In this embodiment, the user characteristics corresponding to User 1, the resource characteristics corresponding to the title of the link clicked by User 1, and the diversity perception characteristics of User 1 for that resource are input into the diversity perception model, resulting in a model perception value of 0.8570 output by the diversity perception model. Therefore, User 1's diversity perception value for that resource is excessive. Table 2 shows the data on resource type, primary category, secondary category, the percentage of primary category impressions within 7 days, the average LCN of primary category, the average LCN of secondary category, and the average LCN of interest point. This data reflects a combination of explicit and implicit features, comprehensively characterizing the diversity of user perception.

[0107] Table 3 shows the evaluation indicators of the diversity perception model, as shown in Table 3:

[0108] Is it [too much]? Overall users Light-living users moderate users Heavy users AUC 0.7162 0.6922 0.6990 0.7378

[0109] Table 3

[0110] AUC (Area Under Curve) is a performance metric used to measure the quality of a learner. The accuracy of the diversity perception model in detecting "excessive resources" is 0.7162 for overall users. For lightly active users, the accuracy is 0.6922. For moderately active users, the accuracy is 0.6990. For heavily active users, the accuracy is 0.7378. A higher AUC value indicates a more accurate detection of "excessive resources" by the diversity perception model. "Lightly active user" refers to a user with moderate activity; "moderately active user" refers to a user with moderate activity; and "heavily active user" refers to a user with heavy activity.

[0111] It should be understood that Figure 2 , Figure 3 and Figure 4 The schematic diagrams shown are merely illustrative and not limiting, and are scalable; those skilled in the art can use them as a basis. Figure 2 , Figure 3 and Figure 4 Even with various obvious changes and / or substitutions to the examples, the resulting technical solutions still fall within the scope of this disclosure.

[0112] This disclosure provides a recommendation ranking method that can be applied to electronic devices. The following will be combined with... Figure 5 The flowchart shown illustrates a recommended sorting method provided by an embodiment of this disclosure. It should be noted that although a logical order is shown in the flowchart, in some cases, the steps shown or described may be performed in a different order.

[0113] S501: Input the user characteristics of the target user, the diversity perception characteristics, and the resource characteristics of the candidate resources into the diversity perception model to obtain the target user's perception value of the candidate resources output by the diversity perception model.

[0114] S502: Generate a recommended ranking list for the target user based on the target user's perceived value of the candidate resources.

[0115] The diversity perception model is obtained by training using the diversity perception model training method described above.

[0116] In this embodiment of the disclosure, the user characteristics of the target user can be obtained through log analysis of the target user. For example, user characteristics include the target user's activity level over a period of time. Another example is the target user's historical click / impression count. Yet another example is the target user's request time. The above are merely illustrative examples and are not intended to limit all possible content of the target user characteristics; they are simply not exhaustive.

[0117] In this embodiment of the disclosure, the resource characteristics of the candidate resource may include the type of resource, the historical number of clicks / impressions of the resource among all users, and the distribution of the resource category in the agile evaluation sample.

[0118] In this embodiment, diversity perception features can be determined by using various statistical values ​​of target features across different recommended evaluation metrics within different preset time periods. These target features can be categorized into explicit and implicit features. Implicit features may include resource title similarity and GCF similarity; explicit features include resource classification features at different levels of refinement, such as generic categories, primary categories, secondary categories, and points of interest; the refinement of generic categories, primary categories, secondary categories, and points of interest gradually increases. Recommended evaluation metrics may include: continuous display without clicks, display percentage, click percentage, and click-through rate. Various statistical values ​​may include: minimum value, maximum value, average value, variance, and distribution.

[0119] In this embodiment of the disclosure, the user characteristics of the target user, the diversity perception characteristics, and the resource characteristics of the candidate resources are input into the diversity perception model. If the perception value of the target user for the first candidate resource output by the diversity perception model is 0.9, then the first candidate resource is sorted to the bottom of the target user's recommendation list based on the perception value of 0.9.

[0120] In this embodiment of the disclosure, the user characteristics of the target user, the diversity perception characteristics, and the resource characteristics of the two candidate resources are input into the diversity perception model. If the target user's perception values ​​for the two candidate resources output by the diversity perception model are 0.8 and 0.6 respectively, then according to the target user's perception values ​​for the two candidate resources, the resource with a perception value of 0.6 is sorted to the top of the recommendation list, and the resource with a perception value of 0.8 is sorted to the bottom of the recommendation list.

[0121] Thus, by generating a recommendation ranking list for the target user based on the target user's perception values ​​of candidate resources output by the diversity perception model, the accuracy of the ranking of candidate resources in the recommendation ranking list can be improved, thereby improving the accuracy of the recommendation.

[0122] In some embodiments, such as Figure 5 As shown, S502 includes:

[0123] S502a: Determine the perceptual diversity adjustment factor based on the target user's perceived value of the candidate resources;

[0124] S502b: Input the perceived diversity adjustment factor into the fusion value model to obtain the estimated value of candidate resources output by the fusion value model. The fusion value model is used to estimate the value of resources.

[0125] S502c: Sort candidate resources according to their estimated value;

[0126] S502d: Generate a recommended ranking list for the target user based on the ranking results of the candidate resources.

[0127] In this embodiment, a perceptual diversity modulating factor is introduced into the fusion value model, and its influence is adaptively learned through evolutionary learning. This perceptual diversity modulating factor is one of the input values ​​of the fusion value model. When the input value of the fusion value model is only the perceptual diversity modulating factor, the perceptual diversity model is equal to the fusion value model.

[0128] Figure 6 The diagram illustrates the generation of a recommended ranking list for the target user, as shown below. Figure 6 As shown, the diverse perception model determines the target user's perceived value of the candidate resources. Based on the target user's perceived value of the candidate resources, a perceptual diversity adjustment factor is determined. This diversity adjustment factor is input into the fusion value model to obtain the estimated value of the candidate resources output by the fusion value model. A recommended ranking list is generated based on the estimated value of the candidate resources.

[0129] In this embodiment of the disclosure, a perceptual diversity adjustment factor is determined based on the target user's perceived value of the candidate resources. This perceptual diversity adjustment factor is solved using formula (2):

[0130]

[0131] Where q represents the diversity perception adjustment factor, x is the predicted result of the diversity perception model on the target user's perceived value, m is the predicted mean of all resources in the sequence generation stage, and s is the standard deviation of the predicted value.

[0132] Figure 7 The diagram illustrates the processing of online evolutionary learning for the fusion value model, as shown below. Figure 7 As shown, online evolutionary learning includes a policy network comprising multiple 'h' values, each representing a parameter of the fusion value model. These parameters include scenario parameters and user immersion state parameters. The fusion value model continuously evolves online through user feedback, application, and exploration.

[0133] In this embodiment, a perceptual diversity adjustment factor is determined based on the target user's perceived values ​​of N candidate resources. The perceptual diversity adjustment factor, information click-through rate adjustment factor, and webpage click-through rate adjustment factor are input into a fusion value model to obtain the estimated value of the candidate resources output by the fusion value model. The candidate resources are then ranked according to their estimated values. Based on the ranking results, a recommended ranking list for the target user is generated. The fusion value model can adjust the weights of its input items according to actual needs. For example, the weights of the perceptual diversity adjustment factor, information click-through rate adjustment factor, and webpage click-through rate adjustment factor can be set to 0.6, 0.2, and 0.2 respectively.

[0134] Thus, by inputting the perceptual diversity adjustment factor into the fusion value model, the estimated value of candidate resources output by the fusion value model is obtained. Based on the estimated value of candidate resources, the candidate resources are ranked. Based on the ranking results of the candidate resources, a recommended ranking list for the target user is generated. This can comprehensively predict the value of resources, improve the accuracy of the ranking of candidate resources in the recommended ranking list, and thus improve the accuracy of the recommendation.

[0135] In some embodiments, S502c includes:

[0136] S502c': Density control is performed on candidate resources whose perceived value is greater than the first threshold and whose estimated value is greater than the second threshold during the ranking process.

[0137] In this embodiment, the first threshold refers to a preset threshold for the perceived value; the second threshold refers to a preset threshold for the estimated value. When a candidate resource has a perceived value greater than the first threshold and an estimated value greater than the second threshold, and this candidate resource has a perceived value of "too many recommendations" and an estimated value that ranks in the top 10% of the comprehensive score in the fusion value model, density control is applied to this specially concerned resource. The density control strategy is as follows: a display interface can show 7 candidate resources, and one of these 7 candidate resources can be a specially concerned resource. Furthermore, after each display interface refresh, only one specially concerned resource can exist among the 7 candidate resources displayed, or the specially concerned resource may not exist on the display interface.

[0138] Thus, by using the diversity perception model and the fusion value model, the value of candidate resources can be measured more comprehensively, which helps to improve the accuracy of the recommendation ranking list and further improve the accuracy of the recommendation ranking.

[0139] This disclosure proposes a diversity-aware recommendation ranking scheme that enables the diversity-aware model to predict the diversity perception of resources among different users, thereby improving the accuracy of recommendations.

[0140] This disclosure provides a diversity perception model training device, such as... Figure 8 As shown, the diversity perception model training device may include: a first acquisition module 801, used to acquire multiple samples and perception labels corresponding to the multiple samples respectively; a first input module 802, used to input user features, resource features and diversity perception features corresponding to the multiple samples respectively into the model to be trained, to obtain the user perception values ​​of the resources corresponding to the multiple samples respectively; and a training module 803, used to train the model to be trained according to the perception values ​​and perception labels corresponding to the multiple samples respectively, to obtain the diversity perception model.

[0141] In some embodiments, the first acquisition module 801 includes: a first acquisition submodule, configured to acquire questionnaire feedback results corresponding to multiple samples respectively, wherein the questionnaire is used to inquire about users' perception experience of resource diversity; and a second acquisition submodule, configured to acquire perception labels corresponding to multiple samples respectively based on the questionnaire feedback results corresponding to multiple samples respectively.

[0142] In some embodiments, the diversity perception model training apparatus includes a questionnaire that includes at least the following: whether the number of recommended resources similar to the current topic is appropriate.

[0143] In some embodiments, the diversity perception model training apparatus further includes: a first determining module 804. Figure 8 (not shown in the image), used to determine multiple target features of multiple samples; second acquisition module 805 ( Figure 8 (Not shown in the image), used to obtain various statistical values ​​of different recommended evaluation indicators within different preset time periods corresponding to each target feature; the second determining module 806 ( Figure 8 (Not shown in the image), used to determine multiple candidate diversity perception features corresponding to multiple samples based on various statistical values ​​of different recommendation evaluation indicators within different preset time periods corresponding to each target feature; the third determination module 807 ( Figure 8 (not shown in the image) is used to determine the diversity perception features corresponding to multiple samples based on multiple candidate diversity perception features corresponding to multiple samples respectively.

[0144] In some embodiments, the diversity-aware model training apparatus includes multiple target features, including classification features of resources at different levels of refinement, and the multiple target features also include at least one of resource title similarity and graph collaborative filtering (GCF) vector similarity.

[0145] In some embodiments, the third determining module 807 includes: a first determining submodule, configured to determine the feature importance of multiple candidate diversity perception features corresponding to multiple samples respectively; and a selecting submodule, configured to select a target number of candidate diversity perception features from the multiple candidate diversity perception features corresponding to multiple samples respectively, according to the feature importance of the multiple candidate diversity perception features corresponding to multiple samples respectively, as the diversity perception features corresponding to multiple samples respectively.

[0146] Those skilled in the art should understand that the functions of each processing module in the diversity perception model training device of this disclosure can be understood with reference to the relevant description of the aforementioned diversity perception model training method. Each processing module in the diversity perception model training device of this disclosure can be implemented by an analog circuit that implements the functions of this disclosure embodiment, or by running software that executes the functions of this disclosure embodiment on an electronic device.

[0147] The diversity perception model training apparatus of this disclosure enables the diversity perception model to predict the diversity perception of resources by different users, thereby improving the accuracy of recommendations.

[0148] This disclosure provides a recommendation sorting device, such as... Figure 9 As shown, the recommendation ranking device may include: a second input module 901, used to input the user characteristics, diversity perception characteristics and resource characteristics of the candidate resources of the target user into the diversity perception model to obtain the perception value of the target user on the candidate resources output by the diversity perception model; and a generation module 902, used to generate a recommendation ranking list of the target user based on the perception value of the target user on the candidate resources; wherein, the diversity perception model is trained by the diversity perception model training method described above.

[0149] In some embodiments, the generation module 902 includes: a second determining submodule, configured to determine a perceptual diversity adjustment factor based on the target user's perceived value of the candidate resources; an output submodule, configured to input the perceptual diversity adjustment factor into a fusion value model to obtain a predicted value of the candidate resources output by the fusion value model, wherein the fusion value model is used to predict the value of the resources; a ranking submodule, configured to rank the candidate resources according to the predicted value of the candidate resources; and a generation submodule, configured to generate a recommended ranking list for the target user based on the ranking result of the candidate resources.

[0150] In some embodiments, the sorting submodule is configured to: perform density control on candidate resources whose perceived value is greater than a first threshold and whose estimated value is greater than a second threshold during sorting.

[0151] Those skilled in the art should understand that the functions of each processing module in the recommendation sorting device of this disclosure embodiment can be understood with reference to the relevant description of the recommendation sorting method described above. Each processing module in the recommendation sorting device of this disclosure embodiment can be implemented by an analog circuit that implements the functions of this disclosure embodiment, or by running software that performs the functions of this disclosure embodiment on an electronic device.

[0152] The recommendation ranking model apparatus of this disclosure can utilize the diversity perception of resources by each user as output by the diversity perception model to determine a more suitable resource ranking list for each user, thereby improving the accuracy of recommendation ranking.

[0153] This disclosure provides a schematic diagram of a scenario for training a diversity perception model, such as... Figure 10 As shown.

[0154] As previously described, the diversity perception model training method provided in this disclosure is applied to electronic devices. Electronic devices are intended to represent various forms of digital computers, such as laptops, desktop computers, workstations, personal digital assistants, servers, blade servers, mainframes, and other suitable computers. Electronic devices can also represent various forms of mobile devices, such as personal digital assistants, cellular phones, smartphones, wearable devices, and other similar computing devices.

[0155] Specifically, the electronic device may perform the following operations:

[0156] Obtain multiple samples and their corresponding perceptual labels;

[0157] The user features, resource features, and diversity perception features corresponding to multiple samples are input into the model to be trained to obtain the user's perception value of resources corresponding to multiple samples.

[0158] The model to be trained is obtained by training the model based on the perceptual values ​​and perceptual labels corresponding to multiple samples.

[0159] The user characteristics, resource characteristics, and diversity perception characteristics corresponding to multiple samples can be obtained from the data source. The data source can be various forms of data storage devices, such as laptops, desktop computers, workstations, personal digital assistants, servers, blade servers, mainframes, and other suitable computers. The data source can also represent various forms of mobile devices, such as personal digital assistants, cellular phones, smartphones, wearable devices, and other similar computing devices. Furthermore, the data source and the user terminal can be the same device.

[0160] It should be understood that Figure 10The scene diagrams shown are merely illustrative and not restrictive; those skilled in the art can interpret them based on... Figure 10 Even with various obvious changes and / or substitutions to the examples, the resulting technical solutions still fall within the scope of this disclosure.

[0161] This disclosure also provides a recommendation sorting device, such as... Figure 11 As shown.

[0162] As previously described, the recommended sorting method provided in this disclosure is applied to electronic devices. Electronic devices are intended to represent various forms of digital computers, such as laptop computers, desktop computers, workstations, personal digital assistants, servers, blade servers, mainframe computers, and other suitable computers. Electronic devices can also represent various forms of mobile devices, such as personal digital assistants, cellular phones, smartphones, wearable devices, and other similar computing devices.

[0163] Specifically, the electronic device may perform the following operations:

[0164] The user characteristics, diversity perception characteristics, and resource characteristics of the candidate resources of the target user are input into the diversity perception model to obtain the target user's perception value of the candidate resources output by the diversity perception model.

[0165] Based on the target user's perceived value of the candidate resources, a recommended ranking list for the target user is generated;

[0166] The diversity perception model is obtained by training using the diversity perception model training method described above.

[0167] The user characteristics, diversity perception characteristics, and resource characteristics of candidate resources of the target user can be obtained from the data source. The data source can be various forms of data storage devices, such as laptops, desktop computers, workstations, personal digital assistants (PDAs), servers, blade servers, mainframes, and other suitable computers. The data source can also represent various forms of mobile devices, such as PDAs, cellular phones, smartphones, wearable devices, and other similar computing devices. Furthermore, the data source and the user terminal can be the same device.

[0168] It should be understood that Figure 11 The scene diagrams shown are merely illustrative and not restrictive; those skilled in the art can interpret them based on... Figure 11 Even with various obvious changes and / or substitutions to the examples, the resulting technical solutions still fall within the scope of this disclosure.

[0169] The acquisition, storage, and application of user personal information involved in the technical solution disclosed herein comply with the provisions of relevant laws and regulations and do not violate public order and good morals.

[0170] According to embodiments of this disclosure, this disclosure also provides an electronic device, a readable storage medium, and a computer program product.

[0171] Figure 12 A schematic block diagram of an example electronic device 1200 that can be used to implement embodiments of the present disclosure is shown. The electronic device is intended to represent various forms of digital computers, such as laptop computers, desktop computers, workstations, personal digital assistants, servers, blade servers, mainframe computers, and other suitable computers. The electronic device may also represent various forms of mobile devices, such as personal digital assistants, cellular phones, smartphones, wearable devices, and other similar computing devices. The components shown herein, their connections and relationships, and their functions are merely illustrative and are not intended to limit the implementation of the present disclosure described and / or claimed herein.

[0172] like Figure 12 As shown, device 1200 includes a computing unit 1201, which can perform various appropriate actions and processes based on a computer program stored in read-only memory (ROM) 1202 or a computer program loaded from storage unit 1208 into random access memory (RAM) 1203. RAM 1203 may also store various programs and data required for the operation of device 1200. The computing unit 1201, ROM 1202, and RAM 1203 are interconnected via bus 1204. Input / output (I / O) interface 1205 is also connected to bus 1204.

[0173] Multiple components in device 1200 are connected to I / O interface 1205, including: input unit 1206, such as keyboard, mouse, etc.; output unit 1207, such as various types of monitors, speakers, etc.; storage unit 1208, such as disk, optical disk, etc.; and communication unit 1209, such as network card, modem, wireless transceiver, etc. Communication unit 1209 allows device 1200 to exchange information / data with other devices through computer networks such as the Internet and / or various telecommunications networks.

[0174] The computing unit 1201 can be various general-purpose and / or special-purpose processing components with processing and computing capabilities. Some examples of the computing unit 1201 include, but are not limited to, a central processing unit (CPU), a graphics processing unit (GPU), various special-purpose artificial intelligence (AI) computing chips, various computing units running machine learning model algorithms, digital signal processors (DSPs), and any suitable processor, controller, microcontroller, etc. The computing unit 1201 performs the various methods and processes described above, such as diversity-aware model training methods / recommendation ranking methods. For example, in some embodiments, the diversity-aware model training method / recommendation ranking method can be implemented as a computer software program tangibly contained in a machine-readable medium, such as storage unit 1208. In some embodiments, part or all of the computer program can be loaded and / or installed on device 1200 via ROM 1202 and / or communication unit 1209. When the computer program is loaded into RAM 1203 and executed by computing unit 1201, one or more steps of the diversity-aware model training method / recommendation ranking method described above can be performed. Alternatively, in other embodiments, computing unit 1201 can be configured to execute the diversity-aware model training method / recommendation ranking method by any other suitable means (e.g., by means of firmware).

[0175] Various embodiments of the systems and techniques described above herein can be implemented in digital electronic circuit systems, integrated circuit systems, field-programmable gate arrays (FPGAs), application-specific integrated circuits (ASICs), application-specific standard products (ASSPs), systems-on-chip (SoCs), complex programmable logic devices (CPLDs), computer hardware, firmware, software, and / or combinations thereof. These various embodiments may include implementations in one or more computer programs that can be executed and / or interpreted on a programmable system including at least one programmable processor, which may be a dedicated or general-purpose programmable processor, capable of receiving data and instructions from a storage system, at least one input device, and at least one output device, and transmitting data and instructions to the storage system, the at least one input device, and the at least one output device.

[0176] The program code used to implement the methods of this disclosure may be written in any combination of one or more programming languages. This program code may be provided to a processor or controller of a general-purpose computer, special-purpose computer, or other programmable data processing apparatus, such that when executed by the processor or controller, the program code causes the functions / operations specified in the flowcharts and / or block diagrams to be implemented. The program code may be executed entirely on a machine, partially on a machine, as a standalone software package partially on a machine and partially on a remote machine, or entirely on a remote machine or server.

[0177] In the context of this disclosure, a machine-readable medium can be a tangible medium that may contain or store a program for use by or in conjunction with an instruction execution system, apparatus, or device. A machine-readable medium can be a machine-readable signal medium or a machine-readable storage medium. A machine-readable medium can be, but is not limited to, electronic, magnetic, optical, electromagnetic, infrared, or semiconductor systems, apparatus, or devices, or any suitable combination of the foregoing. More specific examples of machine-readable storage media include electrical connections based on one or more wires, portable computer disks, hard disks, random access memory, read-only memory, erasable programmable read-only memory (EPROM), flash memory, optical fiber, compact disk read-only memory (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination of the foregoing.

[0178] To provide interaction with a user, the systems and techniques described herein can be implemented on a computer having: a display device (e.g., a cathode ray tube (CRT) or liquid crystal display (LCD) monitor) for displaying information to the user; and a keyboard and pointing device (e.g., a mouse or trackball) through which the user provides input to the computer. Other types of devices can also be used to provide interaction with the user; for example, feedback provided to the user can be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user can be received in any form (including sound input, voice input, or tactile input).

[0179] The systems and technologies described herein can be implemented in computing systems that include backend components (e.g., as a data server), or computing systems that include middleware components (e.g., an application server), or computing systems that include frontend components (e.g., a user computer with a graphical user interface or web browser through which a user can interact with implementations of the systems and technologies described herein), or any combination of such backend, middleware, or frontend components. The components of the system can be interconnected via digital data communication of any form or medium (e.g., a communication network). Examples of communication networks include local area networks (LANs), wide area networks (WANs), and the Internet.

[0180] Computer systems can include clients and servers. Clients and servers are generally located far apart and typically interact via communication networks. Client-server relationships are created by computer programs running on the respective computers and having a client-server relationship with each other. Servers can be cloud servers, servers in distributed systems, or servers incorporating blockchain technology.

[0181] It should be understood that the various forms of processes shown above can be used to rearrange, add, or delete steps. For example, the steps described in this disclosure can be executed in parallel, sequentially, or in different orders, as long as the desired result of the technical solution disclosed in this disclosure can be achieved, and this is not limited herein.

[0182] The specific embodiments described above do not constitute a limitation on the scope of protection of this disclosure. Those skilled in the art should understand that various modifications, combinations, sub-combinations, and substitutions can be made according to design requirements and other factors. Any modifications, equivalent substitutions, and improvements made within the principles of this disclosure should be included within the scope of protection of this disclosure.

Claims

1. A method for training a diversity perception model, comprising: Obtain multiple samples and their corresponding perceptual labels; The user features, resource features, and diversity perception features corresponding to the multiple samples are input into the model to be trained to obtain the user's perception value of the resource corresponding to the multiple samples. The diversity perception features are obtained by: determining multiple target features of the multiple samples; obtaining various statistical values ​​of different recommendation evaluation indicators within different preset time periods corresponding to each target feature; and determining multiple candidate diversity perception features corresponding to the multiple samples based on the various statistical values ​​of different recommendation evaluation indicators within different preset time periods corresponding to each target feature. Based on the multiple candidate diversity perception features corresponding to the multiple samples respectively, determine the diversity perception features corresponding to the multiple samples respectively; The multiple target features include resource classification features at different levels of refinement, and the multiple target features also include at least one of resource title similarity and graph collaborative filtering (GCF) vector similarity; The model to be trained is trained based on the perceived values ​​and perceived labels corresponding to the multiple samples to obtain a diversity perception model.

2. The method according to claim 1, wherein, Obtaining the perceptual labels corresponding to the multiple samples respectively includes: Obtain the questionnaire feedback results corresponding to the multiple samples respectively. The questionnaire is used to ask users about their perception of resource diversity. Based on the survey questionnaire feedback results corresponding to the multiple samples, the perception labels corresponding to the multiple samples are obtained.

3. The method according to claim 2, wherein, The questionnaire should include at least the following: Is the number of recommended resources similar to the current topic appropriate? 4. The method according to claim 1, wherein, The step of determining the diversity perception features corresponding to the multiple samples based on the multiple candidate diversity perception features corresponding to the multiple samples includes: Determine the feature importance of multiple candidate diversity perception features corresponding to the multiple samples respectively; Based on the feature importance of the multiple candidate diversity perception features corresponding to the multiple samples respectively, a target number of candidate diversity perception features are selected from the multiple candidate diversity perception features corresponding to the multiple samples respectively, and these are used as the diversity perception features corresponding to the multiple samples respectively.

5. A recommendation ranking method, comprising: The user characteristics, diversity perception characteristics, and resource characteristics of the candidate resources of the target user are input into the diversity perception model to obtain the perception value of the target user for the candidate resources output by the diversity perception model. Based on the target user's perception of the candidate resources, a recommended ranking list for the target user is generated; The diversity perception model is obtained using the diversity perception model training method according to any one of claims 1 to 4.

6. The method according to claim 5, wherein, The step of generating a recommended ranking list for the target user based on the target user's perception of the candidate resources includes: Based on the target user's perceived value of the candidate resources, a perceptual diversity adjustment factor is determined; The perceived diversity adjustment factor is input into the fusion value model to obtain the estimated value of the candidate resource output by the fusion value model. The fusion value model is used to estimate the value of the resource. The candidate resources are ranked according to their estimated values. Based on the ranking results of the candidate resources, a recommended ranking list for the target user is generated.

7. The method according to claim 6, wherein, The step of ranking the candidate resources based on their estimated values ​​includes: Density control is performed on candidate resources whose perceived value is greater than a first threshold and whose estimated value is greater than a second threshold during sorting.

8. A diversity perception model training device, comprising: The first acquisition module is used to acquire multiple samples and the perceptual labels corresponding to the multiple samples respectively; The first input module is used to input the user features, resource features and diversity perception features corresponding to the multiple samples into the model to be trained, so as to obtain the user's perception value of the resource corresponding to the multiple samples. The training module is used to train the model to be trained based on the perceptual values ​​and perceptual labels corresponding to the multiple samples respectively, so as to obtain a diversity perceptual model; The device further includes: a first determining module, configured to determine multiple target features of the plurality of samples; a second acquiring module, configured to acquire various statistical values ​​of different recommendation evaluation indicators within different preset time periods corresponding to each of the target features; a second determining module, configured to determine multiple candidate diversity perception features corresponding to the plurality of samples based on the various statistical values ​​of different recommendation evaluation indicators within different preset time periods corresponding to each of the target features; and a third determining module, configured to determine the diversity perception features corresponding to the plurality of samples based on the multiple candidate diversity perception features corresponding to the plurality of samples; the plurality of target features include resource classification features at different levels of refinement, and the plurality of target features also include at least one of resource title similarity and graph collaborative filtering (GCF) vector similarity.

9. The apparatus according to claim 8, wherein, The first acquisition module includes: The first acquisition submodule is used to acquire the questionnaire feedback results corresponding to the multiple samples respectively, wherein the questionnaire is used to ask users about their perception experience of resource diversity. The second acquisition submodule is used to acquire the perception labels corresponding to the multiple samples based on the questionnaire feedback results corresponding to the multiple samples respectively.

10. The apparatus according to claim 9, wherein, The questionnaire should include at least the following: Is the number of recommended resources similar to the current topic appropriate? 11. The apparatus according to claim 8, wherein, The third determining module includes: The first determining submodule is used to determine the feature importance of multiple candidate diversity perception features corresponding to the multiple samples respectively; The selection submodule is used to select a target number of candidate diversity perception features from the candidate diversity perception features corresponding to the multiple samples, according to the feature importance of the candidate diversity perception features corresponding to the multiple samples respectively, and use them as the diversity perception features corresponding to the multiple samples respectively.

12. A recommendation sorting device, comprising: The second input module is used to input the user characteristics, diversity perception characteristics and resource characteristics of the target user and the candidate resources into the diversity perception model to obtain the perception value of the target user for the candidate resources output by the diversity perception model. The generation module is used to generate a recommended ranking list for the target user based on the target user's perception value of the candidate resources; The diversity perception model is obtained using the diversity perception model training method according to any one of claims 1 to 4.

13. The apparatus according to claim 12, wherein, The generation module includes: The second determining submodule is used to determine the perceptual diversity adjustment factor based on the target user's perception value of the candidate resources; The output submodule is used to input the perceived diversity adjustment factor into the fusion value model to obtain the estimated value of the candidate resource output by the fusion value model. The fusion value model is used to estimate the value of the resource. The sorting submodule is used to sort the candidate resources according to their estimated values. The generation submodule is used to generate a recommended ranking list for the target user based on the ranking results of the candidate resources.

14. The apparatus according to claim 13, wherein, The sorting submodule is used for: Density control is performed on candidate resources whose perceived value is greater than a first threshold and whose estimated value is greater than a second threshold during sorting.

15. An electronic device comprising: At least one processor; as well as A memory communicatively connected to the at least one processor; wherein, The memory stores instructions that can be executed by the at least one processor to enable the at least one processor to perform the method of any one of claims 1-7.

16. A non-transitory computer-readable storage medium storing computer instructions, wherein, The computer instructions are used to cause the computer to perform the method according to any one of claims 1-7.

17. A computer program product comprising a computer program that, when executed by a processor, implements the method according to any one of claims 1-7.

Images