Recommendation model training method and device

Abstract

The present disclosure provides a recommendation model training method and device, which relates to the field of computer technology, especially to the field of big data and deep learning technology. The specific implementation scheme is: obtaining a multi-objective fusion recommendation model; on the basis of verification parameters, superimposing multiple groups of different parameter perturbations to obtain multiple groups of exploration parameters; obtaining user feedback data and recommended video scene data corresponding to each group of exploration parameters, and obtaining user feedback data and recommended video scene data corresponding to verification parameters; calculating user feedback reward values and video recommendation scene reward values; for each group of exploration parameters, calculating the total reward value based on user feedback reward values and video recommendation scene reward values; determining the preferred exploration parameters, and adjusting the model parameters according to the parameter perturbations corresponding to the preferred exploration parameters. Ultimately, while ensuring that the video recommendation scene meets the expected goals, the user's viewing time and viewing experience are improved, and there is no need to manually formulate complex partial order rules, thereby reducing the implementation complexity.

Description

Technical Field

[0001] The present disclosure relates to the field of computer technology, and in particular to the field of big data and deep learning technology. Background Art

[0002] The sorting module in the video stream recommendation system usually optimizes multi-objective modeling based on click-through rate, completion rate, number of interactions, etc. The fusion of multiple objectives becomes an important part after multi-objective modeling. The fusion of different objectives then scores and sorts the video resources, which directly determines the final recommendation results. Summary of the Invention

[0003] The present disclosure provides a recommendation model training method and device.

[0004] According to one aspect of the present disclosure, a recommendation model training method is provided, comprising:

[0005] Acquire a multi-objective fusion recommendation model, wherein the multi-objective fusion recommendation model includes a plurality of model parameters to be adjusted;

[0006] Determine the current model parameters as verification parameters, and on the basis of the verification parameters, superimpose multiple sets of different parameter perturbations to obtain multiple sets of exploration parameters;

[0007] Obtaining user feedback data and recommended video scene data corresponding to each set of exploration parameters, and obtaining user feedback data and recommended video scene data corresponding to the verification parameters;

[0008] For each set of exploration parameters, a user feedback reward value is calculated based on the user feedback data corresponding to the set of exploration parameters and the user feedback data corresponding to the verification parameters, and a video recommendation scene reward value is calculated based on the recommended video scene data corresponding to the set of exploration parameters and the recommended video scene data corresponding to the verification parameters;

[0009] For each set of exploration parameters, the total reward value is calculated based on the user feedback reward value and the video recommendation scene reward value corresponding to the set of exploration parameters;

[0010] A preferred exploration parameter is determined according to the total reward value, and the model parameter is adjusted according to a parameter perturbation corresponding to the preferred exploration parameter.

[0011] According to one aspect of the present disclosure, a video recommendation method is provided, comprising:

[0012] Identify target users and candidate videos;

[0013] Determining a fusion factor and a fusion feature according to the target user and each candidate video;

[0014] Inputting the fusion factor and fusion feature into a pre-trained multi-objective fusion recommendation model to obtain a recommendation score for each candidate video;

[0015] Recommend candidate videos to the target user based on the recommendation score.

[0016] According to one aspect of the present disclosure, a recommendation model training device is provided, comprising:

[0017] A first acquisition module is used to acquire a multi-objective fusion recommendation model, where the multi-objective fusion recommendation model includes a plurality of model parameters to be adjusted;

[0018] A perturbation module is used to determine the current model parameters as verification parameters, and on the basis of the verification parameters, superimpose multiple sets of different parameter perturbations to obtain multiple sets of exploration parameters;

[0019] A second acquisition module is used to obtain user feedback data and recommended video scene data corresponding to each set of exploration parameters, and obtain user feedback data and recommended video scene data corresponding to the verification parameters;

[0020] A first calculation module calculates, for each set of exploration parameters, a user feedback reward value based on user feedback data corresponding to the set of exploration parameters and user feedback data corresponding to the verification parameters, and calculates a video recommendation scene reward value based on recommended video scene data corresponding to the set of exploration parameters and recommended video scene data corresponding to the verification parameters;

[0021] A second calculation module is used to calculate a total reward value for each set of exploration parameters based on the user feedback reward value and the video recommendation scene reward value corresponding to the set of exploration parameters;

[0022] An adjustment module is configured to determine a preferred exploration parameter according to the total reward value, and adjust the model parameters according to a parameter perturbation corresponding to the preferred exploration parameter.

[0023] According to one aspect of the present disclosure, a video recommendation device is provided, comprising:

[0024] A first determination module is used to determine a target user and candidate videos;

[0025] A second determination module is used to determine a fusion factor and a fusion feature according to the target user and each candidate video;

[0026] A recommendation module is configured to recommend candidate videos to the target user based on the recommendation score.

[0027] According to another aspect of the present disclosure, there is provided an electronic device, comprising:

[0028] at least one processor; and

[0029] a memory communicatively connected to the at least one processor; wherein,

[0030] The memory stores instructions that can be executed by the at least one processor, and the instructions are executed by the at least one processor to enable the at least one processor to perform the recommendation model training method.

[0031] According to another aspect of the present disclosure, a non-transitory computer-readable storage medium storing computer instructions is provided, wherein the computer instructions are used to cause the computer to execute a recommendation model training method.

[0032] According to another aspect of the present disclosure, a computer program product is provided, including a computer program, wherein the computer program is executed by a processor to perform a recommendation model training method.

[0033] It should be understood that the contents described in this section are not intended to identify the key or important features of the embodiments of the present disclosure, nor are they intended to limit the scope of the present disclosure. Other features of the present disclosure will become readily understood through the following description. BRIEF DESCRIPTION OF THE DRAWINGS

[0034] The accompanying drawings are provided to facilitate a better understanding of the present invention and do not constitute a limitation of the present disclosure.

[0035] Figure 1 A flowchart of a recommendation model training method provided in an embodiment of the present disclosure;

[0036] Figure 2 A schematic diagram of a recommendation model training method provided in an embodiment of the present disclosure;

[0037] Figure 3 A schematic diagram of a flow chart of a video recommendation method provided by an embodiment of the present disclosure;

[0038] Figure 4 is a block diagram of an apparatus for implementing the recommendation model training method according to an embodiment of the present disclosure;

[0039] Figure 5 is a block diagram of a device for implementing the video recommendation method according to an embodiment of the present disclosure;

[0040] Figure 6 3 is a block diagram of an electronic device used to implement the recommendation model training method of an embodiment of the present disclosure. DETAILED DESCRIPTION

[0041] The following description of exemplary embodiments of the present disclosure is made in conjunction with the accompanying drawings, including various details of the embodiments of the present disclosure to facilitate understanding. These details should be considered as merely exemplary. Therefore, those skilled in the art will recognize that various changes and modifications may be made to the embodiments described herein without departing from the scope and spirit of the present disclosure. Similarly, for the sake of clarity and conciseness, descriptions of well-known functions and structures are omitted in the following description.

[0042] The sorting module in the video streaming recommendation system usually models and optimizes multiple business objectives such as click-through rate, completion rate, and number of interactions. The fusion of multiple objectives becomes an important part after multi-objective modeling. The fusion of different objectives then scores and sorts video resources, which directly determines the final recommendation results.

[0043] For more complex scenarios involving heterogeneous resources (such as short videos and small videos), in addition to meeting business goals such as click-through rate and completion rate, it is also necessary to ensure scenario goals such as the distribution structure and distribution scale of heterogeneous resources, which greatly increases the difficulty of multi-objective integration.

[0044] In the field of video recommendation, videos with a duration of more than 60 seconds but less than a few minutes are called short videos. Videos with a duration of less than one minute, such as between 15 and 30 seconds, are called mini videos. In addition, the playback format (horizontal and vertical) of short videos and mini videos may also be different.

[0045] Existing multi-objective fusion approaches for recommender systems include formula fusion and learning to rank (LTR) methods, such as the pairwise algorithm. Formula fusion requires manual determination of the influence weights of each objective. This approach often results in simple formulas, weak expressiveness, and requires extensive prior experience.

[0046] The fusion effect of LTR methods represented by pairwise is directly affected by partial order rules. Partial order rules are manually formulated and there are many rules between multiple targets. In complex scenarios involving heterogeneous resources, richer prior knowledge and more complicated partial order relationships are required to eliminate the differences between heterogeneous resources.

[0047] It can be seen that in the video recommendation scenario involving heterogeneous resources, the existing formula fusion and LTR fusion methods both require manual formulation of complex partial order rules, which have high implementation complexity.

[0048] In order to solve the above technical problems, the present disclosure provides a recommendation model training method and device.

[0049] In one embodiment of the present disclosure, a recommendation model training method is provided, the method comprising:

[0050] Acquire a multi-objective fusion recommendation model, wherein the multi-objective fusion recommendation model includes a plurality of model parameters to be adjusted;

[0051] Determine the current model parameters as verification parameters, and on the basis of the verification parameters, superimpose multiple sets of different parameter perturbations to obtain multiple sets of exploration parameters;

[0052] Obtaining user feedback data and recommended video scene data corresponding to each set of exploration parameters, and obtaining user feedback data and recommended video scene data corresponding to the verification parameters;

[0053] For each set of exploration parameters, a user feedback reward value is calculated based on the user feedback data corresponding to the set of exploration parameters and the user feedback data corresponding to the verification parameters, and a video recommendation scene reward value is calculated based on the recommended video scene data corresponding to the set of exploration parameters and the recommended video scene data corresponding to the verification parameters;

[0054] For each set of exploration parameters, the total reward value is calculated based on the user feedback reward and video recommendation scenario reward value corresponding to the set of exploration parameters;

[0055] A preferred exploration parameter is determined according to the total reward value, and the model parameter is adjusted according to a parameter perturbation corresponding to the preferred exploration parameter.

[0056] In an embodiment of the present disclosure, during the training of a multi-objective fusion recommendation model, multiple sets of parameter perturbations are generated based on the verification parameters to obtain multiple sets of exploration parameters, each set of exploration parameters corresponds to a group of users, and then user feedback data and recommended video scene data corresponding to each set of exploration parameters, and user feedback data and recommended video scene data corresponding to the verification parameters are collected; for each set of exploration parameters, the total reward value of the user feedback reward value and the video recommendation scene reward value is calculated respectively, and then the preferred exploration parameters are determined, and the model parameters are adjusted according to the parameter perturbations corresponding to the preferred exploration parameters.

[0057] Therefore, the parameter perturbation corresponding to the preferred exploration parameter can represent a better evolutionary direction of the multi-objective fusion recommendation model. During the model training process, the model parameters continue to evolve in a better direction, and ultimately, while ensuring that the video recommendation scenario meets the expected goals, the user's viewing time and viewing experience are maximized.

[0058] Moreover, there is no need to manually formulate complex partial order rules, which greatly reduces the implementation complexity.

[0059] The following describes the recommendation model training method and device provided by the embodiments of the present disclosure.

[0060] See also Figure 1 , Figure 1 A flow chart of a recommendation model training method provided in an embodiment of the present disclosure is shown as follows: Figure 1As shown, the method may include the following steps:

[0061] S101: Obtain a multi-objective fusion recommendation model, where the multi-objective fusion recommendation model includes multiple model parameters to be adjusted.

[0062] In the embodiment of the present disclosure, the model structure of the multi-objective fusion recommendation model can be preset, and the fusion factors and fusion features involved in the fusion can be determined.

[0063] The fusion factors may include user factors, resource feature factors, scenario feature factors, video quality factors, etc. The fusion features may include scenario-side features, user features, resource features, etc.

[0064] Those skilled in the art will understand that in the process of recommending videos to users, the multi-objective fusion recommendation model performs weighted operations on the fusion factors and fusion features based on the model parameters to obtain the recommendation scores, and then recommends videos to users in order of the recommendation scores.

[0065] As an example, the operation process of the multi-objective fusion recommendation model can be expressed by the following formula:

[0066]

[0067]

[0068] Among them, weight i Represents the power exponential weight of the i-th fusion factor, feature x represents the xth fusion feature, core represents the recommendation score, w i0 represents the basic weight of the i-th fusion factor, w ix represents the weight of the xth fusion feature of the i-th fusion factor, n represents the number of fusion features, and factor represents the fusion factor.

[0069] In the video recommendation scenario, the fusion factor and fusion feature are both specific values, and the weight of the fusion feature w ix These are the model parameters that need to be adjusted in the multi-objective fusion recommendation model.

[0070] S102: Determine the current model parameters as verification parameters, and on the basis of the verification parameters, superimpose multiple sets of different parameter perturbations to obtain multiple sets of exploration parameters.

[0071] In the disclosed embodiment, the current model parameters are determined as verification parameters, and a multi-objective fusion recommendation model including the verification parameters is used to recommend videos to the verification group users within a certain period of time.

[0072] In addition, based on the verification parameters, multiple sets of randomly generated parameter perturbations are superimposed to obtain a variety of exploration parameters.

[0073] Online users are grouped into multiple exploration groups. Each exploration group contains multiple online users, and each exploration group corresponds to a set of exploration parameters. In other words, a multi-objective fusion recommendation model that includes exploration parameters is used to recommend videos to exploration group users within a certain period of time.

[0074] S103: Obtain user feedback data and recommended video scene data corresponding to each set of exploration parameters, and obtain user feedback data and recommended video scene data corresponding to verification parameters.

[0075] In an embodiment of the present disclosure, after recommending videos to exploration group users using a multi-objective fusion recommendation model containing exploration parameters within a certain period of time, user feedback data of the exploration group users on the recommended videos within this period of time is collected, and scene data of the recommended videos within this period of time is collected.

[0076] User feedback data may include: the duration of time users spend browsing the recommended video, the number of views, and / or the number of interactions with the recommended video. Interactions may include likes, comments, and reposts.

[0077] Recommended video scene data may include: the video type ratio, video physical length, video vertical category ratio and / or author ecology ratio of recommended videos.

[0078] The video type ratio may be the ratio of different types of videos, for example, in a heterogeneous resource recommendation scenario including short videos and mini videos, the ratio of short videos in the recommended videos.

[0079] Video categories can include sports, film and television, and documentaries. The video category ratio represents the proportion of videos in each category in the recommended videos.

[0080] The author ecosystem can be represented by the age range of video creators, or you can customize different author ecosystems based on business needs. The author ecosystem ratio can then represent the proportion of videos from different video creators in the recommended videos.

[0081] Accordingly, user feedback data and recommended video scene data corresponding to the verification parameters are obtained.

[0082] S104: For each set of exploration parameters, a user feedback reward value is calculated based on the user feedback data corresponding to the set of exploration parameters and the user feedback data corresponding to the verification parameters, and a video recommendation scene reward value is calculated based on the recommended video scene data corresponding to the set of exploration parameters and the recommended video scene data corresponding to the verification parameters.

[0083] In the embodiment of the present disclosure, user feedback data belongs to business data, and the goal of model training is to improve business data as much as possible; recommended video scene data belongs to scene data, and the goal of model training is to meet pre-set scene goals as much as possible.

[0084] In order to measure the pros and cons of each set of exploration parameters, the user feedback data corresponding to each set of exploration parameters can be compared with the user feedback data corresponding to the verification parameters, and the user feedback reward value can be calculated.

[0085] It is easy to understand that if the user feedback data corresponding to a certain set of exploration parameters shows that the average browsing time of users for recommended videos is longer, the average browsing data is more, and / or the average number of interactions is higher than that of the user feedback data corresponding to the verification parameters, it means that the user feedback is more positive when using this set of exploration parameters. In this case, the calculated user feedback reward value is larger.

[0086] Accordingly, the recommended video scene data corresponding to each set of exploration parameters is compared with the recommended video scene data corresponding to the verification parameters, and the video recommendation scene reward value is calculated.

[0087] S105: For each set of exploration parameters, a total reward value is calculated based on the user feedback reward value and the video recommendation scene reward value corresponding to the set of exploration parameters.

[0088] In the disclosed embodiment, for each set of exploration parameters, the user feedback reward value and the video recommendation scene reward value can be directly added together to obtain a total reward value. Alternatively, the user feedback reward value and the video recommendation scene reward value can be weighted to obtain a total reward value.

[0089] S106: Determine the preferred exploration parameter according to the total reward value, and adjust the model parameters according to the parameter perturbation corresponding to the preferred exploration parameter.

[0090] In the disclosed embodiment, the total reward value can reflect the overall quality of the exploration parameters. Therefore, one or more sets of preferred exploration parameters can be determined in descending order of the total reward value, and the model parameters can be adjusted according to the parameter perturbations corresponding to the preferred exploration parameters.

[0091] It can be seen that in the embodiment of the present disclosure, during the training of the multi-objective fusion recommendation model, multiple groups of parameter perturbations are generated based on the verification parameters to obtain multiple groups of exploration parameters, each group of exploration parameters corresponds to a group of users, and then the user feedback data and recommended video scene data corresponding to each group of exploration parameters, and the user feedback data and recommended video scene data corresponding to the verification parameters are collected; for each group of exploration parameters, the total reward value of the user feedback reward value and the video recommendation scene reward value is calculated respectively, and then the preferred exploration parameters are determined, and the model parameters are adjusted according to the parameter perturbations corresponding to the preferred exploration parameters.

[0092] Therefore, the parameter perturbation corresponding to the preferred exploration parameter can represent a better evolutionary direction of the multi-objective fusion recommendation model. During the model training process, the model parameters continue to evolve in a better direction, and ultimately, while ensuring that the video recommendation scenario meets the expected goals, the user's viewing time and viewing experience are maximized.

[0093] Moreover, there is no need to manually formulate complex partial order rules, which greatly reduces the implementation complexity.

[0094] In one embodiment of the present disclosure, user feedback data may be collected separately for different types of video entries.

[0095] In the embodiment of the present disclosure, the videos recommended to users are heterogeneous videos, that is, they include multiple types of videos, such as short videos and small videos. Users may click to enter from the short video entrance or from the small video entrance.

[0096] In one embodiment of the present disclosure, user feedback data may be collected separately for different types of video entries.

[0097] Specifically, in step S104, the step of calculating the user feedback reward value based on the user feedback data corresponding to the set of exploration parameters and the user feedback data corresponding to the verification parameters may include the following steps:

[0098] Step 11: Obtain first user feedback data of the exploration group users corresponding to the group of exploration parameters for the recommended videos under the preset multiple types of video entrances.

[0099] Specifically, for the set of exploration parameters, first user feedback data of recommended videos under different types of video entrances are counted respectively.

[0100] That is, for the recommended videos in the short video portal and the recommended videos in the small video portal, the user feedback data of the exploration group users are collected separately. The user feedback data includes the browsing time, number of views and / or number of interactions of users for the recommended videos.

[0101] Step 12: Obtain second user feedback data of the verification group users corresponding to the verification parameters for the recommended videos under the preset multiple types of video entrances.

[0102] Correspondingly, for the verification parameters, the second user feedback data of the recommended videos under different types of video entrances are counted respectively.

[0103] That is to say, for the recommended videos of the short video entrance and the recommended videos of the small video entrance, the user feedback data of the verification group users are counted separately.

[0104] Step 13: Determine the feedback data difference between the first user feedback data and the second user feedback data for each type of video entry, and calculate the user feedback reward value for the type of video entry based on the feedback data difference.

[0105] In the embodiment of the present disclosure, for each type of video entry, the feedback data difference between the first user feedback data and the second user feedback data is calculated respectively, and then the user feedback reward value for the type of video entry is calculated.

[0106] Step 14: Superimpose the user feedback reward values under different types of video entrances to obtain the user feedback reward value corresponding to the set of exploration parameters.

[0107] For example, the user feedback reward value under the short video entrance and the user feedback reward value under the small video entrance are superimposed to obtain the user feedback reward value corresponding to the group of exploration parameters.

[0108] Correspondingly, for video recommendation scene data, entry types are also distinguished and calculated separately.

[0109] In the above step S104, the step of calculating the video recommendation scene reward value based on the recommended video scene data corresponding to the set of exploration parameters and the recommended video scene data corresponding to the verification parameters may specifically include:

[0110] Step 21: Acquire the first recommended video scene data under the preset multiple types of video entries of the exploration group users corresponding to the group of exploration parameters.

[0111] Specifically, for this set of exploration parameters, the recommended video scene data for different types of video entrances are collected. The recommended video scene data may include: the proportion of video types, video physical length, video vertical category proportion, and / or author ecology proportion of recommended videos.

[0112] That is to say, for the recommended videos of the short video entrance and the recommended videos of the small video entrance, the recommended video scene data are counted separately.

[0113] Step 22: Obtain the second recommended video scene data under the preset multiple types of video entries of the verification group users corresponding to the verification parameters.

[0114] Specifically, for the verification parameters, the recommended video scene data under different types of video entrances are also counted separately.

[0115] Step 23: Determine the video scene data difference between the first recommended video scene data and the second recommended video scene data for each type of video entry, and calculate the video recommendation scene reward value for the type of video entry based on the video scene data difference.

[0116] In the embodiment of the present disclosure, for each type of video entry, the difference between the first recommended video scene data and the second recommended video scene data is calculated respectively, and then the video recommendation scene reward value under the type of video entry is calculated.

[0117] Step 24: Superimpose the video recommendation scenario reward values under different types of video entrances to obtain the video recommendation scenario reward value corresponding to the set of exploration parameters.

[0118] For example, the video recommendation scene reward value under the short video entrance and the video recommendation scene reward value under the small video entrance are superimposed to obtain the video recommendation scene reward value corresponding to the set of exploration parameters.

[0119] In one embodiment of the present disclosure, the target expected ranges of the video type ratio, video physical duration, video vertical category ratio and / or author ecological ratio can be set respectively, and the reward weights of the video type ratio, video physical duration, video vertical category ratio and / or author ecological ratio can be set respectively.

[0120] As an example, the target expected range of the physical length of the video is set to 30 seconds to 2 minutes, and the proportion of video types is set to: short videos account for 60% to 70%.

[0121] The above is only an example. In actual applications, the target expected range of video type ratio, video physical length, video vertical category ratio and / or author ecology ratio can be set according to business needs.

[0122] Accordingly, in one embodiment of the present disclosure, after obtaining the first recommended video scene data under the preset multiple types of video entries of the exploration group users corresponding to the group of exploration parameters, the method further includes:

[0123] For the first recommended video scene data under each type of video entrance, the reward weights of the video type ratio, video physical length, video vertical category ratio and / or author ecological ratio are determined respectively according to the deviation relationship between the video type ratio, video physical length, video vertical category ratio and / or author ecological ratio and the pre-set target expected range.

[0124] Among them, if the video type ratio, video physical length, video vertical category ratio and / or author ecological ratio are within the pre-set target expectation range, the reward weight of the video type ratio, video physical length, video vertical category ratio and / or author ecological ratio is determined to be a positive value.

[0125] If the video type ratio, video physical duration, video vertical category ratio, and / or author ecology ratio are not within the preset target expected range, respectively determine the deviation value of the video type ratio, video physical duration, video vertical category ratio, and / or author ecology ratio from the preset target expected range;

[0126] The reward weights for determining the video type ratio, video physical duration, video vertical category ratio, and / or author ecosystem ratio are negative, and the absolute value of the reward weight is positively correlated with the deviation value.

[0127] As an example, if the video vertical category ratio in the first recommended video scene data corresponding to the exploration parameters is within the corresponding target expected range, it means that this set of exploration parameters can better meet the scene requirements of the video vertical category ratio, so the reward weight corresponding to the video vertical category ratio is set to a positive value.

[0128] As an example, if the physical duration of the video in the first recommended video scene data corresponding to the exploration parameters is not within the corresponding target expected range, it means that this set of exploration parameters cannot meet the scene requirements of the physical duration of the video. Therefore, the reward weight corresponding to the physical duration of the video is set to a negative value, and the absolute value of the reward weight is positively correlated with the deviation value.

[0129] For example, the preset target range for video duration is 1-2 minutes, but the first recommended video scene data corresponding to the exploration parameter has a duration of 5 minutes, which is a significant deviation from the target range. Therefore, the importance of the video duration scenario target needs to be increased, that is, the absolute value of the reward weight for the video duration should be set larger. This way, in the subsequent training process, the video duration scenario target can be achieved as quickly as possible.

[0130] Accordingly, the steps of determining the video scene data difference between the first recommended video scene data and the second recommended video scene data for each type of video entry, and calculating the video recommendation scene reward value for the type of video entry based on the video scene data difference, may specifically include:

[0131] Determine the video scene data difference between the first recommended video scene data and the second recommended video scene data for each type of video entry;

[0132] Based on the difference in video scene data, as well as the reward weights of the video type ratio, video physical duration, video vertical category ratio and / or author ecological ratio in the first recommended video scene data, the video recommendation scene reward value under this type of video entry is calculated.

[0133] In other words, the reward value of the video recommendation scenario is calculated by combining the reward weights of the video type ratio, video physical length, video vertical category ratio and / or author ecology ratio, so as to maintain the balance of various scenario goals during the model training process.

[0134] As an example, the following formula may be used to calculate the user feedback reward value and / or the video recommendation scene reward value:

[0135] reward m= m1 *log(1+w m2 *iff m )

[0136] Among them, m represents the index number, reward m Indicates the reward value of indicator m. The indicator can be an indicator of user feedback data, such as browsing time, number of views and / or number of interactions, or an indicator of video recommendation scenario data, such as the video type ratio of recommended videos, video physical length, video vertical category ratio and / or author ecology ratio. m It represents the deviation ratio of the indicator m, and the calculation formula can be: (exploration group indicator - verification group indicator) / verification group indicator, w m1 and w m2 Represents the reward weight of indicator m.

[0137] For indicators of video recommendation scenario data, the calculation method of reward weight is as mentioned above. For indicators of user feedback data, reward weight can be set in advance according to business needs.

[0138] In one embodiment of the present disclosure, different target expected ranges of video type ratios and / or video physical durations are set for different types of video entries.

[0139] Specifically, under different types of video entrances, differentiated target expected ranges of video type ratios and / or video physical durations are formulated.

[0140] For example, if a user clicks on a short video entrance to enter the browsing process, then the user is likely to prefer short video resources. Therefore, for the short video entrance, you can set an expected range of video proportions in which short videos account for a higher proportion, as well as an expected range of physical duration that is more consistent with the physical duration of short videos.

[0141] For example, for a short video portal, the target range for the proportion of short video resources can be set between 70% and 90%, and the target range for the physical length of videos can be set between 1 and 3 minutes, which is consistent with the physical length of short video resources. Therefore, after model training is completed, users who click on the short video portal to start browsing will be given priority recommendations for short video resources.

[0142] It can be seen that in the embodiment of the present disclosure, the user's preference for video resources of the entry type is taken into consideration. In the process of calculating the reward value of the video recommendation scenario, different entry types are distinguished, and the target effects related to resources of the same type as the entry are strengthened, so as to give priority to recommending resources of the same type as the video entry to users.

[0143] At the same time, target expected ranges are set for the video type ratio, video physical duration, video vertical category ratio and / or author ecology ratio as dynamic constraints for each scenario goal, so as to adaptively adjust the evolution process of multiple scenario goals. Ultimately, in the recommendation process of heterogeneous video resources, the video type ratio, video physical duration, video vertical category ratio and author ecology ratio all reach the expected range, and the user's viewing time, viewing experience and interaction scale are maximized.

[0144] In one embodiment of the present disclosure, after adjusting the model parameters according to the parameter disturbance corresponding to the preferred exploration parameter, the process returns to step S102 and enters a new round of training until the model parameters converge.

[0145] Below is the accompanying drawings Figure 2 The recommendation model training method provided in the embodiment of the present disclosure is further explained.

[0146] Figure 2 A schematic diagram of a recommendation model training method provided by an embodiment of the present disclosure, such as Figure 2 As shown, the verification parameters represent the current model parameters. Based on the verification parameters, p groups of parameter perturbations are generated. The parameter perturbations are superimposed on the verification parameters to obtain multiple groups of exploration parameters. Accordingly, online users are divided into multiple groups, each group of users corresponds to a set of exploration parameters. After the exploration parameters are applied online for a period of time, the user feedback data and recommended video scene data corresponding to each set of exploration parameters are obtained, and the user feedback data and recommended video scene data corresponding to the verification parameters are obtained, and then the reward value is calculated. In the process of calculating the reward value, differentiated scenario goals are formulated for heterogeneous resources. At the same time, dynamic constraints based on the expected range of targets and multi-objective adaptive adjustment of reward weights are adopted to ensure that the video type ratio, video physical duration, video vertical category ratio and author ecological ratio of heterogeneous resources are balanced. Ultimately, while ensuring that each scenario goal reaches the expected range, the user's viewing time, viewing experience and interaction scale are maximized.

[0147] See also Figure 3 , Figure 3 A flow chart of a video recommendation method provided by an embodiment of the present disclosure is shown as follows: Figure 3 As shown, the following steps are included:

[0148] S301: Determine target users and candidate videos.

[0149] S302: Determine a fusion factor and a fusion feature based on the target user and each candidate video.

[0150] In the disclosed embodiments, after determining the target user and candidate videos, the fusion factors and fusion features can be determined. Fusion factors can include user factors, resource feature factors, scenario-specific feature factors, and video quality factors. Fusion features can include scenario-specific features, user features, and resource features.

[0151] The fusion factor and fusion feature are both specific values.

[0152] S303: Input the fusion factor and fusion feature into a pre-trained multi-objective fusion recommendation model to obtain a recommendation score for each candidate video.

[0153] The fusion factor and fusion feature are input into the pre-trained multi-objective fusion recommendation model. The model parameters in the multi-objective fusion recommendation model are the weights of the fusion factor and the fusion feature. The fusion factor and the fusion feature can be weighted based on the model parameters to obtain the recommendation score.

[0154] In the embodiment of the present disclosure, the multi-objective fusion recommendation model can be used Figure 1 The pipeline shown is pre-trained.

[0155] S304: Recommend candidate videos to the target user based on the recommendation scores.

[0156] Specifically, candidate videos may be recommended to the user in descending order of recommendation scores.

[0157] Due to the adoption Figure 1 The process shown here pre-trains a multi-objective fusion recommendation model. During training, the model parameters continuously evolve towards optimal results, ultimately ensuring that video recommendation scenarios meet the desired goals while maximizing user viewing time and experience. Furthermore, there's no need to manually formulate complex partial ordering rules, significantly reducing implementation complexity.

[0158] See also Figure 4 , Figure 4 This is a block diagram of an apparatus for implementing the recommendation model training method according to an embodiment of the present disclosure, comprising:

[0159] A first acquisition module 401 is configured to acquire a multi-objective fusion recommendation model, wherein the multi-objective fusion recommendation model includes a plurality of model parameters to be adjusted;

[0160] A perturbation module 402 is configured to determine the current model parameters as verification parameters and, based on the verification parameters, superimpose multiple sets of different parameter perturbations to obtain multiple sets of exploration parameters;

[0161] The second acquisition module 403 is used to obtain user feedback data and recommended video scene data corresponding to each set of exploration parameters, and obtain user feedback data and recommended video scene data corresponding to the verification parameters;

[0162] A first calculation module 404 calculates, for each set of exploration parameters, a user feedback reward value based on user feedback data corresponding to the set of exploration parameters and user feedback data corresponding to the verification parameters, and calculates a video recommendation scene reward value based on recommended video scene data corresponding to the set of exploration parameters and recommended video scene data corresponding to the verification parameters;

[0163] A second calculation module 405 is configured to calculate a total reward value for each set of exploration parameters based on the user feedback reward value and the video recommendation scene reward value corresponding to the set of exploration parameters;

[0164] The adjustment module 406 is configured to determine a preferred exploration parameter according to the total reward value, and adjust the model parameters according to a parameter perturbation corresponding to the preferred exploration parameter.

[0165] In one embodiment of the present disclosure, the first calculation module 404 is specifically configured to:

[0166] Obtaining first user feedback data of the exploration group users corresponding to the group of exploration parameters for the recommended videos under the preset multiple types of video entrances;

[0167] Obtaining second user feedback data of the verification group users corresponding to the verification parameters for the recommended videos under the preset multiple types of video entrances;

[0168] Determining a feedback data difference between the first user feedback data and the second user feedback data for each type of video entry, and calculating a user feedback reward value for the type of video entry based on the feedback data difference;

[0169] The user feedback reward values under different types of video entrances are superimposed to obtain the user feedback reward value corresponding to the set of exploration parameters.

[0170] In one embodiment of the present disclosure, the user feedback data includes:

[0171] The user's viewing time, number of views, and / or number of interactions with the recommended video.

[0172] In one embodiment of the present disclosure, the first calculation module 404 is specifically configured to:

[0173] Obtaining first recommended video scene data under preset multiple video entry types for users in the exploration group corresponding to the group of exploration parameters;

[0174] Obtain second recommended video scene data under preset multiple types of video entries for users in the verification group corresponding to the verification parameters;

[0175] Determining a video scene data difference between the first recommended video scene data and the second recommended video scene data for each type of video entry, and calculating a video recommendation scene reward value for the type of video entry based on the video scene data difference;

[0176] The video recommendation scenario reward values under different types of video entrances are superimposed to obtain the video recommendation scenario reward value corresponding to the set of exploration parameters.

[0177] In one embodiment of the present disclosure, the recommended video scene data includes:

[0178] Recommended videos include the video type ratio, video physical length, video vertical category ratio, and / or author ecology ratio.

[0179] In one embodiment of the present disclosure, it further includes:

[0180] The expected range setting module is used to set the target expected ranges of video type ratio, video physical duration, video vertical category ratio and / or author ecology ratio respectively;

[0181] The reward weight setting module is used to set the reward weights of video type ratio, video physical length, video vertical category ratio and / or author ecology ratio respectively;

[0182] The device further includes a reward weight determination module, configured to:

[0183] After obtaining the first recommended video scene data under the preset multiple types of video entrances of the exploration group users corresponding to the group of exploration parameters, for the first recommended video scene data under each type of video entrance, the reward weights of the video type ratio, video physical duration, video vertical category ratio and / or author ecological ratio are determined respectively according to the deviation relationship between the video type ratio, video physical duration, video vertical category ratio and / or author ecological ratio and the pre-set target expected range;

[0184] If the video type ratio, video physical length, video vertical category ratio, and / or author ecosystem ratio are within the pre-set target expected range, the reward weight for the video type ratio, video physical length, video vertical category ratio, and / or author ecosystem ratio will be determined as a positive value.

[0185] If the video type ratio, video physical duration, video vertical category ratio, and / or author ecology ratio are not within the preset target expected range, respectively determine the deviation value of the video type ratio, video physical duration, video vertical category ratio, and / or author ecology ratio from the preset target expected range;

[0186] The reward weights for determining the video type ratio, video physical duration, video vertical category ratio, and / or author ecology ratio are negative, and the absolute value of the reward weight is positively correlated with the deviation value.

[0187] In one embodiment of the present disclosure, the first calculation module 404 is specifically configured to:

[0188] Determine a video scene data difference between the first recommended video scene data and the second recommended video scene data for each type of video entry;

[0189] Based on the difference in the video scene data, and the reward weights of the video type ratio, video physical duration, video vertical category ratio and / or author ecological ratio in the first recommended video scene data, the video recommendation scene reward value under this type of video entry is calculated.

[0190] In one embodiment of the present disclosure, different target expected ranges of video type ratios and / or video physical durations are set for different types of video entries.

[0191] It can be seen that in the embodiment of the present disclosure, during the training of the multi-objective fusion recommendation model, multiple groups of parameter perturbations are generated based on the verification parameters to obtain multiple groups of exploration parameters, each group of exploration parameters corresponds to a group of users, and then the user feedback data and recommended video scene data corresponding to each group of exploration parameters, and the user feedback data and recommended video scene data corresponding to the verification parameters are collected; for each group of exploration parameters, the total reward value of the user feedback reward value and the video recommendation scene reward value is calculated respectively, and then the preferred exploration parameters are determined, and the model parameters are adjusted according to the parameter perturbations corresponding to the preferred exploration parameters.

[0192] Therefore, the parameter perturbation corresponding to the preferred exploration parameter can represent a better evolutionary direction of the multi-objective fusion recommendation model. During the model training process, the model parameters continue to evolve in a better direction, and ultimately, while ensuring that the video recommendation scenario meets the expected goals, the user's viewing time and viewing experience are maximized.

[0193] Moreover, there is no need to manually formulate complex partial order rules, which greatly reduces the implementation complexity.

[0194] See also Figure 5 , Figure 5 is a block diagram of a device for implementing the video recommendation method according to an embodiment of the present disclosure, the device including:

[0195] A first determination module 501 is used to determine a target user and candidate videos;

[0196] A second determining module 502 is configured to determine a fusion factor and a fusion feature based on the target user and each candidate video;

[0197] The recommendation module 503 is configured to recommend candidate videos to the target user based on the recommendation scores.

[0198] Due to the adoption Figure 1 The process shown here pre-trains a multi-objective fusion recommendation model. During training, the model parameters continuously evolve towards optimal results, ultimately ensuring that video recommendation scenarios meet the desired goals while maximizing user viewing time and experience. Furthermore, there's no need to manually formulate complex partial ordering rules, significantly reducing implementation complexity.

[0199] According to an embodiment of the present disclosure, the present disclosure also provides an electronic device, a readable storage medium, and a computer program product.

[0200] The present disclosure provides an electronic device, comprising:

[0201] at least one processor; and

[0202] a memory communicatively connected to the at least one processor; wherein,

[0203] The memory stores instructions that can be executed by the at least one processor, and the instructions are executed by the at least one processor to enable the at least one processor to perform the recommendation model training method.

[0204] The present disclosure provides a non-transitory computer-readable storage medium storing computer instructions, wherein the computer instructions are used to enable the computer to execute a recommendation model training method.

[0205] The present disclosure provides a computer program product, including a computer program, which is executed by a processor to perform a recommendation model training method.

[0206] Figure 6 A schematic block diagram of an example electronic device 600 that can be used to implement embodiments of the present disclosure is shown. The electronic device is intended to represent various forms of digital computers, such as laptop computers, desktop computers, workstations, personal digital assistants, servers, blade servers, mainframe computers, and other suitable computers. The electronic device can also represent various forms of mobile devices, such as personal digital assistants, cellular phones, smartphones, wearable devices, and other similar computing devices. The components shown herein, their connections and relationships, and their functions are provided as examples only and are not intended to limit the implementation of the present disclosure described and / or claimed herein.

[0207] like Figure 6As shown, the device 600 includes a computing unit 601, which can perform various appropriate actions and processes according to a computer program stored in a read-only memory (ROM) 602 or a computer program loaded from a storage unit 608 into a random access memory (RAM) 603. Various programs and data required for the operation of the device 600 can also be stored in the RAM 603. The computing unit 601, the ROM 602, and the RAM 603 are connected to each other via a bus 604. An input / output (I / O) interface 605 is also connected to the bus 604.

[0208] Various components in device 600 are connected to I / O interface 605, including an input unit 606, such as a keyboard, mouse, etc.; an output unit 607, such as various types of displays, speakers, etc.; a storage unit 608, such as a magnetic disk, optical disk, etc.; and a communication unit 609, such as a network card, modem, wireless communication transceiver, etc. The communication unit 609 allows device 600 to exchange information / data with other devices via a computer network such as the Internet and / or various telecommunication networks.

[0209] The computing unit 601 can be a variety of general-purpose and / or specialized processing components with processing and computing capabilities. Some examples of the computing unit 601 include, but are not limited to, a central processing unit (CPU), a graphics processing unit (GPU), various dedicated artificial intelligence (AI) computing chips, various computing units that run machine learning model algorithms, digital signal processors (DSPs), and any appropriate processors, controllers, microcontrollers, etc. The computing unit 601 performs the various methods and processes described above, such as the recommendation model training method. For example, in some embodiments, the recommendation model training method can be implemented as a computer software program that is tangibly contained in a machine-readable medium, such as the storage unit 608. In some embodiments, part or all of the computer program can be loaded and / or installed on the device 600 via the ROM 602 and / or the communication unit 609. When the computer program is loaded into the RAM 603 and executed by the computing unit 601, one or more steps of the recommendation model training method described above can be performed. Alternatively, in other embodiments, the computing unit 601 can be configured to perform the recommendation model training method by any other appropriate means (e.g., by means of firmware).

[0210] Various embodiments of the systems and techniques described above can be implemented in digital electronic circuit systems, integrated circuit systems, field programmable gate arrays (FPGAs), application specific integrated circuits (ASICs), application specific standard products (ASSPs), system-on-chip systems (SOCs), complex programmable logic devices (CPLDs), computer hardware, firmware, software, and / or combinations thereof. These various embodiments can include being implemented in one or more computer programs that are executable and / or interpreted on a programmable system that includes at least one programmable processor, which can be a special purpose or general purpose programmable processor that can receive data and instructions from a storage system, at least one input device, and at least one output device, and transmit data and instructions to the storage system, the at least one input device, and the at least one output device.

[0211] The program code for implementing the method of the present disclosure can be written in any combination of one or more programming languages. These program codes can be provided to a processor or controller of a general-purpose computer, a special-purpose computer, or other programmable data processing device so that when the program code is executed by the processor or controller, the functions / operations specified in the flow chart and / or block diagram are implemented. The program code can be executed entirely on the machine, partially on the machine, as a stand-alone software package, partially on the machine and partially on a remote machine, or entirely on a remote machine or server.

[0212] In the context of the present disclosure, a machine-readable medium can be a tangible medium that can contain or store a program for use by or in conjunction with an instruction execution system, device or equipment. A machine-readable medium can be a machine-readable signal medium or a machine-readable storage medium. A machine-readable medium can include, but is not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, device or equipment, or any suitable combination of the foregoing. A more specific example of a machine-readable storage medium can include an electrical connection based on one or more lines, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing.

[0213] To provide interaction with a user, the systems and techniques described herein can be implemented on a computer having: a display device (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor) for displaying information to the user; and a keyboard and pointing device (e.g., a mouse or trackball) through which the user can provide input to the computer. Other types of devices can also be used to provide interaction with the user; for example, the feedback provided to the user can be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user can be received in any form (including acoustic input, voice input, or tactile input).

[0214] The systems and techniques described herein can be implemented in a computing system that includes back-end components (e.g., as a data server), or a computing system that includes middleware components (e.g., an application server), or a computing system that includes front-end components (e.g., a user computer having a graphical user interface or a web browser through which a user can interact with implementations of the systems and techniques described herein), or a computing system that includes any combination of such back-end components, middleware components, or front-end components. The components of the system can be interconnected by any form or medium of digital data communication (e.g., a communication network). Examples of communication networks include a local area network (LAN), a wide area network (WAN), and the Internet.

[0215] A computer system may include a client and a server. The client and server are generally remote from each other and typically interact through a communication network. The client-server relationship arises through computer programs running on the respective computers and having a client-server relationship with each other. The server may be a cloud server, a server in a distributed system, or a server integrated with a blockchain.

[0216] It should be understood that the various forms of the processes shown above can be used to reorder, add, or delete steps. For example, the steps described in this disclosure can be performed in parallel, sequentially, or in a different order, as long as the desired results of the technical solutions disclosed in this disclosure can be achieved. This is not limited herein.

[0217] The above specific embodiments do not constitute a limitation on the scope of protection of this disclosure. Those skilled in the art will appreciate that various modifications, combinations, sub-combinations, and substitutions may be made based on design requirements and other factors. Any modifications, equivalent substitutions, and improvements made within the spirit and principles of this disclosure shall be included within the scope of protection of this disclosure.

Claims

1. A recommendation model training method, comprising: Acquire a multi-objective fusion recommendation model, wherein the multi-objective fusion recommendation model includes a plurality of model parameters to be adjusted; Determine the current model parameters as verification parameters, and on the basis of the verification parameters, superimpose multiple sets of different parameter perturbations to obtain multiple sets of exploration parameters; Obtaining user feedback data and recommended video scene data corresponding to each set of exploration parameters, and obtaining user feedback data and recommended video scene data corresponding to the verification parameters; For each set of exploration parameters, a user feedback reward value is calculated based on the user feedback data corresponding to the set of exploration parameters and the user feedback data corresponding to the verification parameters, and a video recommendation scene reward value is calculated based on the recommended video scene data corresponding to the set of exploration parameters and the recommended video scene data corresponding to the verification parameters; For each set of exploration parameters, the total reward value is calculated based on the user feedback reward value and the video recommendation scene reward value corresponding to the set of exploration parameters; A preferred exploration parameter is determined according to the total reward value, and the model parameter is adjusted according to a parameter perturbation corresponding to the preferred exploration parameter.

2. The method according to claim 1, wherein The step of calculating the user feedback reward value based on the user feedback data corresponding to the set of exploration parameters and the user feedback data corresponding to the verification parameters includes: Obtaining first user feedback data of the exploration group users corresponding to the group of exploration parameters for the recommended videos under the preset multiple types of video entrances; Obtaining second user feedback data of the verification group users corresponding to the verification parameters for the recommended videos under the preset multiple types of video entrances; Determining a feedback data difference between the first user feedback data and the second user feedback data for each type of video entry, and calculating a user feedback reward value for the type of video entry based on the feedback data difference; The user feedback reward values under different types of video entrances are superimposed to obtain the user feedback reward value corresponding to the set of exploration parameters.

3. The method according to claim 1 or 2, wherein: The user feedback data includes: The user's viewing time, number of views, and / or number of interactions with the recommended video.

4. The method according to claim 1, wherein The step of calculating the video recommendation scene reward value based on the recommended video scene data corresponding to the set of exploration parameters and the recommended video scene data corresponding to the verification parameters includes: Obtaining first recommended video scene data under preset multiple video entry types for users in the exploration group corresponding to the group of exploration parameters; Obtain second recommended video scene data under preset multiple types of video entries for users in the verification group corresponding to the verification parameters; Determining a video scene data difference between the first recommended video scene data and the second recommended video scene data for each type of video entry, and calculating a video recommendation scene reward value for the type of video entry based on the video scene data difference; The video recommendation scenario reward values under different types of video entrances are superimposed to obtain the video recommendation scenario reward value corresponding to the set of exploration parameters.

5. The method according to claim 4, wherein The recommended video scene data includes: Recommended videos include the video type ratio, video physical length, video vertical category ratio, and / or author ecology ratio.

6. The method according to claim 5, further comprising: Set target expected ranges for video type ratio, video physical duration, video vertical category ratio, and / or author ecosystem ratio, and set reward weights for each; After obtaining the first recommended video scene data under the preset multiple types of video entries of the exploration group users corresponding to the group of exploration parameters, the method further includes: For the first recommended video scene data under each type of video entry, the reward weights of the video type ratio, video physical length, video vertical category ratio and / or author ecological ratio are determined based on the deviation relationship between the video type ratio, video physical length, video vertical category ratio and / or author ecological ratio and the pre-set target expected range; If the video type ratio, video physical length, video vertical category ratio, and / or author ecosystem ratio are within the pre-set target expected range, the reward weight for the video type ratio, video physical length, video vertical category ratio, and / or author ecosystem ratio will be determined as a positive value. If the video type ratio, video physical duration, video vertical category ratio, and / or author ecology ratio are not within the preset target expected range, respectively determine the deviation value of the video type ratio, video physical duration, video vertical category ratio, and / or author ecology ratio from the preset target expected range; The reward weights for determining the video type ratio, video physical duration, video vertical category ratio, and / or author ecology ratio are negative, and the absolute value of the reward weight is positively correlated with the deviation value.

7. The method according to claim 6, wherein: The step of determining the video scene data difference between the first recommended video scene data and the second recommended video scene data for each type of video entry, and calculating the video recommendation scene reward value for the type of video entry based on the video scene data difference, includes: Determine a video scene data difference between the first recommended video scene data and the second recommended video scene data for each type of video entry; Based on the difference in the video scene data, and the reward weights of the video type ratio, video physical duration, video vertical category ratio and / or author ecological ratio in the first recommended video scene data, the video recommendation scene reward value under this type of video entry is calculated.

8. The method according to any one of claims 6 to 7, wherein: The target expected ranges of video type ratios and / or video physical durations set for different types of video entrances are different.

9. A video recommendation method, comprising: Identify target users and candidate videos; Determining a fusion factor and a fusion feature according to the target user and each candidate video; Inputting the fusion factor and the fusion feature into a pre-trained multi-objective fusion recommendation model to obtain a recommendation score for each candidate video; the pre-trained multi-objective fusion recommendation model is trained using the method of any one of claims 1 to 8; Recommend candidate videos to the target user based on the recommendation score.

10. A recommendation model training device, comprising: A first acquisition module is used to acquire a multi-objective fusion recommendation model, where the multi-objective fusion recommendation model includes a plurality of model parameters to be adjusted; A perturbation module is used to determine the current model parameters as verification parameters, and on the basis of the verification parameters, superimpose multiple sets of different parameter perturbations to obtain multiple sets of exploration parameters; A second acquisition module is used to obtain user feedback data and recommended video scene data corresponding to each set of exploration parameters, and obtain user feedback data and recommended video scene data corresponding to the verification parameters; A first calculation module calculates, for each set of exploration parameters, a user feedback reward value based on user feedback data corresponding to the set of exploration parameters and user feedback data corresponding to the verification parameters, and calculates a video recommendation scene reward value based on recommended video scene data corresponding to the set of exploration parameters and recommended video scene data corresponding to the verification parameters; A second calculation module is used to calculate a total reward value for each set of exploration parameters based on the user feedback reward value and the video recommendation scene reward value corresponding to the set of exploration parameters; An adjustment module is configured to determine a preferred exploration parameter according to the total reward value, and adjust the model parameters according to a parameter perturbation corresponding to the preferred exploration parameter.

11. The device according to claim 10, wherein The first calculation module is specifically configured to: Obtaining first user feedback data of the exploration group users corresponding to the group of exploration parameters for the recommended videos under the preset multiple types of video entrances; Obtaining second user feedback data of the verification group users corresponding to the verification parameters for the recommended videos under the preset multiple types of video entrances; Determining a feedback data difference between the first user feedback data and the second user feedback data for each type of video entry, and calculating a user feedback reward value for the type of video entry based on the feedback data difference; The user feedback reward values under different types of video entrances are superimposed to obtain the user feedback reward value corresponding to the set of exploration parameters.

12. The device according to claim 10 or 11, wherein The user feedback data includes: The user's viewing time, number of views, and / or number of interactions with the recommended video.

13. The device according to claim 10, wherein The first calculation module is specifically configured to: Obtaining first recommended video scene data under preset multiple video entry types for users in the exploration group corresponding to the group of exploration parameters; Obtain second recommended video scene data under preset multiple types of video entries for users in the verification group corresponding to the verification parameters; Determining a video scene data difference between the first recommended video scene data and the second recommended video scene data for each type of video entry, and calculating a video recommendation scene reward value for the type of video entry based on the video scene data difference; The video recommendation scenario reward values under different types of video entrances are superimposed to obtain the video recommendation scenario reward value corresponding to the set of exploration parameters.

14. The device according to claim 13, wherein The recommended video scene data includes: Recommended videos include the video type ratio, video physical length, video vertical category ratio, and / or author ecology ratio.

15. The apparatus according to claim 14, further comprising: The expected range setting module is used to set the target expected ranges of video type ratio, video physical duration, video vertical category ratio and / or author ecology ratio respectively; The reward weight setting module is used to set the reward weights of video type ratio, video physical length, video vertical category ratio and / or author ecology ratio respectively; The device further includes a reward weight determination module, configured to: After obtaining the first recommended video scene data under the preset multiple types of video entrances of the exploration group users corresponding to the group of exploration parameters, for the first recommended video scene data under each type of video entrance, the reward weights of the video type ratio, video physical duration, video vertical category ratio and / or author ecological ratio are determined respectively according to the deviation relationship between the video type ratio, video physical duration, video vertical category ratio and / or author ecological ratio and the pre-set target expected range; If the video type ratio, video physical length, video vertical category ratio, and / or author ecosystem ratio are within the pre-set target expected range, the reward weight for the video type ratio, video physical length, video vertical category ratio, and / or author ecosystem ratio will be determined as a positive value. If the video type ratio, video physical duration, video vertical category ratio, and / or author ecology ratio are not within the preset target expected range, respectively determine the deviation value of the video type ratio, video physical duration, video vertical category ratio, and / or author ecology ratio from the preset target expected range; The reward weights for determining the video type ratio, video physical duration, video vertical category ratio, and / or author ecology ratio are negative, and the absolute value of the reward weight is positively correlated with the deviation value.

16. The device according to claim 15, wherein The first calculation module is specifically configured to: Determine a video scene data difference between the first recommended video scene data and the second recommended video scene data for each type of video entry; Based on the difference in the video scene data, and the reward weights of the video type ratio, video physical duration, video vertical category ratio and / or author ecological ratio in the first recommended video scene data, the video recommendation scene reward value under this type of video entry is calculated.

17. The device according to any one of claims 15-16, wherein: The target expected ranges of video type ratios and / or video physical durations set for different types of video entrances are different.

18. A video recommendation device, comprising: A first determination module is used to determine a target user and candidate videos; A second determination module is used to determine a fusion factor and a fusion feature according to the target user and each candidate video; a score acquisition module, configured to input the fusion factor and fusion feature into a pre-trained multi-objective fusion recommendation model to obtain a recommendation score for each candidate video; the pre-trained multi-objective fusion recommendation model is trained using the method of any one of claims 1 to 8; A recommendation module is configured to recommend candidate videos to the target user based on the recommendation score.

19. An electronic device comprising: at least one processor; as well as a memory communicatively connected to the at least one processor; wherein, The memory stores instructions that can be executed by the at least one processor, and the instructions are executed by the at least one processor to enable the at least one processor to perform the method according to any one of claims 1 to 9.

20. A non-transitory computer-readable storage medium storing computer instructions, wherein: The computer instructions are used to cause the computer to execute the method according to any one of claims 1 to 9.

21. A computer program product comprising a computer program, which, when executed by a processor, implements the method according to any one of claims 1 to 9.

Images