Content recommendation information determination method, device, server and storage medium

Abstract

The present disclosure relates to a method, device, server and storage medium for determining content recommendation information. The method includes: obtaining first sample data obtained by using first content recommendation information and a first content recommendation model, and second sample data obtained by using newly added second content recommendation information and the first content recommendation model; obtaining target sample data other than the first sample data in the second sample data; isolating the target sample data from the first content recommendation model; obtaining a first evaluation coefficient of a first content recommendation result obtained by using the first content recommendation information and the first content recommendation model, and a second evaluation coefficient of a second content recommendation result obtained by using the first content recommendation information and the second content recommendation model; the second content recommendation model is obtained by training the target sample data; and determining the target content recommendation information according to the first evaluation coefficient and the second evaluation coefficient. The method is used to improve the accuracy of determining content recommendation information.

Description

Technical Field

[0001] The present disclosure relates to the field of Internet technology, and in particular to a method, device, server and storage medium for determining content recommendation information. Background Art

[0002] With the development of Internet technology, various content recommendation information or content recommendation models are used when recommending content; however, before using new content recommendation information or content recommendation models, experiments need to be performed on them.

[0003] In the related art, generally, a basic group and an experimental group are set up, and then the experimental effects of the basic group and the experimental group are judged to determine which content recommendation information is better; however, after a period of experimentation, the experimental effects of the basic group and the experimental group are very similar, which cannot reflect the actual experimental effects of the content recommendation information, resulting in a low accuracy rate in determining the content recommendation information. Summary of the invention

[0004] The present disclosure provides a method, device, server and storage medium for determining content recommendation information, so as to at least solve the problem of low accuracy in determining content recommendation information in related technologies. The technical solution of the present disclosure is as follows:

[0005] According to a first aspect of an embodiment of the present disclosure, a method for determining content recommendation information is provided, comprising:

[0006] Acquire first sample data obtained by using first content recommendation information and a first content recommendation model, and obtain second sample data obtained by using newly added second content recommendation information and the first content recommendation model;

[0007] Acquire target sample data other than the first sample data from the second sample data; the target sample data is isolated from the first content recommendation model;

[0008] Obtaining a first evaluation coefficient of a first content recommendation result obtained by using the first content recommendation information and the first content recommendation model, and a second evaluation coefficient of a second content recommendation result obtained by using the first content recommendation information and the second content recommendation model; the second content recommendation model is obtained by training the first content recommendation model using the target sample data;

[0009] Target content recommendation information is determined according to the first evaluation coefficient and the second evaluation coefficient.

[0010] In an exemplary embodiment, the obtaining a first evaluation coefficient of a first content recommendation result obtained by using the first content recommendation information and the first content recommendation model includes:

[0011] Acquire posterior data of a first content recommendation result obtained by using the first content recommendation information and the first content recommendation model;

[0012] A first evaluation coefficient of the first content recommendation result is determined according to the a posteriori data.

[0013] In an exemplary embodiment, determining target content recommendation information according to the first evaluation coefficient and the second evaluation coefficient includes:

[0014] When the second evaluation coefficient is greater than the first evaluation coefficient, the second content recommendation information is determined as target content recommendation information.

[0015] In an exemplary embodiment, before determining the target content recommendation information according to the first evaluation coefficient and the second evaluation coefficient, the method further includes:

[0016] Obtaining a third evaluation coefficient of a third content recommendation result obtained by using the second content recommendation information and the second content recommendation model;

[0017] When the second evaluation coefficient is greater than the first evaluation coefficient, determining the first content recommendation information as the second content recommendation information includes:

[0018] When the second evaluation coefficient is greater than the first evaluation coefficient and less than the third evaluation coefficient, the second content recommendation information is determined as target content recommendation information.

[0019] In an exemplary embodiment, the acquiring target sample data other than the second sample data in the first sample data includes:

[0020] When the first evaluation coefficient is less than a fourth evaluation coefficient, target sample data other than the second sample data in the first sample data is obtained; the fourth evaluation coefficient is an evaluation coefficient corresponding to a fourth content recommendation result obtained by using the second content recommendation information and the first content recommendation model.

[0021] In an exemplary embodiment, after acquiring the target sample data other than the second sample data in the first sample data, the method further includes:

[0022] The first content recommendation model is trained according to the target sample data until a training end condition is met, and the trained first content recommendation model that meets the training end condition is used as the second content recommendation model.

[0023] In an exemplary embodiment, after determining the target content recommendation information according to the first evaluation coefficient and the second evaluation coefficient, the method further includes:

[0024] Determining the content to be recommended and the account to be recommended according to the target content recommendation information;

[0025] The content to be recommended is recommended to the account to be recommended.

[0026] According to a second aspect of an embodiment of the present disclosure, there is provided a device for determining content recommendation information, including:

[0027] A data acquisition unit is configured to acquire first sample data obtained by using first content recommendation information and a first content recommendation model, and second sample data obtained by using newly added second content recommendation information and the first content recommendation model;

[0028] A data screening unit is configured to obtain target sample data other than the first sample data from the second sample data; the target sample data is isolated from the first content recommendation model;

[0029] a coefficient acquisition unit configured to acquire a first evaluation coefficient of a first content recommendation result obtained by using the first content recommendation information and the first content recommendation model, and a second evaluation coefficient of a second content recommendation result obtained by using the first content recommendation information and the second content recommendation model; the second content recommendation model is obtained by training the first content recommendation model using the target sample data;

[0030] The information determination unit is configured to determine target content recommendation information according to the first evaluation coefficient and the second evaluation coefficient.

[0031] In an exemplary embodiment, the coefficient acquisition unit is further configured to acquire a posteriori data of a first content recommendation result obtained by using the first content recommendation information and the first content recommendation model; and determine a first evaluation coefficient of the first content recommendation result based on the posteriori data.

[0032] In an exemplary embodiment, the information determination unit is further configured to determine the second content recommendation information as the target content recommendation information when the second evaluation coefficient is greater than the first evaluation coefficient.

[0033] In an exemplary embodiment, the apparatus further includes a coefficient determination unit configured to execute obtaining a third evaluation coefficient of a third content recommendation result obtained by using the second content recommendation information and the second content recommendation model;

[0034] The information determination unit is further configured to determine the second content recommendation information as target content recommendation information when the second evaluation coefficient is greater than the first evaluation coefficient and less than the third evaluation coefficient.

[0035] In an exemplary embodiment, the data screening unit is further configured to obtain target sample data other than the second sample data in the first sample data when the first evaluation coefficient is less than a fourth evaluation coefficient; the fourth evaluation coefficient is an evaluation coefficient corresponding to a fourth content recommendation result obtained using the second content recommendation information and the first content recommendation model.

[0036] In an exemplary embodiment, the device also includes a model training unit, which is configured to perform training on the first content recommendation model according to the target sample data until a training end condition is reached, and use the trained first content recommendation model that meets the training end condition as the second content recommendation model.

[0037] In an exemplary embodiment, the device further includes a content recommendation unit configured to determine the content to be recommended and the account to be recommended according to the target content recommendation information; and recommend the content to be recommended to the account to be recommended.

[0038] According to a third aspect of an embodiment of the present disclosure, a server is provided, comprising: a processor; and a memory for storing instructions executable by the processor; wherein the processor is configured to execute the instructions to implement a method for determining content recommendation information as described in any one of the embodiments of the first aspect.

[0039] According to a fourth aspect of an embodiment of the present disclosure, a computer-readable storage medium is provided. When instructions in the computer-readable storage medium are executed by a processor of a server, the server is enabled to execute the content recommendation information determination method described in any one of the embodiments of the first aspect.

[0040] According to a fifth aspect of an embodiment of the present disclosure, a computer program product is provided, wherein the computer program product comprises instructions, and when the instructions are executed by a processor of a server, the server is able to execute the method for determining content recommendation information described in any one of the embodiments of the first aspect.

[0041] The technical solution provided by the embodiments of the present disclosure brings at least the following beneficial effects:

[0042] The first sample data obtained by using the first content recommendation information and the first content recommendation model, and the second sample data obtained by using the newly added second content recommendation information and the first content recommendation model are obtained; then, the target sample data other than the first sample data in the second sample data is obtained; the target sample data is used to train the first content recommendation model to obtain the second content recommendation model; then, the first evaluation coefficient of the first content recommendation result obtained by using the first content recommendation information and the first content recommendation model, and the second evaluation coefficient of the second content recommendation result obtained by using the first content recommendation information and the second content recommendation model are obtained; finally, the target content recommendation information is determined according to the first evaluation coefficient and the second evaluation coefficient. In this way, when judging the recommendation effect of the newly added second content recommendation information, the target sample data other than the first sample data obtained by using the first content recommendation information and the first content recommendation model in the second sample data obtained by using the newly added second content recommendation information and the first content recommendation model are separated from the first content recommendation model to prevent the target sample data from being learned by the first content recommendation model and affecting the recommendation effect of the newly added second content recommendation information, resulting in the inability to truly judge the recommendation effect of the second content recommendation information. At the same time, the target sample data is used to train the newly added second content recommendation information and the first content recommendation model in the first content recommendation model to obtain the second content recommendation model, and the second evaluation coefficient of the second content recommendation result obtained by using the first content recommendation information and the second content recommendation model is compared with the first evaluation coefficient of the first content recommendation result obtained by using the first content recommendation information and the first content recommendation model. This can accurately judge the recommendation effect of the target sample data obtained by using the newly added second content recommendation information, and then accurately judge the recommendation effect of using the newly added second content recommendation information, thereby achieving the purpose of accurately judging the recommendation effect of using the newly added second content recommendation information while isolating the target sample data, thereby improving the accuracy of determining the target content recommendation information.

[0043] It is to be understood that the foregoing general description and the following detailed description are exemplary and explanatory only and are not restrictive of the present disclosure. BRIEF DESCRIPTION OF THE DRAWINGS

[0044] The drawings herein are incorporated into and constitute a part of the specification, illustrate embodiments consistent with the present disclosure, and together with the description are used to explain the principles of the present disclosure, and do not constitute improper limitations on the present disclosure.

[0045] Figure 1 The figure is a flowchart of a method for determining content recommendation information according to an exemplary embodiment.

[0046] Figure 2 is a flowchart showing steps of obtaining a first evaluation coefficient according to an exemplary embodiment.

[0047] Figure 3 The figure is a flowchart of another method for determining content recommendation information according to an exemplary embodiment.

[0048] Figure 4 The invention is a flowchart showing the isolation of a content recommendation model and content recommendation information according to an exemplary embodiment.

[0049] Figure 5 The present invention is a flowchart showing a method of determining whether to apply content recommendation information to a content recommendation system according to an analysis result according to an exemplary embodiment.

[0050] Figure 6 The present invention is a block diagram of a device for determining content recommendation information according to an exemplary embodiment.

[0051] Figure 7 It is a block diagram of a server according to an exemplary embodiment. DETAILED DESCRIPTION

[0052] In order to enable ordinary persons in the art to better understand the technical solutions of the present disclosure, the technical solutions in the embodiments of the present disclosure will be clearly and completely described below in conjunction with the accompanying drawings.

[0053] It should be noted that the terms "first", "second", etc. in the specification and claims of the present disclosure and the above-mentioned drawings are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequence. It should be understood that the data used in this way can be interchanged where appropriate, so that the embodiments of the present disclosure described herein can be implemented in an order other than those illustrated or described herein. The embodiments described in the following exemplary embodiments do not represent all embodiments consistent with the present disclosure. Instead, they are merely examples of devices and methods consistent with some aspects of the present disclosure as detailed in the appended claims.

[0054] It should also be noted that the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for display, data for analysis, etc.) involved in this disclosure are all information and data authorized by the user or fully authorized by all parties.

[0055] Figure 1 is a flowchart of a method for determining content recommendation information according to an exemplary embodiment. Figure 1 As shown, the content recommendation information determination method is used in a server and includes the following steps:

[0056] In step S110, first sample data obtained by using the first content recommendation information and the first content recommendation model, and second sample data obtained by using the newly added second content recommendation information and the first content recommendation model are obtained.

[0057] Among them, the first content recommendation information and the first content recommendation model refer to the control group, and the newly added second content recommendation information and the first content recommendation model refer to the first experimental group. The first content recommendation information refers to the basic content recommendation information, specifically the basic content recommendation strategy. The newly added second content recommendation information refers to the content recommendation information that needs to be judged whether it can be put online for application, specifically refers to the newly added content recommendation strategy, such as content recall strategy, content rough sorting strategy, content fine sorting strategy, etc. The first content recommendation model refers to the deep neural network model.

[0058] The sample data refers to the operation data of the account on the content, such as the account clicking on a certain video or a certain picture. The first sample data is the sample data obtained by using the first content recommendation information and the first content recommendation model, and the second sample data is the sample data obtained by using the second content recommendation information and the first content recommendation model. It should be noted that the first sample data and the second sample data are isolated.

[0059] It should be noted that the content mentioned in the present disclosure may refer to videos, pictures, or other things, and the present disclosure does not limit them specifically.

[0060] Specifically, the server uses the first content recommendation information and the first content recommendation model to recommend content, and obtains the account's operation data on the content from the corresponding content operation data as the first sample data. The server uses the second content recommendation information and the first content recommendation model to recommend content, and obtains the account's operation data on the content from the corresponding content operation data as the second sample data.

[0061] In step S120, target sample data other than the first sample data in the second sample data is obtained; the target sample data is isolated from the first content recommendation model.

[0062] The target sample data refers to the sample data obtained by using the newly added second content recommendation information.

[0063] Among them, the target sample data is isolated from the first content recommendation model, which means that the target sample data is not input into the first content recommendation model to participate in the training, so as to avoid the target sample data being learned by the first content recommendation model. It also has the recommendation effect of the newly added second content recommendation information, and thus is basically consistent with the recommendation effect of the newly added second content recommendation information. Therefore, it is impossible to truly judge the actual recommendation effect of the newly added second content recommendation information, resulting in the hindered progress of the iteration of the content recommendation information.

[0064] It should be noted that, in the process of obtaining the second sample data using the newly added second content recommendation information and the first content recommendation model, the second sample data and the first content recommendation model are also isolated from each other.

[0065] In step S130, a first evaluation coefficient of a first content recommendation result obtained by using the first content recommendation information and the first content recommendation model, and a second evaluation coefficient of a second content recommendation result obtained by using the first content recommendation information and the second content recommendation model are obtained; the second content recommendation model is obtained by training the first content recommendation model using the target sample data.

[0066] Among them, the first evaluation coefficient is used to measure the recommendation effect of the first content recommendation information and the first content recommendation model, such as click-through rate, conversion rate, etc.; the second evaluation coefficient is used to measure the recommendation effect of the first content recommendation information and the second content recommendation model, such as click-through rate, conversion rate, etc.

[0067] Among them, the first content recommendation information and the second content recommendation model refer to the second experimental group. It should be noted that in the basic group and the first experimental group, the recommendation effect obtained by the first experimental group may be brought by the second content recommendation information, or it may be brought by the second content recommendation information and the first content recommendation model; therefore, the second experimental group can be used to further determine the recommendation effect of the second content recommendation information.

[0068] The second content recommendation model refers to a model obtained by training the first content recommendation model with target sample data. Both the second content recommendation model and the first content recommendation model refer to deep neural network models, but the corresponding training sample data are different.

[0069] Specifically, the server uses the target sample data to train the first content recommendation model to obtain the trained first content recommendation model as the second content recommendation model; uses the first content recommendation information and the first content recommendation model to perform content recommendation to obtain a first content recommendation result, and obtains the evaluation coefficient of the first content recommendation result according to the evaluation coefficient acquisition instruction as the first evaluation coefficient; uses the first content recommendation information and the second content recommendation model to perform content recommendation to obtain a second content recommendation result, and obtains the evaluation coefficient of the second content recommendation result according to the evaluation coefficient acquisition instruction as the second evaluation coefficient.

[0070] In step S140 , target content recommendation information is determined according to the first evaluation coefficient and the second evaluation coefficient.

[0071] Specifically, when the first evaluation coefficient is less than the second evaluation coefficient, the server determines the newly added second content recommendation information as the target content recommendation information, and applies the target content recommendation information to the content recommendation system.

[0072] In the above content recommendation information determination method, first sample data obtained by using the first content recommendation information and the first content recommendation model, and second sample data obtained by using the newly added second content recommendation information and the first content recommendation model are obtained; then, target sample data other than the first sample data in the second sample data is obtained; the target sample data is used to train the first content recommendation model to obtain the second content recommendation model; then, a first evaluation coefficient of the first content recommendation result obtained by using the first content recommendation information and the first content recommendation model, and a second evaluation coefficient of the second content recommendation result obtained by using the first content recommendation information and the second content recommendation model are obtained; finally, the target content recommendation information is determined according to the first evaluation coefficient and the second evaluation coefficient. In this way, when judging the recommendation effect of the newly added second content recommendation information, the target sample data other than the first sample data obtained by using the first content recommendation information and the first content recommendation model in the second sample data obtained by using the newly added second content recommendation information and the first content recommendation model are separated from the first content recommendation model to prevent the target sample data from being learned by the first content recommendation model and affecting the recommendation effect of the newly added second content recommendation information, resulting in the inability to truly judge the recommendation effect of the second content recommendation information. At the same time, the target sample data is used to train the newly added second content recommendation information and the first content recommendation model in the first content recommendation model to obtain the second content recommendation model, and the second evaluation coefficient of the second content recommendation result obtained by using the first content recommendation information and the second content recommendation model is compared with the first evaluation coefficient of the first content recommendation result obtained by using the first content recommendation information and the first content recommendation model. This can accurately judge the recommendation effect of the target sample data obtained by using the newly added second content recommendation information, and then accurately judge the recommendation effect of using the newly added second content recommendation information, thereby achieving the purpose of accurately judging the recommendation effect of using the newly added second content recommendation information while isolating the target sample data, thereby improving the accuracy of determining the target content recommendation information.

[0073] In an exemplary embodiment, if Figure 2 As shown, in step S130, obtaining a first evaluation coefficient of a first content recommendation result obtained by using the first content recommendation information and the first content recommendation model can be specifically achieved by the following steps:

[0074] In step S210, a posteriori data of a first content recommendation result obtained by using the first content recommendation information and the first content recommendation model is obtained.

[0075] In step S220, a first evaluation coefficient of the first content recommendation result is determined according to the a posteriori data.

[0076] The first content recommendation result refers to the content recommendation result obtained by using the first content recommendation information and the first content recommendation model. The posterior data of the first content recommendation result refers to the feedback data of the account on the first content recommendation result, such as the click rate of the account on the first content recommendation result, the browsing time of the account on the first content recommendation result, etc.

[0077] Specifically, the server uses the first content recommendation information and the first content recommendation model to perform content recommendation and obtain a first content recommendation result; obtains first content operation data corresponding to the first content recommendation result, and parses the first content operation data to obtain posterior data of the first content recommendation result; performs statistical analysis on the posterior data of the first content recommendation result to obtain a corresponding evaluation coefficient, and uses it as a first evaluation coefficient of the first content recommendation result.

[0078] Furthermore, the second evaluation coefficient may be implemented in the following manner: the server obtains a posteriori data of a second content recommendation result obtained by using the first content recommendation information and the second content recommendation model; and determines a second evaluation coefficient of the second content recommendation result based on the posteriori data of the second content recommendation result.

[0079] For example, the server uses the first content recommendation information and the second content recommendation model to perform content recommendation and obtain a second content recommendation result; obtains second content operation data corresponding to the second content recommendation result, and parses the second content operation data to obtain posterior data of the second content recommendation result; performs statistical analysis on the posterior data of the second content recommendation result to obtain a corresponding evaluation coefficient, and uses it as the second evaluation coefficient of the second content recommendation result.

[0080] The technical solution provided by the embodiment of the present disclosure obtains a posteriori data of a first content recommendation result obtained by using the first content recommendation information and the first content recommendation model, and determines a first evaluation coefficient of the first content recommendation result based on the posteriori data. Since the first evaluation coefficient is obtained based on the posteriori data of the first content recommendation result, it can truly reflect the actual effect of the first content recommendation result, thereby facilitating objectively and truly reflecting the recommendation effect of the first content recommendation information and the first content recommendation model.

[0081] In an exemplary embodiment, the above step S140, determining the target content recommendation information according to the first evaluation coefficient and the second evaluation coefficient, specifically includes: when the second evaluation coefficient is greater than the first evaluation coefficient, determining the second content recommendation information as the target content recommendation information.

[0082] Specifically, the server compares the second evaluation coefficient with the first evaluation coefficient. When the second evaluation coefficient is greater than the first evaluation coefficient, it means that the recommendation effect of using the first content recommendation information and the second content recommendation model is better than the recommendation effect of using the first content recommendation information and the first content recommendation model. The second content recommendation model is obtained by training the first content recommendation model with the target sample data obtained based on the newly added second content recommendation information. This further indicates that the recommendation effect of using the newly added second content recommendation information is better than the recommendation effect of using the first content recommendation information. The second content recommendation information is determined as the target content recommendation information.

[0083] Further, when the second evaluation coefficient is less than or equal to the first evaluation coefficient, newly added content recommendation information is obtained through the server, and steps S110 to S140 are repeatedly performed to continue adjusting the content recommendation information and the content recommendation model until the target content recommendation information is determined.

[0084] The technical solution provided by the disclosed embodiment compares the second evaluation coefficient of the second content recommendation result obtained by using the first content recommendation information and the second content recommendation model with the first evaluation coefficient of the first content recommendation result obtained by using the first content recommendation information and the first content recommendation model, and can accurately judge the recommendation effect of the target sample data obtained by using the newly added second content recommendation information, and then accurately judge the recommendation effect of using the newly added second content recommendation information, thereby achieving the purpose of accurately judging the recommendation effect of using the newly added second content recommendation information when isolating the target sample data, and then improving the accuracy of determining the target content recommendation information.

[0085] In an exemplary embodiment, the above step S140, before determining the target content recommendation information according to the first evaluation coefficient and the second evaluation coefficient, further includes: obtaining a third evaluation coefficient of a third content recommendation result obtained by using the second content recommendation information and the second content recommendation model. Then, when the second evaluation coefficient is greater than the first evaluation coefficient, the first content recommendation information is determined as the second content recommendation information, specifically including: when the second evaluation coefficient is greater than the first evaluation coefficient and less than the third evaluation coefficient, the second content recommendation information is determined as the target content recommendation information.

[0086] The third evaluation coefficient is used to measure the recommendation effect of the second content recommendation information and the second content recommendation model, such as click-through rate, conversion rate, etc.

[0087] Specifically, the server obtains the a posteriori data of the third content recommendation result obtained by using the second content recommendation information and the second content recommendation model; and determines the third evaluation coefficient of the third content recommendation result according to the a posteriori data of the third content recommendation result. For example, the server uses the second content recommendation information and the second content recommendation model to perform content recommendation to obtain the third content recommendation result; obtains the third content operation data corresponding to the third content recommendation result, and parses the third content operation data to obtain the a posteriori data of the third content recommendation result; performs statistical analysis on the a posteriori data of the third content recommendation result to obtain the corresponding evaluation coefficient, and uses it as the third evaluation coefficient of the third content recommendation result.

[0088] Further, the server compares the second evaluation coefficient with the first evaluation coefficient and the third evaluation coefficient. When the second evaluation coefficient is greater than the first evaluation coefficient and less than the third evaluation coefficient, it means that the recommendation effect of using the second content recommendation information and the second content recommendation model is better than the recommendation effect of using the first content recommendation information and the second content recommendation model. This further indicates that the recommendation effect of using the newly added second content recommendation information is better than the recommendation effect of using the first content recommendation information. The second content recommendation information is then determined as the target content recommendation information.

[0089] The technical solution provided by the embodiment of the present disclosure determines the second content recommendation information as the target content recommendation information only when the second evaluation coefficient is greater than the first evaluation coefficient and less than the third evaluation coefficient, and comprehensively considers the size relationship among the first evaluation coefficient, the second evaluation coefficient and the third evaluation coefficient, which is conducive to further improving the accuracy of determining the target content recommendation information.

[0090] In an exemplary embodiment, the above-mentioned step S120, obtaining target sample data other than the second sample data in the first sample data, includes: when the first evaluation coefficient is less than the fourth evaluation coefficient, obtaining the target sample data other than the second sample data in the first sample data; the fourth evaluation coefficient is an evaluation coefficient corresponding to the fourth content recommendation result obtained by using the second content recommendation information and the first content recommendation model.

[0091] Among them, the fourth evaluation coefficient is used to measure the recommendation effect of the second content recommendation information and the first content recommendation model, such as click-through rate, conversion rate, etc.

[0092] Specifically, the server obtains the a posteriori data of the fourth content recommendation result obtained by using the second content recommendation information and the first content recommendation model; and determines the fourth evaluation coefficient of the fourth content recommendation result according to the a posteriori data of the fourth content recommendation result. For example, the server uses the second content recommendation information and the first content recommendation model to perform content recommendation to obtain the fourth content recommendation result; obtains the fourth content operation data corresponding to the fourth content recommendation result, and parses the fourth content operation data to obtain the a posteriori data of the fourth content recommendation result; performs statistical analysis on the a posteriori data of the fourth content recommendation result to obtain the corresponding evaluation coefficient, and uses it as the fourth evaluation coefficient of the fourth content recommendation result.

[0093] Further, the server compares the first evaluation coefficient and the fourth evaluation coefficient. When the first evaluation coefficient is smaller than the fourth evaluation coefficient, it means that the recommendation effect of using the second content recommendation information and the first content recommendation model is better than the recommendation effect of using the first content recommendation information and the first content recommendation model. Furthermore, it means that the second sample data obtained by using the second content recommendation information and the first content recommendation model is better than the first sample data obtained by using the first content recommendation information and the first content recommendation model. Then, the sample data other than the second sample data in the first sample data is determined as the target sample data.

[0094] The technical solution provided by the embodiment of the present disclosure determines the sample data other than the second sample data in the first sample data as the target sample data only when the first evaluation coefficient is less than the fourth evaluation coefficient. Comprehensive consideration of the first evaluation coefficient and the fourth evaluation coefficient is beneficial to improving the accuracy of determining the target sample data.

[0095] In an exemplary embodiment, the above step S120, after obtaining the target sample data other than the second sample data in the first sample data, further includes: training the first content recommendation model according to the target sample data until the training end condition is reached, and using the trained first content recommendation model that meets the training end condition as the second content recommendation model.

[0096] The training end condition refers to reaching a preset number of training times, or the loss function reaching convergence.

[0097] Specifically, the server trains the first content recommendation model according to the target sample data to obtain a loss value; when the loss value is greater than or equal to a preset threshold, the server adjusts the model parameters of the first content recommendation model according to the loss value to obtain the first content recommendation model after the model parameters are adjusted; the first content recommendation model after the model parameters are adjusted is repeatedly trained until the loss value obtained according to the trained first content recommendation model is less than the preset threshold, and the trained first content recommendation model is used as the second content recommendation model.

[0098] The technical solution provided by the disclosed embodiment isolates the target sample data from the first content recommendation model, and uses the target sample data alone to train the first content recommendation model to obtain the second content recommendation model. This is beneficial for subsequently comprehensively considering the first evaluation coefficient of the first content recommendation result obtained by using the first content recommendation information and the first content recommendation model, and the second evaluation coefficient of the second content recommendation result obtained by using the first content recommendation information and the second content recommendation model, to determine the target content recommendation information, thereby improving the accuracy of determining the target content recommendation information.

[0099] In an exemplary embodiment, after determining the target content recommendation information according to the first evaluation coefficient and the second evaluation coefficient, the above step S140 further includes: determining the content to be recommended and the account to be recommended according to the target content recommendation information; and recommending the content to be recommended to the account to be recommended.

[0100] For example, the server determines that the contents to be recommended are content A and content B, and the accounts to be recommended are account U1, account U2, and account U3 according to the target content recommendation information, and recommends content A and content B to account U1, account U2, and account U3.

[0101] The technical solution provided by the embodiment of the present disclosure determines the content to be recommended and the account to be recommended according to the determined target content recommendation information, and recommends the content to be recommended to the account to be recommended, which is conducive to improving the accuracy and effect of content recommendation.

[0102] Figure 3 is a flowchart of a method for determining content recommendation information according to an exemplary embodiment. Figure 3 As shown, the content recommendation information determination method is used in a server and includes the following steps:

[0103] In step S310, first sample data obtained by using the first content recommendation information and the first content recommendation model, and second sample data obtained by using the newly added second content recommendation information and the first content recommendation model are obtained.

[0104] In step S320, when the first evaluation coefficient is less than the fourth evaluation coefficient, target sample data other than the second sample data in the first sample data is obtained; the fourth evaluation coefficient is an evaluation coefficient corresponding to the fourth content recommendation result obtained by using the second content recommendation information and the first content recommendation model; the target sample data is isolated from the first content recommendation model.

[0105] In step S330, the first content recommendation model is trained according to the target sample data until a training end condition is met, and the trained first content recommendation model that meets the training end condition is used as the second content recommendation model.

[0106] In step S340, a first evaluation coefficient of a first content recommendation result obtained by using the first content recommendation information and the first content recommendation model, and a second evaluation coefficient of a second content recommendation result obtained by using the first content recommendation information and the second content recommendation model are obtained.

[0107] In step S350, a third evaluation coefficient of a third content recommendation result obtained by using the second content recommendation information and the second content recommendation model is obtained.

[0108] In step S360 , when the second evaluation coefficient is greater than the first evaluation coefficient and less than the third evaluation coefficient, the second content recommendation information is determined as the target content recommendation information.

[0109] The above-mentioned method for determining content recommendation information isolates the target sample data from the first content recommendation model, which is beneficial to preventing the target sample data from being learned by the first content recommendation model and affecting the recommendation effect of the newly added second content recommendation information, resulting in an inability to truly judge the recommendation effect of the second content recommendation information; at the same time, comprehensively considering the size relationship between the first evaluation coefficient, the second evaluation coefficient and the third evaluation coefficient is beneficial to further improve the accuracy of determining the target content recommendation information.

[0110] In order to more clearly illustrate the content recommendation information determination method provided by the embodiment of the present disclosure, the content recommendation information determination method is specifically described below with a specific embodiment. In one embodiment, the present disclosure also provides a content recommendation information determination method, which isolates the sample data of the control group and the experimental group in the experiment to eliminate the impact caused by the isolation, thereby obtaining the real iterative effect of the content recommendation model and the content recommendation information, and outputs the analysis results that are closer to the real effect after isolation, thereby improving the content recommendation effect. The method specifically includes the following contents:

[0111] refer to Figure 4 , designed the test mechanism of control group, experimental group 1, experimental group 2 and experimental group 3. In the comparative test between the control group and experimental group 1, both the control group and experimental group 1 use the first content recommendation model. For the newly added sample data obtained by using the newly added second content recommendation information, it is not input into the first content recommendation model for training, so as to avoid the newly added sample data being learned by the first content recommendation model, and also having the recommendation effect of the newly added second content recommendation information; then check the final recommendation effect of different content recommendation information; similarly, experimental group 2 and experimental group 3 are similar to the control variable method, controlling the sample data of content recommendation information and content recommendation model. In this way, the recommendation effect of isolated sample data can be achieved, so as to better explore and evaluate the recommendation effect of the second content recommendation information.

[0112] refer to Figure 5 After the content recommendation system recommends content, it obtains the corresponding content operation data, attributes it and enters the test platform, where the final recommendation effect can be seen; after data analysis, the analysis results are output. If the core indicators are all positive, iterative recommendation of content recommendation information can be performed; otherwise, the content recommendation information and content recommendation model will continue to be adjusted to continue the experiment.

[0113] The above-mentioned determination of content recommendation information can achieve the following technical effects: (1) it can effectively isolate the sample data of the control group and the experimental group in the experiment and eliminate the impact caused by this; (2) it can output the real iterative effect of the content recommendation model and content recommendation information, so as to obtain analysis results that are closer to the real recommendation effect; (3) it can improve the accuracy of determining content recommendation information, thereby improving the content recommendation effect.

[0114] It should be understood that although Figure 1-Figure 3 The steps in the flowchart are shown in sequence as indicated by the arrows, but these steps are not necessarily executed in the order indicated by the arrows. Unless otherwise specified in this document, there is no strict order restriction for the execution of these steps, and these steps can be executed in other orders. Moreover, Figure 1-Figure 3 At least part of the steps may include multiple steps or multiple stages. These steps or stages are not necessarily performed at the same time, but can be performed at different times. The execution order of these steps or stages is not necessarily sequential, but can be performed in turn or alternately with other steps or at least part of the steps or stages in other steps.

[0115] It can be understood that the same / similar parts between the various embodiments of the above method in this specification can refer to each other, and each embodiment focuses on the differences from other embodiments. For related points, please refer to the description of other method embodiments.

[0116] Figure 6 is a block diagram of a device for determining content recommendation information according to an exemplary embodiment. Figure 6 , the device includes a data acquisition unit 610, a data screening unit 620, a coefficient acquisition unit 630 and an information determination unit 640.

[0117] The data acquisition unit 610 is configured to acquire first sample data obtained by using the first content recommendation information and the first content recommendation model, and second sample data obtained by using the newly added second content recommendation information and the first content recommendation model.

[0118] The data screening unit 620 is configured to obtain target sample data other than the first sample data from the second sample data; the target sample data is isolated from the first content recommendation model.

[0119] The coefficient acquisition unit 630 is configured to execute acquisition of a first evaluation coefficient of a first content recommendation result obtained by using the first content recommendation information and the first content recommendation model, and a second evaluation coefficient of a second content recommendation result obtained by using the first content recommendation information and the second content recommendation model; the second content recommendation model is obtained by training the first content recommendation model using the target sample data.

[0120] The information determining unit 640 is configured to determine target content recommendation information according to the first evaluation coefficient and the second evaluation coefficient.

[0121] In an exemplary embodiment, the coefficient acquisition unit 630 is further configured to acquire a posteriori data of the first content recommendation result obtained by using the first content recommendation information and the first content recommendation model; and determine a first evaluation coefficient of the first content recommendation result according to the posteriori data.

[0122] In an exemplary embodiment, the information determining unit 640 is further configured to determine the second content recommendation information as the target content recommendation information when the second evaluation coefficient is greater than the first evaluation coefficient.

[0123] In an exemplary embodiment, the content recommendation information determination device further includes a coefficient determination unit configured to execute acquisition of a third evaluation coefficient of a third content recommendation result obtained by using the second content recommendation information and the second content recommendation model;

[0124] The information determining unit 640 is further configured to determine the second content recommendation information as the target content recommendation information when the second evaluation coefficient is greater than the first evaluation coefficient and less than the third evaluation coefficient.

[0125] In an exemplary embodiment, the data screening unit 620 is further configured to obtain target sample data other than the second sample data in the first sample data when the first evaluation coefficient is less than a fourth evaluation coefficient; the fourth evaluation coefficient is an evaluation coefficient corresponding to a fourth content recommendation result obtained using the second content recommendation information and the first content recommendation model.

[0126] In an exemplary embodiment, the content recommendation information determination device also includes a model training unit, which is configured to execute training of the first content recommendation model according to the target sample data until the training end condition is reached, and use the trained first content recommendation model that meets the training end condition as the second content recommendation model.

[0127] In an exemplary embodiment, the content recommendation information determination device further includes a content recommendation unit configured to determine the content to be recommended and the account to be recommended according to the target content recommendation information; and recommend the content to be recommended to the account to be recommended.

[0128] Regarding the device in the above embodiment, the specific manner in which each module performs operations has been described in detail in the embodiment of the method, and will not be elaborated here.

[0129] Figure 7 700 is a block diagram of a device 700 for executing a method for determining content recommendation information according to an exemplary embodiment. For example, the device 700 may be a server. Figure 7 , the device 700 includes a processing component 720, which further includes one or more processors, and a memory resource represented by a memory 722 for storing instructions, such as applications, that can be executed by the processing component 720. The application stored in the memory 722 may include one or more modules, each corresponding to a set of instructions. In addition, the processing component 720 is configured to execute the instructions to perform the above method.

[0130] The device 700 may also include a power supply component 724 configured to perform power management of the device 700, a wired or wireless network interface 726 configured to connect the device 700 to a network, and an input / output (I / O) interface 728. The device 700 may operate based on an operating system stored in the memory 722, such as Windows Server, Mac OS X, Unix, Linux, FreeBSD, or the like.

[0131] In an exemplary embodiment, a computer-readable storage medium including instructions is also provided, such as a memory 722 including instructions, which can be executed by a processor of the device 700 to perform the above method. The storage medium can be a computer-readable storage medium, for example, the computer-readable storage medium can be a ROM, a random access memory (RAM), a CD-ROM, a magnetic tape, a floppy disk, an optical data storage device, etc.

[0132] In an exemplary embodiment, a computer program product is also provided. The computer program product includes instructions, and the instructions can be executed by the processor of the device 700 to implement the above method.

[0133] It should be noted that the above-mentioned devices, servers, computer-readable storage media, computer program products, etc. may also include other implementation methods according to the description of the method embodiments. The specific implementation methods can refer to the description of the relevant method embodiments and will not be described one by one here.

[0134] Those skilled in the art will readily appreciate other embodiments of the present disclosure after considering the specification and practicing the invention disclosed herein. The present disclosure is intended to cover any variations, uses or adaptations of the present disclosure that follow the general principles of the present disclosure and include common knowledge or customary techniques in the art that are not disclosed in the present disclosure. The description and examples are to be considered exemplary only, and the true scope and spirit of the present disclosure are indicated by the claims.

[0135] It should be understood that the present disclosure is not limited to the exact structures that have been described above and shown in the drawings, and that various modifications and changes may be made without departing from the scope thereof. The scope of the present disclosure is limited only by the appended claims.

Claims

1. A method for determining content recommendation information, characterized in that: include: Acquire first sample data obtained by using first content recommendation information and a first content recommendation model, and obtain second sample data obtained by using newly added second content recommendation information and the first content recommendation model; the first content recommendation information represents a basic content recommendation strategy, and the second content recommendation information represents a newly added content recommendation strategy; The first sample data and the second sample data represent operation data of an account on content; Acquire target sample data other than the first sample data in the second sample data; The target sample data and the first content recommendation model are isolated from each other; Obtaining a first evaluation coefficient of a first content recommendation result obtained by using the first content recommendation information and the first content recommendation model, and a second evaluation coefficient of a second content recommendation result obtained by using the first content recommendation information and the second content recommendation model; the second content recommendation model is obtained by training the first content recommendation model using the target sample data; The first evaluation coefficient is used to measure the recommendation effect of the first content recommendation information and the first content recommendation model, and the second evaluation coefficient is used to measure the recommendation effect of the first content recommendation information and the second content recommendation model; Target content recommendation information is determined according to the first evaluation coefficient and the second evaluation coefficient.

2. The method for determining content recommendation information according to claim 1, characterized in that: The obtaining a first evaluation coefficient of a first content recommendation result obtained by using the first content recommendation information and the first content recommendation model includes: Acquire posterior data of a first content recommendation result obtained by using the first content recommendation information and the first content recommendation model; A first evaluation coefficient of the first content recommendation result is determined according to the a posteriori data.

3. The method for determining content recommendation information according to claim 1, characterized in that: The determining target content recommendation information according to the first evaluation coefficient and the second evaluation coefficient includes: When the second evaluation coefficient is greater than the first evaluation coefficient, the second content recommendation information is determined as target content recommendation information.

4. The method for determining content recommendation information according to claim 3, characterized in that: Before determining the target content recommendation information according to the first evaluation coefficient and the second evaluation coefficient, the method further includes: Obtaining a third evaluation coefficient of a third content recommendation result obtained by using the second content recommendation information and the second content recommendation model; When the second evaluation coefficient is greater than the first evaluation coefficient, determining the first content recommendation information as the second content recommendation information includes: When the second evaluation coefficient is greater than the first evaluation coefficient and less than the third evaluation coefficient, the second content recommendation information is determined as target content recommendation information.

5. The method for determining content recommendation information according to claim 1, characterized in that: The acquiring target sample data other than the second sample data from the first sample data includes: When the first evaluation coefficient is less than a fourth evaluation coefficient, target sample data other than the second sample data in the first sample data is obtained; the fourth evaluation coefficient is an evaluation coefficient corresponding to a fourth content recommendation result obtained by using the second content recommendation information and the first content recommendation model.

6. The method for determining content recommendation information according to claim 1, characterized in that: After acquiring target sample data other than the second sample data in the first sample data, the method further includes: The first content recommendation model is trained according to the target sample data until a training end condition is met, and the trained first content recommendation model that meets the training end condition is used as the second content recommendation model.

7. The method for determining content recommendation information according to claim 1, characterized in that: After determining the target content recommendation information according to the first evaluation coefficient and the second evaluation coefficient, the method further includes: Determining the content to be recommended and the account to be recommended according to the target content recommendation information; The content to be recommended is recommended to the account to be recommended.

8. A device for determining content recommendation information, characterized in that: include: a data acquisition unit configured to acquire first sample data obtained by using first content recommendation information and a first content recommendation model, and second sample data obtained by using newly added second content recommendation information and the first content recommendation model; the first content recommendation information represents a basic content recommendation strategy, and the second content recommendation information represents a newly added content recommendation strategy; The first sample data and the second sample data represent operation data of an account on content; A data screening unit is configured to obtain target sample data other than the first sample data from the second sample data; The target sample data and the first content recommendation model are isolated from each other; a coefficient acquisition unit configured to acquire a first evaluation coefficient of a first content recommendation result obtained by using the first content recommendation information and the first content recommendation model, and a second evaluation coefficient of a second content recommendation result obtained by using the first content recommendation information and the second content recommendation model; the second content recommendation model is obtained by training the first content recommendation model using the target sample data; The first evaluation coefficient is used to measure the recommendation effect of the first content recommendation information and the first content recommendation model, and the second evaluation coefficient is used to measure the recommendation effect of the first content recommendation information and the second content recommendation model; The information determination unit is configured to determine target content recommendation information according to the first evaluation coefficient and the second evaluation coefficient.

9. The content recommendation information determination device according to claim 8, characterized in that: The coefficient acquisition unit is further configured to acquire a posteriori data of a first content recommendation result obtained by using the first content recommendation information and the first content recommendation model; and determine a first evaluation coefficient of the first content recommendation result according to the posteriori data.

10. The content recommendation information determination device according to claim 8, characterized in that: The information determination unit is further configured to determine the second content recommendation information as target content recommendation information when the second evaluation coefficient is greater than the first evaluation coefficient.

11. The content recommendation information determination device according to claim 10, characterized in that: The apparatus further includes a coefficient determination unit configured to execute acquisition of a third evaluation coefficient of a third content recommendation result obtained by using the second content recommendation information and the second content recommendation model; The information determination unit is further configured to determine the second content recommendation information as target content recommendation information when the second evaluation coefficient is greater than the first evaluation coefficient and less than the third evaluation coefficient.

12. The content recommendation information determination device according to claim 8, characterized in that: The data screening unit is further configured to obtain target sample data other than the second sample data in the first sample data when the first evaluation coefficient is less than a fourth evaluation coefficient; the fourth evaluation coefficient is an evaluation coefficient corresponding to a fourth content recommendation result obtained by using the second content recommendation information and the first content recommendation model.

13. The content recommendation information determination device according to claim 8, characterized in that: The device also includes a model training unit, which is configured to perform training on the first content recommendation model according to the target sample data until a training end condition is reached, and use the trained first content recommendation model that meets the training end condition as the second content recommendation model.

14. The content recommendation information determination device according to claim 8, characterized in that: The device further comprises a content recommendation unit configured to determine the content to be recommended and the account to be recommended according to the target content recommendation information; and recommend the content to be recommended to the account to be recommended.

15. A server, characterized in that: include: processor; a memory for storing instructions executable by the processor; The processor is configured to execute the instructions to implement the content recommendation information determination method as described in any one of claims 1 to 7.

16. A computer-readable storage medium, characterized in that: When the instructions in the computer-readable storage medium are executed by a processor of a server, the server is enabled to execute the content recommendation information determination method according to any one of claims 1 to 7.

17. A computer program product, comprising instructions, characterized in that: When the instruction is executed by a processor of the server, the server is enabled to execute the content recommendation information determination method as described in any one of claims 1 to 7.

Images