Advertisement index prediction method, device, electronic device and storage medium

Abstract

The embodiment of the present invention provides an advertising index prediction method, device, electronic device and storage medium, the method comprising: obtaining state feature data of an advertisement to be placed; inputting the state feature data into a pre-trained advertising index prediction model, so that the advertising index prediction model determines the predicted browsing index after the advertisement to be placed within a first preset period based on the state feature data; and determines whether the advertisement to be placed can be placed based on the predicted browsing index. The method is used to improve the cold start compliance rate of new advertisements under the premise of the lowest possible cold start cost of new advertisements.

Description

Technical Field

[0001] The present invention relates to the field of data analysis technology, and in particular to an advertising index prediction method, device, electronic equipment and storage medium. Background Art

[0002] As the Internet advertising business matures, more and more advertisers choose to display ads to users through the Internet. However, due to the lack of historical delivery information for new ads, it is difficult for delivery platforms to accurately predict the click-through rate and conversion rate of new ads, which has led to the cold start of new ads being a problem that has plagued the advertising industry. The cold start of new ads refers to the delivery of new ads without promotion.

[0003] Before launching a new ad, the delivery platform usually predicts the click-through rate and conversion rate of the new ad. Currently, there are three main ways to cold-start a new ad: The first way is to design strategy rules based on the scenario of the advertising business to increase the click-through rate of the new ad, but this is not suitable for more complex advertising business scenarios and has low versatility; the second way is to learn the initial ad features for the ad and use the initial ad features to improve the ad click-through rate and conversion rate, but learning the initial ad features to make the conversion rate of the new ad cold start meet the requirements requires the use of a large number of mature ads, and the learning cost is high; the third way is to select new ads for tentative exposure, but this method has poor recognition ability for users and ads, and it is difficult to guarantee the cost of the trial.

[0004] In other words, the above cold start method for new ads is difficult to guarantee the prediction accuracy of both the cold start cost of new ads and the conversion rate of the cold start of new ads, that is, it is difficult to guarantee the cold start target rate of new ads. Summary of the invention

[0005] The purpose of the embodiments of the present invention is to provide an advertising index prediction method, device, electronic device and storage medium to improve the new advertisement cold start compliance rate under the premise of the lowest possible new advertisement cold start cost.

[0006] In a first aspect of the present invention, a method for predicting advertising indicators is provided, comprising:

[0007] Obtaining status characteristic data of the advertisement to be delivered;

[0008] Inputting the state feature data into a pre-trained advertising index prediction model, so that the advertising index prediction model determines the predicted browsing index after the advertisement to be placed within a first preset period based on the state feature data; wherein the advertising index prediction model is obtained by training the reinforcement learning network model to be trained based on the sample state feature data before exposure and the sample state feature data after exposure of the sample advertisement and the reward data corresponding to the sample advertisement;

[0009] Based on the predicted browsing index, it is determined whether the advertisement to be delivered can be delivered.

[0010] Optionally, after determining whether the advertisement to be placed can be placed based on the predicted browsing index, the method further includes:

[0011] If the advertisement to be placed can be placed, the predicted return rate corresponding to the advertisement to be placed is calculated based on the predicted browsing index and preset advertisement reference index data.

[0012] Optionally, before obtaining the state characteristic data of the advertisement to be placed, the method further includes:

[0013] Determine the activity level of the users corresponding to the advertisements to be delivered;

[0014] Determining whether the activity level is greater than a preset activity level threshold;

[0015] If not, giving up delivering the advertisement to be delivered to the delivery user;

[0016] If yes, obtaining attribute information of the delivery user and historical browsing information of the advertisements delivered to the delivery user, and using the attribute information and the historical browsing information as user features corresponding to the delivery user;

[0017] The step of obtaining the status characteristic data of the advertisement to be placed includes:

[0018] The content information and the advertisement position of the advertisement to be placed are obtained, and the content information, the advertisement and the user characteristics corresponding to the placement user are determined as the state characteristic data of the advertisement to be placed.

[0019] Optionally, the predicted browsing index includes: a predicted click rate and a predicted conversion rate corresponding to the advertisement to be placed;

[0020] The calculating the predicted rate of return corresponding to the to-be-delivered advertisement based on the predicted browsing index and the preset advertisement reference index data includes:

[0021] Based on the predicted click rate, the predicted conversion rate, the preset cost, the preset minimum return on investment and the preset minimum conversion indicator parameter, the predicted return rate corresponding to the advertisement to be placed is calculated by using preset calculation rules.

[0022] Optionally, the step of calculating the predicted rate of return corresponding to the advertisement to be placed by a preset calculation rule based on the predicted click rate, the predicted conversion rate, the preset cost, the preset minimum return on investment and the preset minimum conversion indicator parameter includes:

[0023] The following formula is used to calculate the predicted rate of return corresponding to the advertisement to be placed based on the predicted click rate, the predicted conversion rate, the preset cost, the preset minimum return on investment and the preset minimum conversion indicator parameter through the preset calculation rules:

[0024]

[0025] Wherein, ECPM is the predicted rate of return corresponding to the advertisement to be placed, pCTCVR=pCTR*pCVR, pCTR is the predicted click-through rate, pCVR is the predicted conversion rate, conversion_price is the preset cost, β max To preset hyperparameters, ROI thre To set the minimum return on investment, pCTCVR thre The preset minimum conversion indicator parameter.

[0026] Optionally, after calculating the predicted rate of return corresponding to the advertisement to be placed based on the predicted browsing index and the preset advertisement reference index data, the method further includes:

[0027] Determining whether to place the advertisement to be placed based on the predicted rate of return;

[0028] If the advertisement to be placed is placed, obtaining the corresponding real return rate after the advertisement to be placed is placed;

[0029] Log the real rate of return.

[0030] In a second aspect of the present invention, a training method for an advertising index prediction model is provided, wherein the reinforcement learning network model to be trained includes an estimation network and a target network, wherein the estimation network and the target network are networks with the same structure, and the training method of the advertising index prediction model includes:

[0031] Obtain pre-exposure sample state feature data, post-exposure sample state feature data, and reward data corresponding to the sample advertisement;

[0032] Inputting the post-exposure sample state feature data into the estimation network to obtain a first sample predicted browsing index after the sample advertisement is placed within a second preset period;

[0033] Inputting the post-exposure sample state feature data into the target network to obtain a second sample predicted browsing index after the sample advertisement is placed within a second preset period;

[0034] Determine a label corresponding to the sample advertisement based on the reward data and the first sample predicted browsing index, the second sample predicted browsing index, a first preset browsing index threshold, and a second preset browsing index threshold, wherein the label is used to identify the delivery value of the sample advertisement;

[0035] Inputting the sample state feature data before exposure into the estimation network to obtain a third sample predicted browsing index after the sample advertisement is released within a second preset period;

[0036] Based on the label, the third sample prediction browsing index and the third preset browsing index threshold, calculating the loss value corresponding to the reinforcement learning network model to be trained;

[0037] Determining whether the loss value is less than a preset loss threshold;

[0038] If yes, determining the trained estimation network as the advertising index prediction model;

[0039] If not, adjust the parameters of the reinforcement learning network model to be trained, and return to the step of obtaining the pre-exposure sample state feature data, post-exposure sample state feature data and reward data corresponding to the sample advertisement.

[0040] Optionally, determining a label corresponding to the sample advertisement based on the reward data and the first sample predicted browsing index, the second sample predicted browsing index, a first preset browsing index threshold, and a second preset browsing index threshold includes:

[0041] The following formula is used to determine the label corresponding to the sample advertisement based on the reward data and the first sample predicted browsing index, the second sample predicted browsing index, the first preset browsing index threshold, and the second preset browsing index threshold:

[0042] Q l [action]=reward+γ*q t_s

[0043] Among them, Q l [action] is the label corresponding to the sample advertisement. The action value is 0 or 1. l [1] represents the browsing index corresponding to the sample advertisement, Q l [0] indicates that the browsing index corresponding to the sample advertisement is not placed, reward is the reward data, γ is the preset reward discount factor, γ∈(0,1), q t_s =Q t_n [eval index ], eval index = argmax(Q e0_n ,Qe1_n ), Q t_n =[Q t0_n ,Q t1_n ], Q e1_n is the predicted browsing index of the first sample, Q e0_n is the first preset browsing index threshold, Q t1_n Predict browsing index for the second sample, Q t0_n is the second preset browsing index threshold.

[0044] Optionally, the calculating the loss value corresponding to the reinforcement learning network model to be trained based on the label, the third sample prediction browsing index and a third preset browsing index threshold includes:

[0045] The following formula is used to calculate the loss value corresponding to the reinforcement learning network model to be trained based on the label, the third sample prediction browsing index and the third preset browsing index threshold:

[0046]

[0047] Among them, Loss is the loss value, n is the sum of the number of ads delivered and the number of ads not delivered, Q e =[Q e0, Q e1 ], Q e1 is the predicted browsing index of the third sample, Q e0 is the third preset browsing index threshold, Q l for the label.

[0048] In a third aspect of the present invention, there is also provided an advertising index prediction device, comprising:

[0049] A data acquisition module, used to acquire status characteristic data of the advertisement to be delivered;

[0050] An indicator prediction module is used to input the state feature data into a pre-trained advertising indicator prediction model, so that the advertising indicator prediction model determines the predicted browsing indicator after the advertisement to be placed within a first preset period based on the state feature data; wherein the advertising indicator prediction model is obtained by training the reinforcement learning network model to be trained based on the sample state feature data before exposure and the sample state feature data after exposure of the sample advertisement and the reward data corresponding to the sample advertisement;

[0051] The delivery determination module is used to determine whether the advertisement to be delivered can be delivered based on the predicted browsing index.

[0052] In another aspect of the present invention, a computer-readable storage medium is provided, wherein a computer program is stored in the computer-readable storage medium, and when the computer program is executed by a processor, the computer program implements any of the above-mentioned advertising indicator prediction methods or advertising indicator prediction model training methods.

[0053] In another aspect of the present invention, a computer program product comprising instructions is provided, which, when executed on a computer, enables the computer to execute any of the above-mentioned advertising index prediction methods or advertising index prediction model training methods.

[0054] Beneficial effects of the embodiments of the present invention:

[0055] The method provided by the embodiment of the present invention is used to obtain the state feature data of the advertisement to be placed; the state feature data is input into the pre-trained advertisement index prediction model, so that the advertisement index prediction model determines the predicted browsing index after the advertisement to be placed is placed within the first preset period based on the state feature data; and determines whether to place the advertisement to be placed based on the predicted browsing index. That is, the advertisement index prediction model obtained by training the reinforcement learning network model to be trained can be used to determine whether to place the advertisement to be placed, thereby performing a new advertisement cold start. While testing the new advertisement, the characteristics of the advertisement and the characteristics of the user are fully utilized, and the indicator prediction accuracy of the new advertisement cold start is improved under the premise of the lowest possible new advertisement cold start cost, thereby improving the new advertisement cold start compliance rate.

[0056] Of course, it is not necessary to achieve all of the advantages described above at the same time to implement any product or method of the present invention. BRIEF DESCRIPTION OF THE DRAWINGS

[0057] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art are briefly introduced below.

[0058] Figure 1 A flow chart of an advertising index prediction method provided by an embodiment of the present invention;

[0059] Figure 2 A flow chart of a method for training an advertising index prediction model provided by an embodiment of the present invention;

[0060] Figure 3 A structural diagram of a reinforcement learning network model to be trained provided in an embodiment of the present invention;

[0061] Figure 4 A schematic diagram of the structure of an advertising index prediction device provided by an embodiment of the present invention;

[0062] Figure 5Another structural schematic diagram of the advertising index prediction device provided by an embodiment of the present invention;

[0063] Figure 6 A schematic diagram of the structure of an electronic device provided by an embodiment of the present invention. DETAILED DESCRIPTION

[0064] The technical solutions in the embodiments of the present invention will be described below in conjunction with the accompanying drawings in the embodiments of the present invention.

[0065] Since it is difficult for existing cold-start methods for new advertisements to simultaneously guarantee both the cold-start cost of new advertisements and the cold-start compliance rate of new advertisements, in order to improve the cold-start compliance rate of new advertisements with the lowest possible cold-start cost of new advertisements, embodiments of the present invention provide an advertisement index prediction method, apparatus, electronic device, computer-readable storage medium, and computer program product.

[0066] The advertising index prediction method provided in the embodiment of the present invention can be applied to any electronic device that can analyze and process advertising data, for example, a computer, a camera, a processing device, etc., and is not specifically limited here.

[0067] Figure 1 A flow chart of the advertising index prediction method provided by the embodiment of the present invention, such as Figure 1 As shown, the method includes:

[0068] Step 101, obtaining status characteristic data of the advertisement to be delivered.

[0069] Step 102: input the state feature data into a pre-trained advertising index prediction model, so that the advertising index prediction model determines the predicted browsing index after the advertisement to be placed is placed within a first preset period based on the state feature data.

[0070] Among them, the advertising indicator prediction model is obtained by training the reinforcement learning network model to be trained based on the sample state feature data before exposure and the sample state feature data after exposure of the sample advertisement and the reward data corresponding to the sample advertisement.

[0071] The predicted browsing index indicates the predicted revenue value corresponding to the exposure of the ad to be placed. The revenue value Q corresponding to the exposure of the ad to be placed can be expressed as Q e1 Indicates that if the ad to be placed does not obtain the revenue value Q corresponding to this exposure, it can be used e0 express.

[0072] The first preset period can be set to one week or one month, etc. according to actual application.

[0073] Step 103: Determine whether the advertisement to be delivered can be delivered based on the predicted browsing index.

[0074] Specifically, we can take action = argmax(Q e0 ,Q e1 ) as the action for selecting the advertisement to be delivered. e0 Greater than Q e1 , confirm that the pending ad will not be delivered. If Q e0 No greater than Q e1 , confirm the delivery of the ads to be delivered.

[0075] The method provided by the embodiment of the present invention is used to obtain the state feature data of the advertisement to be placed; the state feature data is input into the pre-trained advertisement index prediction model, so that the advertisement index prediction model determines the predicted browsing index after the advertisement to be placed is placed within the first preset period based on the state feature data; and determines whether to place the advertisement to be placed based on the predicted browsing index. That is, the advertisement index prediction model obtained by training the reinforcement learning network model to be trained can be used to determine whether to place the advertisement to be placed, thereby performing a new advertisement cold start. While testing the new advertisement, the characteristics of the advertisement and the characteristics of the user are fully utilized, and the indicator prediction accuracy of the new advertisement cold start is improved under the premise of the lowest possible new advertisement cold start cost, thereby improving the new advertisement cold start compliance rate.

[0076] In a possible implementation manner, after executing the step of determining whether the advertisement to be placed can be placed based on the predicted browsing index, step A1 may also be included:

[0077] Step A1: If the advertisement to be placed can be placed, based on the predicted browsing index and the preset advertisement reference index data, the predicted return rate corresponding to the advertisement to be placed is calculated as the index prediction result.

[0078] In the implementation of the present invention, the advertisements to be placed can be placed in APPs, web pages, office software, etc. of terminal devices such as mobile phones, iPads, and computers. Specifically, taking the APP in a mobile phone as an example, the advertisements to be placed can be placed in the opening screen interface of the APP or various business interfaces of the APP.

[0079] In the embodiment of the present invention, when obtaining the state characteristic data of the advertisement to be placed, it is possible to monitor whether an advertisement placement request is received, the advertisement placement request including information such as the placement user corresponding to the advertisement to be placed. If an advertisement placement request is received, the state characteristic data of the advertisement to be placed can be obtained from the log information of the electronic device.

[0080] The delivery users refer to users who may be displayed with the delivered advertisements, and specifically, they may be users of a certain APP and office software, etc. In the embodiment of the present invention, delivery users may be divided into multiple types of users, such as new users, low-activity users, high-activity and low-conversion users, and high-activity and high-conversion users, according to the activity of delivery users and the conversion degree of delivered advertisements.

[0081] Taking application software A as an example, if the advertisement to be delivered is delivered to the user of application software A, the users of application software A can be pre-divided into new users, low-activity users, high-activity and low-conversion users, and high-activity and high-conversion users. New users are users who have newly registered application software A, low-activity users are users whose activity is lower than the first preset activity threshold, high-activity and low-conversion users are users whose activity is higher than the second preset activity threshold and whose conversion rate to the delivered advertisement is lower than the first preset conversion threshold, and high-activity and high-conversion users are users whose activity is higher than the second preset activity threshold and whose conversion rate to the delivered advertisement is higher than the second preset conversion threshold. Users whose activity is between the first preset activity threshold and the second preset activity threshold are other users who are more active than low-activity users and less active than high-activity and low-conversion users and high-activity and high-conversion users. Among them, under the premise that the first preset activity threshold is less than the second preset activity threshold and the first preset conversion threshold is less than the second preset conversion threshold, the first preset activity threshold, the second preset activity threshold, the first preset conversion threshold, and the second preset conversion threshold can be freely set according to application requirements.

[0082] The conversion rate of the delivered ads reflects the click-through rate and retention rate of the delivered ads by the users. The higher the click-through rate and retention rate of the delivered ads by the users, the higher the corresponding conversion rate of the delivered ads.

[0083] Alternatively, the embodiment of the present invention may also classify the delivery users corresponding to the advertisement delivery request after receiving the advertisement delivery request.

[0084] In the embodiment of the present invention, before obtaining the state characteristic data of the advertisement to be placed, the following steps B1-B4 may also be included:

[0085] Step B1, determining the activity level of the user corresponding to the advertisement to be delivered.

[0086] Specifically, the active time of the delivery user in the application software or the number of published contents may be obtained as the activity level of the delivery user.

[0087] Step B2, determining whether the activity level is greater than a preset activity level threshold.

[0088] Specifically, the preset activity level may be a second preset activity level threshold. In this step, it may be determined whether the activity level of the delivery user is greater than the second preset activity level threshold. If so, it may be determined that the delivery user is a high-activity type user, wherein the high-activity type user may include a high-activity low-conversion user and a high-activity high-conversion user.

[0089] Step B3: If not, give up delivering the advertisement to the delivery user.

[0090] If the user corresponding to the advertisement to be delivered is neither a high-activity and low-conversion user nor a high-activity and high-conversion user, that is, the user corresponding to the advertisement to be delivered is a new user or a low-activity user, the calculation of the predicted return rate corresponding to the advertisement to be delivered can be abandoned.

[0091] Step B4: if yes, obtain the attribute information of the delivery user and the historical browsing information of the advertisement delivered to the delivery user, and use the attribute information and the historical browsing information as the user features corresponding to the delivery user.

[0092] The attribute information of the delivery user may include age, gender, etc. The historical browsing information of the advertisement delivered to the delivery user may specifically include the browsing status of each delivered advertisement by the delivery user, and specifically may include the advertisement browsing time, etc.

[0093] In a possible implementation, the obtaining of status characteristic data of the advertisement to be placed may include: obtaining content information and advertisement slot of the advertisement to be placed, and determining the content information, the advertisement, and user characteristics corresponding to the placement user as the status characteristic data of the advertisement to be placed.

[0094] Among them, the content information of the advertisement to be placed includes the name of the advertisement and the text information contained therein, etc. The ad slot of the advertisement to be placed indicates the location information where the advertisement to be placed is placed. For example, if the advertisement to be placed is placed on the opening screen interface of application software A, then the ad slot is the opening screen interface of application software A; if the advertisement to be placed is placed on the service X interface of application software A, then the ad slot is the service X interface of application software A.

[0095] In the embodiment of the present invention, the predicted browsing index may include: a predicted click rate and a predicted conversion rate corresponding to the advertisement to be placed.

[0096] The step of calculating the predicted rate of return corresponding to the advertisement to be placed based on the predicted browsing index and the preset advertisement reference index data comprises step C1:

[0097] Step C1, based on the predicted click rate, the predicted conversion rate, the preset cost, the preset minimum return on investment and the preset minimum conversion index parameter, the predicted return rate corresponding to the advertisement to be placed is calculated by a preset calculation rule.

[0098] Specifically, in the embodiment of the present invention, the following formula may be used to calculate the predicted rate of return corresponding to the advertisement to be placed through a preset calculation rule based on the predicted click rate, the predicted conversion rate, the preset cost, the preset minimum return on investment ratio and the preset minimum conversion index parameter:

[0099]

[0100] Wherein, ECPM is the predicted rate of return corresponding to the advertisement to be placed, pCTCVR=pCTR*pCVR, pCTR is the predicted click-through rate, pCVR is the predicted conversion rate, conversion_price is the preset cost, β max To preset hyperparameters, ROI thre To set the minimum return on investment, pCTCVR thre The preset minimum conversion indicator parameter.

[0101] The preset cost conversion_price can be set to the payment cost that the advertiser is willing to pay for each conversion. max Can be used to prevent ECPM from being too high.

[0102] In the embodiment of the present invention, the return on investment ratio of the advertisement to be placed can be defined as ROI = revenue / cost, where cost represents the payment cost of the advertisement to be placed, and revenue represents the expected income that can be brought by the advertisement to be placed. thre It can be set as the minimum ROI that advertisers can bear for the cold start of new ads. thre Set the size and adjust the trial cost of new ads. You can set different ROIs for high-activity and high-conversion users and high-activity and low-conversion users. thre .

[0103] In the embodiment of the present invention, after calculating the predicted rate of return corresponding to the advertisement to be placed based on the predicted browsing index and the preset advertisement reference index data, the following steps D1-D3 are also included:

[0104] Step D1, determining whether to place the advertisement based on the predicted rate of return.

[0105] Specifically, taking advertising delivery platform A as an example, advertising delivery platform A can use the predicted rate of return as a bid in a bidding scenario where advertisers select advertising delivery targets. If the bid is higher than that of other platforms, that is, the bidding is successful, the new advertisement can be delivered to advertising delivery platform A.

[0106] Step D2: if the advertisement to be placed is placed, obtain the corresponding real rate of return after the advertisement to be placed is placed.

[0107] Step D3, storing the real rate of return in a log.

[0108] Specifically, in an embodiment of the present invention, only the real rate of return can be stored in the log of the advertisement to be placed; the state feature data corresponding to the advertisement to be placed can also be stored in the log, and the state feature data can specifically include state (state data before placing the advertisement), action (placement action), reward (reward data), state_next (state data after placing the advertisement) and other information, wherein reward is the real rate of return corresponding to the advertisement to be placed. For new advertisements that the advertising index prediction model predicts as not needing to be exposed this time, the state, action, reward, state_next and other information of one of the advertisements can be randomly taken and stored in the log for updating the next training advertising index prediction model.

[0109] By adopting the method provided in the embodiment of the present invention, an advertising indicator prediction model is used to determine whether to place an advertisement to be placed, thereby performing a new advertisement cold start. While testing the new advertisement, the characteristics of the advertisement and the characteristics of the user are fully utilized, thereby achieving an improvement in the accuracy of indicator prediction for the cold start of the new advertisement while keeping the cold start cost of the new advertisement as low as possible, thereby improving the compliance rate of the cold start of the new advertisement.

[0110] Figure 2 A flow chart of a method for training an advertising index prediction model provided by an embodiment of the present invention, such as Figure 2 As shown, the reinforcement learning network model to be trained includes an estimation network and a target network, the estimation network and the target network are networks with the same structure, and the method may include:

[0111] Step 201, obtaining pre-exposure sample state feature data, post-exposure sample state feature data and reward data corresponding to the sample advertisement.

[0112] In an embodiment of the present invention, the pre-exposure sample state feature data, post-exposure sample state feature data and reward data of multiple sample advertisements within a preset time period can be obtained from the advertisement delivery log data. The preset time period can be set to the past 30 days, 60 days or 90 days, etc. The preset time period should not be too short. If it is too short, it may result in an insufficient number of sample advertisements, affecting the training effect of the advertisement index prediction model; the preset time period should not be too long. If it is too long, the interest of the delivery user may have changed, which is not conducive to judging the degree of interest of the delivery user in the advertisement, and also affects the training effect of the advertisement index prediction model.

[0113] The pre-exposure sample state characteristic data corresponding to the sample advertisement includes: user characteristics of the delivery user before the sample advertisement is delivered to the corresponding delivery user, content information of the sample advertisement, and advertisement position.

[0114] The post-exposure sample status characteristic data corresponding to the sample advertisement includes: user characteristics of the delivery user after the sample advertisement is delivered to the corresponding delivery user, content information of the sample advertisement, and advertisement position.

[0115] In an embodiment of the present invention, the sample state feature data state before exposure, the sample state feature data state_next after exposure, whether the sample advertisement requires the exposure operation action and the reward data reward corresponding to the sample advertisement can be used to form sample training data [state, action, reward, state_next].

[0116] Step 202: input the post-exposure sample state feature data into the estimation network to obtain a first sample predicted browsing index after the sample advertisement is released within a second preset period.

[0117] The second preset period can be set to one week or one month, etc. according to actual application.

[0118] Figure 3 The structure diagram of the reinforcement learning network model to be trained provided by the embodiment of the present invention is as follows: Figure 3 As shown, the reinforcement learning network model to be trained includes an estimation network and a target network, and the estimation network and the target network are networks with the same structure. The number of layers M of the estimation network is set to 3≤M≤10. If M is set too small, the reinforcement learning network model to be trained will be too simple and difficult to learn useful information from the training samples. If M is too large, the reinforcement learning network model to be trained will be too complex and difficult to converge. After the estimation network and the target network are constructed, the parameters of the estimation network and the target network can be randomly initialized respectively.

[0119] Step 203: input the post-exposure sample state feature data into the target network to obtain a second sample predicted browsing index after the sample advertisement is released within a second preset period.

[0120] Step 204: Determine a label corresponding to the sample advertisement based on the reward data and the first sample predicted browsing index, the second sample predicted browsing index, the first preset browsing index threshold, and the second preset browsing index threshold.

[0121] The tag is used to identify the delivery value of the sample advertisement.

[0122] Specifically, in the embodiment of the present invention, the following formula may be used to determine the label corresponding to the sample advertisement based on the reward data and the first sample predicted browsing index, the second sample predicted browsing index, the first preset browsing index threshold, and the second preset browsing index threshold:

[0123] Q l [action]=reward+γ*q t_s

[0124] Among them, Q l [action] is the label corresponding to the sample advertisement. The action value is 0 or 1. l [1] represents the browsing index corresponding to the sample advertisement, Q l [0] indicates that the browsing index corresponding to the sample advertisement is not placed, reward is the reward data, γ is the preset reward discount factor, γ∈(0,1), q t_s =Q t_n [eval index ], eval index = argmax(Q e0_n ,Q e1_n ), Q t_n =[Q t0_n, Q t1_n ], Q e1_n is the predicted browsing index of the first sample, Q e0_n Predict browsing index for the second sample, Q t1_n is the predicted browsing index of the third sample, Q t0_n A browsing index is predicted for the fourth sample.

[0125] Among them, the first preset browsing index threshold can be the Q value of the sample predicted browsing index after the sample advertisement is not placed within the second preset period, obtained by inputting the post-exposure sample state feature data into the estimation network; the first preset browsing index threshold can also be a pre-set extremely small threshold for measuring the first sample predicted browsing index, for example, the first preset browsing index threshold can be set to 0.001 or 0.002, etc.

[0126] The second preset browsing index threshold value may be the Q value of the sample predicted browsing index after the sample advertisement is not placed within the second preset period, obtained by inputting the post-exposure sample status characteristic data into the target network; the second preset browsing index threshold value may also be a pre-set extremely small threshold value for measuring the second sample predicted browsing index, for example, the second preset browsing index threshold value may be set to 0.001 or 0.002, etc.

[0127] For example, Figure 3 As shown, in the embodiment of the present invention, the state feature data of the sample before exposure state can be input into the estimation network to obtain Q e =[Qe0,Q e1 ], where Q e1 It indicates the Q value corresponding to the predicted sample ad exposure (action = 1), Q e0 It indicates that the Q value of the predicted sample ad will not be exposed this time (action = 0). l =Q e Then, the state feature data state_next of the sample after exposure is input into the estimation network to obtain the first sample prediction browsing index Q corresponding to the case of delivering the sample advertisement. e_n =[Q e0_n, Q e1_n ], where Q e1_m is the first sample predicted browsing index, which can represent the Q value corresponding to the predicted sample advertisement gaining this exposure (action=1), Q e0_n The first preset browsing index threshold may indicate the Q value of the predicted sample advertisement not obtaining this exposure (action=0). Then, eval imdex = argmax(Q e0_n ,Q e1_n ); eval index You can take Q e0_n and Q e1_n The subscript corresponding to the largest value in Q e0_n Greater than Q e1_n , then eval index The value is 0, if Q e0_n Less than Q e1_n , then evalindex The value is 1.

[0128] Then, the state feature data state_next of the sample after exposure is input into the target network to obtain the corresponding second sample prediction browsing index Q under the condition of delivering the sample advertisement. t_n =[Q t0_n, Q t1_n ], where Q t1_n is the second sample predicted browsing index, which can represent the Q value corresponding to the predicted sample advertisement gaining this exposure (action=1), Q t0_n is the second preset browsing index threshold, which may indicate a Q value that the predicted sample advertisement does not obtain this exposure (action=0).

[0129] Then, we can let q t_s =Q t_n [eval index ] to get the label:

[0130] Q l [action]=reward+γ*q t_s

[0131] Among them, action and reward can be directly obtained from the data of sample advertisements. The larger the preset reward discount factor γ is, the more the model focuses on long-term rewards. The smaller the preset reward discount factor γ is, the more the model focuses on short-term rewards.

[0132] Step 205: input the pre-exposure sample state feature data into the estimation network to obtain a third sample predicted browsing index corresponding to the case where the sample advertisement is released within a second preset period.

[0133] Step 206: Calculate the loss value corresponding to the reinforcement learning network model to be trained based on the label, the third sample predicted browsing index and the third preset browsing index threshold.

[0134] The calculating, based on the label, the third sample prediction browsing index and the third preset browsing index threshold, a loss value corresponding to the reinforcement learning network model to be trained includes:

[0135] The following formula is used to calculate the loss value corresponding to the reinforcement learning network model to be trained based on the label, the third sample prediction browsing index and the third preset browsing index threshold:

[0136]

[0137] Among them, Loss is the loss value, n is the sum of the number of ads delivered and the number of ads not delivered, Q e =[Q e0, Qe1 ], Q e1 is the predicted browsing index of the third sample, Q e0 is the third preset browsing index threshold, Q l for the label.

[0138] Among them, the third preset browsing index threshold can be the Q value of the sample predicted browsing index after the sample advertisement is not placed within the second preset period, obtained by inputting the pre-exposure sample state feature data into the estimation network; the third preset browsing index threshold can also be a pre-set extremely small threshold for measuring the third sample predicted browsing index, for example, the third preset browsing index threshold can be set to 0.001 or 0.002, etc.

[0139] Step 207, determining whether the loss value is less than a preset loss threshold.

[0140] Among them, the preset loss threshold can be set according to the actual application scenario and is not specifically limited here.

[0141] Step 208: If yes, the trained estimation network is determined as the advertising index prediction model.

[0142] Step 209, if not, adjust the parameters of the reinforcement learning network model to be trained, and return to the step of obtaining the pre-exposure sample state feature data, post-exposure sample state feature data and reward data corresponding to the sample advertisement.

[0143] By adopting the training method provided by the embodiment of the present invention, an advertising index prediction model can be obtained by training the reinforcement learning network model to be trained. Then the trained advertising index prediction model is used in the advertising index prediction method provided by the embodiment of the present invention to perform a new advertising cold start. By using the advertising index prediction model obtained by the training method provided by the embodiment of the present invention to perform a new advertising cold start, while testing the new advertisement, the characteristics of the advertisement and the characteristics of the user are fully utilized, and the new advertising cold start compliance rate is improved under the premise of the lowest possible new advertising cold start cost.

[0144] Corresponding to the above-mentioned advertising index prediction method, an embodiment of the present invention further provides an advertising index prediction device. The advertising index prediction device provided by the embodiment of the present invention is introduced below. Figure 4 A schematic diagram of a structure of an advertising index prediction device provided by an embodiment of the present invention, such as Figure 4 As shown, the device comprises:

[0145] The data acquisition module 401 is used to acquire the status characteristic data of the advertisement to be delivered;

[0146] The index prediction module 402 is used to input the state feature data into a pre-trained advertising index prediction model, so that the advertising index prediction model determines the predicted browsing index after the advertisement to be placed within a first preset period based on the state feature data; wherein the advertising index prediction model is obtained by training the reinforcement learning network model to be trained based on the sample state feature data before exposure and the sample state feature data after exposure of the sample advertisement and the reward data corresponding to the sample advertisement;

[0147] The delivery determination module 403 is used to determine whether to deliver the advertisement to be delivered based on the predicted browsing index.

[0148] The device provided by the embodiment of the present invention is used to obtain the state feature data of the advertisement to be placed; the state feature data is input into the pre-trained advertisement index prediction model, so that the advertisement index prediction model determines the predicted browsing index after the advertisement to be placed is placed within the first preset period based on the state feature data; and determines whether to place the advertisement to be placed based on the predicted browsing index. That is, the advertisement index prediction model obtained by training the reinforcement learning network model to be trained can be used to determine whether to place the advertisement to be placed, thereby performing a new advertisement cold start. While testing the new advertisement, the characteristics of the advertisement and the characteristics of the user are fully utilized, and the indicator prediction accuracy of the new advertisement cold start is improved under the premise of the lowest possible new advertisement cold start cost, thereby improving the new advertisement cold start compliance rate.

[0149] Optional, see Figure 5 , the device further comprises:

[0150] The rate of return calculation module 501 is used to calculate the predicted rate of return corresponding to the advertisement to be placed based on the predicted browsing index and preset advertisement reference index data if the advertisement to be placed can be placed.

[0151] Optional, see Figure 5 , the device further comprises:

[0152] The user feature determination module 502 is used to determine the activity of the delivery user corresponding to the to-be-delivered advertisement; determine whether the activity is greater than a preset activity threshold; if not, abandon delivery of the to-be-delivered advertisement to the delivery user; if yes, obtain attribute information of the delivery user and historical browsing information of the advertisement delivered to the delivery user, and use the attribute information and historical browsing information as user features corresponding to the delivery user;

[0153] The data acquisition module 401 is specifically used to acquire the content information and advertisement position of the advertisement to be placed, and determine the content information, the advertisement and the user characteristics corresponding to the placement user as the state characteristic data of the advertisement to be placed.

[0154] Optionally, the predicted browsing index includes: a predicted click rate and a predicted conversion rate corresponding to the advertisement to be placed;

[0155] The rate of return calculation module 501 is specifically used to calculate the predicted rate of return corresponding to the advertisement to be placed through preset calculation rules based on the predicted click rate, the predicted conversion rate, the preset cost, the preset minimum return on investment and the preset minimum conversion indicator parameter.

[0156] Optionally, the rate of return calculation module 501 is specifically used to use the following formula to calculate the predicted rate of return corresponding to the advertisement to be placed through a preset calculation rule based on the predicted click rate, the predicted conversion rate, the preset cost, the preset minimum return on investment ratio and the preset minimum conversion indicator parameter:

[0157]

[0158] Wherein, ECPM is the predicted rate of return corresponding to the advertisement to be placed, pCTCVR=pCTR*pCVR, pCTR is the predicted click-through rate, pCVR is the predicted conversion rate, conversion_price is the preset cost, β max To preset hyperparameters, ROI thre To set the minimum return on investment, pCTCVR thre It is the preset minimum conversion indicator parameter.

[0159] Optionally, the device further comprises:

[0160] A feature data storage module (not shown in the figure) is used to determine whether to place the advertisement to be placed based on the predicted rate of return; if the advertisement to be placed is placed, obtain the corresponding actual rate of return after the advertisement to be placed is placed; and store the actual rate of return in a log.

[0161] Corresponding to the above-mentioned training method of the advertising index prediction model, the embodiment of the present invention further provides a training device for the advertising index prediction model. The training device for the advertising index prediction model provided by the embodiment of the present invention is introduced below. The device includes:

[0162] A model training module (not shown in the figure) is used to obtain pre-exposure sample state feature data, post-exposure sample state feature data and reward data corresponding to the sample advertisement; input the post-exposure sample state feature data into the estimation network to obtain a first sample predicted browsing index after the sample advertisement is released within a second preset period; input the post-exposure sample state feature data into the target network to obtain a second sample predicted browsing index after the sample advertisement is released within a second preset period; determine the label corresponding to the sample advertisement based on the reward data and the first sample predicted browsing index, the second sample predicted browsing index, the first preset browsing index threshold and the second preset browsing index threshold, wherein the The label is used to identify the delivery value of the sample advertisement; the sample state feature data before exposure is input into the estimation network to obtain a third sample predicted browsing index after the sample advertisement is delivered within a second preset period; based on the label, the third sample predicted browsing index and the third preset browsing index threshold, the loss value corresponding to the reinforcement learning network model to be trained is calculated; it is determined whether the loss value is less than the preset loss threshold; if so, the estimation network obtained by training is determined as the advertising index prediction model; if not, the parameters of the reinforcement learning network model to be trained are adjusted, and the step of obtaining the sample state feature data before exposure, the sample state feature data after exposure and the reward data corresponding to the sample advertisement is returned to.

[0163] Optionally, the model training module is specifically used to determine the label corresponding to the sample advertisement based on the reward data and the first sample predicted browsing index, the second sample predicted browsing index, the first preset browsing index threshold, and the second preset browsing index threshold by using the following formula:

[0164] Q l [action]=reward+γ*q t_s

[0165] Among them, Q l [action] is the label corresponding to the sample advertisement. The action value is 0 or 1. l [1] represents the browsing index corresponding to the sample advertisement, Q l [0] indicates that the browsing index corresponding to the sample advertisement is not placed, reward is the reward data, γ is the preset reward discount factor, γ∈(0,1), q t_s =Q t_n [eval index ], eval index = argmax(Q e0_n ,Q e1_n ), Q t_n =[Q t0_n ,Q t1_n ], Qe1_n is the predicted browsing index of the first sample, Q e0_n is the first preset browsing index threshold, Q t1_n Predict browsing index for the second sample, Q t0_n is the second preset browsing index threshold.

[0166] Optionally, the model training module is specifically used to calculate the loss value corresponding to the reinforcement learning network model to be trained based on the label, the third sample predicted browsing index and the third preset browsing index threshold by using the following formula:

[0167]

[0168] Among them, Loss is the loss value, n is the sum of the number of ads delivered and the number of ads not delivered, Q e =[Q e0, Q e1 ], Q e1 is the predicted browsing index of the third sample, Q e0 is the third preset browsing index threshold, Q l for the label.

[0169] By using the device provided by the embodiment of the present invention, an advertising index prediction model can be obtained by training the reinforcement learning network model to be trained. Then, the advertising index prediction model obtained by training can be used to cold start a new advertisement. While testing the new advertisement, the characteristics of the advertisement and the characteristics of the user are fully utilized, so as to improve the new advertisement cold start compliance rate under the premise of the lowest possible new advertisement cold start cost.

[0170] The embodiment of the present invention further provides an electronic device, such as Figure 6 As shown, it includes a processor 601, a communication interface 602, a memory 603 and a communication bus 604, wherein the processor 601, the communication interface 602, and the memory 603 communicate with each other through the communication bus 604.

[0171] Memory 603, used for storing computer programs;

[0172] The processor 601 is used to implement the steps of any of the advertising index prediction methods or any of the advertising index prediction model training methods when executing the program stored in the memory 603.

[0173] The communication bus mentioned in the above electronic device can be a Peripheral Component Interconnect (PCI) bus or an Extended Industry Standard Architecture (EISA) bus, etc. The communication bus can be divided into an address bus, a data bus, a control bus, etc. For ease of representation, only one thick line is used in the figure, but it does not mean that there is only one bus or one type of bus.

[0174] The communication interface is used for communication between the above electronic device and other devices.

[0175] The memory may include a random access memory (RAM) or a non-volatile memory, such as at least one disk memory. Optionally, the memory may also be at least one storage device located away from the aforementioned processor.

[0176] The above-mentioned processor can be a general-purpose processor, including a central processing unit (CPU), a network processor (NP), etc.; it can also be a digital signal processor (DSP), an application specific integrated circuit (ASIC), a field programmable gate array (FPGA) or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components.

[0177] In another embodiment provided by the present invention, a computer-readable storage medium is also provided, in which a computer program is stored. When the computer program is executed by a processor, it implements any of the advertising indicator prediction methods described in the above embodiments or any advertising indicator prediction model training method.

[0178] In another embodiment provided by the present invention, a computer program product including instructions is also provided, which, when executed on a computer, enables the computer to execute any of the advertising index prediction methods or any of the advertising index prediction model training methods described in the above embodiments.

[0179] In the above embodiments, it can be implemented in whole or in part by software, hardware, firmware or any combination thereof. When implemented by software, it can be implemented in whole or in part in the form of a computer program product. The computer program product includes one or more computer instructions. When the computer program instructions are loaded and executed on a computer, the process or function described in the embodiment of the present invention is generated in whole or in part. The computer can be a general-purpose computer, a special-purpose computer, a computer network, or other programmable device. The computer instructions can be stored in a computer-readable storage medium, or transmitted from one computer-readable storage medium to another computer-readable storage medium. For example, the computer instructions can be transmitted from a website site, computer, server or data center to another website site, computer, server or data center by wired (e.g., coaxial cable, optical fiber, digital subscriber line (DSL)) or wireless (e.g., infrared, wireless, microwave, etc.). The computer-readable storage medium can be any available medium that a computer can access or a data storage device such as a server or data center that includes one or more available media integrated. The available medium can be a magnetic medium (e.g., a floppy disk, a hard disk, a tape), an optical medium (e.g., a DVD), or a semiconductor medium (e.g., a solid-state hard disk Solid State Disk (SSD)), etc.

[0180] It should be noted that, in this article, relational terms such as first and second, etc. are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Moreover, the terms "include", "comprise" or any other variants thereof are intended to cover non-exclusive inclusion, so that a process, method, article or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or device. In the absence of further restrictions, the elements defined by the sentence "comprise a ..." do not exclude the existence of other identical elements in the process, method, article or device including the elements.

[0181] Each embodiment in this specification is described in a related manner, and the same or similar parts between the embodiments can be referred to each other, and each embodiment focuses on the differences from other embodiments. In particular, for the device, electronic device, computer-readable storage medium, and computer program product embodiments, since they are basically similar to the method embodiments, the description is relatively simple, and the relevant parts can be referred to the partial description of the method embodiments.

[0182] The above description is only a preferred embodiment of the present invention and is not intended to limit the protection scope of the present invention. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present invention are included in the protection scope of the present invention.

Claims

1. A method for predicting advertising indicators, characterized in that: include: Determine the activity level of the users corresponding to the advertisements to be delivered; Determining whether the activity level is greater than a preset activity level threshold; If not, giving up delivering the advertisement to be delivered to the delivery user; If yes, obtaining attribute information of the delivery user and historical browsing information of the advertisements delivered to the delivery user, and using the attribute information and the historical browsing information as user features corresponding to the delivery user; Acquire state characteristic data of the advertisement to be delivered, wherein the state characteristic data includes user characteristics corresponding to the delivery user; Input the state feature data into a pre-trained advertising index prediction model, so that the advertising index prediction model determines the predicted browsing index after the advertisement to be placed is placed within a first preset period based on the state feature data; wherein the predicted browsing index represents the predicted revenue value corresponding to the exposure of the advertisement to be placed, and the advertising index prediction model is obtained by training the reinforcement learning network model to be trained based on the sample state feature data before and after the exposure of the sample advertisement and the reward data corresponding to the sample advertisement; Based on the predicted browsing index, it is determined whether the advertisement to be delivered can be delivered.

2. The method according to claim 1, characterized in that After determining whether the advertisement to be placed can be placed based on the predicted browsing index, the method further includes: If the advertisement to be placed can be placed, the predicted rate of return corresponding to the advertisement to be placed is calculated based on the predicted browsing index and preset advertisement reference index data.

3. The method according to claim 1, characterized in that The step of obtaining the status characteristic data of the advertisement to be delivered includes: The content information and the advertisement position of the advertisement to be placed are obtained, and the content information, the advertisement position and the user characteristics corresponding to the placement user are determined as the state characteristic data of the advertisement to be placed.

4. The method according to claim 2, characterized in that: The predicted browsing index includes: the predicted click rate and predicted conversion rate corresponding to the advertisement to be placed; The calculating the predicted rate of return corresponding to the advertisement to be placed based on the predicted browsing index and the preset advertisement reference index data includes: Based on the predicted click rate, the predicted conversion rate, the preset cost, the preset minimum return on investment and the preset minimum conversion indicator parameter, the predicted return rate corresponding to the advertisement to be placed is calculated by using preset calculation rules.

5. The method according to claim 4, characterized in that The step of calculating the predicted rate of return corresponding to the advertisement to be placed based on the predicted click rate, the predicted conversion rate, the preset cost, the preset minimum return on investment ratio and the preset minimum conversion index parameter by using a preset calculation rule includes: The following formula is used to calculate the predicted rate of return corresponding to the advertisement to be placed based on the predicted click rate, the predicted conversion rate, the preset cost, the preset minimum return on investment and the preset minimum conversion indicator parameter through the preset calculation rules: Wherein, ECPM is the predicted rate of return corresponding to the advertisement to be placed, pCTCVR=pCTR*pCVR, pCTR is the predicted click-through rate, pCVR is the predicted conversion rate, conversion_price is the preset cost, β max To preset hyperparameters, ROI thre To set the minimum return on investment, pCTCVR thre It is the preset minimum conversion indicator parameter.

6. The method according to claim 2, characterized in that After calculating the predicted return rate corresponding to the to-be-delivered advertisement based on the predicted browsing index and the preset advertisement reference index data, the method further includes: Determining whether to place the advertisement to be placed based on the predicted rate of return; If the advertisement to be placed is placed, obtaining the corresponding real return rate after the advertisement to be placed is placed; Log the real rate of return.

7. A method for training an advertising index prediction model, characterized in that: The reinforcement learning network model to be trained includes an estimation network and a target network, wherein the estimation network and the target network are networks with the same structure, and the method includes: Obtaining pre-exposure sample state feature data, post-exposure sample state feature data, and reward data corresponding to the sample advertisement, wherein the pre-exposure sample state feature data corresponding to the sample advertisement includes: user features of the delivery user before the sample advertisement is delivered to the corresponding delivery user, and the post-exposure sample state feature data corresponding to the sample advertisement includes: user features of the delivery user after the sample advertisement is delivered to the corresponding delivery user; Inputting the post-exposure sample state feature data into the estimation network to obtain a first sample predicted browsing index after the sample advertisement is placed within a second preset period; Inputting the post-exposure sample state feature data into the target network to obtain a second sample predicted browsing index after the sample advertisement is placed within a second preset period; Determine a label corresponding to the sample advertisement based on the reward data and the first sample predicted browsing index, the second sample predicted browsing index, a first preset browsing index threshold, and a second preset browsing index threshold, wherein the label is used to identify the delivery value of the sample advertisement; Inputting the sample state feature data before exposure into the estimation network to obtain a third sample predicted browsing index after the sample advertisement is released within a second preset period; Based on the label, the third sample prediction browsing index and the third preset browsing index threshold, calculating the loss value corresponding to the reinforcement learning network model to be trained; Determining whether the loss value is less than a preset loss threshold; If yes, determining the trained estimation network as the advertising index prediction model; If not, adjust the parameters of the reinforcement learning network model to be trained, and return to the step of obtaining the pre-exposure sample state feature data, post-exposure sample state feature data and reward data corresponding to the sample advertisement.

8. The method according to claim 7, characterized in that The step of determining a label corresponding to the sample advertisement based on the reward data and the first sample predicted browsing index, the second sample predicted browsing index, a first preset browsing index threshold, and a second preset browsing index threshold includes: The following formula is used to determine the label corresponding to the sample advertisement based on the reward data and the first sample predicted browsing index, the second sample predicted browsing index, the first preset browsing index threshold, and the second preset browsing index threshold: Q l [action]=reward+γ*q t_s Among them, Q l [action] is the label corresponding to the sample advertisement. The action value is 0 or 1. l [1] represents the browsing index corresponding to the sample advertisement, Q l [0] indicates that the browsing index corresponding to the sample advertisement is not placed, reward is the reward data, γ is the preset reward discount factor, γ∈(0,1), q t_s =Q t_n [eval index ], eval index = argmax(Q e0_n ,Q e1_n ), Q t_n =[Q t0_n ,Q t1_n ], Q e1_n is the predicted browsing index of the first sample, Q e0_n is the first preset browsing index threshold, Q t1_n Predict browsing index for the second sample, Q t0_n is the second preset browsing index threshold.

9. The method according to claim 8, characterized in that The calculating, based on the label, the third sample prediction browsing index and the third preset browsing index threshold, a loss value corresponding to the reinforcement learning network model to be trained includes: The following formula is used to calculate the loss value corresponding to the reinforcement learning network model to be trained based on the label, the third sample prediction browsing index and the third preset browsing index threshold: Among them, Loss is the loss value, n is the sum of the number of ads delivered and the number of ads not delivered, Q e =[Q e0, Q e1 ], Q e1 is the predicted browsing index of the third sample, Q e0 is the third preset browsing index threshold, Q l for the label.

10. An advertising index prediction device, characterized in that: include: A user feature determination module is used to determine the activity of the user corresponding to the advertisement to be delivered; Determining whether the activity level is greater than a preset activity level threshold; If not, giving up delivering the advertisement to be delivered to the delivery user; If yes, obtaining attribute information of the delivery user and historical browsing information of the advertisements delivered to the delivery user, and using the attribute information and the historical browsing information as user features corresponding to the delivery user; A data acquisition module, used to acquire state characteristic data of the advertisement to be delivered, wherein the state characteristic data includes user characteristics corresponding to the delivery user; An indicator prediction module is used to input the state feature data into a pre-trained advertising indicator prediction model, so that the advertising indicator prediction model determines the predicted browsing indicator after the advertisement to be placed is placed within a first preset period based on the state feature data; wherein the predicted browsing indicator represents the predicted revenue value corresponding to the exposure of the advertisement to be placed, and the advertising indicator prediction model is obtained by training the reinforcement learning network model to be trained based on the sample state feature data before and after the exposure of the sample advertisement and the reward data corresponding to the sample advertisement; The delivery determination module is used to determine whether the advertisement to be delivered can be delivered based on the predicted browsing index.

11. An electronic device, characterized in that: It includes a processor, a communication interface, a memory and a communication bus, wherein the processor, the communication interface and the memory communicate with each other through the communication bus; Memory, used to store computer programs; A processor, for implementing the method described in any one of claims 1-6 or 7-9 when executing a program stored in a memory.

12. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the method described in any one of claims 1-6 or 7-9 is implemented.

Images