Advertisement traffic quality evaluation method, resource investment determination method and electronic equipment

Abstract

The invention provides an advertisement traffic quality evaluation method, a resource investment determination method and electronic equipment. The method comprises the following steps: acquiring traffic data sent by an advertisement platform server; performing joint prediction on the traffic data by using a full-space multi-task network ESMM model to obtain a click rate and a conversion rate; and calculating a click conversion rate according to the click rate and the conversion rate, and determining a predicted quality score according to the click conversion rate. In the implementation process of the scheme, the click rate and the conversion rate are obtained by performing joint prediction on the traffic data through the full-space multi-task network ESMM model, the problem of inaccurate conversion rate prediction caused by sparse samples in a traditional method is effectively improved, the ESMM model combines modeling click and conversion tasks, and the prediction accuracy of the conversion rate is improved. According to the technical scheme of the invention, the click data can be utilized to assist the prediction of the conversion rate, and the problem of data sparsity of the conversion event is effectively relieved, so that the behavior path of the user from click to conversion can be better captured, and the accuracy of evaluating the quality of traffic is improved.

Description

Technical Field

[0001] This application relates to the technical fields of digital advertising, machine learning, data mining, and automated decision-making. Specifically, it relates to a method for evaluating the quality of advertising traffic, a method for determining resource investment, and an electronic device. Background Art

[0002] In the field of digital advertising, the evaluation of advertising traffic quality is an important link to ensure the advertising delivery effect and user satisfaction. Traditional advertising traffic quality evaluation methods usually rely on single-dimensional data analysis, such as evaluating only based on click-through rate or conversion rate. Although this single-dimensional evaluation method is simple and intuitive, it lacks effective modeling of the correlation between these two types of data. This data silo phenomenon leads to the inability to make full use of the internal connection between the two types of data, thus affecting the accuracy of traffic quality evaluation. Summary of the Invention

[0003] The purpose of the embodiments of this application is to provide a method for evaluating the quality of advertising traffic, a method for determining resource investment, and an electronic device, which are used to improve the problem of low accuracy in evaluating the quality of traffic.

[0004] The embodiments of this application provide a method for evaluating the quality of advertising traffic, including: obtaining traffic data sent by an advertising platform server; using the Entire Space Multi-Task Model (ESMM) to perform joint prediction on the traffic data to obtain the click-through rate and conversion rate; calculating the click-through conversion rate based on the click-through rate and conversion rate, and determining the predicted quality score based on the click-through conversion rate. In the implementation process of the above solution, the ESMM is used to perform joint prediction on the traffic data to obtain the click-through rate and conversion rate, effectively improving the problem of inaccurate conversion rate prediction caused by sample sparsity in traditional methods. By jointly modeling the click and conversion tasks, the ESMM can use click data to assist in the prediction of the conversion rate, effectively alleviating the problem of data sparsity of conversion events. Therefore, it can better capture the user's behavior path from click to conversion, thereby improving the accuracy of evaluating the quality of traffic.

[0005] Optionally, in the embodiments of the present application, the Entire Space Multi-Task Model (ESMM) is used to jointly predict traffic data to obtain the click-through rate and conversion rate, including: extracting sample features from the traffic data, where the sample features include brand portrait features, user portrait features, and user behavior sequence features; inputting the brand portrait features, user portrait features, and user behavior sequence features into the ESMM model to obtain the click-through rate and conversion rate output by the ESMM model through joint prediction based on the traffic data. In the implementation process of the above solution, by simultaneously considering the brand portrait features, user portrait features, and user behavior sequence features through the ESMM model, the ESMM model can jointly predict multiple tasks (such as click-through rate (CTR) and conversion rate (CVR)), and can simultaneously model in the entire sample space, so as to more comprehensively understand the user's behavior pattern and improve the accuracy of the ESMM model in predicting the user's click and conversion behavior.

[0006] Optionally, in the embodiments of the present application, the ESMM model includes: a click-through rate prediction module and a conversion rate prediction module; inputting the brand portrait features, user portrait features, and user behavior sequence features into the ESMM model includes: using the click-through rate prediction module to determine the click-through rate according to the brand portrait features, user portrait features, and user behavior sequence features; using the conversion rate prediction module to determine the conversion rate according to the brand portrait features, user portrait features, and user behavior sequence features. In the implementation process of the above solution, the click-through rate and conversion rate tasks are combined through the multi-task learning mechanism of the ESMM model, so as to jointly estimate the click-through rate and conversion rate in the entire sample space (i.e., all exposed samples). Compared with the traditional method of only estimating the conversion rate on click samples, the ESMM can more accurately capture the information in unclicked samples, avoiding the problem of sample selection bias, especially in those scenarios with a low click-through rate but a high conversion rate, enabling the click-through rate prediction and conversion rate prediction to promote each other. The model can better understand the user's potential purchase behavior while learning the click-through rate, thereby improving the accuracy of click-through rate and conversion rate prediction.

[0007] Optionally, in the embodiments of the present application, before using the full-space multi-task network ESMM model to jointly predict traffic data, it further includes: obtaining positive sample data and negative sample data, where the positive sample data is user behavior data of users who place orders within the terminals corresponding to the advertising platform server, and the negative sample data is user behavior data of users who do not place orders within the terminals corresponding to the advertising platform server; using the positive sample data and the negative sample data to train the ESMM model. Traditional model training usually only focuses on a single task (such as click-through rate prediction or conversion rate prediction), while ignoring the full utilization of un-converted users. In the implementation process of the above solution, through the effective utilization of the negative sample data (user behavior data of users who do not place orders within the terminals corresponding to the advertising platform server) by the ESMM model, this enables the model to better understand the characteristics of un-converted users, thereby improving the generalization ability of the model. Further, by jointly training with positive and negative sample data, the model can better handle the diversity and complexity of user behavior, enhancing the robustness and stability of the model.

[0008] The embodiments of the present application further provide a method for determining resource investment, including: obtaining traffic data sent by the advertising platform server; using the above-described advertising traffic quality evaluation method to predict the quality of the traffic data, obtaining a predicted quality score; determining resource investment information according to the predicted quality score. In the implementation process of the above solution, by introducing the advertising traffic quality evaluation method to predict the quality of the traffic data and determining the resource investment information according to the predicted quality score, an intelligent resource investment mechanism is realized, which can avoid investing resources in low-quality traffic (such as robot traffic, low-conversion-rate traffic), avoid resource waste, and can also provide more targeted resource investment for users, enhancing the user experience.

[0009] Optionally, in the embodiments of the present application, the resource investment information includes: final bid information; determining the bid information according to the predicted quality score includes: obtaining the average quality score; dividing the predicted quality score by the average quality score to obtain a bid coefficient; parsing the initial price of the advertising platform server from the traffic data and multiplying the initial price by the bid coefficient to obtain the final bid information. In the implementation process of the above solution, by comparing the predicted quality score (a predicted value reflecting the advertising quality) with the average quality score and calculating the bid coefficient, it can more accurately reflect the quality level of the advertisement, because high-quality advertisements will obtain a higher bid coefficient, thereby increasing the final bid information and increasing the opportunity for advertisement display; while advertisements with lower quality will obtain a lower bid coefficient, avoiding unnecessary cost expenditures. This dynamic optimization of the bid strategy avoids the problem of high cost and low effect that may occur in traditional fixed bid strategies. Through real-time adjustment of the bid information, the system can better adapt to market changes and optimize the advertisement placement effect.

[0010] The embodiment of the present application also provides an advertising traffic quality evaluation device, including: a traffic data acquisition module for acquiring traffic data sent by an advertising platform server; a data joint prediction module for jointly predicting the traffic data using the ESMM model of the full-space multi-task network to obtain click-through rate and conversion rate; a prediction quality determination module for calculating the click-through conversion rate based on the click-through rate and conversion rate and determining the prediction quality score according to the click-through conversion rate.

[0011] Optionally, in the embodiment of the present application, the data joint prediction module includes: a sample feature extraction sub-module for extracting sample features from the traffic data, and the sample features include brand portrait features, user portrait features, and user behavior sequence features; a traffic data prediction sub-module for inputting the brand portrait features, user portrait features, and user behavior sequence features into the ESMM model to obtain the click-through rate and conversion rate output by the ESMM model through joint prediction of the traffic data.

[0012] Optionally, in the embodiment of the present application, the ESMM model includes: a click-through rate estimation module and a conversion rate estimation module; the traffic data prediction sub-module includes: a click-through rate determination unit for using the click-through rate estimation module to determine the click-through rate according to the brand portrait features, user portrait features, and user behavior sequence features; a conversion rate determination unit for using the conversion rate estimation module to determine the conversion rate according to the brand portrait features, user portrait features, and user behavior sequence features.

[0013] Optionally, in the embodiment of the present application, the advertising traffic quality evaluation device further includes: a sample data acquisition module for acquiring positive sample data and negative sample data, where the positive sample data is user behavior data of users who place orders within the terminal corresponding to the advertising platform server, and the negative sample data is user behavior data of users who do not place orders within the terminal corresponding to the advertising platform server; an ESMM model training module for training the ESMM model using the positive sample data and negative sample data.

[0014] The embodiment of the present application also provides an advertising resource investment determination device, including: a traffic data acquisition module for acquiring traffic data sent by an advertising platform server; a traffic quality prediction module for predicting the quality of the traffic data using the above-mentioned advertising traffic quality evaluation method to obtain a prediction quality score; a resource investment information determination module for determining resource investment information according to the prediction quality score.

[0015] Optionally, in the embodiments of the present application, the resource investment information includes: final bid information; the resource investment information determination module includes: a quality score acquisition sub-module for acquiring the average quality score; a bid coefficient determination sub-module for dividing the predicted quality score by the average quality score to obtain the bid coefficient; and a final bid acquisition sub-module for parsing the initial price of the advertising platform server from the traffic data and multiplying the initial price by the bid coefficient to obtain the final bid information.

[0016] The embodiments of the present application also provide an electronic device, including: a processor and a memory, the memory stores machine-readable instructions executable by the processor, and when the machine-readable instructions are run by the processor, the methods described above are executed.

[0017] The embodiments of the present application also provide a computer-readable storage medium, on which a computer program is stored, and when the computer program is run by the processor, the methods described above are executed.

[0018] The embodiments of the present application also provide a computer program product, including: a computer program or computer instructions, and when the computer program or computer instructions are run by the processor, the methods described above are executed. Description of the Drawings

[0019] In order to more clearly illustrate the technical solutions of the embodiments of the present application, the drawings required to be used in the embodiments of the present application will be briefly introduced below. It should be understood that the following drawings only show some embodiments of the present application, and therefore should not be regarded as a limitation of the scope. For those of ordinary skill in the art, other related drawings can be obtained based on these drawings without creative efforts.

[0020] Figure 1 A schematic flowchart of the advertising traffic quality evaluation method provided by the embodiments of the present application;

[0021] Figure 2 A schematic diagram of the training process of the ESMM model provided by the embodiments of the present application;

[0022] Figure 3 A schematic flowchart of the resource investment determination method provided by the embodiments of the present application;

[0023] Figure 4 A schematic structural diagram of the advertising traffic quality evaluation device provided by the embodiments of the present application;

[0024] Figure 5 A schematic structural diagram of the electronic device provided by the embodiments of the present application. Detailed Embodiments

[0025] To make the objectives, technical solutions, and advantages of the embodiments of this application clearer, the following will clearly and completely describe the technical solutions in the embodiments of this application with reference to the accompanying drawings in the embodiments of this application. It should be understood that the accompanying drawings in the embodiments of this application are only for the purpose of illustration and description, and are not used to limit the protection scope of the embodiments of this application. In addition, it should be understood that the schematic drawings are not drawn to actual scale. The flowcharts used in the embodiments of this application illustrate the operations implemented according to some embodiments of the embodiments of this application. It should be understood that the operations in the flowchart may not be implemented in sequence, and steps without logical context may be reversed or implemented simultaneously. In addition, those skilled in the art can add one or more other operations to the flowchart or remove one or more operations from the flowchart under the guidance of the content of the embodiments of this application.

[0026] In addition, the described embodiments are only a part of the embodiments of this application, rather than all of the embodiments. The components of the embodiments of this application generally described and illustrated in the accompanying drawings here can be arranged and designed in various different configurations. Therefore, the following detailed description of the embodiments of this application provided in the accompanying drawings is not intended to limit the scope of the claimed embodiments of this application, but merely represents the selected embodiments of this application.

[0027] It can be understood that "first" and "second" in the embodiments of this application are used to distinguish similar objects. Those skilled in the art can understand that words such as "first" and "second" do not limit the quantity and execution order, and words such as "first" and "second" do not necessarily mean different. In the description of the embodiments of this application, the term "and / or" is only a description of the association relationship of associated objects, indicating that three relationships may exist. For example, A and / or B may represent: A exists alone, A and B exist simultaneously, and B exists alone. In addition, the character " / " in this article generally represents an "or" relationship between the associated objects before and after. The term "plural" refers to two or more (including two). Similarly, "multiple groups" refers to two or more groups (including two groups).

[0028] It should be noted that the advertising traffic quality evaluation method provided by the embodiments of this application can be executed by an electronic device. Here, the electronic device refers to a device terminal or a server with the function of executing a computer program. Examples of the device terminal include: smart phones, personal computers, tablet computers, personal digital assistants, or mobile Internet devices, etc. A server refers to a device that provides computing services through a network. Examples of the server include: x86 servers and non-x86 servers. Non-x86 servers include: mainframes, minicomputers, and UNIX servers.

[0029] In the current digital advertising field, the evaluation of the quality of advertising traffic is an important link to ensure the effectiveness of advertising placement and user satisfaction. Traditional methods for evaluating the quality of advertising traffic usually rely on data analysis from a single dimension, while ignoring the potential impact of another metric. This one-sided evaluation method fails to comprehensively reflect the actual quality of advertising traffic. Especially in scenarios where user behavior is complex and variable, the evaluation results from a single dimension may deviate significantly from the actual situation. Although this single-dimensional evaluation method is simple and intuitive, it lacks effective modeling of the correlation between these two types of data. In some cases, advertising traffic with a high click-through rate may be judged as low-quality due to a low conversion rate, while advertising traffic with a low click-through rate may be judged as high-quality due to a high conversion rate. Such inconsistent evaluation results have brought great troubles to advertisers and platform providers. Especially in cross-platform and cross-device advertising placement scenarios, the single-dimensional evaluation method is difficult to effectively capture the dynamic changes in user behavior, resulting in an even larger gap between the evaluation results and the actual effects. Therefore, this data silo phenomenon leads to the inability to fully utilize the internal connection between the two types of data, thereby affecting the accuracy of traffic quality evaluation.

[0030] To address the above issues, please refer to Figure 1 the flowchart of the advertising traffic quality evaluation method provided by the embodiments of the present application shown in; the core idea of this advertising traffic quality evaluation method is to jointly predict the traffic data of the advertising platform through the use of an Entire Space Multi-Task Model (ESMM), thereby obtaining the click-through rate and conversion rate. Then, based on these two, the click-through conversion rate is calculated, and the predicted quality score of the advertising traffic is determined according to the click-through conversion rate. This multi-task learning framework comprehensively considers the joint probability of clicks and conversions, effectively alleviating the problem of data sparsity in conversion events. Therefore, it can better capture the user's behavior path from click to conversion, thereby improving the accuracy of evaluating the quality of traffic. The above advertising traffic quality evaluation method can be applied to the advertiser server, and the implementation manner of this method may include:

[0031] Step S110: Obtain the traffic data sent by the advertising platform server.

[0032] The advertising platform server refers to the computing server that provides advertising placement and management services. This server is usually maintained by the advertising platform operator and is used to process the relevant data and requests of advertisers and display advertisements to target users according to the set conditions. The above advertising platform can be the advertising platform of a search engine company. Suppose the advertisements of a certain e-commerce platform are placed through the advertising alliance platform of Company A Search Engine. Then the server running on this advertising alliance platform is the advertising platform server.

[0033] Traffic data refers to the user behavior data recorded by the advertising platform server during the advertising placement process, including but not limited to the number of impressions, clicks, conversion times (i.e., purchase times), etc. For example, a certain advertisement is shown to 1000 users within a certain period of time, and 50 of them click on the advertisement. These numbers of impressions and clicks are part of the traffic data.

[0034] It can be understood that the traffic data sent by the above-mentioned advertising platform server refers to the user behavior data recorded by the advertising platform server, such as the number of ad impressions and clicks, etc. These data can be transmitted via the Internet to the advertiser server or the server where its analysis system runs, so that the advertiser server can analyze these traffic data to determine whether the advertiser decides to place advertisements on this advertising platform.

[0035] Step S120: Use the Entire Space Multi-Task Model (ESMM) to jointly predict the traffic data to obtain the click-through rate and conversion rate.

[0036] The Entire Space Multi-Task Model (ESMM) is a multi-task learning model that can simultaneously predict multiple related tasks. It can be used to handle the joint prediction of click-through rate and conversion rate. By using a shared underlying network to simultaneously predict the click-through rate and conversion rate, it can make full use of the sample information in the entire space for modeling, improving the sample selection bias and data sparsity problems in traditional models.

[0037] The click-through rate (CTR) refers to the probability that a user actually clicks on an advertisement or content after seeing it. CTR is one of the important indicators to measure the attractiveness of an advertisement or content. For example, an advertisement is shown 1000 times on a web page, and 100 of them are clicked by users. Then the click-through rate of this advertisement is 10%.

[0038] The conversion rate (CVR) refers to the probability that a user actually completes a predefined goal (such as purchase, registration, etc.) after clicking on an advertisement or content. CVR is one of the key indicators to measure the effectiveness of an advertisement or content. For example, an advertisement is clicked 100 times, and 5 of them result in user purchases. Then the conversion rate of this advertisement is 5%.

[0039] The above-mentioned joint prediction of traffic data means that the model simultaneously considers multiple related tasks, such as comprehensively predicting the click-through rate and conversion rate, rather than predicting the click-through rate or conversion rate separately. For example, the ESMM model can simultaneously predict whether a user will click on an advertisement (click-through rate) and whether they will purchase a product after clicking (conversion rate).

[0040] Step S130: Calculate the click-through conversion rate based on the click-through rate and the conversion rate, and determine the predicted quality score according to the click-through conversion rate.

[0041] The click-through conversion rate (CTCVR) refers to the probability that a user will ultimately complete a conversion (such as a purchase) after seeing and clicking on an advertisement. CTCVR is a comprehensive indicator of the click-through rate and the conversion rate, reflecting the overall effect from advertisement display to ultimate conversion. If the click-through rate of an advertisement is 5% and the conversion rate is 20%, then the click-through conversion rate of this advertisement is 1% (i.e., 5% × 20%).

[0042] The predicted quality score is an indicator for evaluating the prediction quality of the ESMM model based on the prediction result of the click-through conversion rate. For example, it is a comprehensive score calculated based on the prediction results (such as the click-through rate, the conversion rate, and / or the click-through conversion rate, etc.) of the ESMM model. This comprehensive score is used to measure the accuracy and effectiveness of the model's prediction. If the predicted result of the click-through conversion rate of a model is very close to the actual observed value, then the predicted quality score of this model will be relatively high, indicating that the model has strong prediction ability.

[0043] It can be understood that when the above solution runs on the advertiser's server, it can effectively help the advertiser utilize the real-time interface (Real-Time API, RTA) capability of the advertising platform server. During the process of jointly predicting traffic data using the entire-space multi-task network ESMM model, the advertiser can also analyze and predict using the user-related data stored on its own advertiser server. For example, jointly predict the traffic data and the user-related data stored on the advertiser server using the ESMM model, so that the advertiser server can decide in real time whether to participate in the bidding process for a certain advertisement display opportunity. Through this refined bidding method, the advertiser can better control the advertising investment cost and improve the return on investment (ROI) of the advertising.

[0044] In the implementation process of the above solution, the click-through rate and the conversion rate are obtained through joint prediction of traffic data using the entire-space multi-task network ESMM model, which can simultaneously consider the relationship between the click-through rate and the conversion rate, effectively improving the problem of inaccurate conversion rate prediction caused by sample sparsity in traditional methods, and thus more accurately predicting the user's click behavior and subsequent conversion behavior. The ESMM model can capture the complex dependency relationship between clicks and conversions by jointly modeling the click and conversion tasks, effectively alleviating the problem of data sparsity of conversion events, and therefore can better capture the user's behavior path from click to conversion, thereby improving the accuracy of evaluating the quality of traffic.

[0045] Optionally, as an alternative implementation of the advertiser server obtaining the traffic data sent by the advertising platform server in step S110 above, the advertiser server can be implemented in the following ways: First, by means of calling an Application Programming Interface (API). For example, the advertiser server can actively request traffic data through an API interface pre-agreed with the advertising platform server. The advertiser server can send requests to the advertising platform server regularly or as needed to obtain the required traffic data, such as click-through rate, display volume, conversion rate, etc. Second, obtain data through data push. For example, the advertising platform server can actively push the traffic data to the advertiser server. The advertising platform server can, when detecting relevant events (such as ad exposure, click, conversion, etc.), immediately send the data to the advertiser server through the push mechanism to ensure the real-time nature of the data. Third, obtain data through a message queue: The advertising platform server can send the traffic data to a message queue (such as Kafka, RabbitMQ, etc.), and the advertiser server obtains this data by subscribing to the message queue. The message queue can ensure the real-time nature and reliability of the data and is suitable for processing a large amount of real-time data.

[0046] As an alternative implementation of step S120 above, the implementation of using the full-space multi-task network ESMM model to jointly predict traffic data to obtain the click-through rate and conversion rate may include:

[0047] Step S121: Extract sample features from the traffic data. The sample features include brand portrait features, user portrait features, and user behavior sequence features.

[0048] Sample features refer to the features extracted from the traffic data for model prediction and / or training. These features can help the ESMM model understand the specific attributes of users and brands, such as brand portrait features, user portrait features, user behavior sequence features, etc. Brand portrait features are the features that describe the image and positioning of a brand in the minds of users, including brand awareness, positioning, history, product line, user reviews, market share, etc.; user portrait features are the features that describe the personal attributes and behavior habits of users, including age, gender, region, consumption habits, etc.; user behavior sequence features are the series of behavior sequences and patterns of users on the shopping platform, such as first browsing products, clicking and swiping, then adding to the shopping cart, and finally placing an order to purchase.

[0049] The implementation manners of the above step S121 include: the first manner of using a neural network model to extract features. For example, encoding the user behavior sequence in the traffic data into a two-dimensional matrix, and then using a convolutional neural network (CNN) model to extract features from the two-dimensional matrix to extract user behavior sequence features of local patterns and global structures from the user behavior sequence; for another example, using a long short-term memory network (LSTM) or a gated recurrent unit (GRU) to capture the temporal dependence in the user behavior sequence, so as to extract user behavior sequence features from the user behavior sequence; for another example, using a Transformer model to capture brand portrait features and / or user portrait features in the traffic data. Due to the self-attention mechanism of the Transformer to simultaneously capture the dependencies at different positions in the user behavior sequence, more complex features can be extracted. The second manner of using a feature engineering method to extract features. For example, screening out key features related to brand portraits, user portraits, and user behavior sequences from the traffic data through feature engineering methods in statistical methods. The above feature engineering methods include information gain, chi-square test, recursive feature elimination (RFE), etc.; for another example, combining different features to generate new features. The geographical features of users can be combined with the popularity features of brands to generate new "geographical-brand influence" features.

[0050] Optionally, if the traffic data contains text information (such as user comments, product descriptions, etc.), a pre-trained Bidirectional Encoder Representation from Transformers (BERT) model can also be used to extract text representation features. The BERT model can understand the text information through the context and generate rich text representation features. These text representation features can be used as sample features and input into the ESMM model. If the traffic data contains multiple types of data (such as text, audio, images, videos, etc. of user behaviors), multi-modal fusion technology can be used to fuse the user's text comments, click behavior sequence features, and product picture features to generate richer user portrait features. Through the above-mentioned various technical means, brand portrait features, user portrait features, and user behavior sequence features can be effectively extracted from the traffic data, thereby providing high-quality input features for the full-space multi-task network ESMM model.

[0051] Step S122: Input the brand portrait features, user portrait features, and user behavior sequence features into the ESMM model to obtain the click-through rate and conversion rate jointly predicted by the ESMM model based on the traffic data.

[0052] As an alternative implementation of the above step S122, the above ESMM model may include: a click-through rate prediction module and a conversion rate prediction module; the implementation of inputting brand portrait features, user portrait features, and user behavior sequence features into the ESMM model may include:

[0053] Step S122a: Use the click-through rate prediction module to determine the click-through rate according to the brand portrait features, user portrait features, and user behavior sequence features.

[0054] The click-through rate prediction module is a calculation module for predicting the probability that a user will click on an advertisement or content after seeing it. For example, on an e-commerce platform, the click-through rate prediction module can predict the probability that a user will click on an advertisement for a certain product based on the user's browsing history and interests.

[0055] An implementation of the above step S122a is, for example: The click-through rate prediction module uses a Multilayer Perceptron (MLP) and uses the formula to calculate the brand portrait features (such as a certain mobile phone brand being a well-known high-end brand), user portrait features (such as user A liking technology products), and user behavior sequence features (such as user A having browsed mobile phone advertisements many times recently), so as to obtain the click-through rate of user A for the advertisement. Among them, pCTR represents the click-through rate, x represents sample features (such as brand portrait features, user portrait features, user behavior sequence features, etc.), y represents the click label (1 represents being clicked by the user, and 0 represents not being clicked by the user), W 3a ,W 2a ,W 1a are all network parameters of the click-through rate prediction module, and Sigmoid and Relu are both activation functions.

[0056] Step S122b: Use the conversion rate prediction module to determine the conversion rate according to the brand portrait features, user portrait features, and user behavior sequence features.

[0057] The conversion rate prediction module is a calculation module for predicting the probability that a user will finally complete a specific behavior (such as purchase, registration, etc.) after clicking on an advertisement or content. For example, on an e-commerce platform, the conversion rate prediction module can predict the probability that a user will finally purchase a certain product based on the user's click history and purchase behavior.

[0058] An implementation of the above step S122b is, for example: The conversion rate prediction module uses a Multilayer Perceptron (MLP) and can use the formula Calculate the brand portrait features (e.g., a certain mobile phone brand is a well-known high-end brand), user portrait features (e.g., user A likes technology products), and user behavior sequence features (e.g., user A has browsed mobile phone ads many times recently) to obtain the conversion rate of user A, that is, the probability of user purchase. Among them, pCVR represents the conversion rate, x represents the sample features, y represents the click label, z represents the conversion label (1 represents the occurrence of a purchase conversion behavior, and 0 represents the non-occurrence of a purchase conversion behavior), W 3b ,W 2b ,W 1b are all network parameters of the conversion rate prediction module, and Sigmoid and Relu are both activation functions.

[0059] Optionally, as an alternative implementation of the above step S130, the above implementation of calculating the click-through conversion rate based on the click-through rate and the conversion rate is, for example: use the following calculation formula for the click-through conversion rate to calculate the click-through rate and the conversion rate to obtain the click-through conversion rate; where pCTVR represents the click-through conversion rate, pCTR represents the click-through rate, pCVR represents the conversion rate, x represents the sample features, y represents the click label, and z represents the conversion label.

[0060] Optionally, as an alternative implementation of the above step S130, the above implementation of determining the prediction quality score based on the click-through conversion rate includes: the first implementation, directly determining the click-through conversion rate as the prediction quality score, that is, the click-through conversion rate is determined as the prediction quality score p i ; the second implementation, determining the value obtained by multiplying the click-through conversion rate by a preset conversion coefficient as the prediction quality score, for example, determining pCTVR×α as the prediction quality score p i , where pCTVR represents the click-through conversion rate and α represents the preset conversion coefficient.

[0061] Please refer to Figure 2 the schematic diagram of the training process of the ESMM model provided by the embodiment of the present application shown; as an alternative implementation of the above advertising traffic quality evaluation method, before using the full-space multi-task network ESMM model to jointly predict traffic data, the ESMM model can also be trained, and the training process of this ESMM model can include:

[0062] Step S210: Obtain positive sample data and negative sample data. The positive sample data is the user behavior data of placing an order in the terminal corresponding to the advertising platform server, and the negative sample data is the user behavior data of not placing an order in the terminal corresponding to the advertising platform server.

[0063] Positive sample data refers to the user behavior data where actual order placement occurs within the terminals corresponding to the advertising platform server. These data are used to train the ESMM model so that the ESMM model can identify which user behaviors are more likely to be converted into actual purchases or orders. For example, user A clicks on an advertisement on the advertising page of an e-commerce platform, then enters the product page and finally places an order to purchase a product. All the user behavior data (such as clicking on the advertisement, browsing the product page, placing an order, etc.) during this process constitutes positive sample data.

[0064] Negative sample data refers to the user behavior data where no order placement occurs within the terminals corresponding to the advertising platform server. These data are used to train the model to identify which user behaviors will not be converted into actual purchases or orders. For example, user B clicks on an advertisement on the advertising page of the same e-commerce platform, browses the product page, but does not place an order and directly exits the platform. All the user behavior data (such as clicking on the advertisement, browsing the product page, leaving the platform, etc.) during this process constitutes negative sample data.

[0065] Specifically, the above-mentioned negative sample data can include: data of brand exposure without order placement and data of no brand exposure without order placement. Brand exposure means that the brand has been exposed by the advertiser through a network communication system (such as an e-commerce system). Among them, data of brand exposure without order placement refers to the user behavior data where, within the terminal corresponding to the advertising platform server, although the user has been exposed to an advertisement of a certain brand (i.e., the brand is exposed), no order placement occurs in subsequent behaviors. For example: User C sees an advertisement of a certain brand on the advertising page of an e-commerce platform, clicks to enter the product page of the brand, browses several products, but finally does not place an order and directly exits the platform. During this process, the behavior data of user C (including advertisement exposure, clicking on the advertisement, browsing the product page, leaving the platform, etc.) constitutes data of brand exposure without order placement. The above-mentioned data of no brand exposure without order placement refers to the user behavior data where, within the terminal corresponding to the advertising platform server, the user has not been exposed to an advertisement of a certain brand (i.e., the brand is not exposed), and no order placement occurs in subsequent behaviors. Such data is usually used to train the model to identify which users will not be converted into actual purchases without being exposed to advertisements. For example: User D browses multiple pages on an e-commerce platform but does not see any advertisement of a specific brand and finally does not place an order and directly exits the platform. During this process, the behavior data of user D (including browsing multiple pages, leaving the platform, etc.) constitutes data of no brand exposure without order placement.

[0066] The above-mentioned terminal refers to the device or platform used by the user, such as a mobile phone, computer, tablet, etc. The user accesses the advertising platform server through these devices and performs relevant operations (such as clicking on an advertisement, browsing a page, etc.). If user A uses an iPhone, then this iPhone is the terminal of user A. User A clicks on an advertisement and places an order on the advertising page through this terminal, and the behavior data of this terminal will be recorded with the consent of user A.

[0067] Step S220: Train the ESMM model using positive sample data and negative sample data.

[0068] The implementation manner of the above step S220 is, for example: First, perform feature engineering processing on the positive sample data and negative sample data respectively to construct the features required for training the ESMM model. For example, extract brand portrait features, brand behavior features, user portrait features, user statistics features and behavior sequence features for the brand, cross features of users and brands, etc. from the positive sample data and negative sample data respectively. It can be understood that the specific implementation manner of the above feature extraction is similar to or the same as the implementation manner of step S122 above, so it will not be elaborated here. In the specific practice process, the specific implementation manner of the above feature extraction may also include: discretization of numerical features, enumeration of feature values, one-hot encoding, pooling method for multi-value features, etc.

[0069] During the training process of the ESMM model, the following loss function formula can be adopted: Train the ESMM model to obtain the trained ESMM model. Among them, L(Θ CTR , Θ CVR ) represents the loss value calculated based on the sample features extracted from the sample data, Θ CTR represents the network parameters of the click-through rate prediction module (such as W 3a , W 2a , W 1a ), Θ CVR represents the network parameters of the conversion rate prediction module (such as W 3b , W 2b , W 1b ), for the i-th sample data, x i represents the sample features of the i-th sample data, y i is the click label of the i-th sample data, z i is the conversion label of the i-th sample data, and l(y i , f(x i ; Θ CTR )) represents the cross-entropy loss function of the i-th sample data.

[0070] It can be understood that the calculation process of the click-through rate prediction module in the above ESMM model is, for example:

[0071]

[0072] The calculation process of the conversion rate prediction module in the above ESMM model is, for example:

[0073]

[0074] Among them, pCTVR represents the click-through conversion rate, pCTR represents the click-through rate, pCVR represents the conversion rate. For the i-th sample data, x i represents the sample feature of the i-th sample data, and y i is the click label of the i-th sample data, and z i is the conversion label of the i-th sample data. Θ CTR represents the network parameters of the click-through rate prediction module, that is, W 3a , W 2a , W 1a are all network parameters of the click-through rate prediction module. Θ CVR represents the network parameters of the conversion rate prediction module, that is, W 3b , W 2b , W 1b are all network parameters of the conversion rate prediction module. Sigmoid and Relu are both activation functions.

[0075] In the implementation process of the above solution, by introducing an advertising traffic quality evaluation method to predict the quality of traffic data and determining resource input information according to the predicted quality score, an intelligent advertising bidding mechanism is realized, which can avoid advertising to low-quality traffic (such as robot traffic, low-conversion-rate traffic), thereby reducing ineffective exposures and clicks, avoiding waste of advertising budgets, helping to improve the satisfaction of advertisers, and also providing more targeted advertisements for users and enhancing the user experience.

[0076] Please refer to Figure 3 the flowchart of the resource input determination method provided by the embodiment of the present application shown; The embodiment of the present application provides a resource input determination method, including:

[0077] Step S310: Obtain traffic data sent by the advertising platform server.

[0078] It can be understood that the implementation manner of the above step S310 is the same as that of the above step S110, so it will not be elaborated here.

[0079] Step S320: Use the advertising traffic quality evaluation method described above to perform quality prediction on the traffic data to obtain a predicted quality score.

[0080] The implementation manner of the above step S320 is the same as that of the above steps S120 to S130, that is, the implementation manner of using the above-described advertising traffic quality evaluation method to perform quality prediction on traffic data includes: using the entire-space multi-task network ESMM model to perform joint prediction on traffic data to obtain click-through rate and conversion rate, then calculating click-through conversion rate based on the click-through rate and conversion rate, and determining the predicted quality score based on the click-through conversion rate. Therefore, if there is any unclear point, reference can be made to the description of the implementation manner of steps S120 to S130 above.

[0081] Step S330: Determine resource input information according to the predicted quality score.

[0082] The above-mentioned resource input information includes: computing resource input information, storage resource input information, and / or bid information; among them, the computing resource input information refers to the computing resource information allocated for this traffic data on the advertiser server (such as the number of CPU cores or the number of GPU cores, etc.), the storage resource input information refers to the storage resource information allocated for this traffic data on the advertiser server, such as memory size and video memory size, etc., and the bid information refers to the price information that the advertiser is willing to pay to the advertising platform for an advertising display opportunity according to the predicted quality score, and this bid information is usually sent from the advertiser server to the advertising platform server.

[0083] Optionally, after determining the resource input information according to the predicted quality score, the advertiser server can also send the resource input information to the advertising platform server so that the advertising platform server can determine whether to display the advertisement corresponding to the bid information. It can be understood that the above-mentioned resource input information is that the above-mentioned advertising platform server can also be responsible for receiving the advertiser's bid information, processing advertising display requests, and deciding whether to display an advertisement according to factors such as the bid and the advertising quality score. For example: Suppose an advertiser submits an advertisement with a bid information of 0.5 yuan. After receiving this information, the advertising platform server will decide whether to display this advertisement when the user searches for "mobile phone" according to factors such as the current bidding situation and the advertising quality score.

[0084] As an alternative implementation manner of the above step S330, the above-mentioned resource input information may include: final bid information; the implementation manner of the above-mentioned determining resource input information according to the predicted quality score may include:

[0085] Step S331: Obtain the average quality score.

[0086] The above-mentioned average quality score can be calculated by the formula where, Let \(\overline{q}\) denote the average quality score, \(N\) denote the total number of participating users, \(x\) denote the sample feature, \(y\) denote the click label, \(z\) denote the conversion label, and \(p(y = 1|,z = 1|x)\) denote the click-through conversion rate (i.e., pCTVR).

[0087] Step S332: Divide the predicted quality score by the average quality score to obtain the bid coefficient.

[0088] An implementation manner of the above step S332 is, for example: using the calculation formula of the bid coefficient Divide the predicted quality score by the average quality score to obtain the bid coefficient; where, the above \(\beta\), \(\gamma\), \(\lambda\) are all hyperparameters calculated by searching in the data offline. \(\beta\) represents the upper bound parameter of the bid coefficient, and its range can be [1, 10), for example, \(\beta\) is taken as 1.3. \(\gamma\) represents the lower bound parameter of the bid coefficient, and \(\lambda\) represents the discount factor parameter of the bid coefficient. The ranges of \(\gamma\) and \(\lambda\) can both be (0, 1], for example, \(\gamma\) is taken as 0.7, and \(\lambda\) is taken as 0.95.

[0089] Step S333: Parse the initial price of the advertising platform server from the traffic data, and multiply the initial price by the bid coefficient to obtain the final bid information.

[0090] An implementation manner of the above step S333 is, for example: Assume that the initial price parsed from the traffic data for the advertising platform server is 0.5 yuan, and the bid coefficient obtained by dividing the predicted quality score by the average quality score is 0.7. Then the final bid information for the advertiser server for this advertising display request is 0.5 × 0.7 = 0.35 yuan.

[0091] Please refer to Figure 4 the structural schematic diagram of the advertising traffic quality evaluation device provided by the embodiment of the present application shown; The embodiment of the present application provides an advertising traffic quality evaluation device 400, including:

[0092] A traffic data acquisition module 410, configured to acquire traffic data sent by the advertising platform server.

[0093] A data joint prediction module 420, configured to perform joint prediction on the traffic data using the entire space multi-task network ESMM model to obtain the click-through rate and the conversion rate.

[0094] A predicted quality determination module 430, configured to calculate the click-through conversion rate according to the click-through rate and the conversion rate, and determine the predicted quality score according to the click-through conversion rate.

[0095] As an optional implementation manner of the above device, the data joint prediction module includes:

[0096] A sample feature extraction sub-module, which is used to extract sample features from traffic data, and the sample features include brand portrait features, user portrait features, and user behavior sequence features.

[0097] A traffic data prediction sub-module, which is used to input the brand portrait features, user portrait features, and user behavior sequence features into the ESMM model, and obtain the click-through rate and conversion rate jointly predicted by the ESMM model according to the traffic data.

[0098] As an alternative implementation of the above device, the ESMM model includes: a click-through rate prediction module and a conversion rate prediction module; the traffic data prediction sub-module includes:

[0099] A click-through rate determination unit, which is used to use the click-through rate prediction module to determine the click-through rate according to the brand portrait features, user portrait features, and user behavior sequence features.

[0100] A conversion rate determination unit, which is used to use the conversion rate prediction module to determine the conversion rate according to the brand portrait features, user portrait features, and user behavior sequence features.

[0101] As an alternative implementation of the above device, the advertisement traffic quality evaluation device further includes:

[0102] A sample data acquisition module, which is used to acquire positive sample data and negative sample data. The positive sample data is the user behavior data of users who place orders in the terminal corresponding to the advertisement platform server, and the negative sample data is the user behavior data of users who do not place orders in the terminal corresponding to the advertisement platform server.

[0103] An ESMM model training module, which is used to train the ESMM model using the positive sample data and the negative sample data.

[0104] An embodiment of the present application provides an advertisement resource investment determination device, including:

[0105] A traffic data acquisition module, which is used to acquire traffic data sent by the advertisement platform server.

[0106] A traffic quality prediction module, which is used to use the above advertisement traffic quality evaluation method to predict the quality of traffic data and obtain a predicted quality score.

[0107] A bid information determination module, which is used to determine resource investment information according to the predicted quality score.

[0108] As an alternative implementation of the above device, the resource investment information includes: final bid information; the resource investment information determination module includes:

[0109] A quality score acquisition sub-module, which is used to acquire an average quality score.

[0110] The bid coefficient determination submodule is used to divide the predicted quality score by the average quality score to obtain the bid coefficient.

[0111] The final bid acquisition submodule is used to parse the initial price of the advertising platform server from the traffic data, and multiply the initial price by the bid coefficient to obtain the final bid information.

[0112] It should be understood that the device corresponds to the above-mentioned advertising traffic quality evaluation method embodiment and can execute the various steps involved in the above-mentioned method embodiment. The specific functions of the device can be found in the description above, and the detailed description is appropriately omitted here. The device includes at least one software function module that can be stored in a memory in the form of software or firmware or solidified in the operating system (OS) of the device.

[0113] See also Figure 5 An electronic device 500 provided in an embodiment of the present application includes: a processor 510 and a memory 520, wherein the memory 520 stores machine-readable instructions executable by the processor 510, and when the machine-readable instructions are executed by the processor 510, the above method is executed.

[0114] The embodiment of the present application also provides a computer-readable storage medium 530, on which a computer program is stored, and the computer program is executed by the processor 510 to execute the above method. The computer-readable storage medium 530 can be implemented by any type of volatile or non-volatile storage device or a combination thereof, such as static random access memory (Static Random Access Memory, referred to as SRAM), electrically erasable programmable read-only memory (Electrically Erasable Programmable Read-Only Memory, referred to as EEPROM), erasable programmable read-only memory (Erasable Programmable Read Only Memory, referred to as EPROM), programmable read-only memory (Programmable Read-Only Memory, referred to as PROM), read-only memory (Read-Only Memory, referred to as ROM), magnetic memory, flash memory, disk or optical disk.

[0115] The embodiment of the present application also provides a computer program product, including: a computer program or a computer instruction, and the computer program or the computer instruction executes the method described above when executed by a processor.

[0116] It should be noted that the various embodiments in this specification are described in a progressive manner. Each embodiment focuses on the differences from other embodiments. For the same or similar parts among the embodiments, reference can be made to each other. For device embodiments, since they are basically similar to method embodiments, the description is relatively simple. For related parts, reference can be made to the corresponding description in the method embodiments.

[0117] In several embodiments provided by the embodiments of the present application, it should be understood that the disclosed devices and methods can also be implemented in other ways. The device embodiments described above are only illustrative. For example, the flowcharts and block diagrams in the accompanying drawings show the possible architectures, functions, and operations of the devices, methods, and computer program products according to multiple embodiments of the present application. In this regard, each block in the flowchart or block diagram may represent a module, a program segment, or a part of code. A module, a program segment, or a part of code contains one or more executable instructions for implementing the specified logical function. It should also be noted that in some alternative implementations, the functions marked in the blocks may also occur in a different order from that marked in the accompanying drawings. For example, two consecutive blocks can actually be executed substantially in parallel, and they can sometimes be executed in the reverse order, which mainly depends on the functions involved.

[0118] In addition, in each embodiment of the embodiments of the present application, the various functional modules can be integrated together to form an independent part, or each module can exist separately, or two or more modules can be integrated to form an independent part. Moreover, in the description of this specification, the descriptions with reference to terms such as "one embodiment", "some embodiments", "example", "specific example", "some examples", etc. mean that the specific features, structures, materials, or characteristics described in connection with the embodiment or example are included in at least one embodiment or example of the embodiments of the present application. In this specification, the schematic expressions of the above terms do not necessarily refer to the same embodiment or example. Moreover, the specific features, structures, materials, or characteristics described can be combined in a suitable manner in any one or more embodiments or examples. In addition, without contradiction, those skilled in the art can combine and combine the different embodiments or examples described in this specification and the features of different embodiments or examples.

[0119] The above description is only an optional implementation manner of the embodiments of the present application, but the protection scope of the embodiments of the present application is not limited thereto. Any person skilled in the art can easily think of changes or substitutions within the technical scope disclosed in the embodiments of the present application, and all should be covered by the protection scope of the embodiments of the present application.

Claims

1. A method for evaluating the quality of advertising traffic, characterized in that: include: Get the traffic data sent by the advertising platform server; Use the full-space multi-task network ESMM model to jointly predict the traffic data to obtain the click rate and conversion rate; A click-through conversion rate is calculated according to the click-through rate and the conversion rate, and a predicted quality score is determined according to the click-through conversion rate.

2. The method according to claim 1, characterized in that The method of using the full-space multi-task network ESMM model to jointly predict the traffic data to obtain the click rate and conversion rate includes: Extracting sample features from the traffic data, wherein the sample features include brand portrait features, user portrait features, and user behavior sequence features; The brand portrait features, the user portrait features, and the user behavior sequence features are input into the ESMM model to obtain the click rate and the conversion rate output by the ESMM model based on the joint prediction of the traffic data.

3. The method according to claim 2, characterized in that The ESMM model includes: a click rate estimation module and a conversion rate estimation module; the step of inputting the brand portrait feature, the user portrait feature, and the user behavior sequence feature into the ESMM model includes: Determine the click rate using the click rate estimation module according to the brand portrait feature, the user portrait feature, and the user behavior sequence feature; The conversion rate estimation module is used to determine the conversion rate according to the brand portrait features, the user portrait features, and the user behavior sequence features.

4. The method according to claim 1, characterized in that Before the full-space multi-task network ESMM model is used to jointly predict the traffic data, the method further includes: Acquire positive sample data and negative sample data, wherein the positive sample data is user behavior data of placing an order in a terminal corresponding to the advertising platform server, and the negative sample data is user behavior data of not placing an order in a terminal corresponding to the advertising platform server; The ESMM model is trained using the positive sample data and the negative sample data.

5. A method for determining resource investment, characterized in that: include: Get the traffic data sent by the advertising platform server; Use the advertisement traffic quality assessment method described in claim 1 to perform quality prediction on the traffic data to obtain a prediction quality score; Resource investment information is determined based on the predicted quality score.

6. The method according to claim 5, characterized in that The resource input information includes: final bid information; the resource input information determined according to the predicted quality score includes: Get the average quality score; Dividing the predicted quality score by the average quality score to obtain a bid coefficient; The initial price of the advertising platform server is parsed from the traffic data, and the initial price is multiplied by the bid coefficient to obtain the final bid information.

7. An advertisement flow quality evaluation device, characterized in that: include: A traffic data acquisition module is used to acquire traffic data sent by the advertising platform server; A data joint prediction module, used to use a full-space multi-task network ESMM model to jointly predict the traffic data to obtain a click rate and a conversion rate; The prediction quality determination module is used to calculate the click-through conversion rate according to the click-through rate and the conversion rate, and determine the prediction quality score according to the click-through conversion rate.

8. An electronic device, characterized in that: include: A processor and a memory, wherein the memory stores machine-readable instructions executable by the processor, and the machine-readable instructions are executed by the processor to perform any method according to claims 1 to 6.

9. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the method according to any one of claims 1 to 6 is executed.

10. A computer program product, characterized in that include: A computer program or a computer instruction, wherein when the computer program or the computer instruction is executed by a processor, the method according to any one of claims 1 to 6 is executed.

Images