Promote data processing methods, model training methods, systems and storage media

Abstract

The present application discloses a promotion data processing method, a model training method, a system, and a storage medium, relating to the field of artificial intelligence. The method comprises the following steps: obtaining a first parameter group and a second parameter group, inputting the first parameter group into a machine learning model to obtain a first prediction result, the first prediction result being used to characterize whether the conversion data of the target object is abnormal data; inputting the second parameter group into a statistical model to obtain a second prediction result, the second prediction result being used to characterize whether the conversion data of the target object is abnormal data; when both the first prediction result and the second prediction result characterize the conversion data of the target object as abnormal data, the conversion data of the target object is removed from the training data set of the conversion rate prediction model. This solution can improve the quality of the training data of the conversion rate prediction model and enhance the prediction accuracy of the conversion rate prediction model.

Description

Technical Field

[0001] The present application relates to the field of artificial intelligence, and in particular to a generalized data processing method, model training method, system, and storage medium. Background Art

[0002] The promotional message delivery system adjusts the exposure of a user based on the conversion rate of their messages. In some cases, when the system determines that a user's promotional message has achieved good conversion results based on the conversion data reported by the user, it will allocate better promotional resources to increase its exposure, for example, by placing the message in a promotional location with particularly high traffic. Alternatively, if the system determines that a user's promotional message has achieved particularly poor conversion results, it will reduce its exposure, for example, by removing it from most promotional locations. However, due to various reasons such as data transmission and system failures, messages reported by users may contain errors or omissions. Therefore, identifying anomalies in conversion rate data is particularly important for the conversion effectiveness and smooth operation of the promotional message delivery system. In related technologies, promotional message delivery systems typically detect year-on-year and month-on-month changes in conversion rates. When the year-on-year and month-on-month changes meet certain conditions, the conversion rate data is determined to be anomaly. However, the thresholds of these methods are usually fixed, resulting in relatively low recognition accuracy.

[0003] In the promotional message delivery system, conversion rate data is used to train the conversion rate prediction model. If these abnormal data are not identified or normal data are misjudged as abnormal data, the quality of the training data of the conversion rate prediction model will be reduced, which will in turn lead to a decrease in the prediction accuracy of the conversion rate prediction model, affecting the smooth operation of the promotional message delivery system. Summary of the Invention

[0004] In order to solve at least one of the above technical problems, the present application provides a promotion data processing method, a model training method, a system and a storage medium, which can improve the accuracy of abnormal data identification to optimize the quality of training data of the conversion rate prediction model.

[0005] According to one aspect of the present application, a promotion data processing method is provided, comprising the following steps:

[0006] Obtaining a first parameter group and a second parameter group, wherein the first parameter group includes multiple statistical characteristics of a target object, the statistical characteristics being obtained by counting the target object's conversion volume time series data and / or click volume time series data, wherein the target object is a promotion account or a promotional message delivered by the promotion account; and the second parameter group includes the target object's estimated conversion volume, current conversion volume, and current click volume;

[0007] Inputting the first parameter group into a machine learning model to obtain a first prediction result, where the first prediction result is used to characterize whether the conversion data of the target object is abnormal data;

[0008] Inputting the second parameter group into a statistical model to obtain a second prediction result, wherein the second prediction result is used to indicate whether the conversion data of the target object is abnormal data;

[0009] When both the first prediction result and the second prediction result indicate that the conversion data of the target object is abnormal data, the conversion data of the target object is removed from the training data set of the conversion rate prediction model.

[0010] According to another aspect of the present application, a promotion data processing method is provided, comprising the following steps:

[0011] Obtaining multiple first parameter groups and second parameter groups corresponding to the first parameter groups, wherein the first parameter groups include multiple statistical features of a target object, the statistical features being obtained by counting time series data of conversions and / or time series data of clicks of the target object, the target object being a promotion account or a promotional message delivered by the promotion account; and the second parameter group including an estimated conversion volume, current conversion volume, and current click volume of the target object;

[0012] Inputting the plurality of first parameter groups into a machine learning model to obtain a first prediction result corresponding to each of the first parameter groups, wherein the first prediction result is used to indicate whether the conversion data of the target object is abnormal data;

[0013] adding the target object represented by the first prediction result as abnormal data to a first abnormal data set;

[0014] Inputting the plurality of second parameter groups into a statistical model to obtain a second prediction result corresponding to each second parameter group, wherein the second prediction result is used to indicate whether the conversion data of the target object is abnormal data;

[0015] adding the target object represented by the second prediction result as abnormal data to a second abnormal data set;

[0016] Intersecting the first abnormal data set and the second abnormal data set to obtain a third abnormal data set;

[0017] The conversion data of the target object in the third abnormal data set is removed from the training data set of the conversion rate prediction model.

[0018] According to another aspect of the present application, a method for training a conversion rate prediction model is provided, comprising the following steps:

[0019] Acquire a first training data set for the conversion rate prediction model, where the first training data set includes conversion data of a plurality of target objects;

[0020] Processing the conversion data in the training data set by the above-mentioned promotion data processing method to obtain a second training data set;

[0021] The conversion rate prediction model is trained based on the second training data set.

[0022] According to another aspect of the present application, a promotion data processing system is provided, comprising:

[0023] a first acquisition unit configured to acquire a first parameter group and a second parameter group, wherein the first parameter group includes multiple statistical characteristics of a target object, the statistical characteristics being obtained by counting the target object's conversion volume time series data and / or click volume time series data, wherein the target object is a promotion account or a promotional message delivered by the promotion account; and the second parameter group includes the target object's estimated conversion volume, current conversion volume, and current click volume;

[0024] a first prediction unit, configured to input the first parameter group into a machine learning model to obtain a first prediction result, wherein the first prediction result is used to indicate whether the conversion data of the target object is abnormal data;

[0025] a second prediction unit, configured to input the second parameter group into a statistical model to obtain a second prediction result, wherein the second prediction result is used to indicate whether the conversion data of the target object is abnormal data;

[0026] The first data processing unit is configured to remove the conversion data of the target object from a training data set of a conversion rate prediction model when both the first prediction result and the second prediction result indicate that the conversion data of the target object is abnormal data.

[0027] According to another aspect of the present application, a training system for a conversion rate prediction model is provided, comprising:

[0028] A second acquisition unit is configured to acquire a first training data set for the conversion rate prediction model, where the first training data set includes conversion data of a plurality of target objects;

[0029] a second data processing unit, configured to process the conversion data in the training data set using the above-mentioned promotion data processing method to obtain a second training data set;

[0030] A training unit is used to train the conversion rate prediction model according to the second training data set.

[0031] According to another aspect of the present application, a data processing system is provided, comprising: a program; a memory for storing the program; and a processor for loading the program to execute the above-mentioned promotion data processing method or conversion rate prediction model training method.

[0032] According to another aspect of the present application, a computer-readable storage medium is provided, in which a program is stored. When the program is executed by a processor, the above-mentioned promotion data processing method or conversion rate prediction model training method is implemented.

[0033] The beneficial effect of the embodiment of the present application is that the feature extraction of the first parameter group is performed by the machine learning model, and the machine learning model can extract the hidden features of the click volume time series data and / or the conversion volume time series data through the first parameter group, thereby predicting whether the conversion rate data of the target object is abnormal; the estimated conversion volume, current conversion volume and current click volume based on the target object are predicted by the statistical model, and it can also be judged whether there is an abnormality in the current conversion data of the target object. Combining the two models and making anomaly judgments from different data dimensions can improve the prediction accuracy. On this basis, these abnormal data are removed from the training data set of the conversion rate prediction model, which can reduce the proportion of abnormal data in the training data set, thereby improving the quality of the training data set, thereby making the prediction model of the promotion message delivery platform more accurate, and the delivery effect of the promotion message better, which can promote the smooth operation of the promotion message delivery platform. BRIEF DESCRIPTION OF THE DRAWINGS

[0034] Figure 1 This is a flowchart of a promotion data processing method provided according to an embodiment of the present application;

[0035] Figure 2 A framework diagram of a promotional message delivery system provided according to an embodiment of the present application;

[0036] Figure 3 A schematic diagram of the structure of a neural network model provided according to an embodiment of the present application;

[0037] Figure 4 A schematic diagram of the structure of a machine learning model provided according to an embodiment of the present application;

[0038] Figure 5 A schematic diagram of the structure of another machine learning model provided according to an embodiment of the present application;

[0039] Figure 6 A schematic diagram of the structure of another machine learning model provided according to an embodiment of the present application;

[0040] Figure 7A flowchart of sub-steps of step 120 provided according to an embodiment of the present application;

[0041] Figure 8 A schematic diagram of the structure of a gradient boosting iterative decision tree model provided according to an embodiment of the present application;

[0042] Figure 9 A flow chart of a training method for a gradient boosting iterative decision tree model according to an embodiment of the present application;

[0043] Figure 10 A flowchart of a training method for a first neural network model provided according to an embodiment of the present application;

[0044] Figure 11 A schematic diagram of setting a whitelist for operators according to an embodiment of the present application;

[0045] Figure 12 A flowchart of sub-steps of step 130 provided according to an embodiment of the present application;

[0046] Figure 13 A flowchart of a method for obtaining a first training sample according to an embodiment of the present application;

[0047] Figure 14 A flowchart of another promotion data processing method provided according to an embodiment of the present application;

[0048] Figure 15 A flowchart of a method for training a conversion rate prediction model according to an embodiment of the present application;

[0049] Figure 16 A block diagram of a promotion data processing system according to an embodiment of the present application;

[0050] Figure 17 A module block diagram of another promotion data processing system provided according to an embodiment of the present application;

[0051] Figure 18 A module block diagram of a training system for a conversion rate prediction model provided according to an embodiment of the present application;

[0052] Figure 19 The following is a schematic diagram of the working principle of a promotion data processing system provided according to an embodiment of the present application. DETAILED DESCRIPTION

[0053] In the following description, reference is made to “some embodiments”, which describes a subset of all possible embodiments, but it will be understood that “some embodiments” may be the same subset or different subsets of all possible embodiments and may be combined with each other without conflict.

[0054] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by those skilled in the art to which this application pertains. The terms used herein are for the purpose of describing the embodiments of this application only and are not intended to limit this application.

[0055] Before further describing the embodiments of the present application in detail, the nouns and terms involved in the embodiments of the present application are explained. The nouns and terms involved in the embodiments of the present application are subject to the following interpretations.

[0056] GBDT (Gradient Boosting Decision Tree) is a gradient boosting iterative decision tree. Its main idea is that each time a classifier is built, it descends the gradient of the loss function of the previously built model. GBDT goes through multiple rounds of iterations, each generating a weak classifier. Each classifier is trained based on the residuals of the previous round's classifier. The final overall classifier is the weighted sum of the weak classifiers trained in each round.

[0057] Consumption: The amount spent by the promoted user on delivering promotional messages.

[0058] Recall Ratio: In machine learning, recall is calculated using the formula TP / (TP + FN), where TP refers to the number of samples that were originally positive and predicted as positive by the model; FN refers to the number of samples that were originally positive and predicted as negative by the model.

[0059] Precision: In machine learning, accuracy is calculated using the formula: TP / (TP + FP), where TP refers to the number of samples that were originally positive and predicted as positive by the model; FP refers to the number of samples that were originally negative and predicted as positive by the model. Both recall and precision can be used to describe a model's accuracy.

[0060] oCPA advertising: short for Optimized Cost per Action, involves bidding based on optimization goals and charging per click or impression. When advertisers select a specific optimization goal (e.g., mobile app activations, website purchases) during the ad delivery process, provide an average price they're willing to pay for that goal, and promptly and accurately transmit performance data, the system leverages a conversion rate prediction model to estimate the conversion value of each click for the advertiser in real time, automatically setting a bid and ultimately charging per click. Furthermore, the conversion rate prediction model within the promotional message delivery system continuously optimizes based on the advertiser's conversion data.

[0061] oCPA advertising is achieved. When advertising consumption / conversion volume>= advertising target price*0.8, the advertising cost is considered to have been achieved.

[0062] CTR (Click Through Rate): This refers to the click-through rate of online promotional messages. It is calculated by dividing the actual number of clicks on the promotional message by the number of impressions of the promotional message. It is a key indicator for measuring the effectiveness of online promotional messages.

[0063] PCTR (Predict Click Through Rate) is the estimated click-through rate (PCTR) of a promotional message delivered under certain circumstances. It is an important component of the ranking model and represents the click-through rate model in the featured ranking process in a multi-level ranking model. The model is highly complex.

[0064] Bid: When placing an ad, a user hopes to obtain a single conversion at a certain cost, where a certain bid is defined as a Bid.

[0065] CVR (Conversion Rate): This is also an indicator to measure the effectiveness of promotional messages. It refers to the conversion rate from users clicking on promotional messages to becoming effectively activated, registered, or paying users. That is, the actual number of conversions of the promotional message divided by the number of clicks on the promotional message.

[0066] pCVR (Predict Conversion Rate): pCVR is the online promotional message delivery system's estimate of the probability of a conversion occurring after a promotional message is clicked in a certain situation. It is an important component of the ranking model and represents the conversion rate model in the featured ranking in a multi-level ranking model. The model is highly complex.

[0067] Conversion anomalies: This refers to situations where the promotion conversion volume does not meet expectations due to factors such as incorrect conversion data uploaded by the promotion user or errors in the advertising attribution logic. For example, the conversion volume is 0, or the conversion volume exceeds the click volume. Advertising attribution is the process of reversely correlating the conversion data uploaded by the customer with the click and exposure data of the advertising system to track the effectiveness of advertising.

[0068] Promotional messages: These are messages that present information about products and services to the audience, or have the potential to encourage consumption or behavior. For example, promotional messages can include commercial advertisements, public service announcements, and messages calling for audience participation in specific activities or policies.

[0069] One-hot encoding is a method for mapping a discrete feature into a Euclidean space. If the offline feature has N enumeration values, the resulting vector is typically an N-dimensional vector, with the element at the position corresponding to the current value set to 1 and all other positions set to 0.

[0070] Artificial intelligence (AI) refers to the theories, methods, techniques, and application systems that use digital computers or machines controlled by digital computers to simulate, extend, and expand human intelligence, to perceive the environment, acquire knowledge, and use that knowledge to achieve optimal results. In other words, AI is a comprehensive field within computer science that seeks to understand the essence of intelligence and produce new intelligent machines that can respond in a manner similar to human intelligence. AI also studies the design principles and implementation methods of various intelligent machines, enabling them to possess the capabilities of perception, reasoning, and decision-making. AI technology is an interdisciplinary discipline encompassing a wide range of fields, encompassing both hardware and software technologies. Foundational AI technologies generally include sensors, specialized AI chips, cloud computing, distributed storage, big data processing, operating / interaction systems, and mechatronics. AI software technologies primarily encompass computer vision, speech processing, natural language processing, and machine learning / deep learning.

[0071] The artificial intelligence technology of this application is primarily applied to the processing of promotional data. In the related art, the identification of conversion data anomalies is achieved through simple threshold comparison. This approach has relatively low prediction accuracy, lacks differentiated threshold settings for different users, and often fails to identify anomalies such as slow declines / increases, resulting in numerous drawbacks. Inaccurate predictions of abnormal data can lead to "dirty data" entering the training set of the conversion rate prediction model, causing the conversion rate prediction model to learn incorrect samples, thereby reducing model accuracy. In a promotional message delivery system, inaccurate conversion rate predictions for promotional messages can lead to the removal of high-quality promotional messages or the allocation of higher-quality resources (e.g., homepage ad placement) to lower-quality promotional messages. As such situations increase, the overall effectiveness of the promotional message delivery platform deteriorates, and promotional users' delivery results in poor conversion rates. Therefore, this solution provides a promotional data processing method that combines machine learning techniques, such as machine learning models and statistical models, to more accurately identify abnormal conversion data. This technology is applied to the training data cleaning of the conversion rate prediction model, reducing the proportion of abnormal data in the training samples and improving the quality of the training data for the conversion rate prediction model. Therefore, the conversion rate prediction model can obtain improved prediction accuracy through high-quality training data. It can be seen that the solution of this application helps improve the performance of the promotion message delivery system in conversion rate prediction and promotes the smooth operation of the promotion message delivery system.

[0072] Reference Figure 1 , which shows a promotion data processing method of this embodiment. The method of this embodiment is applied in Figure 2The promotional message delivery system shown in FIG. In this embodiment, the promotional message delivery system includes a user's terminal 210, a server 220 of the promotional message delivery system, and a media terminal 230. It is understood that terminal 210 can be a device such as a mobile phone, tablet computer, or personal computer. Server 220 can be a single physical server, a distributed server consisting of multiple servers, or a cloud server provided by a cloud service provider. Media terminal 230 can be a device such as a mobile phone, tablet computer, personal computer, or electronic advertising screen of audience 240. It is understood that terminal 210, server 220, and media terminal 230 can communicate via the Internet, a local area network, or the Internet of Things. Specifically, the method of this embodiment is applied to server 220. The promotional message delivery system operates as follows: First, a user publishes a promotion request to the promotional message delivery system through terminal 210. This request typically includes the promotional message itself, the target audience (region, age, gender, or occupation, etc.), and a bid. The promotional message delivery system allocates inventory (in the advertising industry, the amount of time an ad is visible to an audience is called inventory) to the advertiser based on the advertiser's promotional needs and various delivery models, thereby encouraging audience 240 (i.e., viewers of the ad) to click, download, or register. These behaviors of audience 240 serve as the basis for charging for certain promotional models of the promotional message delivery platform. For example, the advertiser's conversion goal is to have audience 240 register for an account, and they offer a price of 5 yuan per conversion. The promotional message delivery platform might deliver this promotional message 1,000 times to several media terminals 230. Assuming a 10% conversion rate, meaning 100 audience members 240 register, the advertiser would pay the promotional message delivery platform 500 yuan. The promotional message delivery system then displays this information to audience 240 through media terminals 230. These media terminals 230 are equipped with various software, which are essentially media. These software can be music, shopping, communication, payment, or any other media capable of displaying promotional messages. The operation of the promotional message delivery system is relatively complex. For example, in the oCPX bidding advertising model, the key formula that determines whether an ad can be exposed is: In this scenario, Based on customer bids and estimated conversion rates , estimated click-through rate and the influence of weight β. It can be seen that the estimated conversion rate affects whether the promotion message can be exposed, and exposure is the prerequisite for conversion. If the estimated conversion rate is inaccurate, it will lead to poor overall promotion message delivery effect of the promotion message delivery system and system operation fluctuations. If the accuracy of the estimated conversion rate of promotion messages needs to be improved, in addition to improving the structure of the conversion rate prediction model, it is also necessary to improve the data quality of the model training data. Therefore, Figure 1 The method in

[15] was used to eliminate abnormal conversion data, thereby improving the quality of training data for the conversion rate prediction model.

[0073] like Figure 1 As shown, the promotion data processing method of this embodiment includes steps 110 to 140.

[0074] Step 110: Obtain a first parameter group and a second parameter group. The first parameter group includes multiple statistical characteristics of the target object, and the statistical characteristics are obtained through the conversion volume time series data and / or click volume time series data of the target object. The target object is a promotion account or a promotion message delivered by a promotion account; the second parameter group includes the estimated conversion volume, current conversion volume and current click volume of the target object.

[0075] It should be understood that both the first and second parameter groups are combinations of parameters, each including multiple parameters of different types. The first parameter group can be represented as a vector, for example, as (a1, a2, ..., an). Here, a1, a2, ..., an each represent a different type of parameter, and n is a positive integer. Similarly, the second parameter group can also be represented as a vector, for example, as (b1, b2, ..., bm), where m is a positive integer. For example, if m=3, b1 can represent the target object's estimated conversion volume (pCVR), b2 can represent the current conversion volume (CVR), and b3 can represent the current click-through rate (CTR). It should be understood that the statistical features in the first parameter group primarily describe the target object's attributes, including click data and conversion data. For example, attributes include the target object's industry and the region of placement. When the target object is the promotional message itself, the target object's attributes can also include content category (animation, movies, cars, etc.) and format category (print ads, video ads, interactive ads, etc.). When the target is a promoted account, its attributes can also include brand awareness (e.g., whether it's a major brand). When describing the target's click and conversion data, metrics such as year-over-year click-through rate, month-over-month click-through rate, year-over-year conversion rate, month-over-month conversion rate, and reach (click-through rate reach, conversion rate reach, etc.) can be used. It's understandable that the above data is derived from time-series data on conversion volume and click volume, and is therefore also referred to as statistical features. These statistical features can, to a certain extent, reflect the time-series characteristics of abnormal data. By performing feature extraction on this data, we can extract the implicit characteristics of abnormal data in the time-series dimension.

[0076] Specifically, the above statistical features can include the following seven categories: enumeration features, continuous features, confidence features, pCTR, pCVR features, year-on-year features, opt (optimization_goal optimization target) statistical features, and achievement features.

[0077] Enumeration features primarily reflect the attributes of the target object and are typically encoded using one-hot encoding. These features can include industry attributes, content attributes, classification methods, or processing time. Because the attributes of the target object are not linear data, they must be encoded. One-hot encoding is performed as follows. For example, if an attribute has three possible values—1, 2, and 3—then 1 is encoded into the vector (1, 0, 0), 2 into the vector (0, 1, 0), and 3 into the vector (0, 0, 1).

[0078] Continuous features are primarily used to reflect the effectiveness of promotional campaigns. They typically use uniform units or dimensionless data to make data comparable across different promotional accounts. These continuous features often fluctuate significantly when anomalies occur. Examples of continuous features include click-through rate (CTR) and conversion rate (CTR) time series.

[0079] Confidence features are used to measure the trustworthiness of other derived statistics. A larger sample size reduces the representativeness error of the statistical estimate, while a smaller sample size increases the error. These features typically include clicks, impressions, conversions, conversions at the same hour yesterday minus conversions at the same hour today, and clicks at the same hour yesterday minus clicks at the same hour today.

[0080] pCTR and pCVR features generally refer to the relationship between pCTR and pCVR and actual data. This category of features includes pCVR-bias (also known as conversion rate prediction bias) and pCTR-bias (also known as click-through rate prediction bias). Among them, pCVR-bias = (actual conversion volume / estimated conversion volume) - 1, and pCTR-bias = (actual click volume / estimated click volume) - 1.

[0081] Year-on-year and month-on-month features mainly count the year-on-year and month-on-month characteristics of conversion rate and click-through rate. For example, the conversion rate ratio between the same hour yesterday and the same hour today, and the click-through rate ratio between the same hour yesterday and the same hour today. In this example, the same hour refers to the same time period, for example, 5:00 PM to 6:00 PM yesterday corresponds to 5:00 PM to 6:00 PM today.

[0082] The opt statistical features are mainly some horizontal comparison features under the same optimization goal, and usually compare the data of a target object with the average values ​​of other target objects and other statistical figures. The optimization goal can be the number of registrations, downloads, or the amount of consumption generated, etc. In this embodiment, the opt statistical features include the ratio of the conversion rate to the average conversion rate under the same optimization goal conditions, the ratio of the conversion rate to the average conversion rate under the same optimization goal conditions in the same industry, the ratio of the average conversion rate of the previous hour to the current hour under the same optimization goal conditions, the ratio of the average click-through rate of the previous hour to the current hour under the same optimization goal conditions, the ratio of the average conversion rate of the previous day to the current day under the same optimization goal conditions in the same industry, and the ratio of the average click-through rate of the previous day to the current day under the same optimization goal conditions in the same industry, etc.

[0083] Achievement features are parameters used to measure the achievement of optimization goals, including the current day's achievement rate and the ratio of the previous day's achievement rate to the current day's achievement rate for the same hour. The achievement rate can be calculated based on the achievement data of multiple promotional messages.

[0084] It should be understood that the selection of the above features can be screened based on the degree of correlation between the features and the final results. The correlation between the features and the final results can be calculated using the Spearman correlation coefficient. It should be understood that the Spearman correlation coefficient can be replaced by the Pearson correlation coefficient or the Kendall correlation coefficient, so the calculation of the correlation coefficient in this embodiment is not limited to the above example. The correlation coefficient describes the degree of correlation between the input features and the output results. The higher the correlation, the greater the impact of the input features on the output results. In this embodiment, the correlation coefficients between all input features and the output results can be calculated, and then the features with correlation coefficients greater than a certain threshold are selected as the input features of the model. At the same time, in order to optimize the accuracy of the model, features with relatively few samples can be removed. For example, for enumeration features, features with a sample proportion of less than 3% can be removed.

[0085] Step 120: Input the first parameter group into the machine learning model to obtain a first prediction result, where the first prediction result is used to characterize whether the conversion data of the target object is abnormal data.

[0086] It should be understood that the machine learning model is a trained model. In this embodiment, the machine learning model can be a machine learning model such as a deep neural network, a classification decision tree, or a joint model composed of these models. Figure 3As shown, a common deep neural network model is used as the machine learning model. It consists of an input layer 310, a hidden layer 320, and an output layer 330 (for example, a sigmoid classifier or a softmax classifier). In this example, the first parameter group is represented by a vector (a1, a2, ..., an). After the vector (a1, a2, ..., an) is input into the input layer 310, feature extraction is performed by the hidden layer 320. The features extracted by the hidden layer 320 are then output through the output layer 330 as the final result. In this embodiment, the hidden layer 320 may include a multi-layer network, and the final output of the output layer 330 is a binary result, namely 0 or 1. When the machine learning model outputs 1 (i.e., the first prediction result is 1), it indicates that the conversion data of the target object is abnormal. When the machine learning model outputs 0 (i.e., the first prediction result is 0), it indicates that the conversion data of the target object is normal. It should be understood that abnormal conversion data includes cases where the reported data deviates significantly from the correct data due to data transmission errors, incorrect data reported by the promotion user, or errors in the promotion user's client program. For example, the deviation ratio is greater than 20%. It should be understood that the machine learning model is trained based on calibrated data. Therefore, the first prediction result referred to in this step is the output of the machine learning model, that is, the first prediction result is a classification judgment of the first parameter group by the machine learning model. The first prediction result is used to indicate whether there are any anomalies in the conversion data of the target user corresponding to the first parameter group.

[0087] Similarly, this embodiment uses a binary classification machine learning model. Classification tree models also perform very well on binary classification problems, so classification tree models can also be used as machine learning models. These classification tree models include the GBDT model (Gradient Boosted Iterative Decision Tree model) and the random forest model.

[0088] Reference Figure 4 In this embodiment, a joint model consisting of a neural network model 410 and a GBDT model 420 (gradient boosting iterative decision tree model) can also be used as a machine learning model.

[0089] like Figure 4 As shown, a stacking structure is adopted in the joint model of this embodiment. The stacking structure usually considers heterogeneous weak learners, that is, models with different structures are used for learning, and then the final output result is determined based on the output results of multiple models with different structures.

[0090] Among them, in this embodiment, the output feature 440 of the leaf node of the GBDT model is used as the input feature of the neural network model 410, and then bagging processing is performed based on the output result of the GBDT model 420 and the output result of the neural network model to obtain the final output. The meaning of bagging processing is to vote based on the two output results to determine the final classification result. For example, the output result 1 indicates that the conversion data of the target object is abnormal, and 0 indicates that the conversion data of the target object is normal. When the output result of the GBDT model and the output result of the neural network model are both 1, the joint model outputs 1. When either the output result of the GBDT model or the output result of the neural network model is 0, the joint model outputs 0. After data inspection, the method is adopted Figure 4 The model of the structure shown has an accuracy of 86.33% and a recall of 41.99% in the training set; an accuracy of 79.39% and a recall of 46.68% in the test set. It can be seen that the model performs well in terms of accuracy.

[0091] Reference Figure 5 The joint model can also use the intermediate layer features of the neural network model 510 as the input features of the GBDT model 520, and perform bagging on the output results of the neural network model 510 and the output results of the GBDT model 520 to obtain the final output result of the joint model. Figure 4 Similar to the example in , when the output results of the neural network model 510 and the GBDT model 520 are both 1, the joint model outputs 1, and in other cases the output result of the joint model is 0.

[0092] Reference Figure 6 , the joint model can also adopt the mode of parallel operation of the neural network model 610 and the GBDT model 620. Specifically, the neural network model 610 and the GBDT model 620 can use completely different input features, partially the same input features, or completely the same input features. Figure 5 and Figure 4 Similar to the model in

[15] , the neural network model 610 and the GBDT model 620 ultimately output the result of the joint model through bagging. That is, the joint model outputs 1 only when both the neural network model 610 and the GBDT model 620 output 1, and otherwise, the joint model outputs 0.

[0093] Step 130: Input the second parameter group into the statistical model to obtain a second prediction result, which is used to characterize whether the conversion data of the target object is abnormal data.

[0094] Specifically, in this embodiment, a Gaussian statistical model can be used as the statistical model of this embodiment. Its main principle is to predict whether the conversion data of the target object is abnormal based on the probability distribution of the conversion data. In this embodiment, the probability distribution of the conversion data can be analyzed based on historical conversion data and the estimated conversion rate. The statistical model of this embodiment is based on the following assumptions: E(CVR) = pCVR, where E(CVR) represents the mathematical expectation of CVR, pCVR is the estimated conversion rate predicted by the model, and the distribution probability of the conversion volume CV is as follows:

[0095] CVR-pCVR~N(0,1), that is, the conversion rate minus the estimated conversion rate, follows a normal distribution between 0 and 1.

[0096] CV-pCV~N(0,click), where click is the click volume, that is, the conversion volume minus the estimated conversion volume follows a normal distribution between 0 and the click volume.

[0097] CV-pCV~N(0,D his (CV'-pCV')), where CV' represents the historical average conversion volume, pCV' represents the historical estimated conversion volume, and D his () represents the calculation function of the empirical variance based on historical statistics.

[0098] CV~ N(pCV,D his (CV'-pCV')), that is, CV is normally distributed between the estimated conversion volume and the empirical variance of historical statistics.

[0099] Based on the above normal distribution, the statistical model used in this solution is represented by the following piecewise function:

[0100] ;

[0101] Among them, Abnorm (CV) indicates whether the current conversion volume data is abnormal. If the current conversion volume data is abnormal, it outputs 1, and if the current conversion volume data is normal, it outputs 0. Among them, F() represents the Gaussian cumulative distribution function, F pCV(CV) indicates that the current conversion volume is greater than the Gaussian cumulative distribution of the estimated conversion volume. I() is an indicator function. When the condition in the indicator function is true, it outputs 1, and when the condition is false, it outputs 0. It can be seen from the above model that when the number of clicks is greater than or equal to 100, the larger value of the two indicator functions is selected, that is, the output result of any of the above two functions is 1, and the output result of the model is 1. If both of the above two indicator functions output 0, the output result of the model is 0. The first sub-function takes effect when the number of clicks is relatively large, and the conversion volume suddenly soars or suddenly drops. The second sub-function takes effect when the number of clicks is less than 100, mainly considering the sudden drop. In some embodiments, the value of α can be 1 / e 4 , the value of β can be 70%.

[0102] It should be understood that other statistical models may be used in other embodiments, such as a multiple regression model, a principal component analysis model, etc. The statistical granularity of the above statistical parameters is the same. For example, if the conversion volume CV for one hour needs to be calculated, the historical data for one hour and the corresponding click volume for that hour need to be selected for statistics.

[0103] Step 140: When both the first prediction result and the second prediction result indicate that the conversion data of the target object is abnormal data, the conversion data of the target object is removed from the training data set of the conversion rate prediction model.

[0104] It should be understood that in this embodiment, the final determination of whether the target object's conversion data is abnormal is based on the prediction results of the machine learning model and the statistical model. In some examples, both the first and second prediction results are binary data, where 1 indicates that the model has determined the target object's conversion data to be abnormal, and 0 indicates that the model has determined the target object's conversion data to be abnormal. In this step, when both the first and second prediction results indicate that the target object's conversion data is abnormal, the target object's conversion data is removed from the training dataset of the conversion rate prediction model. It should be understood that conversion data includes conversion volume, conversion rate, or other data calculated based on the conversion data. Generally, conversion data serves as labels for training samples of the conversion rate prediction model, or as raw data for labels (i.e., labels are calculated from the raw data). Therefore, removing the target object's conversion data can be understood as removing the conversion data, or removing the conversion data and its related samples. For example, in some embodiments, the training samples of the conversion rate prediction model are input features (c1, c2, ..., cx), where x is a positive integer and c1 through cx are parameters of different types. For example, some parameters such as the previous day's click-through rate, the previous day's conversion rate, etc., and the label of the training sample is the current day's conversion rate. In this example, the entire training sample can be directly deleted.

[0105] Regardless of the structure of the conversion rate prediction model, it can be understood that through the method of the above embodiment, it is possible to relatively accurately predict whether the conversion data of the target object is abnormal, and to remove abnormal conversion data from the training data set of the conversion rate prediction model, thereby improving the quality of the training samples of the conversion rate prediction model and thus improving the accuracy of the conversion rate prediction model. In addition, this solution uses a machine learning model and a statistical model to collaborate for prediction. Compared with the fixed threshold comparison method used in related technologies, the method of this embodiment can learn the characteristics of different target objects and can achieve end-to-end abnormality judgment. At the same time, the model of this solution can also learn the situation of slow decline and slow improvement. It improves the accuracy of abnormal judgment of the conversion data of the target object, and thus can improve the data quality of the conversion rate prediction model.

[0106] In addition, this embodiment adopts a combination model of machine learning models and statistical models, which takes advantage of the fact that machine learning models have advantages in learning a large number of parameters, so that a variety of different statistical features can be counted in the time series data of click volume and conversion volume, thereby extracting the characteristics of abnormal data from more dimensions. The statistical model is suitable for the analysis of a small number of fixed parameters. In this embodiment, the statistical model is used to process a small number of fixed parameters such as the estimated conversion volume, the current conversion volume and the current click volume, which can improve the accuracy of the prediction. The machine learning model can analyze data anomalies from the dimension of time series, while the statistical model analyzes the abnormal characteristics of a time point / time period based on the estimated conversion volume, the current conversion volume and the current click volume.

[0107] Reference Figure 4 , in some embodiments, the machine learning model is composed of a gradient boosting iterative decision tree model (GBDT model 420) and a neural network model 410 (hereinafter referred to as the first neural network model);

[0108] Reference Figure 7 Step 120, inputting the first parameter group into the machine learning model to obtain a first prediction result, including:

[0109] Step 710: Input the first parameter group into the gradient boosting iterative decision tree model to obtain the output features of each leaf node in the gradient boosting iterative decision tree model and the third prediction result.

[0110] Specifically, in a gradient boosting iterative decision tree model, each leaf node (i.e., a node without children) represents an LR (Logistic Regression) feature. By traversing the tree, all LR features corresponding to the sample are obtained. Each dimension of the constructed new feature vector is 0 / 1. For example, a gradient boosting iterative decision tree model has two trees, the left tree has three leaf nodes, and the right tree has two leaf nodes. The resulting feature vector is a five-dimensional vector. For input x, assuming it falls on the first node of the left tree, it is encoded as [1,0,0], while if it falls on the second node of the right tree, it is encoded as [0,1], resulting in an overall encoding of [1,0,0,0,1]. This encoding serves as a feature and is input into the logistic regression unit for classification, ultimately outputting the classification result. In this example, the output features of each leaf node can be represented as [1,0,0,0,1]. The third prediction result is the output of the entire gradient boosting iterative decision tree model. For example, 1 can indicate that the target object's conversion data is abnormal, while 0 can indicate that the target object's conversion data is normal.

[0111] Step 720: Input the output features of each leaf node into the first neural network model to obtain a fourth prediction result.

[0112] Reference Figure 3 , the first neural network model can be used Figure 3 The structure shown in FIG. 1 is a diagram in which the input of the first neural network model is the output features of each leaf node of the gradient boosting iterative decision tree model. In the previous example, the input parameter of the first neural network model is a five-dimensional vector. Of course, in the actual gradient boosting iterative decision tree model, the number of leaf nodes is generally greater than 5, so the input parameter of the first neural network model may be an N-dimensional vector, where N is much greater than 5. The output result of the first neural network model is also 0 or 1, where 1 indicates that the conversion data of the target object is abnormal, and 0 indicates that the conversion data of the target object is normal.

[0113] Step 730: Determine the first prediction result according to the third prediction result and the fourth prediction result.

[0114] In this embodiment, various strategies can be adopted to determine the first prediction result. For example, if either the third prediction result or the fourth prediction result is 1, the first prediction result is determined to be 1, or if both the third prediction result and the fourth prediction result are 1, the first prediction result is determined to be 1. It is understood that with the former strategy, the determination conditions are relatively loose, and therefore, the possibility of missing abnormal conversion data is relatively low; with the latter strategy, the determination conditions are more stringent, and it is less likely that normal data will be determined to be abnormal data.

[0115] In some embodiments, step 730, determining the first prediction result based on the third prediction result and the fourth prediction result, the specific determination logic includes:

[0116] When both the third prediction result and the fourth prediction result indicate that the conversion data of the target object is abnormal data, the first prediction result is determined as a value indicating that the conversion data of the target object is abnormal data;

[0117] When at least one of the third prediction result and the fourth prediction result represents that the conversion data of the target object is normal data, the first prediction result is determined as a value representing that the conversion data of the target object is normal data.

[0118] In this embodiment, a relatively strict judgment condition is adopted, requiring that the target object's conversion data be determined as abnormal only when both the first neural network model and the gradient boosting iterative decision tree model simultaneously determine that the target object's conversion data is abnormal. This can avoid misclassifying negative examples as positive examples, which helps improve the overall accuracy of the model.

[0119] Reference Figure 8 This embodiment discloses a gradient boosting iterative decision tree model structure, which is obtained by weighting multiple trained classifiers (weak classifiers). Figure 8 An example of three weak classifiers is provided, but those skilled in the art can set more or fewer weak classifiers to meet the training conditions according to actual needs.

[0120] Among them, Figure 9 As shown in Figure 2, the gradient boosting iterative decision tree model is obtained in the following way:

[0121] Step 910: Obtain a first training sample set and a first label corresponding to each first training sample in the first training sample set, wherein the first training sample has the same data structure as the first parameter group;

[0122] Step 920: Train multiple classifiers based on the first training samples in the first training sample set and the first labels corresponding to the first training samples, wherein the Nth classifier is trained based on the residual of the N-1th classifier, where N is a positive integer greater than 2;

[0123] Step 930: Weight the trained multiple classifiers to obtain a gradient boosting iterative decision tree model.

[0124] In the above embodiment, each round of learning can create a weak classifier, and then each weak classifier can be trained based on the residual of the previous round of training. The following is a more detailed description of the training process of this embodiment:

[0125] Input: training set , the loss function is ， In this embodiment, x N is the training sample, y N Indicates the label corresponding to the training sample. The goal is to output: classification tree .

[0126] Among them, the first weak classifier is trained in the initialization stage to obtain :

[0127]

[0128] right ( Represents the number of iterations, that is, the number of weak classifiers), for the sample , calculate the pseudo residual.

[0129]

[0130] Probability residual Fit a classification tree and get the leaf node area of ​​the mth tree .

[0131] right calculate:

[0132]

[0133] Get the final classification tree:

[0134] ;

[0135] Of course, the above loss functions can be replaced by other loss functions.

[0136] Reference Figure 10 In some embodiments, the first neural network model is obtained by:

[0137] Step 1010: Obtain a second training sample set and a second label corresponding to each second training sample in the second training sample set, wherein the second training sample is the output feature of each leaf node after the first training sample is predicted by the gradient boosting iterative decision tree model;

[0138] Step 1020: Train the second neural network model according to the second training samples in the second training sample set and the second labels corresponding to the second training samples to obtain the first neural network model.

[0139] It should be understood that the second neural network model referred to in this embodiment has the same structure as the first neural network model. The second neural network model is an untrained model whose parameters are randomly initialized values. The first neural network model refers to the model obtained after training with the second training sample set, that is, the trained model. In these embodiments, the second training sample is constructed based on the intermediate features of the gradient boosting iterative decision tree model. Among them, the first training sample can be predicted by the trained gradient boosting iterative decision tree model, and then the output features of each leaf node are used as the second training sample, and the label of the first training sample is used as the label of the second training sample (i.e., the second label). Of course, it is understandable that the two models can also be combined for joint training without the need for separate training. The advantage of training the two models separately is that it can reduce the cost of model learning, and the two models can be trained in parallel to increase training efficiency.

[0140] In some embodiments, the promotion data processing method further includes the following steps:

[0141] Obtain a preset list, which includes multiple candidate objects;

[0142] Wherein, step 140, when both the first prediction result and the second prediction result indicate that the conversion data of the target object is abnormal data, the conversion data of the target object is removed from the training data set of the conversion rate prediction model, includes:

[0143] When both the first prediction result and the second prediction result indicate that the conversion data of the target object is abnormal data, and the target object does not match any candidate objects in the preset list, the conversion data of the target object is removed from the training data set of the conversion rate prediction model.

[0144] Reference Figure 11 In some embodiments, the operator 1110 can set a whitelist 1130 in the promotional message delivery system 1120, which is the preset list referred to in this embodiment. When the promotional message delivery system 1120 determines that the conversion data of the target object is abnormal, it will first match it from the whitelist. If the target object exists on the whitelist 1130, the conversion data of the target user will not be eliminated. Among them, the whitelist is mainly for special target objects. For example, the conversion data of this type of target object is indeed abnormal compared with normal data, but it is normal data after technical determination. These data are equivalent to some adversarial samples in the model, and these samples will cause the model to classify incorrectly. By setting a whitelist, these data can be prevented from being mistakenly judged as abnormal data and eliminated. This allows the conversion rate prediction model to learn some implicit features from these data, further improving the accuracy of the conversion rate prediction model.

[0145] In some embodiments, the statistical model is a piecewise function, and the statistical model includes a first sub-function and a second sub-function;

[0146] Reference Figure 12 Step 130: inputting the second parameter group into the statistical model to obtain a second prediction result, including:

[0147] Step 1210: Determine whether the number of clicks is greater than or equal to a first threshold.

[0148] Step 1220: When the number of clicks in the second parameter group is greater than or equal to the first threshold, the current conversion volume and the estimated conversion volume in the second parameter group are input into the first sub-function to obtain a second prediction result;

[0149] Step 1230: When the number of clicks in the second parameter group is less than the first threshold, the current conversion volume and the estimated conversion volume in the second parameter group are input into the second sub-function to obtain a second prediction result.

[0150] In some embodiments, step 1220, inputting the current conversion amount and the estimated conversion amount in the second parameter group into the first sub-function to obtain a second prediction result, includes:

[0151] The maximum value of the output value of the first indicator function and the output value of the second indicator function is used as the second prediction result by the first sub-function;

[0152] When the cumulative distribution probability that the current conversion amount is greater than the estimated conversion amount is less than or equal to the second threshold, the first indicator function outputs a first value; when the cumulative distribution probability is greater than the second threshold, the first indicator function outputs a second value;

[0153] When the quotient of the estimated conversion amount and the current conversion amount is greater than or equal to a third threshold, the second indicator function outputs a first value; when the quotient of the estimated conversion amount and the current conversion amount is less than the third threshold, the second indicator function outputs a second value.

[0154] The piecewise function used in the above embodiment is as follows:

[0155] ;

[0156] Among them, Abnorm (CV) indicates whether the current conversion volume data is abnormal. If the current conversion volume data is abnormal, it outputs 1, and if the current conversion volume data is normal, it outputs 0. Among them, F() represents the Gaussian cumulative distribution function, F pCV(CV) indicates that the current conversion volume is greater than the Gaussian cumulative distribution of the estimated conversion volume. I() is an indicator function, which outputs 1 when the condition in the indicator function is true, and outputs 0 when the condition is false. It can be seen from the above model that when the number of clicks is greater than or equal to 100 (i.e., the first threshold), the larger value of the two indicator functions is selected, that is, the output result of any of the above two functions is 1, and the output result of the model is 1. If both of the above two indicator functions output 0, the output result of the model is 0. This segmentation takes into account the situation where the conversion volume suddenly soars or suddenly drops when the number of clicks is relatively large. When the number of clicks is less than 100, the sudden drop is mainly considered. In some embodiments, the value of α can be 1 / e 4 , the value of β can be 70%. Using a piecewise function can ensure that the confidence of the model's output meets certain conditions when the number of clicks is different.

[0157] In some embodiments, at least some of the statistical features in the first parameter group are obtained by:

[0158] Statistics are collected on the conversion volume time series data and click volume time series data of the target object to obtain at least part of the parameters in the first parameter group.

[0159] It is understandable that year-on-year or month-on-month data such as click-through rate and conversion rate can be obtained based on the time series data of conversion volume and click volume. The month-on-month change in click-through rate between the previous day and the same hour today can be calculated by dividing the click-through rate for the same hour today by the click-through rate for the same hour of the previous day. Therefore, it can be seen that based on the time series data of conversion volume and click volume, various statistical features can be calculated and used as model input. In this way, multiple statistical features can be obtained based on sequence data to increase the input dimension of the model, thereby improving the accuracy of the model.

[0160] Given that abnormal examples of transformed data are relatively scarce, there are insufficient positive examples in the training samples, and it is necessary to construct fake samples to make the model have sufficient accuracy. Figure 13 In this embodiment, part of the first training samples is obtained by:

[0161] Step 1310: Randomly increase or decrease at least part of the first conversion volume time series data and the first click volume time series data of the target object within a preset fluctuation range to obtain second conversion volume time series data and second click volume time series data.

[0162] Step 1320: Perform statistics on the second conversion volume time series data and the second click volume time series data to obtain at least part of the statistical features of the training sample.

[0163] Sample construction involves analyzing failures using the product ID and promotion account ID dimensions, i.e., selecting different target objects. Selecting the product ID dimension is equivalent to targeting a specific promotional message. Selecting the promotion account dimension, on the other hand, encompasses all promotional messages from that promotion account. Statistical analysis of extensive data reveals that typical abnormal conversion data exhibits the following characteristics: the failure lasts from three hours to three days, and the conversion volume drops to zero or approximately 20% of the previous day's level.

[0164] Therefore, based on the above statistical analysis, the method for constructing fake samples is as follows: sampling by product ID and promotion account ID dimensions; for each ID, randomly select N hours within 24 hours and randomly decrease the conversion volume by [70%, 100%], where N is in the range of [3 hours, 3 days]. For example: the conversion volume of the promotion account ID dimension randomly decreases by [70%, 100%]; the conversion volume of the promotion account ID dimension randomly decreases by [70%, 100%] over N hours; the conversion volume of the product ID dimension randomly decreases by [70%, 100%]; and the conversion volume of the product ID dimension randomly decreases by [70%, 100%] over N hours. It can be seen that by determining the fluctuation range of abnormal conversion data through statistical analysis and then creating positive examples through random construction, the number of positive examples can be increased, thereby improving the accuracy of the model.

[0165] Reference Figure 14 , a promotion data processing method, comprising the following steps:

[0166] Step 1410: Obtain multiple first parameter groups and second parameter groups corresponding to the first parameter groups. The first parameter group includes multiple statistical characteristics of the target object, and the target object is a promotion account or a promotion message delivered by the promotion account; the second parameter group includes the estimated conversion volume, current conversion volume, and current click volume of the target object.

[0167] Step 1420: Input multiple first parameter groups into the machine learning model to obtain a first prediction result corresponding to each first parameter group. The first prediction result is used to characterize whether the conversion data of the target object is abnormal data.

[0168] Step 1430: Add the target object represented by the first prediction result as abnormal data to the first abnormal data set.

[0169] Step 1440: Input multiple second parameter groups into the statistical model to obtain a second prediction result corresponding to each second parameter group. The second prediction result is used to characterize whether the conversion data of the target object is abnormal data.

[0170] Step 1450: Add the target object representing the second prediction result as abnormal data to the second abnormal data set.

[0171] Step 1460: Intersect the first abnormal data set and the second abnormal data set to obtain a third abnormal data set.

[0172] Step 1470: Remove the conversion data of the target object in the third abnormal data set from the training data set of the conversion rate prediction model.

[0173] It is understandable that this embodiment is different from Figure 1 The methods adopted in the embodiments are basically the same, the difference is that this embodiment is mainly described from the perspective of batch processing. In this embodiment, predictions are made for the first parameter group and the second parameter group of multiple target objects. When the conversion data of a target object is determined to be abnormal based on the first parameter group, the target object will be added to the first abnormal data set. When the conversion data of a target object is determined to be abnormal based on the second parameter group, the target object will be added to the second abnormal data set. Finally, a third abnormal data set is obtained by taking the intersection of the first abnormal data set and the second abnormal data set, and the conversion data of all target objects belonging to the third abnormal data set are eliminated from the training data obtained by the conversion rate prediction model. Batch operation is conducive to batch screening. Compared with judging one by one, the number of operations is less and the efficiency is higher.

[0174] Reference Figure 15 This embodiment discloses a method for training a conversion rate prediction model, comprising the following steps:

[0175] Step 1510: Obtain a first training data set for a conversion rate prediction model, where the first training data set includes conversion data of multiple target objects.

[0176] Step 1520: Process the conversion data in the training data set using the above-mentioned promotion data processing method to obtain a second training data set.

[0177] Step 1530: Train the conversion rate prediction model based on the second training data set.

[0178] It is understandable that this embodiment utilizes Figure 1 or Figure 14 The method shown performs data cleaning on the training samples in the first training data set, eliminating abnormal conversion data as much as possible, so that the data quality of the final second training data set is improved, thereby increasing the accuracy and recall rate of the conversion rate prediction model obtained by the final training.

[0179] Reference Figure 16 This embodiment discloses a promotion data processing system, including:

[0180] A first acquisition unit 1610 is configured to acquire a first parameter group and a second parameter group. The first parameter group includes multiple statistical features of a target object, the statistical features being obtained by collecting time-series data on conversion volumes and / or click volumes of the target object. The target object is a promotion account or a promotional message delivered by the promotion account. The second parameter group includes the estimated conversion volume, current conversion volume, and current click volume of the target object.

[0181] A first prediction unit 1620 is configured to input the first parameter group into a machine learning model to obtain a first prediction result, where the first prediction result is used to indicate whether the conversion data of the target object is abnormal data;

[0182] A second prediction unit 1630 is configured to input the second parameter group into a statistical model to obtain a second prediction result, where the second prediction result is used to indicate whether the conversion data of the target object is abnormal data;

[0183] The first data processing unit 1640 is configured to remove the conversion data of the target object from the training data set of the conversion rate prediction model when both the first prediction result and the second prediction result indicate that the conversion data of the target object is abnormal data.

[0184] Reference Figure 17 This embodiment discloses a promotion data processing system, including:

[0185] The second acquisition unit 1710 is used to obtain multiple first parameter groups and second parameter groups corresponding to the first parameter groups, where the first parameter group includes multiple statistical features of the target object, and the statistical features are obtained through statistics of the conversion volume time series data and / or click volume time series data of the target object, and the target object is a promotion account or a promotion message delivered by the promotion account; the second parameter group includes the estimated conversion volume, current conversion volume and current click volume of the target object.

[0186] The third prediction unit 1720 is used to input multiple first parameter groups into the machine learning model to obtain a first prediction result corresponding to each first parameter group, and the first prediction result is used to characterize whether the conversion data of the target object is abnormal data.

[0187] The first set processing unit 1730 is configured to add the target object representing the first prediction result as abnormal data to the first abnormal data set.

[0188] The fourth prediction unit 1740 is used to input multiple second parameter groups into the statistical model to obtain a second prediction result corresponding to each second parameter group, where the second prediction result is used to indicate whether the conversion data of the target object is abnormal data.

[0189] The second set processing unit 1750 is configured to add the target object representing the second prediction result as abnormal data to the second abnormal data set.

[0190] The third set processing unit 1760 is configured to intersect the first abnormal data set and the second abnormal data set to obtain a third abnormal data set.

[0191] The data processing unit 1770 is configured to remove the conversion data of the target object in the third abnormal data set from the training data set of the conversion rate prediction model.

[0192] Reference Figure 18 This embodiment discloses a training system for a conversion rate prediction model, including:

[0193] The third acquisition unit 1810 acquires a first training data set for the conversion rate prediction model, where the first training data set includes conversion data of multiple target objects.

[0194] The data cleaning unit 1820 is configured to process the transformed data in the training data set using the generalized data processing method to obtain a second training data set.

[0195] The training unit 1830 is configured to train the conversion rate prediction model according to the second training data set.

[0196] Reference Figure 19 This embodiment discloses a promotion data processing system, the working principle of which is as follows: Figure 19 As shown, the system's functions include sample construction, which uses real CVR time series data to randomly increase, decrease, or set to zero some of the data within a preset range. This allows for the construction of more positive examples, i.e., abnormal CVR time series. Statistical features are generated based on these constructed samples (CVR time series) and normal samples to train a classification model (machine learning model). A Gaussian statistical model is then fitted based on these constructed samples and normal samples. Finally, a diagnosis of the CVR time series is performed based on the classification model and the Gaussian statistical model. The diagnosis methods include full and incremental diagnosis. The results of the diagnosis (i.e., the identification of abnormal conversion data) are used to clean the training data for the conversion rate prediction model. Operators can also configure a whitelist in the system to prevent useful data from being filtered out.

[0197] This embodiment discloses a data processing system, including: a program; a memory for storing the program; and a processor for loading the program to execute the method in the above embodiment.

[0198] This embodiment discloses a computer-readable storage medium, in which a program is stored. When the program is executed by a processor, the method in the above embodiment is implemented.

[0199] The terms "first", "second", "third", "fourth", etc. (if any) in the specification of the present application and the above-mentioned drawings are used to distinguish similar objects and are not necessarily used to describe a specific order or sequential order. It should be understood that the data used in this way can be interchangeable where appropriate, so that the embodiments of the present application described herein can, for example, be implemented in an order other than those illustrated or described herein. In addition, the terms "including" and "having" and any variations thereof are intended to cover non-exclusive inclusions, for example, a process, method, system, product or device that includes a series of steps or units is not necessarily limited to those steps or units clearly listed, but may include other steps or units that are not clearly listed or inherent to these processes, methods, products or devices.

[0200] It should be understood that in this application, "at least one (item)" means one or more, and "plurality" means two or more. "And / or" is used to describe the association relationship of associated objects, indicating that three relationships may exist. For example, "A and / or B" can mean: only A exists, only B exists, and A and B exist at the same time, where A and B can be singular or plural. The character " / " generally indicates that the previous and next associated objects are in an "or" relationship. "At least one of the following items" or similar expressions refers to any combination of these items, including any combination of single items or plural items. For example, at least one of a, b or c can mean: a, b, c, "a and b", "a and c", "b and c", or "a and b and c", where a, b, c can be single or multiple.

[0201] If the integrated unit described in this application is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of this application is essentially or the part that contributes to the prior art or all or part of the technical solution can be embodied in the form of a software product, and the computer software product is stored in a storage medium, including a number of instructions for enabling a computer device (which can be a personal computer, a server, or a network device, etc.) to execute all or part of the steps of the method described in each embodiment of this application. The aforementioned storage medium includes: various media that can store program codes, such as a USB flash drive, a mobile hard disk, a read-only memory (ROM), a random access memory (RAM), a magnetic disk or an optical disk.

[0202] The above is a specific description of the preferred implementation of the present application, but the present application is not limited to the above embodiments. Those skilled in the art can make various equivalent modifications or substitutions without violating the spirit of the present application. These equivalent modifications or substitutions are all included in the scope defined by the claims of the present application.

Claims

1. A promotion data processing method, characterized in that: The following steps are involved: Obtaining a first parameter group and a second parameter group, wherein the first parameter group includes multiple statistical characteristics of a target object, the statistical characteristics being obtained by counting the target object's conversion volume time series data and / or click volume time series data, wherein the target object is a promotion account or a promotional message delivered by the promotion account; and the second parameter group includes the target object's estimated conversion volume, current conversion volume, and current click volume; Inputting the first parameter group into a machine learning model to obtain a first prediction result, where the first prediction result is used to characterize whether the conversion data of the target object is abnormal data; When the current click volume in the second parameter group is greater than or equal to a first threshold, the current conversion volume and the estimated conversion volume in the second parameter group are input into a first sub-function in a statistical model to obtain a second prediction result, wherein the statistical model is a piecewise function, and the statistical model includes the first sub-function and the second sub-function, and is used to predict whether the conversion data of the target object is abnormal based on the probability distribution of the conversion data in a piecewise manner, and the second prediction result is used to indicate whether the conversion data of the target object is abnormal data; When the current click volume in the second parameter group is less than the first threshold, the current conversion volume and the estimated conversion volume in the second parameter group are input into the second sub-function to obtain the second prediction result; When both the first prediction result and the second prediction result indicate that the conversion data of the target object is abnormal data, the conversion data of the target object is removed from the training data set of the conversion rate prediction model.

2. The promotion data processing method according to claim 1, characterized in that: The machine learning model is composed of a gradient boosting iterative decision tree model and a first neural network model; Inputting the first parameter group into the machine learning model to obtain a first prediction result includes: Inputting the first parameter group into the gradient boosting iterative decision tree model to obtain output features of each leaf node in the gradient boosting iterative decision tree model and a third prediction result; Inputting the output features of each leaf node into the first neural network model to obtain a fourth prediction result; The first prediction result is determined according to the third prediction result and the fourth prediction result.

3. The promotion data processing method according to claim 2, characterized in that: The determining the first prediction result according to the third prediction result and the fourth prediction result includes: When both the third prediction result and the fourth prediction result indicate that the conversion data of the target object is abnormal data, determining the first prediction result as a value indicating that the conversion data of the target object is abnormal data; When at least one of the third prediction result and the fourth prediction result indicates that the conversion data of the target object is normal data, the first prediction result is determined as a value indicating that the conversion data of the target object is normal data.

4. The promotion data processing method according to claim 2, characterized in that: The gradient boosting iterative decision tree model is obtained in the following way: Obtaining a first training sample set and a first label corresponding to each first training sample in the first training sample set, wherein the first training sample has the same data structure as the first parameter group; Training a plurality of classifiers based on the first training samples in the first training sample set and the first labels corresponding to the first training samples, wherein the Nth classifier is trained based on the residual of the N-1th classifier, where N is a positive integer greater than 2; The trained multiple classifiers are weighted to obtain the gradient boosting iterative decision tree model.

5. The promotion data processing method according to claim 4, characterized in that: The first neural network model is obtained by: Obtaining a second training sample set and a second label corresponding to each second training sample in the second training sample set, wherein the second training sample is an output feature of each leaf node after the first training sample is predicted by the gradient boosting iterative decision tree model; The second neural network model is trained according to the second training samples in the second training sample set and the second labels corresponding to the second training samples to obtain the first neural network model.

6. The promotion data processing method according to claim 1, characterized in that: The method further comprises the following steps: Obtaining a preset list, wherein the preset list includes a plurality of candidate objects; Wherein, when both the first prediction result and the second prediction result indicate that the conversion data of the target object is abnormal data, the conversion data of the target object is removed from the training data set of the conversion rate prediction model, including: When both the first prediction result and the second prediction result indicate that the conversion data of the target object is abnormal data, and the target object does not match any of the candidate objects in the preset list, the conversion data of the target object is removed from the training data set of the conversion rate prediction model.

7. The promotion data processing method according to claim 1, characterized in that: The step of inputting the current conversion amount and the estimated conversion amount in the second parameter group into the first sub-function to obtain the second prediction result includes: Using the first sub-function, the maximum value of the output value of the first indicator function and the output value of the second indicator function is used as the second prediction result; When the cumulative distribution probability that the current conversion amount is greater than the estimated conversion amount is less than or equal to a second threshold, the first indicator function outputs a first value; when the cumulative distribution probability is greater than the second threshold, the first indicator function outputs a second value; When the quotient of the estimated conversion amount and the current conversion amount is greater than or equal to a third threshold, the second indicator function outputs the first value; when the quotient of the estimated conversion amount and the current conversion amount is less than the third threshold, the second indicator function outputs the second value.

8. The promotion data processing method according to claim 4, characterized in that: Part of the first training samples is obtained by: Randomly increase or decrease at least part of the first conversion volume time series data and the first click volume time series data of the target object within a preset fluctuation range to obtain second conversion volume time series data and second click volume time series data; Statistics are performed on the second conversion volume time series data and the second click volume time series data to obtain at least part of the statistical features of the training sample.

9. A promotion data processing method, characterized in that: The following steps are involved: Obtaining multiple first parameter groups and second parameter groups corresponding to the first parameter groups, wherein the first parameter groups include multiple statistical features of a target object, the statistical features being obtained by counting time series data of conversions and / or time series data of clicks of the target object, the target object being a promotion account or a promotional message delivered by the promotion account; and the second parameter group including an estimated conversion volume, current conversion volume, and current click volume of the target object; Inputting the plurality of first parameter groups into a machine learning model to obtain a first prediction result corresponding to each first parameter group, wherein the first prediction result is used to indicate whether the conversion data of the target object is abnormal data; adding the target object represented by the first prediction result as abnormal data to a first abnormal data set; When the current click volume in the second parameter group is greater than or equal to a first threshold, the current conversion volume and the estimated conversion volume in the second parameter group are input into a first sub-function in a statistical model to obtain a second prediction result, wherein the statistical model is a piecewise function, and the statistical model includes the first sub-function and the second sub-function, and is used to predict whether the conversion data of the target object is abnormal based on the probability distribution of the conversion data in a piecewise manner, and the second prediction result is used to indicate whether the conversion data of the target object is abnormal data; When the current click volume in the second parameter group is less than the first threshold, the current conversion volume and the estimated conversion volume in the second parameter group are input into the second sub-function to obtain the second prediction result; adding the target object represented by the second prediction result as abnormal data to a second abnormal data set; Intersecting the first abnormal data set and the second abnormal data set to obtain a third abnormal data set; The conversion data of the target object in the third abnormal data set is removed from the training data set of the conversion rate prediction model.

10. A method for training a conversion rate prediction model, characterized in that: The following steps are involved: Acquire a first training data set for the conversion rate prediction model, where the first training data set includes conversion data of a plurality of target objects; Processing the conversion data in the training data set by the promotion data processing method according to any one of claims 1 to 9 to obtain a second training data set; The conversion rate prediction model is trained based on the second training data set.

11. A promotion data processing system, characterized in that: include: a first acquisition unit configured to acquire a first parameter group and a second parameter group, wherein the first parameter group includes multiple statistical characteristics of a target object, the statistical characteristics being obtained by counting the target object's conversion volume time series data and / or click volume time series data, wherein the target object is a promotion account or a promotional message delivered by the promotion account; and the second parameter group includes the target object's estimated conversion volume, current conversion volume, and current click volume; a first prediction unit, configured to input the first parameter group into a machine learning model to obtain a first prediction result, wherein the first prediction result is used to indicate whether the conversion data of the target object is abnormal data; a second prediction unit, configured to input the current conversion volume and the estimated conversion volume in the second parameter group into a first sub-function in a statistical model to obtain a second prediction result when the current click volume in the second parameter group is greater than or equal to a first threshold, wherein the statistical model is a piecewise function including the first sub-function and the second sub-function, and is configured to predict whether the conversion data of the target object is abnormal based on a probability distribution of the conversion data in a piecewise manner, and the second prediction result is used to indicate whether the conversion data of the target object is abnormal data; The second prediction unit is further configured to input the current conversion volume and the estimated conversion volume in the second parameter group into the second sub-function to obtain the second prediction result when the current click volume in the second parameter group is less than the first threshold; The first data processing unit is configured to remove the conversion data of the target object from a training data set of a conversion rate prediction model when both the first prediction result and the second prediction result indicate that the conversion data of the target object is abnormal data.

12. A data processing system, characterized in that: include: program; a memory for storing the program; A processor, configured to load the program to execute the method according to any one of claims 1 to 9.

13. A computer-readable storage medium, characterized in that The computer-readable storage medium stores a program, and when the program is executed by a processor, the method according to any one of claims 1 to 9 is implemented.

Images