Screening method and screening system

Abstract

The embodiment of the present application provides a screening method and a screening system, the screening method comprises the following steps: obtaining historical flow data of a plurality of target objects from a storage system, and extracting historical values of target variables of the plurality of target objects from the historical flow data; generating a predicted selection probability corresponding to the target objects according to the historical values of the target variables; inputting the predicted selection probability and the historical values of the target variables into an established expectation value model, performing screening analysis on the target objects, and generating expected values of the target variables of the plurality of target objects; and screening the target objects whose expected values of the target variables are greater than a preset threshold value, and storing the corresponding identification information of the target objects into a database. According to the predicted selection probability of the target objects, the expected values of the target variables are obtained by using the expectation value model, and the target objects are screened according to the expected values of the target variables, so that the required target objects can be quickly and accurately screened, and the screening quality is better.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of computer, in particular to a screening method and a screening system. BACKGROUND

[0002] At present, screening of target items can apply various machine learning models, such as logistic regression, random forest, GBDT (Gradient Boosting Decision Tree, constructing multiple decision regression trees to predict classification) and the like. Based on different definitions of target variables, different machine learning algorithms and models are used in model framework design.

[0003] At present, two kinds of machine learning methods are mainly used, one is a regression method based on continuous variables, and the other is a classification prediction method based on whether a transaction is completed. The model based on the regression method has poor stability, and it is difficult to improve the model accuracy for regression prediction of target variables of a large number of target items; the learning based on the classification method mainly predicts the possibility of transaction of target items, and cannot directly give the prediction result of the target variable of the target item, and does not consider the influence of the classification variable on the target variable.

[0004] Therefore, the inventor believes that the above-mentioned screening method of target items has great limitations, and it is difficult to accurately and quickly calculate the prediction result of the target variable related to the selection of the target item by using the existing machine learning model, and there is a problem that it is difficult to select a suitable target item. SUMMARY

[0005] Therefore, the present application provides a screening method and a screening system, which calculates a prediction selection probability of a target item according to a historical value of a target variable of the target item, and then calculates an expected value of the target variable based on the prediction selection probability by using an expected value model, so as to screen and store the target item according to the expected value of the target variable, and the qualified target item required can be quickly screened out.

[0006] According to a first aspect of the present application, a screening method is provided, comprising:

[0007] obtaining historical traffic data of a plurality of target items from a storage system, and extracting historical values of target variables of the plurality of target items from the historical traffic data;

[0008] generating a prediction selection probability corresponding to the target item according to the historical values of the target variables;

[0009] inputting the prediction selection probability and the historical values of the target variables into an established expected value model, performing screening analysis on the target item, and generating expected values of the target variables of the plurality of target items; and

[0010] The target item whose expected value of the target variable is greater than the preset threshold is screened out, and the corresponding identification information is stored in a database.

[0011] Preferably, the historical traffic data includes target item exposure, target item selection and exposure channel data of the target item.

[0012] Preferably, the target variable of the target item is a thousand times of a conversion rate of the target item per thousand exposures, which is a thousand times of a ratio of the target item selection to the target item exposure.

[0013] Preferably, the screening method further comprises: extracting a plurality of attribute data of the target item from the historical traffic data, and classifying a plurality of the target items according to the attribute data.

[0014] Preferably, the attribute data of the target item is continuously processed by using a discrete transformation method based on the target variable of the target item to obtain the predicted selection probability, and the target item corresponds to different predicted selection probabilities and different expected values of the target variable under different classifications.

[0015] Preferably, the historical value of the target variable of the target item under a specified classification of the attribute data is a ratio of a number of positive samples to the target item exposure.

[0016] Preferably, the historical value of the target variable greater than zero is recorded as a positive sample, otherwise as a negative sample.

[0017] Preferably, the attribute data includes item category and exposure channel.

[0018] Preferably, the screening method further comprises:

[0019] The predicted selection probability and the expected value of the target variable of the target item under different exposure channels are calculated.

[0020] The target item exposure is set to decrease in a gradient manner on different exposure channels according to descending order of the expected value of the target variable.

[0021] Preferably, continuously processing the attribute data of the target item by using a discrete transformation method based on the target variable of the target item comprises:

[0022] A specific value of the attribute data and the corresponding target item exposure are combined to form a pair, and the pair is arranged in descending order of the target item exposure.

[0023] acquire historical values of the target variable of the target item under each category and count the number of historical values of the target variable as positive samples;

[0024] integrate part of the number pairs in the number pair column corresponding to the attribute data into one number pair;

[0025] For the processed number pair column, calculate the average value of the historical value of the target variable corresponding to the specific value of each attribute data.

[0026] Preferably, the process of establishing the expected value model comprises:

[0027] Load the data source to obtain historical traffic data and a predicted selected probability of the target item;

[0028] According to the database time partition, divide the data source into a training data set and a test data set, and divide the training data set into a training set and a validation set according to a ratio of 7:3;

[0029] Use a random forest model to adjust the model parameters according to the performance of the model on the validation set through cross-validation;

[0030] Serialize and save the model file;

[0031] Generate the expected value of the target variable according to the predicted selected probability and the historical value of the target variable under a certain time granularity.

[0032] According to the second aspect of the present application, a screening system is provided, comprising:

[0033] A data acquisition unit is configured to acquire historical traffic data of a plurality of target items from a storage system and extract historical values of target variables of the plurality of target items therefrom;

[0034] A prediction unit is configured to generate a predicted selected probability corresponding to the target item according to the historical values of the target variables;

[0035] An expected value calculation unit is configured to input the predicted selected probability and the historical values of the target variables into an established expected value model, perform screening analysis on the target item, and generate expected values of target variables of the plurality of target items; and

[0036] A screening unit is configured to screen out the target item whose expected value of the target variable is greater than a preset threshold value, and store the corresponding identification information of the target item into a database.

[0037] Preferably, the historical traffic data comprises target item exposure, target item selection, and target item exposure channel data.

[0038] Preferably, the target variable of the target item is a thousand times of a click-through rate of the target item, which is a thousand times of a ratio of a selected amount of the target item to an exposure amount of the target item.

[0039] Preferably, the screening system further comprises a classification unit configured to extract attribute data of the target item from the historical traffic data, and classify the target item according to the attribute data.

[0040] Preferably, the attribute data of the target item is continuously processed by a method based on a discrete transformation of the target variable of the target item, to obtain the predicted click probability, and the target item corresponds to different predicted click probabilities and different expected values of the target variable in different classifications.

[0041] Preferably, the historical value of the target variable of the target item corresponding to a specified classification of the attribute data is a ratio of a number of positive samples to the exposure amount of the target item.

[0042] Preferably, the prediction unit comprises:

[0043] a pair generation unit configured to generate a pair of a specific value of attribute data and a corresponding exposure amount of the target item, and arrange the pair in descending order of the exposure amount of the target item;

[0044] a statistics unit configured to obtain the historical value of the target variable of the target item in each classification and count a number of positive samples of the historical value of the target variable;

[0045] a collation unit configured to integrate part of the pairs in a pair list of the attribute data into one pair;

[0046] a mean value calculation unit configured to calculate an average value of the historical value of the target variable corresponding to the specific value of each attribute data for the processed pair list.

[0047] According to a third aspect of the present disclosure, a computer readable storage medium is provided, which stores computer instructions, and the computer instructions are executed to implement the screening method as described above.

[0048] According to a fourth aspect of the present disclosure, a screening device is provided, which comprises a memory configured to store computer instructions, and a processor coupled to the memory, and the processor is configured to execute the computer instructions stored in the memory to implement the screening method as described above.

[0049] The embodiment of the present application has the following advantages or beneficial effects: the predicted selection probability of the target item is obtained by the historical value of the target variable of the target item, and the expected value model is used to obtain the expected value of the target variable according to the predicted selection probability, and the target item is screened according to the high or low of the expected value of the target variable. The predicted selection probability of the target item is added in the calculation process of the expected value of the target variable, so that the expected value of the target variable is more accurate, and the screened target item is more in line with the demand, thereby improving the screening quality of the target item.

[0050] Another preferred embodiment of the present application has the following advantages or beneficial effects: the discrete transformation method based on the target variable is used to continuously process the classification variable, and then the predicted selection probability is calculated, the historical transaction data of the target item is fully utilized, the accuracy of the predicted selection probability of the target item is improved, and the stability of the expected value model is improved.

[0051] Another preferred embodiment of the present application has the following advantages or beneficial effects: the expected value model is used to obtain the predicted selection probability of the target item according to the classification problem, and then the expected value of the target variable is calculated according to the product of the target variable value and the predicted selection probability under a certain time granularity by using the regression problem, the accuracy and stability of the model are improved, the prediction effect of the model is improved, and the screening of the target item is more reasonable. BRIEF DESCRIPTION OF DRAWINGS

[0052] The above and other objects, features and advantages of the present application will become more apparent from the following description of embodiments of the present application taken in conjunction with the accompanying drawings, in which:

[0053] Figure 1 A flowchart of the screening method in the embodiment of the present application is shown;

[0054] Figure 2a A flowchart of the screening method in the embodiment of the present application is shown; Figure 1 A specific flowchart of step S102 is shown;

[0055] Figure 2b A schematic diagram of the continuous processing process of the attribute data by using the discrete transformation method is shown;

[0056] Figure 2c A flowchart of the establishment of the expected value model of the screening method in the embodiment of the present application is shown;

[0057] Figure 3 A flowchart of the screening method in the embodiment of the present application is shown;

[0058] Figure 4 A line chart of the comparison of the business promotion rates before and after the screening method of the embodiment of the present application is shown;

[0059] Figure 5A structure diagram of the screening system in the embodiment of the present application is shown;

[0060] Figure 6 A structure diagram of the screening system in the embodiment of the present application is shown;

[0061] Figure 7 A structure diagram of the prediction unit of the screening system in the embodiment of the present application is shown;

[0062] Figure 8 A structure diagram of the screening device according to the embodiment of the present application is shown. DETAILED DESCRIPTION

[0063] The present application is described below based on embodiments, but the present application is not limited to only these embodiments. In the following detailed description of the present application, some specific details are described in detail. The present application can also be completely understood without the description of these details by those skilled in the art. In order to avoid confusion of the essence of the present application, the well-known methods, processes, and procedures are not described in detail. In addition, the drawings are not necessarily drawn to scale.

[0064] Figure 1 A flowchart of the screening method in the embodiment of the present application is shown, and the specific steps include S101-S104.

[0065] In step S101, historical flow data of a plurality of target items is obtained from a storage system, and historical values of target variables of the plurality of target items are extracted therefrom.

[0066] In step S102, a prediction selection probability corresponding to the target item is generated according to the historical values of the target variables.

[0067] In step S103, the prediction selection probability and the historical values of the target variables are input into an established expectation value model, and the target items are analyzed for screening, to generate expectation values of the target variables of the plurality of target items.

[0068] In step S104, the target items with the expectation values of the target variables greater than a preset threshold value are screened out, and the corresponding identification information thereof is stored in a database.

[0069] In the embodiment, an expectation value model is established, a prediction selection probability of a target item is generated according to historical values of a target variable of the target item, and an expectation value of the target variable is calculated based on the prediction selection probability by using the expectation value model. The target items are screened according to the expectation value of the target variable, and the identification information of the target items with better quality is stored.

[0070] Firstly, the historical traffic data of the target item is obtained from the storage system, and then the historical value of the target variable of the target item is extracted from the obtained historical traffic data of the target item. The source of the historical traffic data of the target item is, for example, the traffic data of the entire Jingdong website, the sales record data, the target item review data, the product front-end page burying point data, and the transaction data of a specific business scenario.

[0071] In one embodiment, the historical traffic data includes the target item exposure amount, the target item selection amount, and the exposure channel data of the target item. The target variable of the target item is, for example, the thousand times exposure conversion rate of the target item, which is one thousand times the ratio of the target item transaction amount to the target item exposure amount. At this time, the historical value of the target variable is the historical thousand times exposure conversion rate of the target item.

[0072] Then, the prediction selection probability corresponding to the target item is generated according to the historical value of the target variable, which can be converted into a classification problem of machine learning, and the selection probability is calculated by using the classification problem. For example, the probability that the historical value of the target variable is greater than zero is taken as the prediction selection probability of the target item.

[0073] Then, the established expected value model is used to calculate the expected value of the target variable of the target item. In this embodiment, the prediction selection probability of the target item obtained in the previous step is taken as a parameter and brought into the model, and then the expected value of the target variable is solved in combination with the historical value of the target variable of the target item. Steps S102-S103 are the core content of this embodiment. Although a classification model is used, the expected value of the target variable can be calculated according to the prediction selection probability of the target item, which simplifies the operation and has high reliability of the result.

[0074] Finally, an expected value threshold is set, for example, 0.8, and according to step S103, each target item corresponds to an expected value of a target variable. These calculation results are screened, the target items with the expected value of the target variable greater than the preset threshold are selected out, and the corresponding identification information of the selected high-quality target items is stored in the database for subsequent calling.

[0075] In one embodiment, the target variable of the target item is the thousand times exposure conversion rate of the target item. When the expected value of the thousand times exposure conversion rate is calculated by using the expected value model, the product of the prediction selection probability of the target item and the historical thousand times exposure conversion rate of the target item can be used as the expected value of the thousand times exposure conversion rate. The historical thousand times exposure conversion rate is, for example, the thousand times exposure conversion rate of the previous day or the thousand times exposure conversion rate of the previous 5 days.

[0076] The screening method of the embodiment can be applied to screening of high-quality products or qualified products, etc. For example, a plurality of target objects need to be screened from a batch of products including multiple models. First, historical data is called, and historical values of target variables corresponding to the target objects are obtained according to the historical data. Then, a prediction selection probability of a certain model product is calculated. Finally, the expected value model is substituted to generate the expected value of the target variable. The products with the expected value greater than the threshold value are screened as high-quality products, thereby completing the screening of the target objects. The corresponding mark information of the target objects is stored, and in the future, the target objects can be directly called to find the model of the target objects.

[0077] In the embodiment, the prediction selection probability of the target object is obtained through the historical value of the target variable of the target object, and the expected value model is used to obtain the expected value of the target variable according to the prediction selection probability, and the target object is screened according to the high and low of the expected value of the target variable. In the calculation process of the expected value of the target variable, the prediction selection probability of the target object is added, so that the expected value of the target variable is more accurate, and the screened target object is more in line with the demand, thereby improving the screening quality of the target object.

[0078] In one embodiment, when calculating the prediction selection probability of the target object, a method based on discrete transformation of the target variable of the target object is used to perform continuous processing on the attribute data of the target object, so as to obtain the prediction selection probability. The attribute data of the target object is also obtained from the historical traffic data, including the product category, the exposure channel, etc. The ratio of the number of positive samples of the historical value of the target variable corresponding to the target object under the classification of the specified attribute data to the exposure amount of the target object is taken as the prediction selection probability. For specific steps, refer to the description of Figure 2a .

[0079] Figure 2a The specific flow chart of step S102 shown in Figure 1 The specific flow chart of step S102 shown in

[0080] In step S1021, the specific value of the attribute data and the corresponding exposure amount of the target object are combined into a pair, and arranged in descending order according to the exposure amount of the target object.

[0081] The historical traffic data of the target item is obtained to acquire its attribute data. This historical traffic data includes the target item's exposure, selection count, and exposure channels. The target item can be categorized according to its attribute data, resulting in different types. Different categories correspond to different predicted selection probabilities and expected values ​​for different target variables. In this step, the exposure of the target item within the category corresponding to a specific value of a given attribute data is counted. In one embodiment, the specific value of the attribute data is, for example, the name of a specific target item within a certain item category. The exposure and selection counts corresponding to this target item are counted, and then the specific target item and its corresponding exposure are paired, sorted in descending order of exposure count, i.e.: {{X1,m1}, {X2,m2}, ..., {X... N ,m N}}, where X i Let m be the i-th value of the categorical variable. i For X i The corresponding exposure amounts are m1>m2>…>m N .

[0082] In step S1022, the historical values ​​of the target variable for each category of the target item are obtained and the number of samples with historical values ​​of the target variable being positive is counted.

[0083] When the attribute data is an item category, it can include primary categories, secondary categories, tertiary categories, etc., and each category includes multiple target items. Each target item corresponds to a target variable. When the target variable is the conversion rate per thousand impressions of the target item, the conversion rate per thousand impressions is denoted as P, the number of times the target item is exposed is denoted as U, and the number of selected items is denoted as N.

[0084] In this embodiment, historical values ​​of the target variable that are greater than zero are recorded as positive samples, and those that are not are recorded as negative samples. For example, if the target variable is defined as the conversion rate per thousand exposures of the target item, a value greater than 0 is a positive sample and is recorded as 1, and otherwise it is a negative sample and is recorded as 0.

[0085] In step S1023, some pairs in the corresponding number pair column of attribute data are combined into one number pair. The first n variables {X1, X2, ..., X...} are taken. n}, the remaining {X n+1 ,…,X N All merged into {X} others The processed variable list is: {{X1,m1}, {X2,m2}, ..., {X...} n ,m n},

[0086] In step S1024, for the processed pair of columns, the average value of the historical value of the target variable corresponding to the specific value of each attribute data is calculated.

[0087] For the processed variable column {{X1,m1}, {X2,m2}, …, {X n n}, The average value of the historical value of the target variable y corresponding to each value {X i} is calculated, and the calculation formula is: Finally, X i is replaced by p i , and the data source is established.

[0088] In this embodiment, X represents attribute data, for example, item category or traffic channel, and the target item is divided into first-level category, second-level category, and third-level category. X i represents the specific attribute value corresponding to the first-level category, second-level category, and third-level category, such as sports and outdoor, sports goods, beauty and skin care, and cleaning supplies, or the specific value corresponding to the exposure channel, such as WeChat and applet. m i represents the number or exposure of X i .

[0089] In this embodiment, a discrete transformation method based on the target variable is used to replace the traditional one-hot encoding for the classification variables such as target item category and exposure channel. This method can effectively perform continuous processing on the classification variables based on historical data, thereby improving the overall effect of the model.

[0090] Figure 2b A schematic diagram of the continuous processing process of attribute data using the discrete transformation method is shown. In combination with Figure 2a , as shown in Figure 2b , in this embodiment, X represents the first-level category, second-level category, and third-level category of the target item, and the exposure channel. X i represents the specific attribute value, such as sports and outdoor, sports goods, beauty and skin care, and cleaning supplies, and WeChat and applet. m i represents the number of X i , i.e., the number, and y represents the number of positive samples of the target variable.

[0091] In the initial data set, there are 3 specific X i under the first-level category, 2, 3, and 1 second-level categories under each first-level category, and 4 specific X i under the exposure channel. Each target item corresponds to a target variable value. The numbers in the first column represent the SKU or number of the target item.

[0092] ​Data cleaning is performed on the initial data set, and classification and calculation are performed according to different classification variables, Figure 2b The prediction selection probability corresponding to each classification variable of the first-level category, the second-level category and the exposure channel is given.

[0093] The first-level category corresponds to three X i , that is, outdoor sports, beauty and skin care, household appliances, and the corresponding m is 5, 4, 1, and the corresponding target variable and y is 3, 2, 1 (the number of positive samples), where y is, for example, the number of times of thousand times exposure conversion rate greater than zero, and the average value of the historical value of the target variable is: p1=3 / 5=0.6, p2=2 / 4=0.5, p3=1 / 1=1. The calculated p is used as the prediction selection probability, so the prediction selection probability of the sports and outdoor target product is 0.6, the prediction selection probability of the beauty and skin care target product is 0.5, and the prediction selection probability of the household appliance target product is 1.

[0094] Similarly, the prediction selection probability of the target product corresponding to different attribute data can be calculated, and finally a new data set is counted as p i represents X i , and is arranged into a transformed data set for subsequent operation.

[0095] The model design scheme of the embodiment abstracts the screening problem of the target product into a specific algorithm model, converts it into a binary classification problem to calculate the prediction selection probability of the target product, and creatively uses the prediction selection probability to calculate the expected value of the thousand times exposure conversion rate of the target product in the future, combines the regression problem, and is more in line with the demand, and the model operation is more accurate and stable.

[0096] Figure 2c A flowchart of an expected value model of the screening method in the embodiment of the application is shown, and specifically includes the following steps.

[0097] In step S201, the data source is loaded, and the historical traffic data and the prediction selection probability of the target product are obtained. The data source is directly loaded from the hive using the spark sql statement (such as Figure 2a-2b The data source is divided into a training data set and a test data set according to the database time partition. The training set data is used for training the model, and the test set data is used for verifying the model.

[0098] In step S202, the data source is divided into a training data set and a test data set according to the database time partition, and the training data set is divided into a training set and a validation set according to 7:3. The validation set and the test set are used to adjust the parameters of the model in the model verification process.

[0099] In step S203, a random forest model is adopted to adjust the model parameters through cross-validation according to the performance of the model on the validation set. The random forest model here is only a model selection optimization.

[0100] In step S204, the model file is serialized and saved. The trained and validated data are saved for subsequent operations.

[0101] In step S205, the expected value of the target variable is generated according to the prediction selected probability and the historical value of the target variable at a certain time granularity.

[0102] The final solution of the expected value model is defined as the expected value of the thousand exposure conversion rate, which is represented by a score value, i.e., score i =p i *C t-1 , wherein score i is the expected value of the thousand exposure conversion rate of the target item i, p i is the prediction selected probability of the model, i.e., the prediction selected probability of the target item i, C t-1 is the thousand exposure conversion rate of the target item i at t-1 time, and t is the time granularity, which can be day, week, month, quarter, etc. Here, C t-1 is the historical thousand exposure conversion rate, which can also represent the historical value of the target variable of the target item.

[0103] In the embodiment, the model design scheme is two-stage. First, the screening problem of the item is converted into a machine learning classification problem to calculate the prediction selected probability of a single target item. Second, the expected value of the future thousand exposure conversion rate of the target item is obtained by multiplying the historical (e.g., the previous day) thousand exposure conversion rate of the target item by the prediction selected probability, so that the reliability of the obtained expected value result is good.

[0104] Figure 3 A flowchart of the screening method summarized in the embodiment of the application is shown, which specifically includes the following steps.

[0105] In step S301, historical traffic data of a plurality of target items is obtained from a storage system, and historical values of target variables of the plurality of target items are extracted therefrom.

[0106] In step S302, prediction selected probabilities corresponding to the target items are generated according to the historical values of the target variables.

[0107] In step S303, a plurality of attribute data of the target items are extracted from the historical traffic data, and the plurality of target items are classified according to the attribute data.

[0108] In step S304, the predicted selected probability and the historical value of the target variable are input into the established expected value model to perform screening analysis on the target items, and expected values of the target variables of the target items are generated.

[0109] In step S305, the target items whose expected values of the target variables are greater than the preset threshold are screened out, and the corresponding identification information is stored in the database.

[0110] In step S306, the predicted selected probability and the expected value of the target variable of the target item in different exposure channels are calculated.

[0111] In step S307, the target item exposure amount with gradient reduction is set in different exposure channels according to the descending order of the expected value of the target variable.

[0112] The embodiment is a more perfect screening method than the foregoing embodiments. Steps S301-S302 and S304-S305 are the same as steps S101-S104 of Figure 1 , which will not be described here.

[0113] In step S303, multiple attribute data of the target items are extracted from the historical traffic data, and the multiple target items are classified according to the attribute data.

[0114] The target items include multiple attribute data, and the target items can be classified according to the attribute data. The target items correspond to different predicted selected probabilities and different expected values of the target variables under different classifications. In combination with the descriptions of Figure 2a and Figure 2b , the predicted selected probability of the target items under different attribute data classifications can be obtained, and the historical value of the target variable under different classifications can also be obtained. Therefore, the expected values of different target variables can be obtained after calculation in the model, and there is an expected value for each type of attribute data, which can be used for screening according to actual needs. For example, the expected value of the target variable corresponding to the target items under the first-level category is used for item screening.

[0115] Steps S306-S307 involve the exposure channels of the target items. In step S306, the predicted selected probability and the expected value of the target variable of the target item in different exposure channels are calculated. The exposure channel is taken as an attribute data, the conversion rate per thousand exposures and the predicted selected probability of the target items corresponding to each channel are calculated, and the exposure channel with a higher expected value of the conversion rate per thousand exposures is selected for exposure of the target items.

[0116] In step S307, the target item exposure quantity with gradient reduction is set on different exposure channels according to descending order of the expected value of the target variable. The expected value of the target variable calculated by taking the exposure channel as attribute data is sorted in descending order, the exposure quantity of the target item is more in the channel with high expected value, and the exposure quantity of the target item is reduced in the exposure channel with low expected value. When the target item needs to be exposed, the exposure can be performed in this way, and the high-quality product is exposed more.

[0117] In one embodiment, the screening method can further include display of the target item, for example, the expected value of the target variable is sorted in descending order, and the target item is displayed to the client according to the sorting order. The expected value greater than the threshold value screened in step S304 is sorted in descending order, and the target item is displayed on the client according to the sorting order, so that the high-quality target item can also be observed on the mobile device.

[0118] Here, the deployment of the corresponding expected value model is: deploying the Scala script and the model file into the spark platform, and writing the target item thousand-time exposure conversion rate expected value predicted by the model into a hive file for storage. The expected value is sorted in descending order according to the prediction, and the target item is screened and displayed according to the SKU of the head target item.

[0119] In the embodiment of the application, the influence of the exposure channel on the thousand-time exposure conversion rate of the target item is fully utilized, the exposure channel is taken as an attribute data for processing, and a large amount of target item exposure is performed on the exposure channel with high expected value of the predicted thousand-time exposure conversion rate, so as to improve the selection rate of the target item. The embodiment considers the influence of the attribute data on the target variable, so that the target item obtained finally is more high-quality.

[0120] Figure 4 A line graph showing the comparison of business improvement rates before and after the screening method of the embodiment of the application is shown. As shown in FIG. 8, the horizontal coordinate represents the number of experiments (the number of experimental groups), and the vertical coordinate represents the business improvement rate. Figure 4

[0121] In one embodiment, the effect verification adopts an AB test scheme to iteratively correct the expected value model. In the embodiment, the original business strategy is taken as a control group, that is, the head target item is recommended according to the average thousand-time exposure rate of the target item in the last week. And a plurality of experimental groups including the screening method of the application are set.

[0122] As shown in FIG. 9, the horizontal coordinate represents the number of experiments (the number of experimental groups), and the vertical coordinate represents the business improvement rate. Figure 4 ​As shown: In the first experiment, the experimental group used the traditional regression method, and the business improvement rate of the experimental group was -15.6% higher than that of the control group; in the second experiment, the experimental group used a multi-classification model, and the business improvement rate of the experimental group was -1.3% higher than that of the control group; in the third experiment, a binary classification model was used, and the business improvement rate of the experimental group was 7.8% higher than that of the control group; in the fourth experiment, a binary classification model was used, and the expected value of the conversion rate per thousand impressions was calculated, and the business improvement rate of the experimental group was 10.8% higher than that of the control group; in the fifth experiment, based on the fourth experiment, the target item category feature was added, and a discrete transformation method based on the target variable was used, and the business improvement rate of the experimental group was 15.1% higher than that of the control group; in the sixth experiment, based on the fifth experiment, traffic channel feature was added, and a discrete transformation method based on the target variable was used, and the business improvement rate of the experimental group was 22.5% higher than that of the control group.

[0123] The first and second comparative experiments show that the classification model is more effective than the regression model; the second and third comparative experiments show that the binary classification model is more suitable and more stable; the third and fourth comparative experiments show that using the expected value of the conversion rate per thousand impressions can bring a 2% increase in business rate compared to simply using the conversion probability for target item recommendation.

[0124] Experiments four, five, and six show that the discrete transformation method based on the target variable significantly improves business performance. Furthermore, the model utilizing the impact of category information and traffic channel information on the conversion rate of target items demonstrates the most significant improvement in business rate. In summary, the screening method of this invention, in the group-buying business, ultimately improves the conversion rate of target items by 22.5% per thousand impressions.

[0125] In one embodiment, cluster modeling can be used for the attribute data, and the final aggregated results can be used as the final output to predict the expected value of the target variable.

[0126] In another embodiment, regression prediction can be performed directly on the conversion rate per thousand exposures of the target item to build a model of a more complex system and solve for the expected value of the target variable.

[0127] It should be noted that the present invention is not limited to the algorithm rules provided in the above embodiments; other algorithm rules can also be used to practice the screening method provided in the embodiments of the present invention.

[0128] Figure 5 A structural diagram of the screening system in an embodiment of the present invention is shown.

[0129] The screening system 500 includes a data acquisition unit 501, a prediction unit 502, an expected value calculation unit 503, and a screening unit 504.

[0130] The data acquisition unit 501 is configured to acquire historical traffic data of the target items from a storage system, and extract historical values of target variables of the target items from the historical traffic data;

[0131] The prediction unit 502 is configured to generate a predicted selection probability of the target items according to the historical values of the target variables.

[0132] The expected value calculation unit 503 is configured to input the predicted selection probability and the historical values of the target variables into an established expected value model, perform screening analysis on the target items, and generate expected values of the target variables of the target items.

[0133] The screening unit 504 is configured to screen out the target items with expected values of the target variables greater than a preset threshold, and store identification information of the target items into a database.

[0134] The historical traffic data includes exposure of the target items, transaction volume of the target items, and exposure channel data of the target items, the target variable of the target items is a thousand times conversion rate of the target items, and the thousand times conversion rate is a thousand times of a ratio of selected volume of the target items to exposure of the target items.

[0135] In the embodiment, the predicted selection probability of the target items is obtained according to the historical values of the target variables of the target items, the expected values of the target variables are obtained according to the predicted selection probability by using the expected value model, and the target items are screened according to the expected values of the target variables. The predicted selection probability of the target items is added in the calculation process of the expected values of the target variables, so that the expected values of the target variables are more accurate, and the screened target items are more in line with the demand, thereby improving the screening quality of the target items.

[0136] Figure 6 A structure diagram of the screening system is shown. Figure 6 The embodiment shown in the figure is based on the embodiment shown in Figure 5 The classification unit 601 is added to the embodiment shown in the figure.

[0137] The classification unit 601 is configured to extract a plurality of attribute data of the target items from the historical traffic data, and classify the target items according to the attribute data. After the classification, the predicted selection probability and the expected values of the target variables corresponding to the target items in different classifications can be acquired respectively.

[0138] It should be understood that the system and method of the embodiment of the present application are corresponding, and therefore, the system is described in a relatively brief manner.

[0139] Figure 7 A structure diagram of the prediction unit 502 of the screening system in the embodiment of the present application is shown.

[0140] The prediction unit 502 is configured to generate a prediction selected probability corresponding to the target item according to the historical value of the target variable. In an embodiment, the prediction unit 502 adopts a method based on a discrete transformation of the target variable of the target item, performs a continuous processing on the attribute data of the target item to obtain the prediction selected probability, and the target item corresponds to different prediction selected probabilities and different expected values of the target variable under different classifications. The ratio of the number of positive samples of the historical value of the target variable corresponding to the target item under the specified classification of the attribute data to the exposure amount of the target item is taken as the prediction selected probability.

[0141] Specifically, the prediction unit 502 includes a pair generation unit 5021, a statistical unit 5022, a collation unit 5023 and a mean value calculation unit 5024.

[0142] The pair generation unit 5021 is configured to form a pair of the specific value of the attribute data and the corresponding exposure amount of the target item, and arrange the pairs in descending order of the exposure amount of the target item.

[0143] The statistical unit 5022 is configured to obtain the historical value of the target variable of the target item under each classification and count the number of positive samples of the historical value of the target variable.

[0144] The collation unit 5023 is configured to integrate part of the pairs in the pair column of the attribute data into one pair.

[0145] The mean value calculation unit 5024 is configured to calculate the average value of the historical value of the target variable corresponding to the specific value of each attribute data for the processed pair column.

[0146] Figure 8 A structure diagram of the screening device of the embodiment of the application is shown. Figure 8 The device shown is only an example and should not constitute any limitation on the function and use range of the embodiment of the application.

[0147] Reference Figure 8The screening device 800 includes a processor 801, a memory 802 and an input / output device 803 connected through a bus. The memory 802 includes a read-only memory (ROM) and a random access memory (RAM), and stores various computer instructions and data required for performing system functions. The processor 801 reads various computer instructions from the memory 802 to perform various appropriate actions and processes. The input / output device includes an input part of a keyboard, a mouse, etc.; an output part of a cathode ray tube (CRT), a liquid crystal display (LCD), etc., and a speaker, etc.; a storage part of a hard disk, etc.; and a communication part of a network interface card such as a LAN card, a modem, etc. The memory 802 also stores computer instructions for performing operations specified by the screening method of the embodiment of the present application: obtaining historical flow data of a plurality of target items from a storage system, and extracting historical values of target variables of the plurality of target items therefrom; generating a predicted selection probability corresponding to the target items according to the historical values of the target variables; inputting the predicted selection probability and the historical values of the target variables into an established expected value model to perform screening analysis on the target items, and generating expected values of the target variables of the plurality of target items; and screening out target items whose expected values of the target variables are greater than a preset threshold, and storing identification information corresponding thereto into a database.

[0148] Accordingly, the embodiment of the present application provides a computer readable storage medium storing computer instructions, which, when executed, implement the operations specified by the screening method.

[0149] The flowcharts and block diagrams in the drawings illustrate the possible system framework, functions and operations of the system, method and device of the embodiment of the present application. The blocks on the flowcharts and block diagrams can represent a module, a program segment or only a piece of code, which are executable instructions for implementing specified logic functions. It should also be noted that the executable instructions for implementing the specified logic functions can be recombined to generate new modules and program segments. Therefore, the blocks and the block sequence in the drawings are only used to better illustrate the processes and steps of the embodiment, and should not be regarded as a limitation on the present application itself.

[0150] The various modules or units of the system can be implemented by hardware, firmware or software. The software, for example, includes coded programs formed by various programming languages such as JAVA, C / C++ / C#, SQL, etc. Although the steps and the sequence of the steps of the embodiment of the present application are given in the method and the method diagram, the executable instructions for implementing the specified logic functions of the steps can be recombined to generate new steps. The sequence of the steps should not be limited to the sequence of the steps in the method and the method diagram, and can be adjusted at any time according to the needs of the functions. For example, some of the steps can be performed in parallel or in reverse order.

[0151] The system and method according to the present application can be deployed on a single or multiple servers. For example, different modules can be deployed on different servers respectively, forming dedicated servers. Alternatively, the same functional units, modules or systems can be deployed on multiple servers in a distributed manner to alleviate the load pressure. The servers include but are not limited to multiple PC machines, PC servers, blade machines, supercomputers, etc. connected in the same local area network and through the Internet.

[0152] The above description is merely preferred embodiments of the present application, but not for limiting the present application. For the ordinary skilled in the art, any modifications, equivalent replacements, improvements, etc. made within the spirit and principle of the present application shall fall into the scope of protection of the present application.

Claims

1. A screening method, characterized by, The method comprises the following steps: obtaining historical traffic data of a plurality of target items from a storage system, and extracting historical values of a target variable of the plurality of target items from the historical traffic data, the target variable of the target item being a thousand-exposure conversion rate of the target item, and the thousand-exposure conversion rate being a thousand times of a ratio of a selected amount of the target item to an exposure amount of the target item; generating a predicted selection probability corresponding to the target item according to the historical values of the target variable; inputting the predicted selection probability and the historical values of the target variable into an established expected value model to perform screening analysis on the target item, and generating expected values of the target variable of the plurality of target items; and screening out the target item whose expected value of the target variable is greater than a preset threshold, and storing identification information corresponding to the target item into a database, wherein the generating of the predicted selection probability corresponding to the target item according to the historical values of the target variable comprises: counting the exposure amount of the target item under a classification corresponding to a specific value of attribute data, the specific value of attribute data being a target item name under a certain item category, forming a pair of the specific target item and the corresponding target item exposure amount, and arranging the pair in descending order of the target item exposure amount; obtaining the historical values of the target variable of the target item under each classification, recording the historical values of the target variable greater than zero as positive samples, otherwise recording as negative samples, and counting the number of positive samples of the historical values of the target variable; integrating part of the pairs in the pair column corresponding to the attribute data into one pair; for the processed pair column, calculating the ratio of the number of positive samples of the historical values of the target variable corresponding to the specific value of the specified attribute data to the target item exposure amount as the predicted selection probability.

2. The screening method according to claim 1, characterized in that, The historical traffic data comprises target item exposure amount, target item selected amount and target item exposure channel data.

3. The screening method according to claim 1, characterized by, Further comprising: extracting a plurality of attribute data of the target item from the historical traffic data, and classifying the plurality of target items according to the attribute data.

4. The screening method according to claim 3, characterized in that, Using a method based on discrete transformation of the target variable of the target item, the attribute data of the target item is continuously processed to obtain the predicted selection probability, and the target item corresponds to different predicted selection probabilities and different expected values of the target variable under different classifications.

5. The screening method according to claim 1, wherein The attribute data comprises item category and exposure channel.

6. The screening method according to claim 5, characterized in that, Further comprising: calculating the predicted selection probability and the expected value of the target variable of the target item on different exposure channels; setting the target item exposure amount with gradient reduction on different exposure channels according to the descending order of the expected value of the target variable.

7. The screening method according to claim 1, wherein The establishment process of the expected value model comprises: loading a data source to obtain historical traffic data and a predicted selection probability of a target item; dividing the data source into a training data set and a test data set according to a database time partition, and dividing the training data set into a training set and a validation set according to a ratio of 7:3; using a random forest model to adjust model parameters through cross-validation according to the performance of the model on the validation set; serializing and saving the model file; Generate an expected value of the target variable according to the predicted selection probability and a historical value of the target variable under a certain time granularity.

8. A screening system characterized by, Comprise: A data acquisition unit for acquiring historical traffic data of a plurality of target items from a storage system and extracting historical values of a target variable of the plurality of target items therefrom, the target variable of the target item being a thousand-exposure conversion rate of the target item, the thousand-exposure conversion rate being a thousand times of a ratio of a selected amount of the target item to an exposure amount of the target item; A prediction unit for generating a predicted selection probability corresponding to the target item according to the historical values of the target variable; An expected value calculation unit for inputting the predicted selection probability and the historical values of the target variable into an established expected value model, performing screening analysis on the target item, and generating expected values of the target variable of the plurality of target items; And A screening unit for screening the target item whose expected value of the target variable is greater than a preset threshold value and storing identification information corresponding thereto into a database, Wherein, the prediction unit comprises: A pair generation unit for counting the exposure amount of the target item under a classification corresponding to a specific value of attribute data, the specific value of attribute data being a target item name under a certain item category, forming a pair of specific target item and corresponding target item exposure amount, and arranging in descending order according to the target item exposure amount; A statistical unit for acquiring historical values of the target variable of the target item under each classification, recording the historical values of the target variable greater than zero as positive samples, otherwise as negative samples, and counting the number of positive samples of the historical values of the target variable; A collation unit for integrating part of the pairs in the pair column corresponding to the attribute data into one pair; A mean calculation unit for calculating, for the processed pair column, the ratio of the number of positive samples of the historical values of the target variable corresponding to the specific value of the attribute data to the target item exposure amount as the predicted selection probability.

9. The screening system of claim 8, wherein, The historical traffic data comprises target item exposure amount, target item selected amount and exposure channel data of the target item.

10. The screening system of claim 8, wherein, Further comprise: A classification unit for extracting a plurality of attribute data of the target item from the historical traffic data and classifying a plurality of the target items according to the attribute data.

11. The screening system of claim 10, wherein, Adopting a method based on discrete transformation of the target variable of the target item, the attribute data of the target item is continuously processed to obtain the predicted selection probability, the target item corresponds to different predicted selection probabilities and different expected values of the target variable under different classifications.

12. A computer-readable storage medium, characterized in that, The computer readable storage medium stores computer instructions, the computer instructions are executed to realize the screening method of any one of claims 1 to 7.

13. A screening device characterized by, Comprise: A memory for storing computer instructions; A processor coupled to the memory, the processor is configured to execute the screening method of any one of claims 1 to 7 based on the computer instructions stored in the memory.

Images