Target user screening method, readable storage medium and computing device

Abstract

The invention discloses a method for screening a target user, a readable storage medium and computing equipment, and the method comprises the steps: predicting the external characteristics of any alternative user of a target mechanism according to the internal characteristics of the alternative user, and obtaining the predicted external characteristics; predicting a first conversion rate of the alternative user according to the predicted external feature, and determining whether the alternative user is a preliminary screening target according to the first conversion rate and query cost for an external mechanism; if the alternative user is a preliminary screening target, acquiring external features of the alternative user; predicting a second conversion rate of the alternative user according to the internal characteristics and the external characteristics of the alternative user; if the second conversion rate is higher than the second threshold value, the alternative user is used as the target user, the conversion rate before the external features are actually obtained and after the external features are obtained can be predicted, whether the external features are obtained or not is determined according to the predicted conversion rate, the cost of screening the target user can be reduced, and a good screening effect is achieved.

Description

TECHNICAL FIELD

[0001] The embodiment of the present specification belongs to the technical field of data processing, and particularly relates to a method for screening target users, a readable storage medium and a computing device. BACKGROUND

[0002] In many scenarios, an organization needs to push specific functions, services or information to users, in order to further understand and use their own products.

[0003] Generally, due to the need for certain overhead in the pushing process, in order to avoid excessive overhead, the organization will screen target users according to self-set standards and only push to target users.

[0004] In this process of screening target users, the organization often predicts the willingness of each user to understand the pushing content according to the user characteristics it holds, and then selects users with high predicted conversion rate as target users.

[0005] Obviously, the more user characteristics the organization holds, the deeper its understanding of users, and the more accurate its prediction of user willingness. Therefore, some organizations will choose to use user characteristics held by external organizations to assist in prediction in order to expand the scope of pushing. However, the user characteristics of external organizations introduce additional overhead. In the process of screening target users, the comprehensive cost of the organization is always difficult to control.

[0006] Therefore, there is an urgent need for a scheme for screening target users to at least partially solve the above problems. SUMMARY

[0007] The purpose of the present application is to provide a method for screening target users, a readable storage medium and a computing device, comprising:

[0008] The first aspect of the present specification provides a method for screening target users, the method comprising:

[0009] For any alternative user of a target organization, according to a number of internal characteristics of the alternative user, predict the external characteristics of the alternative user to obtain predicted external characteristics; wherein the internal characteristics are held by the target organization, and the external characteristics are held by an external organization;

[0010] According to the predicted external characteristics, predict the first conversion rate of the alternative user;

[0011] According to the first conversion rate and the query cost for the external organization, determine whether the alternative user is a primary screening target;

[0012] If the alternative user is a primary screening target, obtain the external characteristics of the alternative user;

[0013] predict a second conversion rate of the alternative user according to a plurality of internal features of the alternative user and the external feature;

[0014] if the second conversion rate is higher than a second threshold, take the alternative user as a target user.

[0015] The second aspect of the present specification provides a computer readable storage medium, which stores a computer program, and when the computer program is executed in a computer, the computer program causes the computer to execute the method according to the first aspect.

[0016] The third aspect of the present specification provides a computing device, which includes a memory and a processor, the memory stores executable code, and when the processor executes the executable code, the method according to the first aspect is implemented.

[0017] The scheme for screening target users provided by the embodiment can predict the conversion rate after obtaining the external feature before actually obtaining the external feature, and then decide whether to obtain the external feature according to the predicted conversion rate, so that a parameter-controllable and highly adjustable external feature obtaining scheme for multiple types of delivery activities is proposed, the cost of screening target users can be reduced, and a better screening effect can be achieved. BRIEF DESCRIPTION OF DRAWINGS

[0018] In order to more clearly illustrate the technical solutions of the embodiments of the present specification, the following will briefly introduce the drawings needed to be used in the embodiment description. Obviously, the drawings in the following description are only some embodiments described in the present specification, and for those skilled in the art, other drawings can also be obtained according to these drawings without creative labor.

[0019] Figure 1 is a schematic diagram of the influence of the number of user features on the screening condition and the screening process in an embodiment of the present specification;

[0020] Figure 2 is a schematic diagram of the relationship between the target organization and the external organization in an embodiment of the present specification;

[0021] Figure 3 is a flowchart of a method for screening target users in an embodiment of the present specification;

[0022] Figure 4 is a flowchart of a training method of a conversion rate prediction model in an embodiment of the present specification. DETAILED DESCRIPTION

[0023] To enable those skilled in the art to better understand the technical solutions in this specification, the technical solutions in the embodiments of this specification will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this specification, and not all embodiments. Based on the embodiments in this specification, all other embodiments obtained by those skilled in the art without creative effort should fall within the scope of protection of this specification.

[0024] Typically, users register their personal information with organizations providing online services. These organizations can then construct various user profiles based on this registration information and the user's usage of their services. When new content (such as features, services, or information) is available, target organizations can filter their user base based on the correlation between user profiles and the content (e.g., if a user's browsing frequency of information in a certain field reaches a certain threshold, they can be considered interested in that field, and therefore more likely to be interested in the relevant content). This allows for precise targeting of users. This approach reduces advertising costs and avoids user aversion to advertising campaigns.

[0025] In the process of screening target users, the target users selected by the target organization based on its own user characteristics may be missed compared with the real user group who are interested in the content. The more user characteristics used in the screening, the less the target organization will miss the real user group.

[0026] For example, such as Figure 1 As shown, the target organization can construct five screening conditions "A>x1", "A>x1", "A>x1", "A>x1", and "A>x1" by combining three user features A, B, and C.<x2 AND B> Given the user characteristics A, B, C, D, and E, the target organization can construct filtering conditions that include not only the aforementioned five filtering conditions but also those resulting from newly added user characteristics, such as "D>x8" and "E>x9". Furthermore, users meeting any of these filtering conditions can be identified as target users, and more filtering conditions allow the target organization to identify even more target users.

[0027] Therefore, in order to screen out as many target users as possible while ensuring the accuracy of identification, the target organization will choose to obtain more types of user characteristics from external organizations and combine external characteristics (user characteristics held by external organizations) with internal characteristics (user characteristics held by the target organization itself) to screen its own users.

[0028] In existing technologies, methods for obtaining external features can be divided into two types:

[0029] One approach is to acquire all the external characteristics of all its users. In other words, when a target organization needs to decide whether to target n users, it acquires the feature data of all n users' external characteristics from external organizations.

[0030] Secondly, based on empirical parameters or standards set by experts, a threshold for external parameters is determined, and then it is decided whether to acquire external features. Taking the example that the target organization needs to decide whether to target n users, the target organization provides the aforementioned threshold to the external organization, and the external organization only provides the feature data of the users whose external features meet the threshold among these n users to the target organization.

[0031] However, the aforementioned methods in the existing technology each have their own problems. Obtaining external features requires a certain query cost, and the more users obtained, the higher the cost. In the first method, obtaining the external features of all users undoubtedly greatly increases the total cost of the campaign. The second method relies on experience or expert judgment, and its stability and effectiveness are difficult to guarantee when faced with diverse campaign needs and a wide variety of user features.

[0032] Therefore, this specification provides a method for screening target users, offering a solution that balances campaign costs and campaign effectiveness.

[0033] Figure 2 A schematic diagram illustrating the relationship between the target mechanism and external mechanisms in one embodiment of this specification is shown. The target mechanism is the executor of the method for screening target users. Figure 2 As shown, the target organization and the external organization share the same group of users, but the user characteristics held by the target organization and the external organization for the same users differ. That is, the external characteristics of the external organization referred to in this specification are user characteristics that the target organization does not possess. For example, the internal characteristics held by the target organization may include the user's registration period, behavioral characteristics, login habits, etc.; while the external characteristics held by the external organization may include the user's consumption characteristics, application installation status, etc.

[0034] Based on the above relationship Figure 3 A flowchart illustrating a method for screening target users according to an embodiment of this specification is shown.

[0035] How Figure 3 The method shown can be performed by the target organization and includes the following steps:

[0036] Step S301: For any candidate user of the target institution, according to several internal features of the candidate user, the external features of the candidate user are predicted to obtain predicted external features; wherein, the internal features are held by the target institution, and the external features are held by an external institution.

[0037] Before step S301 is performed, the target institution can determine potential users of the target institution who will perform a delivery activity in the future as several candidate users, that is, users who may need to determine whether to perform a delivery activity by using external features. The following will be described as an example. Figure 3 The delivery activity performed on the target user in the method shown in Figure 3 is called a target delivery activity. The range of candidate users can be determined by the target institution according to the type of the target delivery activity, for example, the candidate users can be all users of the target institution, or can be obtained by pre-screening from all users according to a threshold value set by experience. The present specification does not make specific limitations here.

[0038] It should be noted that for each candidate user, the processing measures performed by the target institution in the method shown in Figure 3 are the same, and the following will only take one candidate user as an example to describe the method of screening target users from candidate users.

[0039] Unlike the two methods of obtaining external features in the prior art, neither all candidate users' external features are obtained in the embodiment, nor a fixed threshold is set for screening. In order to balance the delivery cost and the delivery effect, the external features of the candidate user are obtained according to the income and cost of the target delivery activity for the candidate user in the embodiment. Wherein, the income can be understood as the conversion rate of the user after the target delivery activity is delivered, and the cost is determined by the consumption of the target delivery activity and the external features. For a single user, the conversion rate can represent the probability of successfully converting the user. Specifically, successful conversion in actual application scenarios can be represented as - the user makes specific conversion behavior. For example, the user participates in the delivery activity within a predetermined time range, the user browses the relevant web page of the delivery activity within a predetermined time range, etc. According to the type of the delivery activity and the needs of the implementer in the embodiment, the conversion behavior can be defined by the implementer. Correspondingly, the size of the conversion rate can represent the probability of the user making the conversion behavior.

[0040] To realize the prediction of the conversion rate in the subsequent step, in this embodiment, the external feature of the candidate user is first predicted. It should be noted that in the prior art, it is common to directly predict the conversion rate of the user by using the internal feature of the user. For example, in some implementations, the target institution can take the internal feature of the user owned by itself as sample data, determine a label according to the historical conversion behavior of the user for a specific type of delivery activity, train by using a common machine learning model, and obtain a basic screening model for the type of delivery activity. The basic screening model can output the conversion rate of the type of delivery activity according to the internal feature of the user.

[0041] However, in the subsequent step S305 in this embodiment, the purpose of the prediction of the first conversion rate to be performed is not only to predict whether the candidate user will successfully convert, but also to predict whether the first conversion rate of the candidate user will be different from the basic conversion rate predicted according to the internal feature after introducing the external feature as an auxiliary. Therefore, the first conversion rate in this embodiment cannot be determined by using the implementation in the prior art.

[0042] Of course, it should be noted that the internal feature and the external feature are owned by different institutions, and the target institution cannot directly obtain the external feature of the user without the provision of the external institution. Therefore, the prediction of the first conversion rate is also difficult to realize.

[0043] However, on the other hand, the external feature and the internal feature can be regarded as the mapping of the user characteristics of the same user in different feature spaces, that is, the external feature and the internal feature of the same user have certain connection and commonality.

[0044] Therefore, in this embodiment, the internal feature of the candidate user is used to predict the external feature of the candidate user, and the predicted external feature is used to predict the first conversion rate in the subsequent step.

[0045] Among them, the specific type of external feature can be determined by the target structure in advance, as long as the external feature is owned by the external institution and the user feature can be obtained in the subsequent step.

[0046] Further, since the external institution may own multiple different types of features, in some implementations, the target institution can determine the feature corresponding to the highest prediction accuracy as the external feature required to be predicted in step S301 according to the prediction accuracy when the external feature is used to fit the conversion rate of the user for the delivery activity.

[0047] In some implementations, the target institution can further determine a feature fusion model, so that the feature fusion model can be used to determine the fusion value of the external features held by the external institution. Further, in step S301, the fusion value of the external features of the candidate user is predicted according to the internal features of the candidate user, as the predicted external features.

[0048] Thus, compared with predicting for a single external feature, the fusion value of the external features can more comprehensively contain the feature information of the candidate user. The influence on the prediction accuracy of the first conversion rate caused by the selection of external features is avoided.

[0049] On the other hand, the fusion value of the external features can uniformly represent the external features with different parameter dimensions as a continuous variable, which is more convenient for subsequent fitting, prediction and other calculations using the predicted external features.

[0050] To ensure that the predicted external features are representative of the real external features, in some implementations, the value of the external features of each first sample user (the external features of the first sample user are real external features obtained from the external institution) and the internal features can be determined. Further, the internal features of the first sample user are input into the feature fitting model to obtain the fitted external features of the first sample user, so that the corresponding fitted external features of the first sample user and the value of the real external features are as close as possible, and the parameters of the feature fitting model are adjusted.

[0051] The feature fitting model is used to predict the value of the external features according to the internal features. Thus, the internal features of the first sample user are used as input samples, and the value of the real external features of the first sample user is used as a label to train the parameters of the feature fitting model, so that the feature fitting model can learn the correlation between the internal features and the external features, and has the function of predicting the external features according to the internal features.

[0052] The feature fitting model can be a common machine learning model, such as a neural network model, a logistic regression model, etc., which is not limited in the present specification.

[0053] Further, if the predicted external feature in step S301 is a predicted value of a fusion value of a plurality of external features, in some implementations, before step S301, a plurality of first sample users can be pre-collected, and a plurality of external features of each first sample user (the external features of the first sample user are real external features obtained from an external agency) and a plurality of internal features of each first sample user can be determined. Then, the plurality of external features of the first sample user can be input into the feature fusion model to obtain a sample fusion feature of the first sample user, and the plurality of internal features of the first sample user can be input into the feature fitting model to obtain a fitted fusion feature of the first sample user, so as to minimize the difference between the fitted fusion feature and the sample fusion feature corresponding to the first sample user, and at least adjust the parameters of the feature fitting model.

[0054] The feature fusion model can be understood as an encoding model, which is used to encode the input plurality of external features with different parameter dimensions into a continuous variable, so as to facilitate the prediction in step S301. The feature fusion model can be, for example, an autoencoder model, which can be pre-trained using the plurality of external features of each first sample user.

[0055] Step S303: predicting a first conversion rate of the candidate user according to the predicted external feature.

[0056] As described above, the parameters of the feature fitting model are pre-adjusted, and then the predicted external feature is obtained by using the feature fitting model, so that the predicted external feature can represent the real external feature of the candidate user.

[0057] Further, to ensure that the first conversion rate obtained according to the predicted external feature can represent the conversion rate determined according to the real external feature of the candidate user, before step S303, a conversion rate prediction model corresponding to the external feature can also be pre-trained, so that the conversion rate prediction model learns the correlation between the real external feature and the conversion rate. Then, in step S303, the external feature is input into the conversion rate prediction model, and the output first conversion rate can simulate the conversion rate predicted according to the internal feature.

[0058] Therefore, by combining step S301 and step S303, the embodiment can simulate the process of predicting the conversion rate according to the external feature without obtaining the external feature, only by using the internal feature and the pre-trained model.

[0059] A training method of a conversion rate prediction model is given below. In some implementations, external features of a plurality of second sample users and historical conversion behaviors of each second sample user are obtained, a conversion behavior label of each second sample user is determined according to the historical conversion behavior, external features of any second sample user are taken as input of the conversion rate prediction model, and the conversion rate prediction model is trained to minimize the difference between the output of the conversion rate prediction model and the label of the second sample user. Thus, the trained conversion process prediction model can learn the correlation between external features and conversion rate.

[0060] The conversion rate prediction model can be any common machine learning model, such as a neural network model, a logistic regression model, etc., which is not limited in the present specification.

[0061] It should be noted that the number of second sample users used to train the conversion rate prediction model and the number of first sample users used to train the feature fitting model are much smaller than the total number of users of the target institution. Thus, even if the target institution needs to pay a certain cost to obtain the external features of the second sample users and the first sample users, the cost consumption is much smaller than the technical solution of obtaining the external features of all users in the prior art.

[0062] Step S305: determining whether the alternative user is a primary screening target according to the first conversion rate and the query cost of the external institution.

[0063] As described above, the first conversion rate can represent the probability of successfully converting the alternative user. Thus, the expected revenue for the alternative user can be determined according to the first conversion rate. On the other hand, the query cost of the external institution is public data. By combining the expected revenue and the query cost, it can be determined whether the alternative user is a primary screening target. The primary screening target is the alternative user whose external features need to be obtained in subsequent steps. On the other hand, if any alternative user is not determined to be a primary screening target, the processing of the alternative user is terminated in step S305, i.e., the alternative user will not be further determined to be a target user.

[0064] For the target institution, a higher first conversion rate indicates that the alternative user is more likely to be determined to be a target user in subsequent steps. Further, the expected revenue represents the possible revenue brought by the user conversion behavior after the alternative user is targeted. If the expected revenue can cover the query cost, it means that the cost consumption of obtaining external features, conducting targeted activities, etc. for the alternative user is recoverable.

[0065] Unlike the technical solutions in the prior art which rely on empirical parameters to determine whether to obtain external features, in the embodiment, a clear and interpretable method for determining the initial screening target is proposed using the first conversion rate and the query cost. Compared with the empirical parameters relying on intuition, the initial screening target determined by the method in the embodiment can be closer to the real target user.

[0066] In addition, when facing different types of delivery activities, the method provided in the embodiment can use different conversion behaviors corresponding to each type of delivery activity (for example, if the delivery activity is a flow diversion activity, the conversion behavior can be whether to click on a specified page, if the delivery activity is a promotion activity, the conversion behavior can be whether to place an order, etc.) to pre-train each model (feature fusion model, feature fitting model, conversion rate prediction model, etc.) used in the process of determining the first conversion rate. Thus, the method provided in the embodiment can flexibly adjust the selection strategy of the initial screening target according to the different types of delivery activities, and can adapt to various marketing directions.

[0067] In combination with the foregoing steps S301-S305, the embodiment provides a complete solution for predicting expected revenue and determining whether to obtain external features based on the balance between expected revenue and cost. Compared with various technical solutions in the prior art, the cost of obtaining external features can be reduced without losing real target users as much as possible.

[0068] In some implementations, the expected revenue for the candidate user can be determined according to the first conversion rate and the average conversion revenue, and when the expected revenue is higher than the query cost and the difference between the expected revenue and the query cost is higher than a threshold value, the candidate user is determined as the initial screening target.

[0069] The difference between the expected revenue and the query cost can be regarded as the activity cost of the delivery activity, and when the expected revenue is sufficient to cover the cost brought by obtaining the external feature, “obtaining the external feature of the candidate user” will bring positive revenue to the target institution, and thus the candidate user satisfying the foregoing condition is determined as the initial screening target.

[0070] In other implementations, the expected growth revenue can be determined according to the difference between the first conversion rate and the preset basic conversion rate, the return on investment can be determined according to the ratio of the expected growth revenue to the query cost for the external institution, and if the return on investment is higher than a preset return on investment threshold, the candidate user is determined as the initial screening target.

[0071] Wherein, the base conversion rate can be determined according to the historical conversion behaviors of the same type of historical delivery activities of the target delivery activity of all users of the target organization. Thus, if the expected growth benefit is positive, the first conversion rate of the candidate user is obviously higher than the average value. Further, the return rate is determined according to the ratio of the expected growth benefit to the query cost. The greater the predicted growth benefit, the greater the return rate, and the cost consumption for the candidate user is more likely to be recovered in subsequent delivery activities. Thus, the candidate user with a return rate higher than a preset return rate threshold is determined as a preliminary screening target.

[0072] Step S307: If the candidate user is a preliminary screening target, the external features of the candidate user are obtained.

[0073] If the candidate user is determined as a preliminary screening target, the target organization can obtain the external features of the candidate user from an external organization. That is, the external features obtained in step S307 are the real external features of the candidate user.

[0074] Step S309: The second conversion rate of the candidate user is predicted according to the internal features and the external features of the candidate user.

[0075] After obtaining the external features of the candidate user, the pre-established comprehensive screening model can be used to predict the conversion rate of the candidate user, i.e., the second conversion rate, in combination with the real external features and internal features of the candidate user.

[0076] Wherein, the comprehensive screening model can be a common machine learning model. Specifically, the internal features and the external features of each third sample user can be taken as input, and the label corresponding to each third sample user is determined according to the conversion behavior of each third sample user, so as to train the comprehensive screening model with the objective of minimizing the difference between the conversion rate output by the comprehensive screening model and the label. The comprehensive screening model can be, for example, a neural network model, a logistic regression model, etc., which is not limited in the present specification.

[0077] It should be noted that the external features used for training the comprehensive screening model are also real external features obtained from an external organization. Thus, it can be ensured that the second conversion rate predicted by the comprehensive screening model has a high fitting ability for the real conversion behavior of the user.

[0078] Here, the difference between the first conversion rate and the second conversion rate is further described.

[0079] In the process of predicting conversion rates, step S303 predicts the first conversion rate by using internal features to predict external features, and then uses the predicted external features to predict the first conversion rate. In this case, the target organization actually only holds internal features and each prediction model. In step S309, the second conversion rate is predicted by using both internal and external features. In this case, the target organization holds both internal and external features.

[0080] In terms of design concept, step S303 is to use internal features to simulate external features to predict conversion rate without knowing the external features, in order to obtain a different prediction result than when using internal features alone to predict conversion rate; while step S309 is to use the acquired internal features and external features to make the most accurate conversion rate prediction.

[0081] Step S311: If the second conversion rate is higher than the second threshold, the candidate user is selected as the target user.

[0082] After obtaining the second conversion rate, users whose second conversion rate is higher than the preset second threshold can be identified as target users. Furthermore, the target organization can then conduct subsequent advertising campaigns targeting these target users.

[0083] The second threshold can be set by the target organization according to the activity type of the corresponding campaign, and this specification does not impose specific restrictions here.

[0084] like Figure 3 The method shown can predict the conversion rate after obtaining external features before actually obtaining them, and then decide whether to obtain external features based on the predicted conversion rate. It proposes an external feature acquisition scheme with controllable parameters and strong adjustability for multiple types of campaigns, which can reduce the cost of selecting target users and achieve better screening results.

[0085] In addition, to further reduce the cost of acquiring external features, in some implementations, before step S301, a basic screening model can be used to determine the basic conversion rate of each candidate user of the target organization based on several internal features of each candidate user. Based on the basic conversion rate, several target users are determined among the candidate users, and the candidate users who are not determined as target users are used as alternative users.

[0086] It is mentioned in the introduction corresponding to step S301 in the foregoing that "the target institution can determine potential users of the user to be carried out in the user as a number of alternative users", and this implementation manner has the problem of excessive consumption of cost. Therefore, in this embodiment, the potential users are taken as candidate users, and a batch of target users are first screened out from the candidate users by using the basic screening model, and the candidate users who are not selected (not determined as target users) are determined as alternative users.

[0087] In this embodiment, the alternative users are the candidate users who are not selected, in other words, the alternative users in this embodiment are part of the potential users. Further, the number of the primary screening targets determined according to the part of the potential users will be much smaller than the number of the primary screening targets determined according to all the potential users. Therefore, in this embodiment, the number of external features required to be obtained is much smaller than that of the corresponding part of the technical solution in step S301, and thus the consumption of the cost of obtaining external features can be reduced.

[0088] On the other hand, since the target users determined from the candidate users by using the basic screening model will still be determined as target users under more abundant screening dimensions, this implementation manner of this embodiment will not affect the screening accuracy of the target users.

[0089] In summary, the method in this embodiment can reduce the acquisition cost of external features while ensuring the screening accuracy.

[0090] The specific method of screening target users from candidate users can be the same as the method of screening by using internal features in the prior art, and the present specification does not make specific limitations here. For example, the target institution can select a basic screening model corresponding to the type of the delivery activity according to the type of the delivery activity, and pre-set a conversion threshold for the conversion index, obtain the conversion index corresponding to each user by using the basic screening model, and screen the users with the corresponding conversion index higher than the conversion threshold as target users, that is, the target users can be screened from the candidate users.

[0091] In addition, it has been proposed in the foregoing that the predicted external features referred to in steps S301 and S303 can be a predicted value for a single external feature, or a predicted value for a fusion value of a plurality of external features. In the above part of the specification, the method is introduced as shown in Figure 3 For example, the predicted external feature is a predicted value for a single external feature. In actual use scenarios, it is obvious that the prediction of the conversion rate in combination with a plurality of external features will bring higher accuracy, and the implementation manner of the method will be given below when the predicted external feature is a predicted value for a fusion value of a plurality of external features. Figure 3

[0092] Specifically, the implementation manner of the method is as shown in Figure 3 ​In step S301, a fusion value of the external features of the candidate user is predicted according to the internal features of the candidate user as a predicted external feature; then, in step S305, the external features of the candidate user are obtained if the candidate user is a screening target; further, in step S309, a second conversion rate of the candidate user is predicted according to the internal features and the external features of the candidate user. Figure 3 In step S307, the external features of the candidate user are obtained if the candidate user is a screening target. Figure 3 In step S309, a second conversion rate of the candidate user is predicted according to the internal features and the external features of the candidate user.

[0093] In addition, according to the introduction of step S303, the embodiment proposes to pre-train a conversion rate prediction model for predicting the first conversion rate according to the predicted external feature, and the specific type of the conversion rate prediction model is not limited in the embodiment.

[0094] However, considering that there is a strong correlation between different user features in actual application scenarios. For example, for user feature A and user feature B, if the value of user feature A of a user is high, the value of user feature B of the user is usually low. Corresponding to the method shown in Figure 3 For a candidate user, the external features and the internal features thereof may also have such a correlation. Therefore, if the external features are directly used as input and the conversion behavior is used as label to train the conversion rate prediction model, the accuracy of the conversion rate prediction model will be affected due to the influence of the internal features on the external features and the conversion rate.

[0095] Therefore, in some implementations, before step S303, a causal effect function representing the causal relationship between the external features and the conversion behavior can be determined according to the internal features, the external features and the conversion behavior label of each of a plurality of second sample users, wherein the causal effect function takes the internal features as input and outputs a causal effect value corresponding to the internal features.

[0096] The causal effect function can represent the correlation between the external features and the conversion rate after excluding the interference of the internal features. Therefore, the conversion rate prediction model can be represented by the following formula:

[0097] F(x) = θ(x') · x + g(x') (1)

[0098] wherein F(·) is the conversion rate prediction model; x represents the external features; θ(·) is the causal effect function; x' represents the internal features; θ(x') is the causal effect value corresponding to the internal features x'; and g(x') represents the correlation between the internal features x' and the conversion rate, and for the external features, g(x') is equivalent to the noise interference caused by irrelevant variables.

[0099] Further, in step S303, a causal effect value corresponding to the alternative user is determined according to a plurality of internal features of the alternative user by using the causal effect function, and a first conversion rate of the alternative user is predicted according to the causal effect value corresponding to the alternative user and the predicted external feature.

[0100] Thus, the predicted first conversion rate can more accurately represent the conversion rate determined according to the external feature by substituting the predicted external feature into the aforementioned conversion rate prediction model as a substitute value of the real external feature.

[0101] Further, Figure 4 A flowchart of a training method of a conversion rate prediction model in an embodiment of the present specification is shown, and the training process of the conversion rate prediction model includes k rounds of residual estimation and a step of fitting a causal effect function after completing the residual estimation, wherein the residual estimation of any round specifically includes:

[0102] S401: The plurality of second sample users are divided into mutually exclusive model training groups and residual estimation groups.

[0103] The specific process of training the conversion rate prediction model using the second sample users can be divided into three stages:

[0104] In the first stage, a first prediction model that outputs a conversion rate estimation value according to internal features as input, and a second prediction model that outputs a fusion feature estimation value according to internal features as input are trained.

[0105] In the second stage, the first prediction model and the second prediction model are used to determine the conversion rate residual and the fusion feature residual of the plurality of second sample users, respectively.

[0106] In the third stage, a causal effect function is fitted using the conversion rate parameter and the fusion feature residual.

[0107] As can be seen from the above three stages, each round of residual estimation corresponds to the first stage and the second stage described above. The first stage is the training process of the first prediction model and the second prediction model, and the second stage is the prediction process (or verification process) using the first prediction model and the second prediction model. In a typical data analysis process, to avoid overfitting and other problems, the same sample data should not be used for training and prediction of the same model.

[0108] Thus, in each round of residual estimation in the present embodiment, the plurality of second sample users are first divided into mutually exclusive model training groups and residual estimation groups. The second sample users in the model training groups are used for training the first prediction model and the second prediction model. The second sample users in the residual estimation groups are used to calculate their own conversion rate residual and fusion feature residual.

[0109] It should be noted that for each round of residual estimation, step S401 will be re-executed to re-divide the model training group and the residual estimation group.

[0110] S403: According to the internal features of each second sample user in the model training group and the conversion behavior label, a first prediction model is trained, and according to the internal features of each second sample user and the sample fusion features, a second prediction model is trained; wherein the first prediction model takes internal features as input and outputs conversion rate estimation value; the second prediction model takes internal features as input and outputs fusion feature estimation value; for any second sample user, the sample fusion features of the second sample user are determined by the feature fusion model according to the external features of the second sample user.

[0111] After determining the model training group, the first prediction model and the second prediction model can be trained using the data of the second sample users in the model training group. Specifically, when training the first prediction model, the internal features of any second sample user can be taken as input, so as to adjust the parameters of the first prediction model to minimize the difference between the model output and the conversion behavior label of the second sample user; when training the second prediction model, the internal features of any second sample user can be taken as input, so as to adjust the parameters of the second prediction model to minimize the difference between the model output and the sample fusion features of the second sample user.

[0112] Thus, the first prediction model trained can learn the correlation between internal features and conversion rate, and the second prediction model can learn the correlation between internal features and external features.

[0113] It should be noted that for each round of residual estimation, step S403 will be re-executed to re-train the parameters of the first prediction model and the second prediction model.

[0114] S405: For any second sample user in the residual estimation group, the conversion rate residual corresponding to the second sample user is determined according to the difference between the conversion rate estimation value of the second sample user and the conversion behavior label, and the fusion feature residual corresponding to the second sample user is determined according to the difference between the fusion feature estimation value of the second sample user and the sample fusion features; wherein the conversion rate estimation value of the second sample user is determined by the first prediction model, and the fusion feature estimation value of the second sample user is determined by the second prediction model.

[0115] After training the first prediction model and the second prediction model, the conversion rate estimation value and the fusion feature estimation value of each second sample user in the residual estimation group can be determined using the trained first prediction model and the second prediction model.

[0116] For any second sample user, the conversion behavior label of the second sample user is the real value actually observed, i.e., the value of the conversion behavior label is the result after receiving the influence of the internal feature. On the other hand, the conversion rate estimate represents the conversion rate obtained only by the influence of the internal feature. Thus, the conversion rate residual obtained by subtracting the conversion rate estimate from the conversion behavior label can represent the value of the conversion rate after removing the influence of the internal feature, i.e., the conversion rate obtained only by the influence of the external feature (assuming that the external feature and the internal feature are all possible factors that can affect the conversion rate). Similarly, the fusion feature residual can represent the value of the external feature when the external feature is not affected by the internal feature.

[0117] Thus, in this embodiment, the external feature and the conversion rate after removing the influence of the internal feature are obtained by residual estimation respectively.

[0118] After completing the k rounds of residual estimation, the subsequent step of fitting the causal effect function can be performed:

[0119] Step S407: determining the causal effect function representing the causal relationship from the external feature to the conversion behavior according to each fusion feature residual in the k residual estimation groups and the conversion rate residual corresponding to each fusion feature residual.

[0120] It should be noted that if the number of samples in the residual estimation group in each round is m, then k*m fusion feature residuals and the conversion rate residual corresponding to each fusion feature residual can be obtained.

[0121] As described above, the fusion feature residual and the conversion rate residual both exclude the influence of the internal feature. Thus, the conversion rate prediction model can be determined with the external feature as the independent variable and the conversion rate as the dependent variable. In the conversion rate prediction model, the conversion relationship between the independent variable and the dependent variable is the causal effect function. The formula corresponding to the conversion rate prediction model is formula (1) described above. By adjusting the parameters of the conversion rate prediction model with the fusion feature residual as the independent variable, the conversion rate residual as the corresponding label, and the internal feature as an irrelevant variable, the causal effect function representing the causal relationship from the external feature to the conversion behavior can be determined.

[0122] Further, when predicting the first conversion rate of the candidate user in step S303, the causal effect value θ(x') is determined according to substituting the internal feature of the candidate user into the causal effect function, the noise interference g(x') is determined according to the internal feature, and the predicted external feature of the candidate user is taken as the independent variable x in formula (1), so that the first conversion rate F(x) can be predicted.

[0123] In addition, in the step S407 of training the conversion rate prediction model, in determining the causal effect function according to the fusion feature residual and the conversion rate residual, it is equivalent to fusing each group of residuals (one group of residuals includes a pair of fusion feature residual and conversion rate residual from the same second sample user in the same round), fitting the causal effect function according to the fusion result.

[0124] In actual application scenarios, the implementer needs to screen out users with high conversion rates, i.e., the data of users with high conversion rates is more instructive.

[0125] Therefore, in the step S407 as shown in the figure, Figure 4 for any second sample user in any residual estimation group, the base conversion rate of the second sample user is determined according to the internal features of the second sample user; the fusion weight corresponding to the second sample user is determined according to the base conversion rate of the second sample user; the causal effect function representing the causal relationship between the external feature and the conversion behavior is determined according to the fusion feature residual in the k residual estimation groups, the conversion rate residual corresponding to each fusion feature residual, and the fusion weight corresponding to each fusion feature residual.

[0126] Therefore, in the process of determining the causal effect function, the residual of the second sample user with high base conversion rate corresponds to a higher weight, which ensures that the prediction of the causal effect value of the causal effect function for the user with high conversion rate is more accurate, and accordingly ensures that the prediction of the first conversion rate of the candidate user who is more likely to convert in the subsequent step is more accurate.

[0127] In addition, a method for determining the second conversion rate is given below. In some implementations, before the step S309, a decision tree model with internal feature threshold and / or external feature threshold as splitting condition and conversion rate as output of leaf node can be determined according to the internal features, external features and conversion behavior labels of the third sample users by using a decision tree algorithm. Further, in the step S309, the second conversion rate of the candidate user is predicted according to the internal features and the external features of the candidate user by using the decision tree model.

[0128] Specifically, the decision tree model trained according to the third sample users can use common decision tree model training frameworks such as Light Gradient Boosting Machine (LightGBM) and eXtreme Gradient Boosting (XGBoost), and this specification will not be repeated.

[0129] After the real external features of the candidate user are obtained, the most accurate conversion rate prediction can be obtained in combination with the internal features, i.e., the second conversion rate. Finally, in this embodiment, whether the candidate user is the target user of the launched activity is determined according to the second conversion rate

[0130] In the 1990s, it was quite obvious to distinguish whether an improvement in a technology was in hardware (e.g., improvement in circuit structures of diodes, transistors, switches, etc.) or in software (improvement in method flow). However, as technology has evolved, many improvements in method flow today can be considered as direct improvements in hardware circuit structures. Designers almost always obtain the corresponding hardware circuit structures by programming the improved method flow into hardware circuits. Therefore, it cannot be said that an improvement in a method flow cannot be implemented by hardware entity modules. For example, a programmable logic device (PLD) (e.g., a field programmable gate array (FPGA)) is an integrated circuit whose logic function is determined by user programming of the device. A digital system is "integrated" on a PLD by the designer programming it, rather than by asking a chip manufacturer to design and fabricate a custom integrated circuit chip. Moreover, instead of manually fabricating integrated circuit chips, this programming is now mostly implemented by "logic compiler" software, which is similar to software compilers used in program development, and the original code to be compiled is written in a specific programming language, which is called a hardware description language (HDL), and there are many such languages, such as ABEL (Advanced Boolean Expression Language), AHDL (Altera Hardware Description Language), Confluence, CUPL (Cornell University Programming Language), HDCal, JHDL (Java Hardware Description Language), Lava, Lola, MyHDL, PALASM, RHDL (Ruby Hardware Description Language), etc., and the most commonly used are VHDL (Very-High-Speed Integrated Circuit Hardware Description Language) and Verilog. Those skilled in the art should be aware that, as long as the method flow is logically programmed in the above-mentioned hardware description languages and programmed into an integrated circuit, a hardware circuit implementing the logical method flow can be easily obtained.

[0131] The controller can be implemented in any suitable way, for example, the controller can take the form of, for example, a microprocessor or processor and a computer readable medium storing computer readable program code, such as software or firmware, executable by the (micro)processor, logic gates, switches, an application specific integrated circuit (ASIC), a programmable logic controller and an embedded microcontroller, examples of which include but are not limited to the following microcontrollers: ARC 625D, Atmel AT91SAM, Microchip PIC18F26K20 and Silicone Labs C8051F320, the memory controller can also be implemented as part of the control logic of the memory. Those skilled in the art will also know that, in addition to being implemented in pure computer readable program code form, the controller can perfectly well be implemented to perform the same functions in the form of logic gates, switches, an application specific integrated circuit, a programmable logic controller and an embedded microcontroller, etc. by means of logical programming of the method steps. Such a controller can thus be considered a hardware component, and the means comprised therein for performing the various functions can also be considered structures within the hardware component. Alternatively, or even, the means for performing the various functions can be considered both software modules implementing the method and structures within the hardware component.

[0132] The systems, apparatuses, modules or units illustrated by the above embodiments can be specifically implemented by computer chips or entities, or by products with certain functions. A typical implementation device is a server system. Of course, the present application does not exclude that with the development of computer technology in the future, computers implementing the functions of the above embodiments can be personal computers, laptop computers, vehicle-mounted human-computer interaction devices, cellular phones, camera phones, smart phones, personal digital assistants, media players, navigation devices, electronic mail devices, game consoles, tablet computers, wearable devices, or combinations of any of these devices.

[0133] Although the method operations of the embodiments of the present disclosure are described in a particular, sequential order, one or more of the method operations can be omitted, or the method operations can be performed in an order other than the described order. Additionally, one or more of the method operations can be performed concurrently, or with partial concurrence. Furthermore, one or more of the method operations can be performed by different entities, or over different time periods. The term "including" as used herein is intended to mean "comprising," such that the process, method, article, or apparatus that includes elements in addition to those specified. As used in this description, the term "coupled" means a direct or indirect connection, which can be physical or logical. The term "coupled" does not relate to a direct connection or wiring.

[0134] For the sake of description, the above-described apparatus is described as various modules to describe the functions of the apparatus. Of course, when implementing one or more embodiments of the present disclosure, the functions of the modules can be implemented in one or more software and / or hardware, or the modules implementing the same functions can be combined into a plurality of sub-modules or sub-units. The apparatus embodiments described above are merely illustrative, for example, the division of the units is merely a logical function division, and actual implementation can have another division manner, for example, a plurality of units or components can be combined or integrated into another system, or some features can be ignored or not executed. In addition, the coupling or direct coupling or communication connection between the units or components shown or discussed can be indirect coupling or communication connection through some interfaces, devices or units, and can be electrical, mechanical or other forms.

[0135] The present disclosure is described with reference to the flowcharts and / or block diagrams of the methods, apparatus (systems) and computer program products according to embodiments of the present disclosure. It should be understood that each flow and / or block in the flowcharts and / or block diagrams can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing apparatus to produce a machine, so that the instructions executed by the processor of the computer or other programmable data processing apparatus generate a means for implementing the functions specified in the flowcharts and / or block diagrams. Figure 1 The functions of one or more flows and / or blocks Figure 1 The apparatus that implements the functions specified in one or more flows and / or blocks.

[0136] These computer program instructions can also be stored in a computer- readable memory that can direct a computer or other programmable data processing apparatus to function in a particular manner, such that the instructions stored in the computer-readable memory produce an article of manufacture including instructions which implement the Figure 1 function specified in the flow or flows and / or blocks Figure 1 of the block or blocks.

[0137] These computer program instructions can also be loaded onto a computer or other programmable data processing apparatus to cause a series of operational steps to be performed on the computer or other programmable apparatus to produce a computer implemented process such that the instructions which execute on the computer or other programmable apparatus provide steps for implementing the Figure 1 function specified in the flow or flows and / or blocks Figure 1 of the block or blocks.

[0138] In a typical configuration, a computing device includes one or more processors (CPUs), input / output interfaces, network interfaces, and memory.

[0139] The memory can include non-persistent memory and / or volatile memory, such as random access memory (RAM) and / or cache memory, non-volatile memory, such as read-only memory (ROM), EPROM, and / or flash memory. The memory is an example of computer-readable media.

[0140] Computer-readable media includes permanent and non-permanent, moveable and non- moveable media that can be implemented in any method or technology for storage of information such as computer readable instructions, data structures, program modules or other data. Examples of computer storage media include, but are not limited to, phase change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technology, compact disc read-only memory (CD-ROM), digital versatile discs (DVDs) or other optical storage, magnetic cassettes, magnetic tape, magnetic disk storage or other magnetic storage devices, graphene storage or other magnetic storage devices, or any other non-transmission medium that can be used to store information accessible to computing devices. According to the definition herein, computer-readable media does not include transitory media, such as modulated data signals and carrier waves.

[0141] Those skilled in the art will appreciate that the one or more embodiments described herein can be provided as a method, a system or a computer program product. Accordingly, the one or more embodiments described herein can take the form of an entirely hardware embodiment, an entirely software embodiment or an embodiment combining software and hardware aspects. Furthermore, the one or more embodiments described herein can take the form of a computer program product on one or more computer-usable storage media (including, but not limited to, disk storage, CD-ROMs, optical storage devices, etc.) embodying computer readable code.

[0142] The one or more embodiments described herein can be described in the general context of computer-executable instructions, such as program modules, being executed by a computer. Generally, program modules include routines, programs, objects, components, data structures, etc., that perform particular tasks or implement particular abstract data types. The one or more embodiments described herein can also be practiced in distributed computing environments where tasks are performed by remote processing devices that are linked through a communications network. In a distributed computing environment, program modules can be located in both local and remote computer storage media including memory storage devices.

[0143] The various embodiments described in this specification are described in the context of progressive embodiments, with each embodiment building on the previous one. The same or similar parts between embodiments are cross-referenced as appropriate. Each embodiment focuses on the differences between that embodiment and the previous one. In particular, the system embodiments are described relatively simply, as they are substantially similar to the method embodiments. In the description of the specification, the description of the terms "one embodiment", "some embodiments", "example", "specific example" or "some examples" means that the specific feature, structure, material or characteristic being described in connection with that embodiment or example is included in at least one embodiment or example in the specification. Illustrative representations of the above terms in the specification are not necessarily referring to the same embodiment or example. Moreover, the specific features, structures, materials or characteristics being described can be combined in any suitable manner in one or more embodiments or examples. Furthermore, the skilled person can combine and combine features of different embodiments or examples and characteristics of different embodiments or examples in a manner that is not mutually contradictory.

[0144] The above description merely provides examples of the one or more embodiments described in this specification and does not limit the one or more embodiments described in this specification. The one or more embodiments described in this specification can have various modifications and changes. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the one or more embodiments described in this specification should be included in the scope of the claims.

Claims

1. A method for screening target users, the method comprising: For any candidate user of the target organization, based on several internal characteristics of the candidate user, predict the external characteristics of the candidate user to obtain the predicted external characteristics; wherein, the internal characteristics are held by the target organization, and the external characteristics are held by an external organization; Based on the predicted external characteristics, predict the first conversion rate of the candidate user; Based on the first conversion rate and the query cost for the external organization, determine whether the candidate user is a preliminary screening target; If the candidate user is the initial screening target, obtain the external characteristics of the candidate user; Based on several internal characteristics of the candidate user and the external characteristics, predict the second conversion rate of the candidate user; If the second conversion rate is higher than the second threshold, the candidate user will be used as the target user.

2. The method of claim 1, further comprising: Using a basic screening model, the basic conversion rate of each candidate user in the target organization is determined based on several internal characteristics of each candidate user. Based on the aforementioned basic conversion rate, several target users are identified from among the candidate users, and the candidate users who are not identified as target users are used as alternative users.

3. The method as described in claim 1, wherein predicting the external characteristics of the candidate user based on several internal characteristics of the candidate user to obtain the predicted external characteristics specifically includes: Based on several internal characteristics of the candidate user, predict the fusion value of several external characteristics of the candidate user as the predicted external characteristics; If the candidate user is a target for initial screening, the external characteristics of the candidate user are obtained, specifically including: If the candidate user is the initial screening target, obtain the aforementioned external characteristics of the candidate user; Based on several internal characteristics and the external characteristics of the candidate user, the second conversion rate of the candidate user is predicted, specifically including: Based on several internal characteristics and several external characteristics of the candidate user, predict the second conversion rate of the candidate user.

4. The method of claim 3, further comprising: Several external features of the first sample user are input into the feature fusion model to obtain the sample fusion features of the first sample user, and several internal features of the first sample user are input into the feature fitting model to obtain the fitted fusion features of the first sample user. With the goal of minimizing the difference between the fitted fusion features corresponding to the first sample user and the sample fusion features, at least the parameters of the feature fitting model are adjusted. Based on several internal characteristics of the candidate user, predict the fusion value of several external characteristics of the candidate user as the predicted external characteristics, specifically including: Input several internal features of the candidate user into the feature fitting model, predict the fusion value of several external features of the candidate user, and obtain the predicted external features of the candidate user.

5. The method of claim 4, further comprising: Based on several internal features, several external features, and conversion behavior labels of several second sample users, a causal effect function representing the causal relationship from external features to conversion behavior is determined, wherein the causal effect function takes internal features as input and the causal effect value corresponding to the internal features as output. Based on the predicted external characteristics, the first conversion rate of the candidate user is predicted, specifically including: Using the causal effect function, the causal effect value corresponding to the candidate user is determined based on several internal characteristics of the candidate user; Based on the causal effect value corresponding to the candidate user and the predicted external characteristics, predict the first conversion rate of the candidate user.

6. The method as described in claim 5, wherein based on several internal characteristics, several external characteristics, and conversion behavior labels of several second sample users, a causal effect function representing the causal relationship from external characteristics to conversion behavior is determined, specifically including: Perform residual estimation for k rounds, where the residual estimation for any round specifically includes: The aforementioned second sample users are divided into mutually exclusive model training groups and residual estimation groups; A first prediction model is trained based on several internal features and conversion behavior labels of each second sample user in the model training group, and a second prediction model is trained based on several internal features and sample fusion features of each second sample user; wherein, the first prediction model takes internal features as input and outputs a conversion rate estimate; the second prediction model takes internal features as input and outputs a fusion feature estimate; for any second sample user, the sample fusion feature of the second sample user is determined using the feature fusion model based on several external features of the second sample user; For any second sample user in the residual estimation group, the conversion rate residual corresponding to the second sample user is determined based on the difference between the conversion rate estimate and the conversion behavior label of the second sample user, and the fusion feature residual corresponding to the second sample user is determined based on the difference between the fusion feature estimate and the sample fusion feature of the second sample user; wherein, the conversion rate estimate of the second sample user is determined using the first prediction model, and the fusion feature estimate of the second sample user is determined using the second prediction model; Based on the residuals of each fusion feature in the k residual estimation groups and the conversion rate residual corresponding to each fusion feature residual, determine the causal effect function representing the causal relationship from external features to conversion behavior.

7. The method as described in claim 6, wherein a causal effect function representing the causal relationship from external features to transformation behavior is determined based on the residuals of each fusion feature in the k residual estimation groups and the conversion rate residual corresponding to each fusion feature residual, specifically including: For any second sample user in any residual estimation group, determine the basic conversion rate of the second sample user based on several internal characteristics of the second sample user; The fusion weight corresponding to the second sample user is determined based on the basic conversion rate of the second sample user. Based on the residuals of each fusion feature in the k residual estimation groups, the conversion rate residual corresponding to each fusion feature residual, and the fusion weight corresponding to each fusion feature residual, the causal effect function representing the causal relationship from external features to conversion behavior is determined.

8. The method of claim 1, further comprising: Based on the internal features, external features, and conversion behavior labels of several third-sample users, a decision tree algorithm is used to determine a decision tree model with internal feature thresholds and / or external feature thresholds as splitting conditions and conversion rate as the output of the leaf nodes. Based on several internal characteristics and the external characteristics of the candidate user, the second conversion rate of the candidate user is predicted, specifically including: Using the decision tree model, the second conversion rate of the candidate user is predicted based on several internal characteristics and the external characteristics of the candidate user.

9. A computer-readable storage medium having a computer program stored thereon, which, when executed in a computer, causes the computer to perform the method as described in any one of claims 1-8.

10. A computing device comprising a memory and a processor, the memory storing executable code, wherein the processor, when executing the executable code, implements the method as claimed in any one of claims 1-8.

Images