Data processing method, device, equipment, storage medium and computer program product
By calculating the response parameters of feature information in model training and filtering out important feature information, the problem of redundant features in massive feature information affecting model training is solved, and the effect of improving model training efficiency and accuracy is achieved.
Patent Information
- Application Number
- CN202210170919.X
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2022-02-23
- Publication Date
- 2025-03-18
- Estimated Expiration
- 2042-02-23
AI Technical Summary
During the model training process, there is too much redundant feature information in the massive feature information, which affects the accuracy and training efficiency of the model, and there is currently a lack of effective solutions.
By obtaining the response parameters of positive sample data and negative sample data under multiple different feature information, the difference in response parameters of each feature information is calculated, and the difference is weighted to obtain the feature importance mark and filter out the feature information that meets the preset conditions.
Effectively remove redundant feature information from massive feature information, improve the training efficiency and accuracy of the model, and improve the effectiveness of model application.
Smart Images

Figure CN114579547B_ABST
Abstract
Description
Technical Field
[0001] The present application relates to the field of data processing, and specifically to a data processing method, apparatus, device, storage medium and computer program product. Background Art
[0002] During the model training process, the selection of feature information of sample data is an important part of subsequent model training.
[0003] However, the existence of too much redundant feature information interferes with both model accuracy and training efficiency. How to select the right number of feature groups in a reasonable and effective way from the massive amount of feature information, maximize the role of the features, and improve the effectiveness of model application is a difficult problem that is often overlooked in model training, and there is currently no good solution. Summary of the invention
[0004] The embodiments of the present application provide a data processing method, apparatus, device, storage medium and computer program product to achieve the effect of efficiently removing redundant feature information from massive feature information.
[0005] In a first aspect, an embodiment of the present application provides a data processing method, the method comprising:
[0006] Obtaining first response parameters of positive sample data under multiple different feature information, and second response parameters of negative sample data under multiple different feature information;
[0007] For each piece of feature information, respectively determine the difference between the first response parameter and the second response parameter;
[0008] For each feature information, weighted calculation is performed on each difference to obtain the feature importance identification of the feature information;
[0009] From a plurality of different feature information, filter out the feature information that meets the preset conditions in the feature importance identifier to obtain the target feature information;
[0010] The positive sample data is sample data in which the first user responds positively to the preset object; and the negative sample data is sample data in which the first user responds negatively to the preset object.
[0011] In a second aspect, an embodiment of the present application provides a data processing device, the device comprising:
[0012] An acquisition module, used to acquire first response parameters of positive sample data under multiple different feature information, and second response parameters of negative sample data under multiple different feature information;
[0013] A first determination module, configured to determine, for each piece of feature information, a difference between the first response parameter and the second response parameter;
[0014] The second determination module is used to perform weighted calculation on each difference for each feature information to obtain a feature importance identifier of the feature information;
[0015] A screening module is used to screen out feature information that meets preset conditions in the feature importance identifier from multiple different feature information to obtain target feature information; wherein the positive sample data is sample data in which the first user responds positively to the preset object; and the negative sample data is sample data in which the first user responds negatively to the preset object.
[0016] In a third aspect, an embodiment of the present application provides an electronic device, comprising: a processor and a memory storing computer program instructions; when the processor executes the computer program instructions, the data processing method as described in the first aspect is implemented.
[0017] In a fourth aspect, an embodiment of the present application provides a computer storage medium, on which computer program instructions are stored. When the computer program instructions are executed by a processor, the data processing method described in the first aspect is implemented.
[0018] In a fifth aspect, an embodiment of the present application provides a computer program product. When instructions in the computer program product are executed by a processor of an electronic device, the electronic device executes the data processing method as described in the first aspect.
[0019] The data processing method, apparatus, device, storage medium and computer program product of the embodiments of the present application obtain first response parameters of positive sample data under multiple different feature information, and second response parameters of negative sample data under multiple different feature information, and then for each feature information, respectively determine the difference between the first response parameter and the second response parameter, and then perform weighted calculation on each difference to obtain a feature importance identifier of the feature information, and based on the identifier, select target feature information from different feature information, thereby efficiently achieving the purpose of removing redundant feature information from massive feature information.
[0020] It should be understood that the foregoing general description and the following detailed description are exemplary and explanatory only and are not restrictive of the present application. BRIEF DESCRIPTION OF THE DRAWINGS
[0021] The drawings herein are incorporated into the specification and constitute a part of the specification, illustrate embodiments consistent with the present application, and together with the specification are used to explain the principles of the present application, and do not constitute an improper limitation on the present application.
[0022] Figure 1 It is a flowchart of a data processing method provided by an embodiment of the first aspect of the present application;
[0023] Figure 2 This is a schematic diagram of the structure of the data processing system involved in the first embodiment of the present application.
[0024] Figure 3 is a structural schematic diagram of a data processing device provided in an embodiment of the second aspect of the present application;
[0025] Figure 4 It is a schematic diagram of the structure of an electronic device provided in an embodiment of the third aspect of the present application. DETAILED DESCRIPTION
[0026] In order to make those of ordinary skill in the art better understand the technical scheme of the present application, the technical scheme in the embodiments of the present application will be clearly and completely described below in conjunction with the accompanying drawings. It should be understood that the specific embodiments described herein are only intended to explain the present application, rather than to limit the present application. For those skilled in the art, the present application can be implemented without the need for some of these specific details. The following description of the embodiments is only to provide a better understanding of the present application by illustrating the examples of the present application.
[0027] It should be noted that the terms "first", "second", etc. in the specification and claims of the present application and the above-mentioned drawings are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequence. It should be understood that the data used in this way can be interchanged where appropriate, so that the embodiments of the present application described here can be implemented in an order other than those illustrated or described here. The embodiments described in the following exemplary embodiments do not represent all embodiments consistent with the present application. On the contrary, they are only examples consistent with some aspects of the present application as detailed in the attached claims.
[0028] As described in the background technology section, in order to effectively remove redundant feature information from massive feature information, the embodiments of the present application provide a data processing method, apparatus, device, storage medium and computer program product, by obtaining the first response parameters of the positive sample data under multiple different feature information, and the second response parameters of the negative sample data under multiple different feature information, and then for each feature information, respectively determine the difference between the first response parameter and the second response parameter, and then perform weighted calculation on each difference to obtain a feature importance identifier of the feature information, and based on the identifier, select the target feature information from different feature information, thereby efficiently achieving the purpose of removing redundant feature information from massive feature information.
[0029] The following first introduces the data processing method provided in the embodiment of the present application.
[0030] Figure 1 FIG. 1 is a flow chart of a data processing method provided by an embodiment of the present application. Figure 1 As shown, the data processing method provided in the embodiment of the present application specifically includes the following steps 110 to 140:
[0031] Step 110: Obtain first response parameters of positive sample data under multiple different feature information, and second response parameters of negative sample data under multiple different feature information.
[0032] The positive sample data may be sample data of a positive response of the first user to a preset object.
[0033] The negative sample data may be sample data in which the first user responds negatively to a preset object.
[0034] Step 120: For each piece of characteristic information, determine the difference between the first response parameter and the second response parameter.
[0035] Step 130: For each piece of feature information, perform weighted calculation on each difference to obtain a feature importance identifier of the feature information.
[0036] Step 140 : Filter out feature information that meets preset conditions in feature importance identifiers from a plurality of different feature information to obtain target feature information.
[0037] In an embodiment of the present application, by obtaining the first response parameters of the positive sample data under multiple different feature information, and the second response parameters of the negative sample data under multiple different feature information, and then for each feature information, the difference between the first response parameter and the second response parameter is determined, and then each difference is weightedly calculated to obtain a feature importance identifier of the feature information, and based on the identifier, the target feature information is selected from different feature information, thereby efficiently achieving the purpose of removing redundant feature information from massive feature information.
[0038] The following is a detailed description of the data processing method provided in the embodiment of the present application:
[0039] First, step 110 is introduced, which is to obtain first response parameters of positive sample data under multiple different feature information, and second response parameters of negative sample data under multiple different feature information.
[0040] The positive sample data may be sample data of a positive response of the first user to a preset object.
[0041] In some embodiments of the present application, the first user may be a user who responds to a preset object.
[0042] The preset object may be a pre-set object. Here, the preset object may be an activity, and the corresponding first user may be a user who participates in the activity.
[0043] In some embodiments of the present application, a positive response may be a positive response. For example, taking an online marketing activity as an example, a positive response may be a consumption behavior in the activity. The corresponding positive sample data may be the first user who participated in the activity and consumed in the online marketing activity.
[0044] In some embodiments of the present application, the negative sample data may be sample data in which the first user responds negatively to a preset object.
[0045] In some embodiments of the present application, a negative response may be a negative response. For example, taking an online marketing activity as an example, a negative response may be that no consumption behavior is performed in the activity. The corresponding negative sample data may be the first user who participated in the activity and did not perform consumption behavior in the online marketing activity.
[0046] In some embodiments of the present application, the characteristic information may be information corresponding to a preset object and used to characterize the characteristics of the preset object. For example, taking the preset object as an online marketing activity, the characteristic information here may be, but is not limited to: the number of times the sharing is used, the number of times the public payment is used, the number of times the discount details page is used, the number of times my red envelope is used, the number of times the search page is used, the number of times the payment control is used, and the number of times the bill is used.
[0047] In some embodiments of the present application, the first response parameter may be a response parameter of the positive sample data under each feature information. The first response parameter here may include the first response times and the first responder number.
[0048] In some embodiments of the present application, the first number of responses may be the number of responses of the positive sample data under each feature information.
[0049] In an example, the preset object is an online marketing activity, the characteristic information is the number of sharing uses, and the positive sample data is user A, user B and user C. If user A has shared once, user B has not used sharing, and user C has used sharing twice, then the first response number is 3.
[0050] In some embodiments of the present application, the first number of responders may be the number of responders of the positive sample data under each feature information.
[0051] In an example, the preset object is an online marketing activity, the characteristic information is the number of sharing uses, and the positive sample data is user A, user B and user C. If user A uses sharing once, user B does not use sharing, and user C uses sharing twice, then the number of first responders is 2 (that is, users A and C).
[0052] In some embodiments of the present application, the second response parameter may be a response parameter of the negative sample data under each feature information. The second response parameter here may include the second response times and the second number of responders.
[0053] In some embodiments of the present application, the second number of responses may be the number of responses of the negative sample data under each feature information. Specific examples may refer to the first number of responses, which will not be described in detail here.
[0054] In some embodiments of the present application, the second number of responders may be the number of responders of the negative sample data under each feature information. Specific examples may refer to the first number of responders, which will not be described in detail here.
[0055] Then, step 120 is introduced to determine the difference between the first response parameter and the second response parameter for each feature information.
[0056] In some embodiments of the present application, after the first response parameter and the second response parameter are determined, the difference between the first response parameter and the second response parameter may be determined for each piece of feature information.
[0057] In some embodiments of the present application, in order to further remove redundant feature information from massive feature information, before step 120, the data processing method involved above may further include:
[0058] A first number of positive sample data and a second number of negative sample data are obtained.
[0059] In some embodiments of the present application, the first number may be the number of positive sample data.
[0060] The second number may be the number of negative sample data.
[0061] Correspondingly, step 120 may specifically include:
[0062] For each feature information, determining a first ratio between the first response parameter and the first quantity;
[0063] For each piece of characteristic information, determining a second ratio of a second response parameter to the second quantity;
[0064] A difference between the first ratio and the second ratio is determined.
[0065] In some embodiments of the present application, the first ratio may be a ratio of the first response parameter to the first quantity.
[0066] The second ratio may be a ratio of the second response parameter to the second quantity.
[0067] In an embodiment of the present application, for each characteristic information, by determining a first ratio of a first response parameter and a first quantity, and a second ratio of a second response parameter and a second quantity, the difference between the first ratio and the second ratio can be accurately determined, and then redundant characteristic information can be accurately removed based on the difference.
[0068] In some embodiments of the present application, in order to accurately determine the first ratio, determining the first ratio of the first response parameter and the first quantity for each piece of characteristic information may specifically include:
[0069] For each piece of feature information, a first sub-ratio of the first number of responders to the first quantity and a second sub-ratio of the first number of responses to the first quantity are determined.
[0070] In some embodiments of the present application, the first sub-ratio may be a ratio of the first number of responders to the first quantity.
[0071] The second sub-ratio may be a ratio of the first number of responses to the first quantity.
[0072] In some embodiments of the present application, in order to accurately determine the second ratio, determining the second ratio of the second response parameter and the second quantity for each feature information may specifically include:
[0073] For each piece of feature information, a third sub-ratio of the second number of responders and the second quantity, and a fourth sub-ratio of the second number of responses and the second quantity are determined.
[0074] In some embodiments of the present application, the third sub-ratio may be a ratio of the second number of respondents to the second quantity.
[0075] The fourth sub-ratio may be a ratio of the second number of responses to the second quantity.
[0076] In an embodiment of the present application, corresponding to each characteristic information, a first sub-ratio of the first number of responders and the first quantity, a second sub-ratio of the first number of responses and the first quantity, a third sub-ratio of the second number of responders and the second quantity, and a third sub-ratio of the second number of responses and the second quantity can be determined, so that an accurate first ratio and second ratio can be obtained, and then redundant characteristic information can be removed based on the accurate first ratio and second ratio.
[0077] In some embodiments of the present application, in order to accurately determine the difference between the first ratio and the second ratio, the determining the difference between the first ratio and the second ratio may specifically include:
[0078] A first difference between the first sub-ratio and the third sub-ratio and a second difference between the second sub-ratio and the fourth sub-ratio are determined.
[0079] In some embodiments of the present application, the first difference may be a difference between the first sub-ratio and the third sub-ratio. Specifically, the difference between the first sub-ratio and the third sub-ratio may be reflected by the ratio of the first sub-ratio to the third sub-ratio.
[0080] The second difference may be a difference between the second sub-ratio and the fourth sub-ratio. Specifically, the difference between the second sub-ratio and the fourth sub-ratio may be reflected by the ratio of the second sub-ratio to the fourth sub-ratio.
[0081] In the embodiment of the present application, by determining a first difference between the first sub-ratio and the third sub-ratio, and a second difference between the second sub-ratio and the fourth sub-ratio, redundant feature information can be removed accurately based on the first difference and the second difference.
[0082] Next, step 130 is introduced, in which for each piece of feature information, weighted calculation is performed on each difference to obtain a feature importance identifier of the feature information.
[0083] In some embodiments of the present application, the feature importance identifier may be an identifier used to characterize the importance of feature information.
[0084] In some embodiments of the present application, in order to accurately perform weighted calculation on each difference, step 130 may specifically include:
[0085] For each piece of feature information, a weighted calculation is performed on the first difference and the second difference to obtain a feature importance identifier of the feature information.
[0086] In some embodiments of the present application, the feature importance indicator of the feature information can be obtained by the following formula (1):
[0087]
[0088] Among them, N i and M l are the values of each positive sample data and negative sample data under a certain feature (i=1,2,…n,l=1,2,…m), A i and B l It indicates whether each positive sample data and negative sample data has this feature, and its value is 0 or 1. n is the number of positive sample data (i.e., the first number), and m is the number of negative sample data (i.e., the second number). ∝ is the weight value (its value can be set according to user needs, and the default value is ∝=0.5).
[0089] The following is a specific example to illustrate the meaning of each character in the above formula (1):
[0090] The preset object is an online marketing activity, the number of positive sample data (i.e., the first number) is 3 (i.e., n=3), and the number of negative sample data (i.e., the second number) is 5 (i.e., m=5). Among them, the positive sample data includes users A, B, and C, and the negative sample data includes users D, E, F, G, and H. User A shared and used it once, user B shared and used it 0 times, user C shared and used it 2 times, user D shared and used it 1 time, user E shared and used it 1 time, user F shared and used it 0 times, user G shared and used it 2 times, and user H shared and used it 1 time. Then N i (where i here is user A, user B and user C) takes values of 1, 0 and 2 respectively (so, represents the sum of the values of each positive sample data under this feature (number of times shared) (i.e., 1+0+2=3), M l (where l here is user D, user E, user F, user G and user H) and its values are 1, 1, 0, 2 and 1 respectively (so, represents the sum of the values of each negative sample data under this feature (number of times shared) (i.e. 1+1+0+2+1=5). Since user A and user C have the feature information of number of times shared, their corresponding A i The value of is 1. User B does not have the feature information of sharing usage times. i The value of is 0 (so, Indicates the sum of whether each positive sample data has this feature, that is, 1+0+1=2). Since users D, E, G, and H have the feature information of sharing usage times, their corresponding B l The value of is 1, user F does not have the feature information of sharing usage times, and its corresponding B l The value of is 0 (so, The sum of whether each negative sample data has this feature is 1+1+0+1+1=4).
[0091] Continuing with the above example, the process of steps 120-130 is described as follows:
[0092] For the characteristic information of the number of times of sharing, the first sub-ratio of the number of first responders to the first quantity is the above The ratio (first sub-ratio) of n (first number of responders) and n (first quantity).
[0093] For the characteristic information of the number of times of sharing, the second sub-ratio of the first number of responses to the first quantity is the above The ratio (second sub-ratio) of n (first number of responses) and n (first quantity).
[0094] For the characteristic information of the number of times of sharing, the third sub-ratio of the second number of responders to the second number is the above The ratio of m (the second number of responders) and m (the second quantity) (the third sub-ratio).
[0095] For the characteristic information of the number of times of sharing, the fourth sub-ratio of the second number of responses to the second quantity is the above The ratio (fourth sub-ratio) of m (second number of responses) and m (second quantity).
[0096] For the feature information of the number of times of sharing, the first difference between the first sub-ratio and the third sub-ratio is the above
[0097] For the feature information of the number of times of sharing, the second difference between the second sub-ratio and the fourth sub-ratio is the above
[0098] Then, a weighted calculation is performed on the first difference and the second difference to obtain a feature importance indicator Y of the feature information of the number of times of sharing and use.
[0099] It should be noted that the higher the value of the feature importance indicator is, the greater the difference between the positive sample data and the negative sample data is, that is, the higher the correlation between the feature information and the response of the first user is, and vice versa.
[0100] It should be noted that in the above formula (1), only when N i 、M l , and The values of (Should The value can be set according to user needs and is not limited here. In the embodiment of the present application, it can be ), the score value Y is meaningful.
[0101] Finally, step 140 is introduced, in which feature information that meets preset conditions in feature importance identifiers is screened out from a plurality of different feature information to obtain target feature information.
[0102] The preset condition may be a condition that a preset feature importance indicator satisfies.
[0103] The target feature information may be feature information whose feature importance identifier satisfies a preset condition and is screened out from different feature information.
[0104] In an example, there are 10 pieces of feature information, and there are 4 pieces of feature information that meet the preset conditions among the screened feature importance identifiers, and these 4 pieces of feature information are target feature information.
[0105] In some embodiments of the present application, in order to accurately filter out target feature information from a plurality of different feature information, step 140 may specifically include:
[0106] The feature importance identifiers are arranged in order from large to small, and the feature information corresponding to the feature importance identifiers with the first N orders is filtered out to obtain the target feature information.
[0107] Wherein, N is a positive integer greater than or equal to 1.
[0108] Taking the preset object as an online marketing activity as an example, Table 1 shows the ranking of the score values corresponding to some feature information:
[0109] Table 1
[0110]
[0111] The negative sample scores in Table 1 above are The ratio of (second response times) and m (second quantity) (fourth sub-ratio), the positive sample score is The ratio of (first response times) to n (first quantity) (second sub-ratio), the ratio of times is the ratio of negative sample scores to positive sample scores. The number of negative samples is The ratio of (number of second responders) to m (second quantity), the number of positive samples is The ratio (first sub-ratio) of n (the first number of responders) and n (the first quantity), the ratio of the number of people is the ratio of the number of negative samples to the number of positive samples, and the score is the Y value in the above formula (1).
[0112] It should be noted that the negative sample scores, positive sample scores, number of negative samples and number of positive samples in Table 1 above are not the data for the same time of the feature, so the value of the ratio of times in Table 1 is not the ratio of the negative sample score to the positive sample score. For example, for the feature information of the number of times of sharing, the negative sample score may be the value obtained for the number of times of sharing once, and the positive sample score may be the value obtained for the number of times of sharing twice, so the obtained ratio of times is not the ratio of the negative sample score to the positive sample score. However, in the actual calculation process, the ratio of times is the ratio of the negative sample score to the positive sample score.
[0113] In an embodiment of the present application, by arranging the feature importance identifiers in order from large to small, the feature information corresponding to the top N feature importance identifiers are screened out to obtain the target feature information. Since the feature importance identifiers are arranged in order from large to small, the feature information corresponding to the feature importance identifiers that are arranged higher has a higher correlation with the response of the first user. In this way, the feature information corresponding to the top N feature importance identifiers is screened out, and the feature information with a high correlation with the response of the first user can be obtained, and redundant feature information with a low correlation is removed.
[0114] In some embodiments of the present application, during the model building process, due to certain objective factors, there are often too few positive sample data or negative sample data, that is, the problem of imbalance between positive sample data and negative sample data. In order to solve the problem of imbalance between positive sample data and negative sample data, after obtaining the first number of positive sample data and the second number of negative sample data, the data processing method involved above may also include:
[0115] The negative sample data is sampled to obtain a third amount of target negative sample data.
[0116] The target negative sample data may be negative sample data obtained by sampling the negative sample data.
[0117] The third number may be the number of target negative sample data.
[0118] In an example, there are 5 negative sample data, namely user D, user E, user F, user G and user H. After sampling these 5 negative sample data, the negative sample data obtained are user D, user E and user F. Then user D, user E and user F are the target negative sample data, and the third quantity is 3.
[0119] Correspondingly, for each piece of characteristic information, determining the second ratio of the second response parameter to the second quantity may specifically include:
[0120] For each feature information, a second ratio of the second response parameter to the third quantity is determined.
[0121] In an embodiment of the present application, by sampling the negative sample data, a third number of target negative sample data is obtained, so that the number of positive sample data and negative sample data can be balanced, further improving the accuracy of removing redundant feature information.
[0122] In some embodiments of the present application, in order to accurately sample the negative sample data, sampling the negative sample data to obtain the third amount of target negative sample data may specifically include:
[0123] Obtaining a fourth amount of historical positive sample data and a fifth amount of historical negative sample data corresponding to the historical preset object;
[0124] Based on the first number, the fourth number, and the fifth number, a third number of target negative sample data is obtained.
[0125] The historical preset object may be an object of the same type as the preset object before the preset object. For example, if the preset object is an online marketing activity, the historical preset object is an online marketing activity of the same type as the online marketing activity before the online marketing activity.
[0126] The historical positive sample data may be positive sample data in a historical preset object.
[0127] The fourth number may be the number of historical positive sample data.
[0128] The historical negative sample data may be negative sample data in a historical preset object.
[0129] The fifth quantity may be the quantity of historical negative sample data.
[0130] In one example, if the preset object is an online marketing activity, the historical preset object is an online marketing activity of the same type as the online marketing activity before the online marketing activity. In the historical online marketing activity, users who have performed consumption behavior are positive sample data, and users who have not performed consumption behavior are negative sample data.
[0131] In some embodiments of the present application, the third quantity of target negative sample data may be obtained based on the first quantity, the second quantity, the fourth quantity, and the fifth quantity according to the following formula (2):
[0132]
[0133] Among them, A i is the number of positive sample data (i.e. the first number), A j is the number of target negative sample data (i.e., the third number), C i is the number of historical positive sample data (i.e., the fourth number) C j is the number of historical negative sample data (i.e., the fifth number).
[0134] In some embodiments of the present application, after the third number of target negative sample data is obtained by formula (2), the third number of negative sample data can be randomly selected from the second number as the target negative sample data, or the third number of negative sample data can be selected as the target negative sample data based on certain feature information. The specific selection method can be set according to user needs and is not limited here.
[0135] In an embodiment of the present application, by obtaining the fourth number of historical positive sample data and the fifth number of historical negative sample data corresponding to the historical preset objects; based on the first number, the fourth number and the fifth number, the third number of target negative sample data can be obtained, so that the number of positive sample data and negative sample data can be balanced to more accurately remove redundant feature information.
[0136] It should be noted that in the embodiment of the present application, the negative sample data is much larger than the positive sample data as an example to illustrate the solution to the problem of imbalance between positive sample data and negative sample data, so only the scheme for sampling the negative sample data is shown here, and the scheme for sampling the positive sample data is not shown. However, in the actual application process, if the positive sample data is much larger than the negative sample data, it is necessary to sample the positive sample data, which can be achieved by sampling in the same way as sampling the negative sample data, which will not be repeated here.
[0137] In some embodiments of the present application, in addition to using the above method to solve the problem of balancing positive sample data and negative sample data, the Metropolis-Hastings sampling method can also be used. The specific Metropolis-Hastings sampling belongs to the existing technology and will not be repeated here.
[0138] In some embodiments of the present application, in order to allow more users to respond to the preset object, after step 140, the above-mentioned data processing method may further include:
[0139] Based on the target feature information of the positive sample data and the target feature information of the negative sample data, a prediction model is trained to obtain a target prediction model.
[0140] The prediction model may be a probability value for predicting that a second user who has not responded to the preset object will respond to the preset object.
[0141] The second user here may be a user who has not responded to the preset object.
[0142] In an example, taking the preset object as an online marketing activity as an example, the second user is a user who has not participated in the online marketing activity.
[0143] The target prediction model may be a model obtained after training the prediction model.
[0144] In an embodiment of the present application, a prediction model is trained based on target feature information of positive sample data and target feature information of negative sample data to obtain a target prediction model, so that the probability value of a second user who has not responded to the preset object responding to the preset object can be predicted based on the target prediction model.
[0145] In some embodiments of the present application, the target prediction model may include a first network and a second network.
[0146] The first network may be a sub-model in the target prediction model.
[0147] The second network may be another sub-model in the target prediction model except the first network.
[0148] In some embodiments of the present application, since the target prediction model is used to predict the probability value of the second user who has not responded to the preset object responding to the preset object, the target prediction model is required to have high interpretability and be easy to use. Therefore, the first network here can be a linear model, and the second network can be a tree model. Since the linear model is simple to use and the tree model has strong interpretability, the two models are combined to form a fusion model with universality and strong interpretability, so as to better predict the probability value of the second user who has not responded to the preset object responding to the preset object.
[0149] After the prediction model is trained based on the target feature information of the positive sample data and the target feature information of the negative sample data to obtain the target prediction model, the data processing method involved above may further include:
[0150] Based on the target feature information corresponding to the second user and the first network, predicting a first probability value that the second user responds to the preset object;
[0151] Based on the target feature information corresponding to the second user and the second network, predicting a second probability value that the second user responds to the preset object;
[0152] The first probability value and the second probability value are weightedly calculated to obtain a probability value of the second user responding to the preset object.
[0153] The first probability value may be a probability value of a second user response prediction object based on the first network prediction.
[0154] The second probability value may be a probability value of a second user response prediction object based on a second network prediction.
[0155] In some embodiments of the present application, the first probability value and the second probability value may be weighted to obtain a probability value of the second user responding to the preset object. Specifically, the probability value of the second user responding to the preset object without responding to the preset object may be predicted by the target prediction model using the following formula (3):
[0156]
[0157] Among them, LR scorei is the score (i.e., the first probability value) of the i-th (i=1, 2, ... n) second user under the linear model, XGBoost score i is the score of the ith user under the tree model (ie, the second probability value); ∝ is the weight value (which can be set according to user needs, with ∝=0.5 by default).
[0158] In an embodiment of the present application, a first probability value of the second user responding to a preset object is predicted based on the target feature information corresponding to the second user and the first network; a second probability value of the second user responding to the preset object is predicted based on the target feature information corresponding to the second user and the second network; and the first probability value and the second probability value are weightedly calculated. In this way, the probability value of the second user responding to the preset object can be accurately obtained, thereby increasing the probability value of the second user wanting the preset object.
[0159] In some embodiments of the present application, in order to allow more people to respond to the preset object, after performing weighted calculation on the first probability value and the second probability value to obtain the probability value of the second user responding to the preset object, the data processing method involved may further include:
[0160] Filter out probability values whose values are greater than or equal to a preset probability value from the probability values to obtain a target probability value;
[0161] The information corresponding to the preset object is pushed to the second user corresponding to the target probability value.
[0162] The preset probability value may be a preset probability value threshold.
[0163] The target probability value may be a probability value selected from the probability values and having a value greater than or equal to a preset probability value.
[0164] In some embodiments of the present application, probability values whose values are greater than or equal to preset probability values can be screened out from the probability values to obtain a target probability value, and then information corresponding to the preset object is pushed to a second user corresponding to the target probability value via SMS or online push, so that the second user can promptly understand the information of the preset object and respond to the preset object, thereby increasing the probability of responding to the preset object.
[0165] In some embodiments of the present application, when information corresponding to a preset object is pushed to a second user corresponding to a target probability value by SMS or online push, the information corresponding to the preset object can be pushed to the second user by an AB experiment. Specifically, for example, if information corresponding to a preset object needs to be pushed to 10 second users in total, the information corresponding to the preset object can be first sent to 5 of the second users using SMS template A, and then the information corresponding to the preset object can be sent to the remaining 5 second users using SMS template B. In this way, based on the subsequent response effects of the 10 second users to the preset object, a push method that is more suitable for the preset object can be selected, thereby further increasing the probability of the second user responding to the preset object.
[0166] In an embodiment of the present application, a target probability value is obtained by screening out probability values whose values are greater than or equal to a preset probability value from the probability values; and information corresponding to the preset object is pushed to a second user corresponding to the target probability value, so that the second user can timely understand the information of the preset object and respond to the preset object, thereby increasing the probability of responding to the preset object.
[0167] In some embodiments of the present application, in order to further improve the accuracy of removing redundant feature information, after pushing the information corresponding to the preset object to the second user corresponding to the target probability value, the data processing method involved above may further include:
[0168] Acquire a sixth number of second users who respond positively to the preset object, and a seventh number of second users who respond negatively to the preset object;
[0169] The positive sample data is updated based on the sixth quantity, and the negative sample data is updated based on the seventh quantity.
[0170] The sixth number may be the number of second users who respond positively to the preset object.
[0171] The seventh number may be the number of second users who make a negative response to the preset object.
[0172] In some embodiments of the present application, after each round of pushing information corresponding to a preset object to a second user corresponding to a target probability value, the response result of the second user to whom the information is pushed to the preset object can be obtained as a supplement to the massive feature information to optimize the redundant feature information label screening information. Specifically, the sixth number of second users who respond positively to the preset object and the seventh number of second users who respond negatively to the preset object can be obtained, and then the positive sample data is updated based on the sixth number, and the negative sample data is updated based on the seventh number.
[0173] In one example, if there are n positive sample data and m negative sample data, the positive sample data can be updated using the sixth quantity, and the negative sample data can be updated based on the seventh quantity. If N′ i is the value of each positive sample data under a certain feature after update, M′ l A′ is the value of each negative sample data under this feature after update. i Is each positive sample data after update whether it has this feature, B′ l To determine whether each negative sample data after update has this feature, the feature importance indicator Y′ is updated as follows:
[0174]
[0175] In an embodiment of the present application, by obtaining the sixth number of second users who respond positively to the preset object, and the seventh number of second users who respond negatively to the preset object; updating the positive sample data based on the sixth number, and updating the negative sample data based on the seventh number, the massive feature information can be continuously optimized, the accuracy of removing redundant feature information can be improved, and more representative target feature information that can be used to build a prediction model can be obtained.
[0176] In some embodiments of the present application, in order to enable users to understand the technical solutions of the embodiments of the present application in more detail, the technical solutions of the embodiments of the present application are described in detail below by taking an online marketing activity as an example:
[0177] Figure 2 A framework diagram of a data processing system for implementing a data processing method provided in an embodiment of the present application, such as Figure 2 As shown, the data processing system includes: a redundant feature processing module 210, an online scene prediction model construction module 220, a user reach module 230 and an effect verification and feature optimization module 240.
[0178] The following uses an online marketing campaign as an example to explain the functions of each module in detail:
[0179] 1. Redundant feature processing module 210
[0180] In some embodiments of the present application, the redundant feature processing module 210 is used to remove redundant feature information from the massive feature information to obtain target feature information, which corresponds to steps 110 to 140 in the above embodiment, and will not be described in detail here.
[0181] In this way, by comparing and analyzing the response parameters of positive sample data and negative sample data under each feature information, using the difference as an indicator of feature importance, high-quality feature information is extracted from massive feature information, redundant feature information is removed, and the efficiency and effectiveness of subsequent prediction model training are improved.
[0182] 2. Online scene prediction model construction module 220
[0183] In some embodiments of the present application, the online scenario prediction model construction module 220 mainly implements two parts: one is to sample negative sample data in online marketing activities to obtain target negative sample data; the other is to train the prediction model using target feature information to obtain the target prediction model. The specific implementation process of each part has been introduced in the above embodiments and will not be repeated here.
[0184] In this way, by sampling the negative sample data, the imbalance problem between positive sample data and negative sample data is solved, and the construction of positive sample data and negative sample data is completed. At the same time, by integrating the tree model and the linear model, a fusion prediction model with strong interpretability is formed, and the prediction model results are optimized and adjusted based on the reach results. The probability of high potential users (i.e., second users) in online scenarios fused with multiple algorithms responding to preset objects is calculated.
[0185] 3. User Reach Module 230
[0186] In some embodiments of the present application, the user contact module 230 is used to send the information of the preset object to the second user by SMS or online push after obtaining the probability value of the second user responding to the preset object based on the target prediction model. The specific implementation process has been introduced in the above embodiment and will not be repeated here.
[0187] 4. Effect inspection and feature optimization module 240
[0188] In some embodiments of the present application, the effect verification and feature optimization module 240 is used to record the response result of the second user to the preset object after each user accesses the module, as a supplement to the massive feature information, optimize the screening of redundant feature information labels, and improve the prediction model effect data. The specific implementation process has been introduced in the above embodiment and will not be repeated here.
[0189] The data processing system proposed in the embodiment of the present application aims to form a complete solution for user screening, reaching, and optimization through methods such as redundant feature information removal, data migration sampling, and multi-model fusion, so as to provide differentiated data services to users in a concise and effective manner, enhance the adaptability between users and online scenarios, improve user experience, and improve the efficiency and effectiveness of user operations of business parties.
[0190] It should be noted that the data processing method provided in the embodiment of the present application can be executed by a data processing device or a control module in the data processing device for executing the data processing method. In the embodiment of the present application, the data processing device provided in the embodiment of the present application is described by taking the data processing device executing the data processing method as an example.
[0191] Based on the same inventive concept as the above-mentioned data processing method, the present application also provides a data processing device. Figure 3 The data processing device provided in the embodiment of the present application is described in detail.
[0192] Figure 3 It is a structural block diagram of a data processing device according to an exemplary embodiment.
[0193] like Figure 3 As shown, the data processing device 300 may include:
[0194] An acquisition module 310 is used to acquire first response parameters of positive sample data under multiple different feature information, and second response parameters of negative sample data under multiple different feature information;
[0195] A first determination module 320, configured to determine, for each piece of feature information, a difference between the first response parameter and the second response parameter;
[0196] The second determination module 330 is used to perform weighted calculation on each difference for each feature information to obtain a feature importance identifier of the feature information;
[0197] The screening module 340 is used to screen out the feature information that meets the preset conditions in the feature importance identification from multiple different feature information to obtain target feature information; wherein the positive sample data is the sample data in which the first user responds positively to the preset object; and the negative sample data is the sample data in which the first user responds negatively to the preset object.
[0198] In an embodiment of the present application, the first response parameters of the positive sample data under multiple different feature information and the second response parameters of the negative sample data under multiple different feature information are obtained through an acquisition module, and then the difference between the first response parameter and the second response parameter is determined for each feature information based on the first determination module, and then the respective differences are weightedly calculated based on the second determination module to obtain a feature importance identifier of the feature information, and the target feature information is selected from different feature information based on the identifier through a screening module, thereby efficiently achieving the purpose of removing redundant feature information from massive feature information.
[0199] In some embodiments of the present application, in order to further remove redundant feature information from massive feature information, the acquisition module may also be used to:
[0200] Acquire a first quantity of the positive sample data and a second quantity of the negative sample data;
[0201] Correspondingly, the first determination module may specifically include:
[0202] A first determining unit, configured to determine, for each piece of characteristic information, a first ratio of a first response parameter to a first quantity;
[0203] A second determining unit, configured to determine, for each piece of characteristic information, a second ratio of a second response parameter to the second quantity;
[0204] The third determining unit is configured to determine a difference between the first ratio and the second ratio.
[0205] In some embodiments of the present application, the first response parameter includes the number of first responders and the first response times; the second response parameter includes the number of second responders and the second response times.
[0206] In some embodiments of the present application, in order to accurately determine the first ratio, the first determination unit can be specifically used to: for each feature information, determine a first sub-ratio of the first number of responders and the first quantity, and a second sub-ratio of the first number of responses and the first quantity.
[0207] In some embodiments of the present application, in order to accurately determine the second ratio, the second determination unit can be specifically used to: for each feature information, determine a third sub-ratio of the second number of responders and the second quantity, and a fourth sub-ratio of the second number of responses and the second quantity.
[0208] In some embodiments of the present application, the third determination unit may be specifically configured to: determine a first difference between the first sub-ratio and the third sub-ratio, and a second difference between the second sub-ratio and the fourth sub-ratio.
[0209] Correspondingly, the second determination module may be specifically configured to: for each piece of feature information, perform weighted calculation on the first difference and the second difference to obtain a feature importance identifier of the feature information.
[0210] In some embodiments of the present application, in order to further improve the accuracy of determining the target feature information, the data processing device involved above may also include:
[0211] A sampling module, used for sampling the negative sample data to obtain a third amount of target negative sample data;
[0212] Correspondingly, the second determining unit may be specifically configured to:
[0213] For each feature information, a second ratio of the second response parameter to the third quantity is determined.
[0214] In some embodiments of the present application, in order to accurately sample negative sample data, the module may be specifically used for:
[0215] Obtaining a fourth amount of historical positive sample data and a fifth amount of historical negative sample data corresponding to the historical preset object;
[0216] Based on the first number, the fourth number, and the fifth number, a third number of target negative sample data is obtained.
[0217] In some embodiments of the present application, in order to accurately filter out target feature information from a plurality of different feature information, the screening module may be specifically used to:
[0218] The feature importance identifiers are arranged in descending order, and feature information corresponding to the feature importance identifiers with the first N orders is screened out to obtain target feature information; wherein N is a positive integer greater than or equal to 1.
[0219] In some embodiments of the present application, in order to allow more users to respond to the preset object, the data processing device involved above may further include:
[0220] The model training module is used to train a prediction model based on the target feature information of the positive sample data and the target feature information of the negative sample data to obtain a target prediction model; the prediction model is used to predict the probability value of a second user who has not responded to the preset object responding to the preset object.
[0221] In some embodiments of the present application, the target prediction model includes a first network and a second network; the data processing device involved above may also include:
[0222] A first prediction module, configured to predict a first probability value of the second user responding to the preset object based on the target feature information corresponding to the second user and the first network;
[0223] A second prediction module, configured to predict a second probability value of the second user responding to the preset object based on the target feature information corresponding to the second user and the second network;
[0224] The third determining module is used to perform weighted calculation on the first probability value and the second probability value to obtain the probability value that the second user responds to the preset object.
[0225] In some embodiments of the present application, in order to allow more people to respond to the preset object, the data processing device involved above may further include:
[0226] A fourth determination module is used to filter out probability values whose values are greater than or equal to a preset probability value from the probability values to obtain a target probability value;
[0227] The information push module is used to push the information corresponding to the preset object to the second user corresponding to the target probability value.
[0228] In some embodiments of the present application, in order to further improve the accuracy of removing redundant feature information, the acquisition module may also be specifically used to:
[0229] Acquire a sixth number of second users who respond positively to the preset object, and a seventh number of second users who respond negatively to the preset object;
[0230] The data processing device mentioned above may also include:
[0231] An updating module is used to update the positive sample data based on the sixth quantity, and to update the negative sample data based on the seventh quantity.
[0232] Figure 4 A schematic diagram of the hardware structure of an electronic device provided in an embodiment of the present application is shown.
[0233] The electronic device may include a processor 401 and a memory 402 storing computer program instructions.
[0234] Specifically, the processor 401 may include a central processing unit (CPU), or an application specific integrated circuit (ASIC), or may be configured to implement one or more integrated circuits of the embodiments of the present application.
[0235] Memory 402 may include a large capacity memory for data or instructions. By way of example and not limitation, memory 402 may include a hard disk drive (HDD), a floppy disk drive, a flash memory, an optical disk, a magneto-optical disk, a magnetic tape, or a universal serial bus (USB) drive or a combination of two or more of these. In appropriate cases, memory 402 may include a removable or non-removable (or fixed) medium. In appropriate cases, memory 402 may be inside or outside of an integrated gateway disaster recovery device. In a specific embodiment, memory 402 is a non-volatile solid-state memory.
[0236] The memory may include read-only memory (ROM), random access memory (RAM), magnetic disk storage media devices, optical storage media devices, flash memory devices, electrical, optical or other physical / tangible memory storage devices. Thus, typically, the memory includes one or more tangible (non-transitory) computer-readable storage media (e.g., memory devices) encoded with software including computer-executable instructions, and when the software is executed (e.g., by one or more processors), it is operable to perform the operations described with reference to the method according to an aspect of the present disclosure.
[0237] The processor 401 implements any one of the data processing methods in the above embodiments by reading and executing computer program instructions stored in the memory 402 .
[0238] In one example, the electronic device may further include a communication interface 403 and a bus 410. Figure 4 As shown, the processor 401, the memory 402, and the communication interface 403 are connected via a bus 410 and communicate with each other.
[0239] The communication interface 403 is mainly used to implement communication between various modules, devices, units and / or equipment in the embodiments of the present application.
[0240] Bus 410 includes hardware, software or both, and the parts of electronic equipment are coupled to each other. For example, but not limitation, bus may include accelerated graphics port (AGP) or other graphics bus, enhanced industrial standard architecture (EISA) bus, front side bus (FSB), hypertransport (HT) interconnection, industrial standard architecture (ISA) bus, infinite bandwidth interconnection, low pin count (LPC) bus, memory bus, micro channel architecture (MCA) bus, peripheral component interconnection (PCI) bus, PCI-Express (PCI-X) bus, serial advanced technology attachment (SATA) bus, video electronics standard association local (VLB) bus or other suitable bus or two or more of these combinations. In appropriate cases, bus 410 may include one or more buses. Although the present application embodiment describes and shows a specific bus, the application considers any suitable bus or interconnection.
[0241] The electronic device executes the data processing method in the embodiment of the present application, thereby realizing the combination Figure 1 Describe the data processing methods.
[0242] In addition, in combination with the data processing method in the above embodiments, the present application embodiment can provide a computer storage medium for implementation. The computer storage medium stores computer program instructions; when the computer program instructions are executed by a processor, any one of the data processing methods in the above embodiments is implemented.
[0243] It should be clear that the present application is not limited to the specific configuration and processing described above and shown in the figures. For the sake of simplicity, a detailed description of the known method is omitted here. In the above embodiments, several specific steps are described and shown as examples. However, the method process of the present application is not limited to the specific steps described and shown, and those skilled in the art can make various changes, modifications and additions, or change the order between the steps after understanding the spirit of the present application.
[0244] The functional blocks shown in the above-described block diagram can be implemented as hardware, software, firmware or a combination thereof. When implemented in hardware, it can be, for example, an electronic circuit, an application specific integrated circuit (ASIC), appropriate firmware, a plug-in, a function card, etc. When implemented in software, the elements of the present application are programs or code segments that are used to perform the required tasks. The program or code segment can be stored in a machine-readable medium, or transmitted on a transmission medium or a communication link by a data signal carried in a carrier wave. "Machine-readable medium" can include any medium capable of storing or transmitting information. Examples of machine-readable media include electronic circuits, semiconductor memory devices, ROM, flash memory, erasable ROM (EROM), floppy disks, CD-ROMs, optical disks, hard disks, optical fiber media, radio frequency (RF) links, etc. The code segment can be downloaded via a computer network such as the Internet, an intranet, etc.
[0245] It should also be noted that the exemplary embodiments mentioned in this application describe some methods or systems based on a series of steps or devices. However, this application is not limited to the order of the above steps, that is, the steps can be performed in the order mentioned in the embodiment, or in a different order from the embodiment, or several steps can be performed simultaneously.
[0246] Aspects of the present disclosure are described above with reference to the flowchart and / or block diagram of the method, device (system) and computer program product according to the embodiment of the present disclosure. It should be understood that each box in the flowchart and / or block diagram and the combination of each box in the flowchart and / or block diagram can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, or other programmable data processing device to produce a machine so that these instructions executed by the processor of the computer or other programmable data processing device enable the implementation of the function / action specified in one or more boxes of the flowchart and / or block diagram. Such a processor can be, but is not limited to, a general-purpose processor, a special-purpose processor, a special application processor, or a field programmable logic circuit. It can also be understood that each box in the block diagram and / or flowchart and the combination of boxes in the block diagram and / or flowchart can also be implemented by dedicated hardware that performs a specified function or action, or can be implemented by a combination of dedicated hardware and computer instructions.
[0247] The above is only a specific implementation of the present application. Those skilled in the art can clearly understand that for the convenience and simplicity of description, the specific working processes of the systems, modules and units described above can refer to the corresponding processes in the aforementioned method embodiments, and will not be repeated here. It should be understood that the protection scope of the present application is not limited to this. Any technician familiar with the technical field can easily think of various equivalent modifications or replacements within the technical scope disclosed in this application, and these modifications or replacements should be included in the protection scope of this application.
Claims
1. A data processing method, characterized in that: The method comprises: Obtaining first response parameters of positive sample data under multiple different feature information, and second response parameters of negative sample data under multiple different feature information; For each piece of feature information, respectively determine the difference between the first response parameter and the second response parameter; For each feature information, weighted calculation is performed on each difference to obtain the feature importance identification of the feature information; From a plurality of different feature information, feature information that meets a preset condition in the feature importance identifier is screened out to obtain target feature information; the target feature information is used for model training; Among them, the positive sample data is sample data of the first user responding positively to the preset object; the negative sample data is sample data of the first user responding negatively to the preset object; the preset object is an online marketing activity, the sample data of the positive response is the first user who participated in the online marketing activity and made consumption behavior in the online marketing activity, and the sample data of the negative response is the first user who participated in the online marketing activity and did not make consumption behavior in the online marketing activity; the characteristic information includes at least one of the following: the number of sharing uses, the number of public payment uses, the number of discount details page uses, the number of my red envelope uses, the number of search page uses, the number of payment control uses and the number of bill uses; the first response parameter includes the first response number and the number of first responders, and the second response parameter includes the second response number and the number of second responders.
2. The method according to claim 1, characterized in that Before determining the difference between the first response parameter and the second response parameter for each piece of feature information, the method further includes: Acquire a first quantity of the positive sample data and a second quantity of the negative sample data; The step of determining the difference between the first response parameter and the second response parameter for each piece of feature information includes: For each feature information, determining a first ratio between the first response parameter and the first quantity; For each piece of characteristic information, determining a second ratio of the second response parameter to the second quantity; A difference between the first ratio and the second ratio is determined.
3. The method according to claim 2, characterized in that The step of determining, for each piece of characteristic information, a first ratio of a first response parameter to a first quantity comprises: For each piece of characteristic information, determining a first sub-ratio of the first number of responders to the first quantity, and a second sub-ratio of the first number of responses to the first quantity; The step of determining, for each piece of characteristic information, a second ratio between the second response parameter and the second quantity comprises: For each piece of feature information, a third sub-ratio of the second number of responders and the second quantity, and a fourth sub-ratio of the second number of responses and the second quantity are determined.
4. The method according to claim 3, characterized in that Determining the difference between the first ratio and the second ratio includes: determining a first difference between the first sub-ratio and the third sub-ratio, and a second difference between the second sub-ratio and the fourth sub-ratio; For each feature information, weighted calculation is performed on each difference to obtain a feature importance identifier of the feature information, including: For each piece of feature information, a weighted calculation is performed on the first difference and the second difference to obtain a feature importance identifier of the feature information.
5. The method according to claim 2, characterized in that: After obtaining the first amount of the positive sample data and the second amount of the negative sample data, the method further includes: Sampling the negative sample data to obtain a third amount of target negative sample data; The step of determining, for each piece of characteristic information, a second ratio between the second response parameter and the second quantity comprises: For each feature information, a second ratio of the second response parameter to the third quantity is determined.
6. The method according to claim 5, characterized in that The step of sampling the negative sample data to obtain a third amount of target negative sample data includes: Obtaining a fourth amount of historical positive sample data and a fifth amount of historical negative sample data corresponding to the historical preset object; Based on the first number, the fourth number, and the fifth number, obtaining a third number of target negative sample data; The obtaining a third quantity of target negative sample data based on the first quantity, the fourth quantity, and the fifth quantity includes: The third number of target negative sample data is obtained according to the following formula: Among them, A i is the first quantity, A j is the third quantity, C i is the fourth quantity, C j The fifth quantity.
7. The method according to any one of claims 1 to 6, characterized in that: The step of selecting the feature information that meets the preset conditions in the feature importance identifier from the plurality of different feature information to obtain the target feature information includes: The feature importance identifiers are arranged in descending order, and feature information corresponding to the feature importance identifiers with the first N orders is screened out to obtain target feature information; wherein N is a positive integer greater than or equal to 1.
8. The method according to claim 1, characterized in that After selecting the feature information that meets the preset conditions in the feature importance identifier from the plurality of different feature information to obtain the target feature information, the method further includes: Based on the target feature information of the positive sample data and the target feature information of the negative sample data, a prediction model is trained to obtain a target prediction model; the target prediction model includes a first network and a second network, the first network is a linear model, and the second network is a tree model; The prediction model is used to predict the probability value of a second user who has not responded to the preset object responding to the preset object; After training the prediction model based on the target feature information to obtain the target prediction model, the method further includes: Based on the target feature information corresponding to the second user and the first network, predicting a first probability value that the second user responds to the preset object; Based on the target feature information corresponding to the second user and the second network, predicting a second probability value that the second user responds to the preset object; Performing weighted calculation on the first probability value and the second probability value to obtain a probability value of the second user responding to the preset object; Specifically, the probability value of the second user who did not respond to the preset object responding to the preset object is calculated by the following formula: Among them, LR score i is the score of the i-th second user under the linear model, that is, the first probability value, i = 1, 2, ... n, XGBoost score i is the score of the i-th second user under the tree model, that is, the second probability value; ∝ is the weight value.
9. The method according to claim 8, characterized in that After performing weighted calculation on the first probability value and the second probability value to obtain the probability value of the second user responding to the preset object, the method further includes: Filter out probability values whose values are greater than or equal to a preset probability value from the probability values to obtain a target probability value; The information corresponding to the preset object is pushed to the second user corresponding to the target probability value.
10. The method according to claim 9, characterized in that After pushing the information corresponding to the preset object to the second user corresponding to the target probability value, the method further includes: Acquire a sixth number of second users who respond positively to the preset object, and a seventh number of second users who respond negatively to the preset object; The positive sample data is updated based on the sixth number, and the negative sample data is updated based on the seventh number.
11. A data processing device, characterized in that: The device comprises: An acquisition module, used to acquire first response parameters of positive sample data under multiple different feature information, and second response parameters of negative sample data under multiple different feature information; A first determination module, configured to determine, for each piece of feature information, a difference between the first response parameter and the second response parameter; The second determination module is used to perform weighted calculation on each difference for each feature information to obtain a feature importance identifier of the feature information; A screening module is used to screen out the feature information that meets the preset conditions in the feature importance identifier from a plurality of different feature information to obtain target feature information; the target feature information is used for model training; Among them, the positive sample data is sample data of the first user responding positively to the preset object; the negative sample data is sample data of the first user responding negatively to the preset object; the preset object is an online marketing activity, the sample data of the positive response is the first user who participated in the online marketing activity and made consumption behavior in the online marketing activity, and the sample data of the negative response is the first user who participated in the online marketing activity and did not make consumption behavior in the online marketing activity; the characteristic information includes at least one of the following: the number of sharing uses, the number of public payment uses, the number of discount details page uses, the number of my red envelope uses, the number of search page uses, the number of payment control uses and the number of bill uses; the first response parameter includes the first response number and the number of first responders, and the second response parameter includes the second response number and the number of second responders.
12. A data processing device, characterized in that: The device comprises: a processor and a memory storing computer program instructions; when the processor executes the computer program instructions, the data processing method according to any one of claims 1 to 10 is implemented.
13. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores computer program instructions, and when the computer program instructions are executed by a processor, the data processing method according to any one of claims 1 to 10 is implemented.
14. A computer program product, characterized in that When the instructions in the computer program product are executed by a processor of an electronic device, the electronic device executes the data processing method according to any one of claims 1 to 10.
Citation Information
Patent Citations
Data evaluation method based on dimension model and computer program product
CN113570259A
Recommendation Model Training Method and Related Apparatus
US20210326729A1