Target crowd acquisition method based on multi-source data, electronic equipment and storage medium
By constructing a judgment matrix and using linear regression or XGBoost model, combining Internet interest and behavioral characteristic data, accurately identifying people susceptible to dangerous Internet behaviors, solving the identification problems in the existing technology and improving the prevention effect.
Patent Information
- Application Number
- CN202411229462.0
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-09-03
- Publication Date
- 2024-12-20
AI Technical Summary
The existing technology is difficult to accurately identify and early warning of people susceptible to dangerous Internet behavior, resulting in potential threats to information and property security.
By obtaining the Internet interest feature data, Internet behavior feature data and social attribute data of candidate users, a judgment matrix is constructed, and the probability of users being the target population is predicted using linear regression model or XGBoost model, and the optimization matrix is adjusted through consistency test and scale value. , determine whether the user is the target group.
Accurate identification and early warning of the target population has been achieved, and the prevention effect of dangerous Internet behavior has been improved.
Smart Images
Figure CN119167299A_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to the field of computer technology, and in particular to a method for acquiring a target population based on multi-source data, an electronic device and a storage medium. Background Art
[0002] With the vigorous development of network technology, users use terminal devices to obtain various information services such as various communication services and Internet services. However, some dangerous Internet behaviors that harm users' property have also emerged and become increasingly serious, showing characteristics such as diversity and high concealment, which will pose potential dangers to users' information and / or property security. Therefore, if the susceptible population of such dangerous Internet behaviors can be accurately identified and early warnings can be issued, it can prevent dangerous Internet behaviors to a certain extent. Summary of the invention
[0003] In view of the above technical problems, the technical solution adopted by the present invention is:
[0004] The embodiment of the present invention provides a method for acquiring a target population based on multi-source data, the method comprising the following steps:
[0005] S100, obtaining a first prediction value Y1 and a second prediction value Y2 and social attribute data corresponding to the Internet interest feature data and the Internet behavior feature data of the candidate user, wherein the first prediction value Y1 is a probability that the candidate user corresponding to the Internet interest feature data is predicted to be the target population, and the second prediction value Y2 is a probability that the candidate user corresponding to the Internet behavior feature data is predicted to be the target population;
[0006] S200, obtaining a first weight of a target feature of the candidate user, where the target feature includes the first prediction value, the second prediction value, and the social attribute data;
[0007] S300, acquiring an initial scale value set based on a first weight of the target feature, and acquiring an initial judgment matrix of the target feature based on the initial scale value set;
[0008] S400, performing consistency check on the current judgment matrix, if it passes, executing S500; otherwise, adjusting the current scale value set, and adjusting the current judgment matrix based on the adjusted scale value set, and executing S400;
[0009] S500, obtaining a second weight of the target feature based on the current judgment matrix, and obtaining a target value of the candidate user based on the target feature and the corresponding second weight; executing S600;
[0010] S600: If the target value of the candidate user is greater than a set threshold, the candidate user is set as a target population.
[0011] Optionally, S300 specifically includes:
[0012] S301, sorting the first weights of the target features in ascending or descending order to obtain a first weight sequence L={L1, L2, . . . , L p , ..., L q}, L p is the pth weight in L, p ranges from 1 to q, and q is the number of weights in L;
[0013] S302, obtain the initial scaling factor f0=d 1q / (S max -1), d 1q =|△L1-△L q ∣,S max is the maximum scale value in the preset scale set;
[0014] S303, obtain the kth weight L in L k and the g-th weight L k+1 The scale value S kg =[|△L k -△L g | / f0]+1, [] indicates rounding, k and g range from 1 to q-1, and k≠g; forming the initial scale value set S0={S kg};
[0015] S304: Based on S0, construct an initial judgment matrix of the target feature.
[0016] Optionally, in S400, adjusting the current scale value set, and adjusting the current judgment matrix based on the adjusted scale value set specifically includes:
[0017] S401, if (S c -△b)>S2, execute S402, otherwise, exit the current control program; S c is the maximum scale value in the current scale value set, △b is the scale difference between two adjacent scale values, and S2 is the second smallest scale value in the preset scale value set;
[0018] S402, obtain the current scaling factor fc=d 1q / (S c -△b-1);
[0019] S403, obtain L k and L k+1 Adjustment scale value S between ckg =[|△L k -△L g | / fc]+1; get the adjusted scale value set Sc={S c kg};
[0020] S404: Based on Sc, adjust the current judgment matrix.
[0021] Optionally, the preset scale value set is {1, 2, ..., 9}.
[0022] Optionally, the Internet interest characteristic data is APP installation data, and the Internet behavior characteristic data is financial transaction data.
[0023] Optionally, the first prediction value and the second prediction value are obtained based on a linear regression model or an XGBoost model.
[0024] Optionally, the first weight is obtained based on an XGBoost model.
[0025] Optionally, the first weight is obtained through an XGBoost model and expert experience.
[0026] An embodiment of the present invention also provides a non-transitory computer-readable storage medium, in which at least one instruction or at least one program is stored, and wherein the at least one instruction or the at least one program is loaded and executed by a processor to implement the aforementioned method.
[0027] An embodiment of the present invention further provides an electronic device, including a processor and the aforementioned non-transitory computer-readable storage medium.
[0028] The present invention has at least the following beneficial effects:
[0029] The target population acquisition method based on multi-source data provided by the embodiment of the present invention can construct a judgment matrix based on the Internet interest feature data, Internet behavior feature data and social attribute data of the candidate users to obtain the probability that the candidate users are the target population, and can accurately identify the target population. BRIEF DESCRIPTION OF THE DRAWINGS
[0030] In order to more clearly illustrate the technical solutions in the embodiments of the present invention, the following briefly introduces the drawings required for use in the description of the embodiments. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without creative work.
[0031] Figure 1A flowchart of a method for acquiring a target population based on multi-source data provided by an embodiment of the present invention. DETAILED DESCRIPTION
[0032] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative work are within the scope of protection of the present invention.
[0033] The embodiment of the present invention provides a method for acquiring a target population based on multi-source data, such as Figure 1 As shown, the method may include the following steps:
[0034] S100, obtaining a first prediction value Y1 and a second prediction value Y2 and social attribute data corresponding to the Internet interest feature data and Internet behavior feature data of the candidate user.
[0035] In an embodiment of the present invention, the Internet interest characteristic data may be mobile Internet behavior characteristic data. In an embodiment of the present invention, the Internet behavior characteristic data may be mobile Internet APP installation behavior, Internet browsing behavior, etc.
[0036] In an embodiment of the present invention, the first prediction value Y1 may be a probability that the candidate user corresponding to the prediction based on the Internet interest feature data is the target population, and the second prediction value Y2 is a probability that the candidate user corresponding to the prediction based on the Internet behavior feature data is the target population. The first prediction value Y1 and the second prediction value Y2 may be obtained based on a linear regression model or an XGBoost model, and may be generated after independent modeling from different types of Internet vendors.
[0037] In the embodiment of the present invention, the social attribute data may at least include age, gender, place of residence, education, occupation, user level, size of social circle, peak time of Internet traffic, etc. The social attribute data may be obtained through the population archive data provider.
[0038] S200, obtaining a first weight of a target feature of the candidate user, where the target feature includes the first prediction value, the second prediction value, and the social attribute data.
[0039] In one embodiment of the present invention, the first weight may be obtained through a model, and specifically, may be obtained through an XGBoost model.
[0040] Specifically, it can be obtained through the following steps:
[0041] S201, obtain a sample data set, including positive samples and negative samples. Each sample includes a first prediction value, a second prediction value, social attribute data and a sample label corresponding to the corresponding user. If the sample label is a first set value, it means that the user corresponding to the sample is a target population. If the sample label is a second set value, it means that the user corresponding to the sample is not a target population. The first set value is, for example, 10, and the second set value is, for example, 01.
[0042] S202, inputting the positive and negative sample data of the current batch into the initial XGBoost model, and obtaining a corresponding probability prediction value for each sample;
[0043] S203, obtaining the current loss function value based on the probability prediction values and sample labels of all samples in the current batch, and determining whether the current loss function value meets the preset model training end condition, if so, executing step S205, otherwise, executing step S204;
[0044] S204, calculating partial derivatives based on the current loss function to obtain current parameter values, updating weights corresponding to each neural network based on the current parameter values, and using the positive and negative sample data of the next batch as the positive and negative sample data of the current batch, and executing S202;
[0045] S205: Using the weight in the current XGBoost model as the first weight.
[0046] The obtained first weight will be stored in the database. The first weight of the target feature of the candidate user can be obtained by querying the database.
[0047] In the embodiment of the present invention, the weight of each feature is obtained through the XGBoost model, which can improve the accuracy of weight acquisition and avoid subjective bias caused by human experience.
[0048] In another embodiment of the present invention, the first weight is obtained through the XGBoost model and expert experience. Specifically, the following steps may be included:
[0049] S210, obtaining a first candidate weight of the target feature based on the XGBoost model; the step of obtaining the first candidate weight can refer to the aforementioned embodiment.
[0050] S211, obtain the second candidate weight of the target feature by sending a questionnaire to the designated experts. S212, obtain the deviation between the first candidate weight and the second candidate weight, if the deviation is less than the set deviation threshold, use the first candidate weight as the first weight, otherwise, output a prompt message.
[0051] In one embodiment of the present invention, the deviation can be obtained based on the Mahalanobis distance or covariance. If the Mahalanobis distance between the first candidate weight and the second candidate weight is smaller, it means that the deviation between the two is smaller, and vice versa. If the covariance between the first candidate weight and the second candidate weight is larger, it means that the change trend between the two is closer, and the deviation is smaller. Those skilled in the art know that the specific calculation of the Mahalanobis distance and covariance can be prior art.
[0052] In the embodiment of the present invention, the deviation threshold may be set to a custom value, such as an empirical value, which is not particularly limited in the present invention.
[0053] In an embodiment of the present invention, the outputted prompt information may be information indicating that the first candidate weight may be inaccurate. The number of samples, network parameters, etc. may be adjusted based on the prompt information, or the first candidate weight may be adjusted based on the second candidate weight so that the change trend of the first candidate weight is consistent with that of the second candidate weight.
[0054] S300: Acquire an initial scale value set based on a first weight of the target feature, and acquire an initial judgment matrix of the target feature based on the initial scale value set.
[0055] Furthermore, S300 may specifically include:
[0056] S301, sorting the first weights of the target features in ascending or descending order to obtain a first weight sequence L={L1, L2, . . . , L p , ..., L q}, L p is the pth weight in L, p ranges from 1 to q, and q is the number of weights in L, that is, the number of features in the target feature.
[0057] Those skilled in the art know that each weight is the weight of a corresponding feature in the target features.
[0058] S302, obtain the initial scaling factor f0=d 1q / (S max -1), d 1q =|△L1-△L q ∣,S max is the maximum scale value in the preset scale set.
[0059] In an embodiment of the present invention, the preset scale value set is {1, 2, ..., 9}, that is, a scale of 1 to 9.
[0060] S303, obtain the kth weight L in L k and the g-th weight L k+1 The scale value S kg =[|△Lk -△L g | / f0]+1, [] indicates rounding, k and g range from 1 to q-1, and k≠g; forming the initial scale value set S0={S kg}.
[0061] Through S302 and S303, the scale values corresponding to any two weights can fall into the preset scale value set.
[0062] S304: Based on S0, construct an initial judgment matrix of the target feature.
[0063] The judgment matrix may be a judgment matrix of the hierarchical analysis method. Those skilled in the art know that, based on S0, constructing the initial judgment matrix of the target feature may be prior art.
[0064] In a specific example, for example, a scale of 1 to 9 is used, and the target features include 6 features, namely the first predicted value, the second predicted value, household registration, age, gender and education level. The weights of these 6 features are 0.4, 0.3, 0.1, 0.1, 0.05, 0.05, respectively. After sorting in descending order, L = {0.4, 0.3, 0.1, 0.1, 0.05, 0.05} is obtained. Then f0 = (0.4-0.05) / (9-1) = 0.35 / 8. In this way, the scale values between the first weight and the second weight, the first weight and the third weight, ..., the fifth weight and the sixth weight are: 3, 7, 7, 9, 9, 5, 5, 6, 6, 1, 2, 2, 2, 2, 1. In this way, the initial judgment matrix constructed is shown in Table 1 below:
[0065] Table 1
[0066] First prediction value Second prediction value Household registration age gender Education First prediction value 1 3 7 7 9 9 Second prediction value 1 / 3 1 5 5 6 6 Household registration 1 / 7 1 / 5 1 1 2 2 age 1 / 7 1 / 5 1 1 2 2 gender 1 / 9 1 / 6 1 / 2 1 / 2 1 1 Education 1 / 9 1 / 6 1 / 2 1 / 2 1 1
[0067] S400, performing consistency check on the current judgment matrix, if passing, executing S500; otherwise, adjusting the current scale value set, and adjusting the current judgment matrix based on the adjusted scale value set, and executing S400.
[0068] In the embodiment of the present invention, the method for performing consistency check on the current judgment matrix may be the existing technology.
[0069] Further, in S400, adjusting the current scale value set, and adjusting the current judgment matrix based on the adjusted scale value set, specifically includes:
[0070] S401, if (S c -△b)>S2, execute S402, otherwise, exit the current control program, because if it is reduced again, S c-△b is equal to the minimum scale value in the preset scale value set, and the minimum scale value indicates equal importance and cannot reflect the importance judgment; S c is the maximum scale value in the current scale value set, △b is the scale difference between two adjacent scale values, and S2 is the second smallest scale value in the preset scale value set. When the preset scale value is {1, 2, ..., 9}, △b is 1 and S2 is 2.
[0071] S402, obtain the current scaling factor fc=d 1q / (S c -△b-1);
[0072] S403, obtain L k and L k+1 Adjustment scale value S between c kg =[|△L k -△L g | / fc]+1; get the adjusted scale value set Sc={S c kg}.
[0073] S404: Based on Sc, the current judgment matrix is adjusted, that is, the scale values in the current judgment matrix are replaced with the adjusted scale values.
[0074] S500, obtaining a second weight of the target feature based on the current judgment matrix, and obtaining a target value of the candidate user based on the target feature and the corresponding second weight; executing S600.
[0075] Those skilled in the art know that obtaining the second weight of the target feature based on the current judgment matrix may be an existing technology.
[0076] In one embodiment of the present invention, the target value of the candidate user may be the weighted sum of the target feature and the corresponding second weight, that is, Y=a1*x1+a2*x2+…+ap*xp+…+aq*xq, ap is the second weight of the pth feature, and xp is the feature value of the pth feature.
[0077] S600: If the target value of the candidate user is greater than a set threshold, the candidate user is set as a target population.
[0078] In the embodiment of the present invention, the threshold value may be set to a user-defined value.
[0079] The target population acquisition method based on multi-source data provided by the embodiment of the present invention can construct a judgment matrix based on the Internet interest feature data, Internet behavior feature data and social attribute data of the candidate users to obtain the probability that the candidate users are the target population, and can accurately identify the target population.
[0080] An embodiment of the present invention also provides a non-transitory computer-readable storage medium, which can be set in an electronic device to store at least one instruction or at least one program related to implementing a method in a method embodiment. The at least one instruction or the at least one program is loaded and executed by the processor to implement the method provided in the above embodiment.
[0081] An embodiment of the present invention further provides an electronic device, comprising a processor and the aforementioned non-transitory computer-readable storage medium.
[0082] An embodiment of the present invention further provides a computer program product, which includes program code. When the program product is run on an electronic device, the program code is used to enable the electronic device to execute the steps of the method according to various exemplary embodiments of the present invention described above in this specification.
[0083] Although some specific embodiments of the present invention have been described in detail by way of example, it should be understood by those skilled in the art that the above examples are for illustration only and are not intended to limit the scope of the present invention. It should also be understood by those skilled in the art that various modifications may be made to the embodiments without departing from the scope and spirit of the present invention. The scope of the present invention is defined by the appended claims.
Claims
1. A method for acquiring a target population based on multi-source data, characterized in that: The method comprises the following steps: S100, obtaining a first prediction value Y1 and a second prediction value Y2 and social attribute data corresponding to the Internet interest feature data and the Internet behavior feature data of the candidate user, wherein the first prediction value Y1 is a probability that the candidate user corresponding to the Internet interest feature data is predicted to be the target population, and the second prediction value Y2 is a probability that the candidate user corresponding to the Internet behavior feature data is predicted to be the target population; S200, obtaining a first weight of a target feature of the candidate user, where the target feature includes the first prediction value, the second prediction value, and the social attribute data; S300, acquiring an initial scale value set based on a first weight of the target feature, and acquiring an initial judgment matrix of the target feature based on the initial scale value set; S400, performing consistency check on the current judgment matrix, if it passes, executing S500; otherwise, adjusting the current scale value set, and adjusting the current judgment matrix based on the adjusted scale value set, and executing S400; S500, obtaining a second weight of the target feature based on the current judgment matrix, and obtaining a target value of the candidate user based on the target feature and the corresponding second weight; executing S600; S600: If the target value of the candidate user is greater than a set threshold, the candidate user is set as a target population.
2. The method according to claim 1, characterized in that S300 specifically includes: S301, sorting the first weights of the target features in ascending or descending order to obtain a first weight sequence L={L1, L2, . . . , L p , ..., L q }, L p is the pth weight in L, p ranges from 1 to q, and q is the number of weights in L; S302, obtain the initial scaling factor f0=d 1q / (S max -1), d 1q =|△L1-△L q ∣,S max is the maximum scale value in the preset scale set; S303, obtain the kth weight L in L k and the g-th weight L k+1 The scale value S kg =[|△L k -△L g | / f0]+1, [] indicates rounding, k and g range from 1 to q-1, and k≠g; forming the initial scale value set S0={S kg }; S304: Based on S0, construct an initial judgment matrix of the target feature.
3. The method according to claim 2, characterized in that In S400, the current scale value set is adjusted, and the current judgment matrix is adjusted based on the adjusted scale value set, specifically including: S401, if (S c -△b)>S2, execute S402, otherwise, exit the current control program; S c is the maximum scale value in the current scale value set, △b is the scale difference between two adjacent scale values, and S2 is the second smallest scale value in the preset scale value set; S402, obtain the current scaling factor fc=d 1q / (S c -△b-1); S403, obtain L k and L k+1 Adjustment scale value S between c kg =[|△L k -△L g | / fc]+1; get the adjusted scale value set Sc={S c kg }; S404: Based on Sc, adjust the current judgment matrix.
4. The method according to claim 1, characterized in that The preset scale value set is {1, 2, ..., 9}.
5. The method according to claim 1, characterized in that The Internet interest characteristic data is APP installation data, and the Internet behavior characteristic data is financial transaction data.
6. The method according to claim 1, characterized in that The first prediction value and the second prediction value are obtained based on a linear regression model or an XGBoost model.
7. The method according to claim 1, characterized in that The first weight is obtained based on the XGBoost model.
8. The method according to claim 1, characterized in that The first weight is obtained through the XGBoost model and expert experience.
9. A non-transitory computer-readable storage medium, wherein at least one instruction or at least one program is stored in the storage medium, characterized in that: The at least one instruction or the at least one program is loaded and executed by the processor to implement the method according to any one of claims 1 to 8.
10. An electronic device, characterized in that: The invention comprises a processor and the non-transitory computer-readable storage medium as claimed in claim 9.