Target security level identification method, device and electronic equipment based on fusion model

Abstract

The present invention discloses a method, device and electronic device for identifying target security levels based on a fusion model. The method comprises: pre-processing historical target data to obtain a training set; dividing the training set into multiple training subsets; for each training subset, using multiple different models and fusion methods to train the corresponding fusion model; inputting the target data to be identified into each trained fusion model to obtain multiple initial security levels of the target; and fusing and calculating the multiple initial security levels of the target to be identified to obtain the security level of the target. The present invention divides the training set into multiple training subsets; for each training subset, using multiple different models and fusion methods to train, thereby improving the accuracy of the target's initial security level. On this basis, the multiple initial security levels of the target are fused, thereby strengthening the protection of data security in the platform, avoiding data leakage, and improving the data security of the platform.

Description

Technical Field

[0001] The present invention relates to the field of data processing, and in particular to a method, device, electronic device and computer-readable medium for identifying target security levels based on a fusion model. Background Art

[0002] With the development of the Internet, various Internet service platforms have emerged, such as online shopping platforms, online ride-hailing platforms, sharing platforms, maps, music, etc. These platforms have brought great convenience to people's lives, but because they are closely integrated with the Internet, there are some security risks such as fraud and loss of trust. Therefore, the security identification of devices is particularly important in Internet service platforms.

[0003] Currently, tree-based ensemble models such as XGBoost, LightGBM, and GBDT are used to identify devices and determine their security. During the training process, these models are limited by factors such as incomplete data collection and missing scenario data, resulting in some missing data in the training set. When some data in the training set is missing, it is typically filled with a preset default value, allowing the tree model to be trained using the training set filled with the default value. Because the gap between the default value and the true value is often large and does not accurately reflect the true value, this training method affects the model's ranking ability, resulting in inaccurate device security identification and the risk of data leakage from internet service platforms. Summary of the Invention

[0004] In view of this, the main purpose of the present invention is to propose a target security level identification method, device, electronic device and computer-readable medium based on a fusion model, in order to at least partially solve at least one of the above technical problems.

[0005] In order to solve the above technical problems, the first aspect of the present invention proposes a target security level identification method based on a fusion model, the method comprising:

[0006] Preprocess the historical target data to obtain the training set;

[0007] Dividing the training set into a plurality of training subsets;

[0008] For each training subset, multiple different models and fusion methods are used to train the corresponding fusion model;

[0009] Inputting the target data to be identified into each trained fusion model respectively to obtain multiple initial security levels of the target to be identified;

[0010] The multiple initial security levels of the target to be identified are fused and calculated to obtain the security level of the target to be identified.

[0011] According to a preferred embodiment of the present invention, for each training subset, a plurality of different models and fusion methods are used to train the corresponding fusion model, including:

[0012] For each training subset, models of different categories and / or models of the same category with different parameters and fusion methods are used for training; wherein the models and fusion methods used in each training subset divided into the same training set are different from each other.

[0013] According to a preferred embodiment of the present invention, the training of the corresponding fusion model using multiple different models and fusion methods includes:

[0014] Dividing the training subset into a training sub-dataset and a validation sub-dataset;

[0015] By training sub-datasets and validation sub-datasets, multiple models of different categories are trained to obtain multiple groups of sub-fusion models;

[0016] Perform model fusion on multiple groups of sub-fusion models to obtain the corresponding fusion model;

[0017] And / or, the adopting of multiple different models and fusion methods to train the corresponding fusion model includes:

[0018] generating a plurality of first models with different parameters;

[0019] Inputting the training subsets into a plurality of first models for training respectively, to obtain a plurality of groups of sub-fusion models and output results of each sub-fusion model;

[0020] Performing mean fusion on multiple groups of sub-fusion models and corresponding output results to obtain a corresponding fusion model;

[0021] And / or, the adopting of multiple different models and fusion methods to train the corresponding fusion model includes:

[0022] Dividing the training subset into N sub-datasets;

[0023] Input N sub-datasets into N second models with different parameters for training, and obtain N groups of sub-fusion models and N output results output by each sub-fusion model;

[0024] The N groups of sub-fusion models and the corresponding N output results are averaged and fused to obtain a corresponding fusion model.

[0025] According to a preferred embodiment of the present invention, before using multiple different models and fusion methods to train the corresponding fusion model, the method further includes:

[0026] Determine the correlation coefficients for each model;

[0027] Models with correlation coefficients greater than a threshold are considered as models of different categories and / or models of the same category with different parameters.

[0028] According to a preferred embodiment of the present invention, dividing the training set into multiple training subsets includes:

[0029] Balance the training set;

[0030] Samples are extracted multiple times from the balanced training set to form training subsets.

[0031] According to a preferred embodiment of the present invention, preprocessing the historical target data to obtain a training set includes:

[0032] Perform data cleaning on historical target data to obtain wide table variables;

[0033] A descriptive exploratory analysis is performed on the wide table variables, and characteristic data is screened out according to the analysis results to obtain a training set.

[0034] To solve the above technical problems, the second aspect of the present invention provides a target security level identification device based on a fusion model, the device comprising:

[0035] The preprocessing module is used to preprocess the historical target data to obtain the training set;

[0036] A partitioning module, configured to divide the training set into a plurality of training subsets;

[0037] The training module is used to train the corresponding fusion model for each training subset using multiple different models and fusion methods;

[0038] An input module, configured to input the target data to be identified into each trained fusion model to obtain multiple initial security levels of the target to be identified;

[0039] The fusion module is used to fuse and calculate the multiple initial security levels of the target to be identified to obtain the security level of the target to be identified.

[0040] According to a preferred embodiment of the present invention, the training module is used to train each training subset using models of different categories and / or models of the same category with different parameters and fusion methods; wherein the models and fusion methods used in each training subset divided into the same training set are different from each other.

[0041] According to a preferred embodiment of the present invention, the training module includes:

[0042] A first partitioning module is configured to divide the training subset into a training sub-dataset and a validation sub-dataset;

[0043] The first training module is used to train multiple models of different categories using a training sub-dataset and a validation sub-dataset to obtain multiple groups of sub-fusion models;

[0044] The first fusion module is used to fuse multiple groups of sub-fusion models to obtain a corresponding fusion model;

[0045] And / or, the training module includes:

[0046] A generating module, configured to generate a plurality of first models with different parameters;

[0047] A second training module is used to input the training subsets into multiple first models for training, to obtain multiple groups of sub-fusion models and output results of each sub-fusion model;

[0048] A second fusion module is used to perform mean fusion on multiple groups of sub-fusion models and corresponding output results to obtain a corresponding fusion model;

[0049] And / or, the training module includes:

[0050] A third partitioning module is used to divide the training subset into N sub-data sets;

[0051] The third training module is used to input N sub-data sets into N second models with different parameters for training, thereby obtaining N groups of sub-fusion models and N output results output by each sub-fusion model;

[0052] The third fusion module is used to perform average fusion on the N groups of sub-fusion models and the corresponding N output results to obtain a corresponding fusion model.

[0053] According to a preferred embodiment of the present invention, the device further comprises:

[0054] A determination module is used to determine the correlation coefficients of each model;

[0055] The screening module is used to select models with correlation coefficients greater than a threshold as models of different categories and / or models of the same category with different parameters.

[0056] According to a preferred embodiment of the present invention, the division module includes:

[0057] The balancing processing module is used to balance the training set;

[0058] The sampling module is used to extract samples from the balanced training set multiple times to form training subsets.

[0059] According to a preferred embodiment of the present invention, the preprocessing module includes:

[0060] The cleaning module is used to clean the historical target data and obtain wide table variables;

[0061] The analysis and screening module is used to perform descriptive exploratory analysis on the wide table variables and screen out characteristic data based on the analysis results to obtain a training set.

[0062] To solve the above technical problems, the present invention provides an electronic device according to a third aspect, including:

[0063] processor; and

[0064] A memory storing computer executable instructions, which, when executed, cause the processor to perform the above method.

[0065] In order to solve the above technical problems, the fourth aspect of the present invention provides a computer-readable storage medium, wherein the computer-readable storage medium stores one or more programs, and when the one or more programs are executed by a processor, the above method is implemented.

[0066] The present invention divides the training set into multiple groups of training subsets; for each training subset, multiple different models and fusion methods are used for training. The trained fusion model is compatible with the data characteristics of different training subsets and the sorting and prediction capabilities of different models based on different fusion methods, thereby improving the accuracy of the target's initial security level. On this basis, the multiple initial security levels of the target are fused to make the target security level more accurate, thereby strengthening the protection of data security in the Internet service platform, and effectively avoiding the situation in the related technology where the security level of the user target is not accurately determined, causing data leakage in the Internet service platform. BRIEF DESCRIPTION OF THE DRAWINGS

[0067] In order to make the technical problems solved by the present invention, the technical means adopted, and the technical effects achieved more clearly, specific embodiments of the present invention will be described in detail below with reference to the accompanying drawings. However, it should be noted that the drawings described below are only drawings of exemplary embodiments of the present invention. Those skilled in the art can derive drawings of other embodiments based on these drawings without inventive effort.

[0068] Figure 1 This is a flow chart of a method for identifying target security levels based on a fusion model according to an embodiment of the present invention;

[0069] Figure 2 Schematic diagram of an embodiment of the present invention in which a training set D0 is divided into three training subsets D1, D2, and D3, and a corresponding fusion model is trained using multiple different models and fusion methods for each training subset;

[0070] Figure 3Schematic diagram of a fusion model obtained by training multiple SVM models with different parameters in an embodiment of the present invention;

[0071] Figure 4 This is a schematic diagram of the structural framework of a target security level identification device based on a fusion model according to an embodiment of the present invention;

[0072] Figure 5 is a structural block diagram of an exemplary embodiment of an electronic device according to the present invention;

[0073] Figure 6 is a schematic diagram of an embodiment of a computer-readable medium of the present invention. DETAILED DESCRIPTION

[0074] The exemplary embodiments of the present invention will now be described more fully with reference to the accompanying drawings. Although each exemplary embodiment can be implemented in a variety of specific ways, it should not be understood that the present invention is limited to the embodiments set forth herein. Rather, these exemplary embodiments are provided to make the content of the present invention more complete and to more fully convey the inventive concept to those skilled in the art.

[0075] The same reference numerals in the accompanying drawings represent the same or similar elements, components or parts, and thus repeated descriptions of the same or similar elements, components or parts may be omitted below. It should also be understood that although the first, second, third and other numbered adjectives may be used herein to describe various devices, elements, components or parts, these devices, elements, components or parts should not be limited by these adjectives. In other words, these adjectives are only used to distinguish one from another. For example, the first device may also be called the second device, but this does not deviate from the essential technical solution of the present invention. In addition, the terms "and / or" and "and / or" refer to all combinations including any one or more of the listed items.

[0076] See also Figure 1 , Figure 1 This invention provides a target security level identification method based on a fusion model. Figure 1 As shown, the method includes:

[0077] S1. Preprocess the historical target data to obtain a training set;

[0078] In this embodiment, the target may be a terminal device or a server, wherein the target data may be basic information in the terminal device or server, and the basic information may be log information, data transmission information, device status information, etc. of the terminal device or server; the target may also be a user using the terminal device or server, and the target data may include one or more of the following public information: network name, user place of origin, time of last login to the system, publicly released content, user historical behavior information, device location, etc. Among them: target information may include target model, target ID, etc., and behavior information may include: purchase behavior, usage behavior, rental behavior, business browsing time, etc. Buried behavior information may include: click behavior, browsing behavior, etc.

[0079] In addition, the data processing of this scheme can be performed only through user information that cannot identify the user's identity to protect the user's privacy; the protection of user privacy can be achieved by deleting or anonymizing the information that can identify the user in the user information, and the anonymization processing can be performed by encrypting the data.

[0080] In order to train the model for stability and accuracy, this embodiment requires pre-processing the collected historical target data, such as data cleaning, data integration, data transformation, data classification, variable screening, etc. Exemplarily, this step may include:

[0081] S11. Clean the historical target data to obtain wide table variables;

[0082] This embodiment uses data cleaning to remove duplicate and erroneous information and improve data consistency. Exemplary data cleaning includes, but is not limited to, variable missing rate analysis (e.g., screening for features with a missing rate below 95%), outlier processing (e.g., capping and removing outliers), and text variable processing. After data cleaning, historical target data is stored in the same data table, resulting in wide table variables with tens of thousands of dimensions, which can improve the efficiency of iterative calculations during model training.

[0083] S12. Perform descriptive exploratory analysis on the wide table variables, and screen out characteristic data based on the analysis results to obtain a training set.

[0084] In this embodiment, the descriptive exploratory analysis may include: central tendency analysis, dispersion analysis and distribution morphology analysis, and the wide table variable may be subjected to central tendency analysis, dispersion analysis and distribution morphology analysis, and characteristic data may be screened out based on the analysis results of central tendency analysis, dispersion analysis and distribution morphology analysis to obtain a training set; central tendency includes: mean, median and mode; dispersion includes: range, standard deviation, coefficient of variation, percentile, quartile difference and variance; distribution morphology includes: skewness and kurtosis.

[0085] In this step, the descriptive exploratory analysis may also include: analyzing the discrimination (KS) of the target variable in combination with the coverage of each variable, the concentration of feature values, the relevance and significance of the target variable to obtain a first analysis result; analyzing the information value (IV) of the target variable in combination with the coverage of each variable, the concentration of feature values, the relevance and significance of the target variable to obtain a second analysis result; analyzing the feature importance ranking of the tree ensemble model (such as XGBoost, RF, etc.) in combination with the coverage of each variable, the concentration of feature values, the relevance and significance of the target variable to obtain a third analysis result. Furthermore, the first analysis result, the second analysis result and the third analysis result are combined to screen out features (for example, 200) with a coverage rate higher than a threshold and a discrimination degree for the target variable greater than a preset discrimination degree from the wide table variables to form a training set.

[0086] Furthermore, in order to verify the effect of the fusion model, before this step, the historical target data can be divided into a training set and a test set according to the chronological order of data production. For example: the historical user data between January 30, 2020 and January 30, 2021 is used as the training set, and the historical user data between February 1, 2021 and May 1, 2021 is used as the test set. The effect of the fusion model is verified through the test set.

[0087] S2, dividing the training set into multiple training subsets;

[0088] In this step, the training set can be directly divided into multiple training subsets according to the number of samples, for example: the number of samples in the training set can be evenly divided into multiple training subsets; samples can also be randomly selected from the training set to form training subsets; or samples can be selected from the training set according to preset rules to form training subsets.

[0089] In this embodiment, in order to ensure a balanced distribution of positive and negative samples in the training set, it is necessary to first balance the training set, and then take multiple samples from the balanced training set to form training subsets, thereby dividing the training set into multiple groups of training subsets. Exemplarily, the combined inheritance method can be used to balance the training set; wherein, the combined inheritance method uses the prototype chain to implement the inheritance of prototype properties and methods, and at the same time uses the constructor inheritance to implement the inheritance of instance properties, thereby avoiding data imbalance caused by multiple instances sharing a copy of reference type data. When extracting training subsets, an equal number of samples can be randomly extracted to form a training subset. For example: the training set has 1.5 million samples, which are extracted three times, and each sample subset consists of 500,000 randomly extracted samples.

[0090] S3. For each training subset, use multiple different models and fusion methods to train the corresponding fusion model;

[0091] In this embodiment: the models and fusion methods used in the various training subsets divided into the same training set are different from each other, thereby ensuring that the various fusion models after training can be compatible with the data characteristics of different training subsets and the ranking prediction capabilities of different models based on different fusion methods.

[0092] Among them: the multiple different models used in the same training subset can be models of multiple different categories (for example: XGboost model and LR model), or multiple models of the same category with different parameters (for example: logistic regression model with different parameters), or multiple models composed of models of different categories and models of the same category with different parameters (for example: XGboost model, LR model with parameter A, LR model with parameter B). The fusion method is used to fuse the different models of the training subset, so as to give full play to the advantages of each different model. The fusion method may include: mean fusion, average fusion, model fusion, cross fusion, feature fusion, etc. In actual applications, different models and different fusion methods can be arbitrarily combined to ensure that each training subset is trained using different models and fusion methods. The number of different models used in each training subset can be flexibly configured as needed.

[0093] For example, if a training subset is trained using models of different categories, the training of the corresponding fusion model using multiple different models and fusion methods includes:

[0094] S31, dividing the training subset into a training sub-dataset and a validation sub-dataset;

[0095] Use cross-validation method to train multiple different models and obtain multiple sets of output results;

[0096] In order to obtain a stable and reliable model, this embodiment uses a cross-validation method to train multiple different models. In the cross-validation method, the training subset is divided into n data sets. Each time a data set is taken out as a test set, and the other n-1 data sets are used as training sets for model training and parameter adjustment, thereby training n models with n data sets. Among them: the value of n can be determined according to the number of training models, for example: if 5 different models are trained, then n = 5; or, n can be set to a predetermined value, such as n = 5; then no matter how many models there are, the 5-fold cross-validation method will be used for training.

[0097] For example, Figure 2 Taking four different models, XGBoost, GBDT (gradient boosted tree), Extratrees (extreme random tree), and RandomForest (random forest), as examples, the XGBoost, GBDT, Extratrees, and RandomForest models were trained respectively through the 5-fold cross-validation method. After the training was completed, the training subsets were input into the above four trained models respectively to obtain four sets of output results.

[0098] S32, training multiple models of different categories using the training sub-dataset and the validation sub-dataset to obtain multiple groups of sub-fusion models respectively;

[0099] S33, performing model fusion on multiple groups of sub-fusion models to obtain a corresponding fusion model;

[0100] In this example, model fusion can be: using the LR model to perform weight training on the output results of multiple groups of sub-fusion models, thereby overcoming the problem of unbalanced weight distribution caused by manually preset weights.

[0101] The multiple groups of sub-fusion models trained in the above step S32 and the LR model trained in step S33 are used together to generate a corresponding fusion model.

[0102] In one example, if a training subset is trained using models of the same category but with different parameters, the training of the corresponding fusion model using multiple different models and fusion methods includes:

[0103] S301, generating a plurality of first models with different parameters;

[0104] generating a plurality of first models with different parameters;

[0105] The first model can be configured as needed, for example, the first model can adopt an XGboost model. Exemplarily, multiple XGboost models with different parameters can be generated by parameter perturbation, wherein the parameter perturbation refers to training models with large differences by randomly setting different parameters.

[0106] S302: Input the training subsets into multiple first models for training, to obtain multiple groups of sub-fusion models and output results of each sub-fusion model;

[0107] S303, performing mean fusion on multiple groups of sub-fusion models and corresponding output results to obtain a corresponding fusion model;

[0108] In this example, mean fusion is used as the fusion method, that is, different weights are assigned to the multiple groups of sub-fusion models according to a preset algorithm, and the output results corresponding to the multiple groups of sub-fusion models are weighted according to the weights.

[0109] The multiple groups of sub-fusion models with different parameters trained in step S302 and the model weights trained in step S303 are used together to generate a corresponding fusion model.

[0110] In another example, if a training subset is trained using models of the same category but with different parameters, the training of the corresponding fusion model using multiple different models and fusion methods includes:

[0111] S311, dividing the training subset into N sub-data sets;

[0112] The value of N is the same as the number of different models that need to be trained. For example, the training subset D2 can be divided into 5 sub-datasets D by Boostrap with replacement sampling. 21 、D 22 …D 25 , corresponding to training 5 different models.

[0113] S312, inputting the N sub-data sets into N second models with different parameters for training, to obtain N groups of sub-fusion models and N output results output by each sub-fusion model;

[0114] Wherein: The category of the second model is different from that of the first model. For example, the second model can adopt the SVM model. The specific training process can refer to Figure 3 After each sub-dataset is used to train each second model and each sub-fusion model is obtained, any sub-dataset can be used as a test set to evaluate the sub-fusion model.

[0115] S313 , performing average fusion on the N groups of sub-fusion models and the corresponding N output results to obtain a corresponding fusion model.

[0116] In this example, the fusion method adopts average fusion, that is, the output results of all N groups of sub-fusion models are averaged.

[0117] The corresponding fusion model is generated together with the multiple sub-fusion models with different parameters trained in the above step S312 and the model mean trained in step S313.

[0118] Figure 2 The training set D0 is divided into three training subsets D1, D2, and D3. Among them, the training subset D1 uses the S301~S303 method to train 4 XGboost models with different parameters. After mean fusion, the fusion model M1 is obtained. The training subset D2 uses the S311~S313 method to train 5 SVM models with different parameters. After averaging, the fusion model M2 is obtained. The training subset D2 uses the S31~S32 method to train four models of XGBoost, GBDT, Extratrees, and RandomForest. After LR model fusion, the fusion model M3 is obtained.

[0119] In this embodiment, the smaller the correlation between the multiple different models used in each training subset, the better the ranking effect of the trained fusion model. Therefore, for each training subset, before using multiple different models and fusion methods to train the corresponding fusion model, the models can be screened. For example, the correlation coefficients of the different models can be determined first; then, the models with correlation coefficients greater than a threshold are selected as the final different models. The correlation coefficients of the models can be evaluated using cosine similarity or Pearson correlation coefficient.

[0120] S4. Input the target data to be identified into each trained fusion model to obtain multiple initial security levels of the corresponding target to be identified;

[0121] In order to ensure the recognition effect of the fusion model, the target data to be identified can be pre-processed as in step S1, and the pre-processed target data to be identified can be input into each trained fusion model (for example, Figure 2 In the fusion models M1, M2 and M3), each fusion model will output the initial security level of the target to be identified. For example, the initial security level of the target to be identified output by the fusion model M1 is recorded as m1, the initial security level of the target to be identified output by the fusion model M2 is recorded as m2, and the initial security level of the target to be identified output by the fusion model M3 is recorded as m3.

[0122] In this embodiment, the security level of the target to be identified may be a specific score value, or a category value divided based on the score value, which is not specifically limited in the present invention.

[0123] S5. merging and calculating multiple initial security levels of the target to be identified to obtain the security level of the target to be identified.

[0124] like Figure 2The multiple initial security levels of the target to be identified are fused and calculated to form a second-layer model M. The fusion calculation of multiple initial security levels can include mean fusion, average fusion, model fusion, cross fusion, feature fusion, etc. For example, mean fusion can be used for m1, m2, and m3 to obtain the target security level m+.

[0125] In this embodiment, the recognition ability of the model is determined based on the recognition results of each fusion model for each training sample in the corresponding training subset. The recognition ability of the model can be determined by determining the AUC of the model. The AUC (Area Under Curve) is defined as the area enclosed by the coordinate axis under the ROC curve (receiver operating characteristic curve). The features of the training samples in each training subset are ranked according to the proportion of the number of features and the ranking of the corresponding model, and the features in each training sample are scored. The higher the ranking of the proportion of the number of features, the higher the score of the feature. When the ranking of the proportion of the number of features is the same, the higher the corresponding model ranking, the higher the score of the feature. Finally, the total score of each feature in different training subsets is summed up, and the importance ranking of each feature is determined according to the total score of each feature.

[0126] In this embodiment, when performing fusion calculation on each initial security level, the number of features of the training subset used by the fusion model corresponding to each initial level in the training process can be obtained, and the number of features of the training subset corresponding to each fusion model can be weighted according to the proportion of the number of features and the total score corresponding to each feature to obtain the weight of each fusion model. According to the weight of each fusion model and the corresponding initial security level, the security level of the target to be identified is obtained by weighted calculation; the feature with the largest number of features can also be used as the target feature of the fusion model, and the ranking of the target features of each fusion model in the importance ranking of the above features can be combined to determine the ranking weight corresponding to each fusion model. According to the ranking weight and the corresponding initial security level, the security level of the target to be identified is obtained by weighted calculation; or, the total score of the target features of each fusion model can be directly used as the weight, and the total score and the corresponding initial security level can be weighted to obtain the security level of the target to be identified.

[0127] For example, training subset A includes 10 samples, and the total number of features of each sample includes: 20 feature 1s, 15 feature 2s, and 10 feature 3s; training subset B includes 10 samples, and the total number of features of each sample includes: 10 feature 1s, 15 feature 2s, 20 feature 3s, and 1 feature 4; training subset C includes 10 samples, and the total number of features of each sample includes: 15 feature 1s, 20 feature 2s, and 10 feature 3s; the AUC of the model corresponding to training subset A is the first, the AUC of the model corresponding to training subset B is the second, and the AUC of the model corresponding to training subset C is the third. At this time, the features of each training subset are analyzed separately. The scoring can be as follows: training subset A: feature 1: 0.5 points, feature 2: 0.3 points, feature 3: 0.2 points; training subset B: feature 3: 0.4 points, feature 2: 0.25 points, feature 1: 0.15 points, feature 4: 0.1 points; training subset C: feature 2: 0.3 points, feature 1: 0.2 points, feature 3: 0.1 points; the total score of each feature is, feature 1: 0.85 points, feature 2: 0.85 points; feature 3: 0.7 points, feature 4: 0.1 points. The above is only an example. In actual application, the order of magnitude of training subsets and the order of magnitude of features will be much larger, and this solution does not specifically limit this.

[0128] To compare the performance of models trained using the same dataset, this embodiment pre-divided the historical target data into a training set and a test set based on the chronological order of data generation. The training set was used to train each fusion model through steps S2 and S3, and the metrics of each trained model were evaluated using the test set. Table 1 shows the AUC, KS, and Top5-Lif evaluation results for each model. In Table 1, model M+ includes a second-layer model M that fuses the fusion models M1, M2, and M3, as well as the target initial security levels output by M1, M2, and M3.

[0129] XGBoost M1 M2 M3 M+ AUC 0.6612 0.6638 0.6548 0.6631 0.6680 KS 0.2312 0.2343 0.2243 0.2342 0.2386 Top5-Lift 2.12 2.15 2.01 2.14 2.23

[0130] Table 1: Evaluation results of different models on the same test set

[0131] As can be seen from Table 1, in the same test set, the M+ model trained by the present invention has better sorting ability, differentiation ability and local effect improvement ability, making the target security level more accurate, thereby strengthening the protection of data security in the Internet service platform, and effectively avoiding the situation in the related art where the security level of the target determined is not accurate enough, causing data leakage in the Internet service platform.

[0132] Figure 4 This invention is a target security level identification device based on a fusion model, such as Figure 4 As shown, the device includes:

[0133] A preprocessing module 41 is used to preprocess the historical target data to obtain a training set;

[0134] A division module 42 is used to divide the training set into multiple training subsets;

[0135] A training module 43 is used to train a corresponding fusion model for each training subset using multiple different models and fusion methods;

[0136] An input module 44 is used to input the target data to be identified into each trained fusion model to obtain multiple initial security levels of the target to be identified;

[0137] The fusion module 45 is configured to fuse and calculate the multiple initial security levels of the target to be identified to obtain the security level of the target to be identified.

[0138] In a specific embodiment, the training module 43 is used to train each training subset using models of different categories and / or models of the same category with different parameters and fusion methods; wherein the models and fusion methods used in each training subset divided into the same training set are different from each other.

[0139] Optionally, the training module 43 includes:

[0140] A first partitioning module is used to divide the training subset into a training sub-dataset and a validation sub-dataset;

[0141] The first training module is used to train multiple models of different categories using a training sub-dataset and a validation sub-dataset to obtain multiple groups of sub-fusion models;

[0142] The first fusion module is used to fuse multiple groups of sub-fusion models to obtain a corresponding fusion model;

[0143] And / or, the training module 43 includes:

[0144] A generating module, configured to generate a plurality of first models with different parameters;

[0145] A second training module is used to input the training subsets into multiple first models for training, to obtain multiple groups of sub-fusion models and output results of each sub-fusion model;

[0146] A second fusion module is used to perform mean fusion on multiple groups of sub-fusion models and corresponding output results to obtain a corresponding fusion model;

[0147] And / or, the training module 43 includes:

[0148] A third partitioning module is used to divide the training subset into N sub-data sets;

[0149] The third training module is used to input the N sub-data sets into the second model with N different parameters for training, thereby obtaining N groups of sub-fusion models and N output results output by each sub-fusion model;

[0150] The third fusion module is used to perform average fusion on the N groups of sub-fusion models and the corresponding N output results to obtain a corresponding fusion model.

[0151] Furthermore, the device further comprises:

[0152] A determination module is used to determine the correlation coefficients of each model;

[0153] The screening module is used to select models with correlation coefficients greater than a threshold as models of different categories and / or models of the same category with different parameters.

[0154] The division module 42 includes:

[0155] The balancing processing module is used to balance the training set;

[0156] The sampling module is used to extract samples from the balanced training set multiple times to form training subsets.

[0157] The pre-processing module 41 includes:

[0158] The cleaning module is used to clean the historical target data and obtain wide table variables;

[0159] The analysis and screening module is used to perform descriptive exploratory analysis on the wide table variables and screen out characteristic data based on the analysis results to obtain a training set.

[0160] Those skilled in the art will appreciate that the modules in the above device embodiments may be distributed in the device as described, or may be modified accordingly and distributed in one or more devices different from the above embodiments. The modules in the above embodiments may be combined into one module or further split into multiple submodules.

[0161] The following describes an electronic device embodiment of the present invention. This electronic device can be considered a physical implementation of the method and apparatus embodiments of the present invention described above. Details described in the electronic device embodiment of the present invention should be considered supplementary to the above-described method or apparatus embodiments; details not disclosed in the electronic device embodiment of the present invention can be implemented with reference to the above-described method or apparatus embodiments.

[0162] Figure 5 is a structural block diagram of an exemplary embodiment of an electronic device according to the present invention. Figure 5The electronic device shown is only an example and should not limit the functions and scope of use of the embodiments of the present invention.

[0163] like Figure 5 As shown, the electronic device 500 of this exemplary embodiment is represented as a general-purpose data processing target. Components of the electronic device 500 may include, but are not limited to, at least one processing unit 510, at least one storage unit 520, a bus 530 connecting different electronic device components (including the storage unit 520 and the processing unit 510), a display unit 540, and the like.

[0164] The storage unit 520 stores a computer-readable program, which may be a source program or a code of a read-only program. The program may be executed by the processing unit 510, so that the processing unit 510 performs the steps of various embodiments of the present invention. For example, the processing unit 510 may perform the following steps: Figure 1 Steps shown.

[0165] The storage unit 520 may include a readable medium in the form of a volatile storage unit, such as a random access memory unit (RAM) 5201 and / or a cache memory unit 5202, and may further include a read-only memory unit (ROM) 5203. The storage unit 520 may also include a program / utility 5204 having a set (at least one) of program modules 5205, such program modules 5205 including but not limited to: operating the electronic device, one or more application programs, other program modules, and program data, each of which or some combination thereof may include the implementation of a network environment.

[0166] Bus 530 may represent one or more of several types of bus structures, including a memory bus or memory controller, a peripheral bus, an accelerated graphics port, a processing unit, or a local bus using any of a variety of bus architectures.

[0167] The electronic device 500 may also communicate with one or more external devices 100 (e.g., keyboard, display, network device, Bluetooth device, etc.), so that a user can interact with the electronic device 500 via these external devices 100, and / or the electronic device 500 can communicate with one or more other data processing devices (e.g., router, modem, etc.). Such communication may be performed through an input / output (I / O) interface 550, or may be performed through a network adapter 560 with one or more networks (e.g., a local area network (LAN), a wide area network (WAN), and / or a public network, such as the Internet). The network adapter 560 may communicate with other modules of the electronic device 500 via the bus 530. It should be understood that although Figure 5Not shown, other hardware and / or software modules may be used in the electronic device 500, including but not limited to: microcode, device drivers, redundant processing units, external disk drive arrays, RAID electronic devices, tape drives, and data backup storage electronic devices.

[0168] Figure 6 Schematic diagram of a computer readable medium embodiment of the present invention. Figure 6 As shown, the computer program can be stored on one or more computer-readable media. The computer-readable medium can be a readable signal medium or a readable storage medium. The readable storage medium can be, for example, but not limited to, an electronic device, device or component of electricity, magnetism, light, electromagnetic, infrared, or semiconductor, or any combination thereof. More specific examples (non-exhaustive list) of readable storage media include: an electrical connection with one or more wires, a portable disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination thereof. When the computer program is executed by one or more data processing devices, the computer-readable medium is able to implement the above-mentioned method of the present invention, namely: pre-processing the historical target data to obtain a training set; dividing the training set into multiple groups of training subsets; for each training subset, using multiple different models and fusion methods to train the corresponding fusion model; inputting the target data to be identified into each trained fusion model respectively to obtain multiple initial security levels of the corresponding target to be identified; and fusing and calculating the multiple initial security levels of the target to be identified to obtain the security level of the target to be identified.

[0169] Through the description of the above embodiments, it is easy for those skilled in the art to understand that the exemplary embodiments described in the present invention can be implemented by software or by combining software with necessary hardware. Therefore, the technical solution according to the embodiment of the present invention can be embodied in the form of a software product, which can be stored in a computer-readable storage medium (which can be a CD-ROM, USB flash drive, mobile hard disk, etc.) or on a network, and includes several instructions to enable a data processing device (which can be a personal computer, server, or network device, etc.) to execute the above method according to the present invention.

[0170] The computer-readable storage medium may include a data signal propagated in baseband or as part of a carrier wave, wherein the readable program code is carried. Such propagated data signals may take a variety of forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination thereof. The readable storage medium may also be any readable medium other than a readable storage medium, which may send, propagate, or transmit a program for use by or in conjunction with an instruction-executing electronic device, apparatus, or device. The program code contained on the readable storage medium may be transmitted using any suitable medium, including but not limited to wireless, wired, optical cable, RF, etc., or any suitable combination thereof.

[0171] The program code for performing the operations of the present invention may be written in any combination of one or more programming languages, including object-oriented programming languages ​​such as Java, C++, and the like, as well as conventional procedural programming languages ​​such as "C" or similar programming languages. The program code may be executed entirely on the user computing device, partially on the user device, as a stand-alone software package, partially on the user computing device and partially on a remote computing device, or entirely on a remote computing device or server. In cases involving a remote computing device, the remote computing device may be connected to the user computing device via any type of network, including a local area network (LAN) or a wide area network (WAN), or may be connected to an external computing device (e.g., via the Internet using an Internet service provider).

[0172] In summary, the present invention can be implemented by a method, apparatus, electronic device or computer-readable medium that executes a computer program. In practice, a general-purpose data processing device such as a microprocessor or a digital signal processor (DSP) can be used to implement some or all of the functions of the present invention.

[0173] The specific embodiments described above further illustrate the objectives, technical solutions, and beneficial effects of the present invention. It should be understood that the present invention is not inherently related to any specific computer, virtual device, or electronic device, and various general-purpose devices can also implement the present invention. The above description is only a specific embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions, improvements, etc. made within the spirit and principles of the present invention shall be included in the scope of protection of the present invention.

Claims

1. A target security level identification method based on a fusion model, characterized in that: The method comprises: Preprocess the historical target data to obtain the training set; Dividing the training set into a plurality of training subsets; For each training subset, a corresponding fusion model is trained using multiple different models and fusion methods; wherein: the fusion method is used to fuse the different models of the training subset, and the fusion methods include: mean fusion, average fusion, model fusion, cross fusion, and feature fusion; different models and different fusion methods are arbitrarily combined to ensure that each training subset is trained using a different model and a different fusion method; thereby, the models and fusion methods used in each training subset divided into the same training set are different, and each fusion model after training can be compatible with the data features of different training subsets and the ranking prediction capabilities of different models based on different fusion methods; Input the target data to be identified into each trained fusion model to obtain multiple initial security levels of the corresponding target to be identified; Obtain the number of features of the training subset used in the training process of the fusion model corresponding to each initial security level, perform weighted calculation on the number of features of the training subset corresponding to each fusion model according to the proportion of the number of features and the total score corresponding to each feature, obtain the weight of each fusion model, and perform weighted calculation based on the weight of each fusion model and the corresponding initial security level to obtain the security level of the target to be identified; alternatively, obtain the number of features of the training subset used in the training process of the fusion model corresponding to each initial security level, use the feature with the largest number of features as the target feature of the fusion model, determine the ranking weight corresponding to each fusion model based on the ranking of the target features of each fusion model in the above-mentioned feature importance ranking, and perform weighted calculation based on the ranking weight and the corresponding initial security level to obtain the security level of the target to be identified; alternatively, obtain the number of features of the training subset used in the training process of the fusion model corresponding to each initial security level, use the feature with the largest number of features as the target feature of the fusion model, use the total score of the target features of each fusion model as the weight, and perform weighted calculation based on the total score and the corresponding initial security level to obtain the security level of the target to be identified.

2. The method according to claim 1, characterized in that For each training subset, multiple different models and fusion methods are used to train the corresponding fusion model, including: For each training subset, models of different categories and / or models of the same category with different parameters and fusion methods are used for training.

3. The method according to claim 1, characterized in that The use of multiple different models and fusion methods to train the corresponding fusion model includes: Dividing the training subset into a training sub-dataset and a validation sub-dataset; By training sub-datasets and validation sub-datasets, multiple models of different categories are trained to obtain multiple groups of sub-fusion models; Perform model fusion on multiple groups of sub-fusion models to obtain the corresponding fusion model; And / or, the adopting of multiple different models and fusion methods to train the corresponding fusion model includes: generating a plurality of first models with different parameters; Inputting the training subsets into a plurality of first models for training respectively, to obtain a plurality of groups of sub-fusion models and output results of each sub-fusion model; Performing mean fusion on multiple groups of sub-fusion models and corresponding output results to obtain a corresponding fusion model; And / or, the adopting of multiple different models and fusion methods to train the corresponding fusion model includes: Dividing the training subset into N sub-datasets; Input N sub-datasets into N second models with different parameters for training, and obtain N groups of sub-fusion models and N output results output by each sub-fusion model; The N groups of sub-fusion models and the corresponding N output results are averaged and fused to obtain a corresponding fusion model.

4. The method according to claim 2, characterized in that Before using a plurality of different models and fusion methods to train a corresponding fusion model, the method further includes: Determine the correlation coefficients for each model; Models with correlation coefficients greater than a threshold are considered as models of different categories and / or models of the same category with different parameters.

5. The method according to claim 1, wherein The dividing the training set into a plurality of training subsets comprises: Balance the training set; Samples are extracted multiple times from the balanced training set to form training subsets.

6. The method according to claim 1, characterized in that The historical target data is preprocessed to obtain a training set including: Perform data cleaning on historical target data to obtain wide table variables; A descriptive exploratory analysis is performed on the wide table variables, and characteristic data is screened out according to the analysis results to obtain a training set.

7. A target security level identification device based on a fusion model, characterized in that: The device comprises: The preprocessing module is used to preprocess the historical target data to obtain the training set; A partitioning module, configured to divide the training set into a plurality of training subsets; A training module is used to train a corresponding fusion model for each training subset using multiple different models and fusion methods; wherein: the fusion method is used to fuse the different models of the training subset, and the fusion methods include: mean fusion, average fusion, model fusion, cross fusion, and feature fusion; different models and different fusion methods are arbitrarily combined to ensure that each training subset is trained using a different model and a different fusion method; thereby, the models and fusion methods used in each training subset divided into the same training set are different, and each fusion model after training can be compatible with the data features of different training subsets and the ranking prediction capabilities of different models based on different fusion methods; An input module, configured to input the target data to be identified into each trained fusion model to obtain multiple initial security levels of the target to be identified; The fusion module is used to obtain the number of features of the training subset used in the training process of the fusion model corresponding to each initial security level, perform weighted calculation on the number of features of the training subset corresponding to each fusion model according to the proportion of the number of features and the total score corresponding to each feature, obtain the weight of each fusion model, and obtain the security level of the target to be identified by weighted calculation based on the weight of each fusion model and the corresponding initial security level; or, obtain the number of features of the training subset used in the training process of the fusion model corresponding to each initial security level, use the feature with the largest number of features as the target feature of the fusion model, determine the ranking weight corresponding to each fusion model based on the ranking of the target features of each fusion model in the above-mentioned feature importance ranking, and obtain the security level of the target to be identified by weighted calculation based on the ranking weight and the corresponding initial security level; or, obtain the number of features of the training subset used in the training process of the fusion model corresponding to each initial security level, use the feature with the largest number of features as the target feature of the fusion model, use the total score of the target features of each fusion model as the weight, and obtain the security level of the target to be identified by weighted calculation based on the total score and the corresponding initial security level.

8. The device according to claim 7, characterized in that The training module is used to perform training on each training subset using models of different categories and / or models of the same category with different parameters and a fusion method.

9. The device according to claim 7, characterized in that The training module includes: A first partitioning module is used to divide the training subset into a training sub-dataset and a validation sub-dataset; The first training module is used to train multiple models of different categories using a training sub-dataset and a validation sub-dataset to obtain multiple groups of sub-fusion models; The first fusion module is used to fuse multiple groups of sub-fusion models to obtain a corresponding fusion model; And / or, the training module includes: A generating module, configured to generate a plurality of first models with different parameters; A second training module is used to input the training subsets into multiple first models for training, to obtain multiple groups of sub-fusion models and output results of each sub-fusion model; A second fusion module is used to perform mean fusion on multiple groups of sub-fusion models and corresponding output results to obtain a corresponding fusion model; And / or, the training module includes: A third division module is used to divide the training subset into N sub-data sets; The third training module is used to input the N sub-data sets into the second model with N different parameters for training, thereby obtaining N groups of sub-fusion models and N output results output by each sub-fusion model; The third fusion module is used to perform average fusion on the N groups of sub-fusion models and the corresponding N output results to obtain a corresponding fusion model.

10. The device according to claim 8, characterized in that The device further comprises: A determination module is used to determine the correlation coefficients of each model; The screening module is used to select models with correlation coefficients greater than a threshold as models of different categories and / or models of the same category with different parameters.

11. The device according to claim 7, characterized in that The division module includes: The balancing processing module is used to balance the training set; The sampling module is used to extract samples from the balanced training set multiple times to form training subsets.

12. The device according to claim 7, characterized in that The pre-processing module comprises: The cleaning module is used to clean the historical target data and obtain wide table variables; The analysis and screening module is used to perform descriptive exploratory analysis on the wide table variables and screen out characteristic data based on the analysis results to obtain a training set.

13. An electronic device comprising: processor; as well as A memory storing computer executable instructions which, when executed, cause the processor to perform the method according to any one of claims 1 to 6.

14. A computer-readable storage medium, wherein: The computer-readable storage medium stores one or more programs, and when the one or more programs are executed by a processor, the method according to any one of claims 1 to 6 is implemented.

Images