Number card detection model training method and device, equipment, medium and product

Abstract

The embodiment of the invention provides a number card detection model training method and device, equipment, a medium and a product, and is applied to the technical field of network security. The method comprises the steps of training an initial model based on a first data set in a sample number card data set to obtain a to-be-determined model; calculating a recall value of the to-be-determined model based on a second data set in the sample number card data set; when the recall value is smaller than a preset recall threshold value, updating the to-be-determined model based on a preset parameter updating rule until the recall value of the to-be-determined model is larger than or equal to the preset recall threshold value; when the recall value is larger than or equal to the preset recall threshold value, the to-be-determined model is determined as the target number card detection model, and the book effect of improving the number card detection recall sorting consistency is achieved.

Description

Technical Field

[0001] This application relates to the field of network security technology, and in particular, to a training method, device, equipment, medium and product of a card detection model. Background Art

[0002] An Internet of Things card is a Subscriber Identity Module (SIM) card used to connect intelligent devices. With the diversification of intelligent devices, Internet of Things cards have been effectively applied in multiple fields. Since the usage situation of Internet of Things cards cannot be accurately detected in real time, some Internet of Things cards are used illegally or for illegal activities, resulting in risks in the use of Internet of Things cards.

[0003] In the prior art, the processing method for risk cards in Internet of Things cards is as follows: collecting network data or historical processing data for model training to obtain a card detection model; inputting the card data to be detected into the card detection model to obtain a risk determination result corresponding to the card data to be detected, so as to realize the risk detection of Internet of Things cards.

[0004] Since the card detection method in the prior art only performs simple feature comparison for a single classification target, there is a situation where the detection results of the model are unevenly distributed, resulting in the technical problem of low consistency of card detection recall ranking in the prior art. Summary of the Invention

[0005] Embodiments of this application provide a training method, device, equipment, medium and product of a card detection model, so as to achieve the technical effect of improving the consistency of card detection recall ranking.

[0006] In a first aspect, an embodiment of this application provides a training method of a card detection model, including:

[0007] Training an initial model based on a first data set in a sample card data set to obtain a model to be determined;

[0008] Calculating a recall value of the model to be determined based on a second data set in the sample card data set;

[0009] When the recall value is less than a preset recall threshold, updating the model to be determined based on a preset parameter update rule until the recall value of the model to be determined is greater than or equal to the preset recall threshold;

[0010] When the recall value is greater than or equal to the preset recall threshold, determining the model to be determined as a target card detection model;

[0011] Wherein, the recall value is used to indicate the level of recall ranking consistency of the model to be determined on the second data set.

[0012] In a possible implementation, updating the to-be-determined model based on the preset parameter update rule includes:

[0013] Updating the model parameters in the initial model based on the preset parameter update rule to obtain the initial model after parameter update;

[0014] Training the initial model after parameter update based on the first data set to obtain the to-be-determined model after update.

[0015] In a possible implementation, calculating the recall value of the to-be-determined model based on the second data set in the sample number card data set includes:

[0016] Inputting the second data set into the to-be-determined model to obtain the prediction result corresponding to each sample number card data in the second data set, where the to-be-determined model includes multiple classification tasks, and the prediction result includes the predicted class label of the sample number card data in multiple classification tasks and the predicted probability value corresponding to each predicted class label;

[0017] Generating a third data set based on the prediction result of each sample number card data and the sample number card data;

[0018] Calculating the similarity between the second data set and the third data set to obtain the recall value of the to-be-determined model.

[0019] In a possible implementation, generating a third data set based on the prediction result of each sample number card data and the sample number card data includes:

[0020] For each sample number card data, obtaining the predicted class label and the predicted probability value of the sample number card data in multiple classification tasks based on the prediction result of the sample number card data;

[0021] Calculating the predicted probability mean value corresponding to the sample number card data based on the predicted probability values of the sample number card data in each classification task;

[0022] Generating a to-be-determined data set based on each sample number card data and its corresponding multiple predicted class labels;

[0023] Sorting the multiple sample number card data in the to-be-determined data set in ascending order or descending order based on the predicted probability mean value to obtain the third data set.

[0024] In a possible implementation, calculating the similarity between the second data set and the third data set to obtain the recall value of the to-be-determined model includes:

[0025] Obtain the first N sample number card data of the second data set and their corresponding actual results, and determine them as the actual set; wherein, the actual results include the actual class identifiers of the sample number card data for each classification task.

[0026] Obtain the first N sample number card data of the third data set and their corresponding multiple predicted class identifiers, and determine them as the prediction set.

[0027] Based on the intersection of the actual set and the prediction set, calculate the similarity between the actual set and the prediction set, and determine it as the recall value of the model to be determined.

[0028] Wherein, N is a positive integer greater than 1.

[0029] In a possible implementation manner, before training the initial model based on the first data set in the sample number card data set to obtain the model to be determined, the method further includes:

[0030] Extract features from the historical sample number card data to obtain multiple initial data features.

[0031] Based on the multiple initial data features, perform feature combination to obtain multiple initial combined features.

[0032] Perform feature screening on the multiple initial combined features to obtain multiple target combined features.

[0033] Based on the multiple target combined features, perform data screening and marking on the historical sample number card data to obtain the sample number card data set.

[0034] Divide the sample number card data set into a first data set and a second data set according to a preset ratio.

[0035] Wherein, the multiple sample number card data in the first data set and the second data set are evenly distributed on multiple combined features.

[0036] In a second aspect, an embodiment of the present application provides a training device for a number card detection model, including:

[0037] A first processing module, configured to train an initial model based on the first data set in the sample number card data set to obtain a model to be determined.

[0038] A second processing module, configured to calculate the recall value of the model to be determined based on the second data set in the sample number card data set.

[0039] A third processing module, configured to update the model to be determined based on a preset parameter update rule until the recall value of the model to be determined is greater than or equal to the preset recall threshold when the recall value is less than the preset recall threshold;

[0040] A fourth processing module, configured to determine the model to be determined as the target SIM card detection model when the recall value is greater than or equal to the preset recall threshold;

[0041] Wherein, the recall value is used to indicate the consistency between the prediction result of the model to be determined and the actual result of the second data set.

[0042] In a possible implementation manner, the third processing module is further configured to:

[0043] Update the model parameters in the initial model based on a preset parameter update rule to obtain an initial model after parameter update;

[0044] Train the initial model after parameter update based on the first data set to obtain an updated model to be determined.

[0045] In a possible implementation manner, the second processing module is further configured to:

[0046] Input the second data set into the model to be determined to obtain the prediction result corresponding to each sample SIM card data in the second data set, wherein the model to be determined includes multiple classification tasks, and the prediction result includes the predicted class label of the sample SIM card data in multiple classification tasks and the predicted probability value corresponding to each predicted class label;

[0047] Generate a third data set based on the prediction result of each sample SIM card data and the sample SIM card data;

[0048] Calculate the similarity between the second data set and the third data set to obtain the recall value of the model to be determined.

[0049] In a possible implementation manner, the second processing module is further configured to:

[0050] For each sample SIM card data, obtain the predicted class label and predicted probability value of the sample SIM card data in multiple classification tasks based on the prediction result of the sample SIM card data;

[0051] Calculate the predicted probability mean value corresponding to the sample SIM card data based on the predicted probability values of the sample SIM card data in each classification task;

[0052] Generate a data set to be determined based on each sample SIM card data and its corresponding multiple predicted class labels;

[0053] Sort the multiple sample number card data in the to-be-determined data set in ascending or descending order based on the mean of the predicted probabilities to obtain the third data set.

[0054] In a possible implementation manner, the second processing module is further configured to:

[0055] Obtain the first N sample number card data in the second data set and their corresponding actual results, and determine them as the actual set; wherein, the actual results include the actual class identifiers of the sample number card data for each classification task.

[0056] Obtain the first N sample number card data in the third data set and their corresponding multiple predicted class identifiers, and determine them as the predicted set.

[0057] Calculate the similarity between the actual set and the predicted set based on the intersection of the actual set and the predicted set, and determine it as the recall value of the to-be-determined model.

[0058] Wherein, N is a positive integer greater than 1.

[0059] In a possible implementation manner, the device further includes an acquisition module, configured to:

[0060] Extract features from historical sample number card data to obtain multiple initial data features.

[0061] Perform feature combination based on multiple initial data features to obtain multiple initial combined features.

[0062] Perform feature screening on multiple initial combined features to obtain multiple target combined features.

[0063] Perform data screening and marking on the historical sample number card data based on multiple target combined features to obtain the sample number card data set.

[0064] Divide the sample number card data set into a first data set and a second data set according to a preset ratio.

[0065] Wherein, the multiple sample number card data in the first data set and the second data set are evenly distributed on multiple combined features.

[0066] In a third aspect, an embodiment of the present application provides an electronic device, including: a memory, a processor;

[0067] The memory stores computer execution instructions.

[0068] The processor executes the computer execution instructions stored in the memory, so that the processor executes the above first aspect and various possible implementation manners in the first aspect.

[0069] Fourthly, an embodiment of the present application provides a computer-readable storage medium, in which computer-executable instructions are stored. When the computer-executable instructions are executed by a processor, they are used to implement the first aspect and various possible implementation manners of the first aspect as described above.

[0070] Fifthly, an embodiment of the present application provides a computer program product, including a computer program. When the computer program is executed by a processor, it implements the first aspect and various possible implementation manners of the first aspect as described above.

[0071] The training method, device, equipment, medium and product of the SIM card detection model provided by the embodiments of the present application. In the training method of the SIM card detection model, an initial model is trained using a first data set in a sample SIM card data set to obtain a to-be-determined model; a recall value of the to-be-determined model is calculated using a second data set in the sample SIM card data set; it is determined whether the recall value is greater than or equal to a preset recall threshold. If so, the to-be-determined model can be directly determined as the target SIM card detection model. If not, the to-be-determined model is updated based on a preset parameter update rule until the recall value of the to-be-determined model is greater than or equal to the preset recall threshold. The recall value is used to represent the level of recall ranking consistency of the to-be-determined model for the second data set, so as to continuously optimize the to-be-determined model through the preset recall threshold and continuously improve the recall ranking consistency of the to-be-determined model; compared with the prior art, the present application uses a data set to verify the recall ranking consistency of the to-be-determined model and continuously optimizes the recall value of the to-be-determined model using the preset parameter update rule, thereby achieving the technical effect of improving the SIM card recall ranking consistency. BRIEF DESCRIPTION OF THE DRAWINGS

[0072] The accompanying drawings herein are incorporated into the specification and constitute a part of this specification, showing embodiments consistent with the present application and used together with the specification to explain the principles of the present application.

[0073] Figure 1 Schematic flowchart of the training method of the SIM card detection model provided by the present application Figure 1 ;

[0074] Figure 2 Schematic flowchart of the training method of the SIM card detection model provided by the present application Figure 2 ;

[0075] Figure 3 Schematic structural diagram of the SIM card detection system provided by the present application;

[0076] Figure 4 Schematic single-task structural diagram of an initial model provided by an embodiment of the present application;

[0077] Figure 5 Schematic structural diagram of the training device of the SIM card detection model provided by the present application;

[0078] Figure 6 Schematic diagram of the structure of the electronic device provided for this application.

[0079] Through the above-mentioned drawings, specific embodiments of this application have been shown, and there will be more detailed descriptions hereinafter. These drawings and textual descriptions are not intended to limit the scope of the concept of this application in any way, but to illustrate the concept of this application to those skilled in the art by referring to specific embodiments. Detailed implementation manners

[0080] Here, the exemplary embodiments will be described in detail, and the examples are shown in the drawings. When the following description refers to the drawings, unless otherwise indicated, the same numbers in different drawings represent the same or similar elements. The implementation manners described in the following exemplary embodiments do not represent all implementation manners consistent with this application. On the contrary, they are merely examples of devices and methods consistent with some aspects of this application as detailed in the appended claims.

[0081] First, the nouns involved in this application are explained:

[0082] Multi-gate Mixture of Experts (MMOE) model: A model architecture for multi-task learning (MTL). Its core idea is to achieve information sharing between multiple tasks and task-specific feature extraction by sharing expert networks and task-specific gates.

[0083] Expert networks: Refers to multiple independent neural networks in the MMOE model, which are used to extract shared feature representations.

[0084] Gates: In the MMOE model, each task has an independent gate network, which is used to generate the weights of the expert networks and control the contributions of different expert networks to the tasks.

[0085] Weight Sharing Network (WSN): A mechanism for achieving information sharing between multiple tasks or modules by sharing weights. In multi-task learning, WSN can be used to dynamically adjust the weight allocation of different tasks and improve the generalization ability and performance of the model.

[0086] Task Tower: It is a neural network structure for specific tasks in the MMOE architecture. Each task tower is designed specifically for a certain task and is responsible for processing data and features related to that task. By using multiple task towers, MMOE can effectively share underlying features while maintaining the independence between different tasks.

[0087] In the prior art, the main method for sim card detection in the Internet of Things is as follows: extracting the features of the current sim card, inputting the sim card features into the corresponding sim card detection model to obtain the detection result for the current sim card, so as to realize the risk determination of different sim cards in the Internet of Things.

[0088] However, since the sim card detection in the prior art obtains the detection result through simple feature comparison, when detecting a large amount of sim card data, it is easy to have a situation where a large number of sim card detection results are concentrated in a specific area, specific type or specific user, resulting in the technical problem of poor recall ranking consistency in sim card recall.

[0089] In view of the above technical problems, the present application proposes the following technical concept: considering the recall ranking consistency of the output result of the sim card detection model during the training of the sim card detection model, and continuously adjusting the sim card detection model to ensure the recall ranking consistency of the sim card detection model. Specifically: using the first data set to train the initial model to obtain a model to be determined; calculating the recall value of the model to be determined using the second data set, and determining the high or low recall ranking consistency of the model to be determined by comparing the recall value with a preset recall threshold; when the recall ranking consistency of the model to be determined is low, updating the model to be determined using a preset parameter update rule to achieve the purpose of increasing the recall value; when the recall value corresponding to the model to be determined indicates that the recall ranking consistency of the current model to be determined is high, determining the model to be determined as the target sim card detection model, thereby achieving the technical effect of improving the recall ranking consistency of sim card detection.

[0090] The following uses specific embodiments to elaborate in detail on the technical solution of the present application and how the technical solution of the present application solves the above technical problems. These specific embodiments below can be combined with each other, and the same or similar concepts or processes may not be repeated in some embodiments. The following will describe the embodiments of the present application in conjunction with the accompanying drawings.

[0091] Figure 1 Flow schematic of the training method for the sim card detection model provided by the present application Figure 1 As Figure 1 shown, the method includes:

[0092] S101. Based on the first data set in the sample sim card data set, train the initial model to obtain a model to be determined.

[0093] In this step, the initial model is a multi-task learning model, which includes multiple classification tasks.

[0094] Exemplarily, the multiple classification tasks in the initial model are respectively: predicting the area corresponding to the sample ID card data, predicting the user corresponding to the sample ID card data, and predicting the risk type corresponding to the sample ID card data.

[0095] Optionally, before training the model based on the first data set in the sample ID card data set, it is also necessary to obtain the sample ID card data set. A possible implementation manner for obtaining the sample ID card data set is as follows:

[0096] S1011. Extract features from the historical sample ID card data to obtain multiple initial data features.

[0097] In this step, the extraction method of the initial data features can be: collecting the data features that appear in the historical sample ID card data, calculating the frequency according to the occurrence frequency corresponding to each data feature, and taking the data features with high frequency occurrences as the initial data features.

[0098] It should be noted that in the embodiments of the present application, the feature extraction method of the historical sample ID card data features is not limited.

[0099] S1012. Perform feature combination based on the multiple initial data features to obtain multiple initial combined features.

[0100] In this step, the feature combination method can be: using an automated combination or a manual combination method to combine the multiple initial data features to generate multiple initial combined features.

[0101] Exemplarily, the initial data features are respectively divided into three categories: ID card attributes, user information, and ID card risk processing information; the multiple initial data features corresponding to each of these three categories are combined to obtain multiple initial combined features.

[0102] S1013. Perform feature screening on the multiple initial combined features to obtain multiple target combined features.

[0103] In this step, it is necessary to perform feature screening on the obtained multiple initial combined features, select the initial combined features with high correlation with the variables in the first data set as the target combined features, and at the same time delete the duplicate initial combined features.

[0104] Exemplarily, a possible implementation manner for feature screening is as follows:

[0105] a1. Select target variables from the first data set.

[0106] a2. Calculate the correlation coefficient between each initial combined feature and the target variable.

[0107] a3. Obtain the absolute value of the correlation coefficient, and measure the importance of the initial combined feature to the target variable according to the size of the absolute value.

[0108] a4. Sort multiple initial combined features in descending order according to the absolute value, and screen out the initial combined features with high correlation with the target variable as the to-be-determined combined features.

[0109] a5. Calculate the correlation coefficient matrix corresponding to multiple to-be-determined combined features.

[0110] a6. Traverse each pair of to-be-determined combined features in the matrix. If the absolute value of the correlation coefficient exceeds the threshold, delete the to-be-determined combined features with low correlation with the target variable, so as to remove redundancy; use the undeleted to-be-determined combined features as the target combined features.

[0111] S1014. Based on multiple target combined features, perform data screening and marking on historical sample number card data to obtain a sample number card data set.

[0112] In this step, a possible implementation method for data screening is:

[0113] b1. Determine the target variable, and use the target variable as the mark corresponding to each sample number card data.

[0114] In this step, the target variable can contain multiple pieces of information. When using a multi-task learning model to detect sample number card data, the target variable can contain the prediction target corresponding to each classification task.

[0115] Exemplarily, there are three classification tasks, namely: risk type, number card area, and number card customer. Then the target variable needs to contain the risk type, number card area, and number card customer corresponding to each sample number card data.

[0116] b2. Use multiple target combined features to perform data screening on historical sample number card data.

[0117] The screening method can be: screen the data set according to the preset screening rules to ensure that the screened data is evenly distributed on each type of target variable.

[0118] Exemplarily, when the target variable is the number card area, there are three number card areas; when performing data screening, it is necessary to ensure that the screened data is evenly distributed in terms of quantity on the three number card areas.

[0119] b3. In the screened data set, add marks to each piece of data, including customer, risk type, and area.

[0120] S1015. Divide the sample number card dataset into a first dataset and a second dataset according to a preset ratio.

[0121] In this step, the multiple sample number card data in the first dataset and the second dataset are evenly distributed on multiple combined features.

[0122] Exemplarily, the way to divide the dataset according to the preset ratio can be:

[0123] Divide the dataset by stratified sampling. When the preset ratio is 4:1, select the target variable as the stratification basis, and use stratified sampling to ensure that the two divided datasets maintain a uniform distribution in terms of data distribution.

[0124] S102. Calculate the recall value of the model to be determined based on the second dataset in the sample number card dataset.

[0125] In this step, the recall value is used to indicate the level of recall ranking consistency of the model to be determined on the second dataset.

[0126] Among them, the calculation method of the recall value is to obtain the prediction result of the model to be determined for the second dataset, compare the prediction result with the actual result corresponding to the second dataset, calculate the similarity between the prediction result and the actual result, and determine the recall value based on the similarity.

[0127] It should be noted that the specific calculation method of the recall value in this step will be further explained in the following embodiments, and no redundant elaboration will be made here.

[0128] S103. When the recall value is less than the preset recall threshold, update the model to be determined based on the preset parameter update rule until the recall value of the model to be determined is greater than or equal to the preset recall threshold.

[0129] Optionally, a possible implementation way to update the model to be determined based on the preset parameter update rule is:

[0130] S1031. Update the model parameters in the initial model based on the preset parameter update rule to obtain the initial model after parameter update.

[0131] In this step, the purpose of updating the model parameters is to adjust the output of the initial model for each category corresponding to each classification task to be balanced through the way of updating the model parameters; the recall consistency indicates that among the multiple classification tasks of the initial model, there are differences in the recall rates of multiple categories in each classification task, resulting in an unbalanced distribution of the final output results of the model among multiple categories; therefore, it is necessary to adjust the parameters for each category of each classification task. The specific method can be:

[0132] d1. For each classification task, determine multiple class identifiers corresponding to the classification task.

[0133] d2. Set a preset weight for each class identifier.

[0134] d3. Adjust the preset weight corresponding to each class identifier to update the model parameters.

[0135] S1032. Train the initial model after parameter update based on the first data set to obtain the to-be-determined model after update.

[0136] In this step, the initial model after parameter update needs to be retrained based on the first data set. The way of model training can be multi-round iterative training. After model training, the to-be-determined model corresponding to the initial model is obtained.

[0137] S104. When the recall value is greater than or equal to the preset recall threshold, determine the to-be-determined model as the target SIM card detection model.

[0138] In this step, the way to determine the to-be-determined model as the target SIM card detection model can be: save the to-be-determined model in a preset format and store it.

[0139] The training method of the SIM card detection model provided in the embodiments of the present application uses the first data set in the sample SIM card data set to train the initial model to obtain the to-be-determined model; uses the second data set in the sample SIM card data set to calculate the recall value of the to-be-determined model; determines whether the recall value is greater than or equal to the preset recall threshold. If so, the to-be-determined model can be directly determined as the target SIM card detection model. If not, update the to-be-determined model based on the preset parameter update rule until the recall value of the to-be-determined model is greater than or equal to the preset recall threshold. The recall value is used to represent the level of recall ranking consistency of the to-be-determined model for the second data set, so as to continuously optimize the to-be-determined model through the preset recall threshold and continuously improve the recall ranking consistency of the to-be-determined model. Compared with the prior art, the present application uses the data set to verify the recall ranking consistency of the to-be-determined model and uses the preset parameter table update rule to continuously optimize the recall value of the to-be-determined model, thereby achieving the technical effect of improving the SIM card recall ranking consistency.

[0140] Figure 2 is the process schematic of the training method of the SIM card detection model provided by the present application Figure 2 , on the basis of the above Figure 1 shown embodiments, this embodiment further explains the calculation of the recall value in step S102. As Figure 2 shown, the method includes:

[0141] S201. Input the second dataset into the model to be determined, and obtain the prediction results corresponding to each sample ID card data in the second dataset.

[0142] In this step, the model to be determined includes multiple classification tasks. The prediction results include the predicted class labels of the sample ID card data for multiple classification tasks, and the predicted probability values corresponding to each predicted class label.

[0143] Exemplarily, the multiple classification tasks are respectively: predicting the area corresponding to the sample ID card data, predicting the user corresponding to the sample ID card data, and predicting the risk type corresponding to the sample ID card data.

[0144] Then the prediction results include: the predicted area label, the predicted probability value corresponding to the predicted area label; the predicted user label, and the predicted probability value corresponding to the predicted user label; the predicted risk type label, and the predicted probability value corresponding to the predicted risk type label.

[0145] For example: the prediction results include: Area A, probability value 0.92; User 00001, probability value 0.99; Risk type one, probability value 0.94. They are respectively the predicted class labels and predicted probability values for each classification task.

[0146] S202. Generate a third dataset based on the prediction results of each sample ID card data and the sample ID card data.

[0147] Optionally, a possible implementation of generating the third dataset is:

[0148] S2021. For each sample ID card data, based on the prediction results of the sample ID card data, obtain the predicted class labels and predicted probability values of the sample ID card data for multiple classification tasks.

[0149] In this step, the predicted class label refers to the prediction result of the sample ID card data for each classification task.

[0150] Exemplarily, there is a classification task: predicting the area corresponding to the sample ID card data. This classification task will take the area with the highest predicted probability value as the prediction result, and simultaneously output the label of this area and the predicted probability value corresponding to this area.

[0151] S2022. Calculate the predicted probability mean value corresponding to the sample ID card data based on the predicted probability values of the sample ID card data for each classification task.

[0152] Exemplarily, the predicted probability values of the sample number card data on three classification tasks and the corresponding predicted class identifiers are as follows: the predicted class identifier for the first classification task is Region B, and the predicted probability value is 0.93; the predicted class identifier for the second classification task is User 000002, and the predicted probability value is 0.92; the predicted class identifier for the third classification task is Risk Type Two, and the predicted probability value is 0.91; the mean value of the three predicted probability values is calculated, and the obtained predicted probability mean is 0.92.

[0153] S2023. Generate a dataset to be determined based on each sample number card data and its corresponding multiple predicted class identifiers.

[0154] In this step, the dataset to be determined contains the sample number card data and the multiple predicted class identifiers corresponding to each sample number card data.

[0155] S2024. Sort the multiple sample number card data in the dataset to be determined in ascending order or descending order based on the predicted probability mean to obtain a third dataset.

[0156] In this step, use the predicted probability mean corresponding to each sample number card data to sort the multiple sample number card data in the dataset to be determined, and the obtained third dataset is a dataset in descending order or ascending order.

[0157] Among them, the sample number card data in the third dataset is the same as the sample number card data in the second dataset, and the multiple predicted class identifiers corresponding to each sample number card data in the third dataset are equivalent to the multiple actual class identifiers corresponding to each sample number card data in the second dataset. The multiple actual class identifiers and the multiple predicted class identifiers both refer to the classification marks of each sample number card data.

[0158] S203. Calculate the similarity between the second dataset and the third dataset to obtain the recall value of the model to be determined.

[0159] Optionally, a possible implementation method for obtaining the recall value through similarity calculation is as follows:

[0160] S2031. Obtain the first N sample number card data in the second dataset and their corresponding actual results, and determine them as the actual set.

[0161] In this step, the actual results include the actual class identifiers of the sample number card data for each classification task.

[0162] S2032. Obtain the first N sample number card data in the third dataset and their corresponding multiple predicted class identifiers, and determine them as the predicted set.

[0163] S2033. Calculate the similarity between the actual set and the predicted set based on the intersection of the actual set and the predicted set, and determine it as the recall value of the model to be determined.

[0164] It should be noted that N is a positive integer greater than 1.

[0165] In this step, the calculation of the recall value is shown in Formula 1:

[0166]

[0167] Among them, recall@K refers to the recall value, and Topk pre -call refers to the predicted set;

[0168] Topk call refers to the actual set.

[0169] In this embodiment, the second data set is used to calculate the recall value of the model to be determined, so as to detect the recall consistency of the model to be determined.

[0170] Based on the above embodiments, the embodiment of the present application provides a SIM card detection system based on a multi-task learning model. Figure 3 It is a schematic structural diagram of the SIM card detection system provided by the present application. As Figure 3 shown, the system includes:

[0171] A data processing module 301, configured to collect historical risk SIM card data, extract behavior data, user data, and risk processing data corresponding to each SIM card in the historical risk SIM card data, and perform data cleaning and processing to obtain historical sample SIM card data.

[0172] A feature selection module 302, configured to extract data features from the historical sample SIM card data to obtain a plurality of initial data features; combine the plurality of initial data features to obtain a plurality of initial combined features; and perform feature screening on the plurality of initial combined authentications to obtain a plurality of target combined features.

[0173] A model design module 303, configured to use the target combined authentication to perform data screening and marking on the historical sample SIM card data to obtain a sample SIM card data set; and perform model design based on the MMOE model to obtain an initial model.

[0174] Figure 4 It is a schematic single-task structure diagram of the initial model provided by the embodiment of the present application. As Figure 4 shown, this embodiment provides an initial model, and the single learning task structure of this model includes:

[0175] A weight sharing network 401, an expert network 402, a task tower 403, an input layer 404, and an output layer 405.

[0176] Among them, data is input into the expert network through the input layer, and the feature vectors extracted by each expert network in the initial model for this data are obtained; the feature vectors in multiple expert networks are input into the weight-sharing network layer, and weight learning is performed using the weight-sharing network. The feature vectors after weight learning are input into the multi-layer task tower for multi-layer output calculation, and the calculated prediction results are output through the output layer for prediction result output.

[0177] The model training module 304 trains the initial model based on the first dataset in the sample number card dataset to obtain a to-be-determined model, calculates the recall value of the to-be-determined model based on the second dataset in the sample number card dataset, and when the recall value is greater than or equal to the preset recall value, determines the to-be-determined model as the target number card detection model. Among them, the calculation method of the recall value is to input the second dataset into the to-be-determined model to obtain the prediction result of the to-be-determined model, sort the multiple sample number card data in the second dataset according to the prediction result, and generate a third dataset; take out multiple sample number card data with the same quantity from the third dataset and the second dataset to form two sets, calculate the similarity between the two sets, and determine the similarity value as the recall value of the to-be-determined model.

[0178] Among them, the initial model training is implemented based on Figure 4 the model structure shown, and the calculation formula based on Figure 4 the structure is shown in Formula 2:

[0179]

[0180] Among them, L refers to the processing status value corresponding to each number card data, and this status value includes: 0, 1, 2; where 0 represents exposure, 1 represents click, and 2 represents disposal. W-mmoe refers to the target number card detection model; F refers to the data feature corresponding to each number card data; softmax refers to the normalization activation function; \(\hat{y}_{dvr}\) refers to the predicted processing conversion rate for each number card data.

[0181] The loss function corresponding to the target number card detection model is shown in Formula 3:

[0182]

[0183] Among them, L dvr is the loss value, which is used to continuously optimize the model; \(\hat{y}_{dvr}\) refers to the predicted processing conversion rate for each number card data; y dvr refers to the actual processing conversion rate for each number card data.

[0184] It should be noted that, compared with the prior art, the W-mmoe model in this embodiment refers to adding a weight-sharing network on the basis of the MMOE model to learn the weight changes of multiple classification tasks, perform weight fusion and update, and send the reallocated weights to each classification task to achieve the purpose of optimizing the model.

[0185] Exemplarily, this embodiment provides a method for comparing the recall values of the W-mmoe model and the MMOE model. By calculating the recall values of the two models under different data scales, the improvement change of the recall value of the W-mmoe model compared with the MMOE model is calculated.

[0186] When the second dataset and the third dataset are determined, the numbers of the prediction set and the actual set taken from the second dataset and the third dataset are divided into four levels, which are 100, 300, 500, and 1000 respectively. Under the numbers of the four levels, the recall values of the MMOE model are 0.6534, 0.6632, 0.7125, and 0.7511 respectively; the recall values of the W-mmoe model are 0.6756, 0.6839, 0.7436, and 0.7784 respectively. Then the average value of the recall value improvement ratio of the W-mmoe model calculated is 5.6%.

[0187] The card number detection module 305 processes the card number data in the current time period based on the target card number detection model and outputs the prediction result corresponding to each card number data. The prediction result includes the area corresponding to the card number data, the risk type, and the user to whom the risk card number belongs.

[0188] Figure 5 It is a schematic structural diagram of the training device for the card number detection model provided by this application. As Figure 6 shown, the training device for the card number detection model provided in this embodiment includes:

[0189] The first processing module 501 is used to train the initial model based on the first dataset in the sample card number dataset to obtain a model to be determined;

[0190] The second processing module 502 is used to calculate the recall value of the model to be determined based on the second dataset in the sample card number dataset;

[0191] The third processing module 503 is used to update the model to be determined based on the preset parameter update rule when the recall value is less than the preset recall threshold until the recall value of the model to be determined is greater than or equal to the preset recall threshold;

[0192] The fourth processing module 504 is used to determine the model to be determined as the target card number detection model when the recall value is greater than or equal to the preset recall threshold;

[0193] Among them, the recall value is used to indicate the consistency between the prediction result of the model to be determined and the actual result of the second dataset.

[0194] In a possible implementation, the third processing module 503 is further configured to:

[0195] Update the model parameters in the initial model based on a preset parameter update rule to obtain the initial model after parameter update;

[0196] Train the initial model after parameter update based on the first dataset to obtain the model to be determined after update.

[0197] In a possible implementation, the second processing module 502 is further configured to:

[0198] Input the second dataset into the model to be determined to obtain the prediction result corresponding to each sample number card data in the second dataset. Among them, the model to be determined includes multiple classification tasks, and the prediction result includes the predicted class identifier of the sample number card data on multiple classification tasks and the predicted probability value corresponding to each predicted class identifier;

[0199] Generate a third dataset based on the prediction result of each sample number card data and the sample number card data;

[0200] Calculate the similarity between the second dataset and the third dataset to obtain the recall value of the model to be determined.

[0201] In a possible implementation, the second processing module 502 is further configured to:

[0202] For each sample number card data, based on the prediction result of the sample number card data, obtain the predicted class identifier and predicted probability value of the sample number card data on multiple classification tasks;

[0203] Calculate the predicted probability mean value corresponding to the sample number card data based on the predicted probability values of the sample number card data on each classification task;

[0204] Generate a dataset to be determined based on each sample number card data and its corresponding multiple predicted class identifiers;

[0205] Sort the multiple sample number card data in the dataset to be determined in ascending or descending order based on the predicted probability mean value to obtain the third dataset.

[0206] In a possible implementation, the second processing module 502 is further configured to:

[0207] Obtain the first N sample number card data in the second dataset and their corresponding actual results, and determine them as the actual set; among them, the actual result includes the actual class identifier of the sample number card data for each classification task;

[0208] Obtain the first N sample number card data of the third data set and their corresponding multiple prediction class identifiers, and determine them as the prediction set;

[0209] Based on the intersection of the actual set and the prediction set, calculate the similarity between the actual set and the prediction set, and determine it as the recall value of the model to be determined;

[0210] Wherein, N is a positive integer greater than 1.

[0211] In a possible implementation manner, the device further includes an obtaining module, configured to:

[0212] Extract features from the historical sample number card data to obtain multiple initial data features;

[0213] Based on the multiple initial data features, perform feature combination to obtain multiple initial combined features;

[0214] Perform feature screening on the multiple initial combined features to obtain multiple target combined features;

[0215] Based on the multiple target combined features, perform data screening and marking on the historical sample number card data to obtain a sample number card data set;

[0216] Divide the sample number card data set into a first data set and a second data set according to a preset ratio;

[0217] Wherein, the multiple sample number card data in the first data set and the second data set are evenly distributed on the multiple combined features.

[0218] The test case generation device provided in this embodiment can execute the method provided in the above method embodiment, and its implementation principle and technical effect are similar, which will not be elaborated here in this embodiment.

[0219] Figure 6 It is a schematic structural diagram of an electronic device provided in this application. As Figure 6 shown, the electronic device provided in this embodiment includes: at least one processor 601 and a memory 602. Optionally, the device further includes a communication component 603. Among them, the processor 601, the memory 602, and the communication component 603 are connected through a bus 604.

[0220] In the specific implementation process, at least one processor 601 executes the computer execution instructions stored in the memory 602, so that at least one processor 601 executes the above-mentioned training method of the number card detection model or the test case generation method.

[0221] The specific implementation process of the processor 601 can refer to the above method embodiment, and its implementation principle and technical effect are similar, which will not be elaborated here in this embodiment.

[0222] In the above embodiments, it should be understood that the processor may be a central processing unit (CPU for short), or other general-purpose processors, digital signal processors (DSP for short), application specific integrated circuits (ASIC for short), etc. The general-purpose processor may be a microprocessor or any conventional processor, etc. The steps of the method disclosed in combination with the invention can be directly implemented by a hardware processor, or implemented by a combination of hardware and software modules in the processor.

[0223] The memory may include a high-speed memory (Random Access Memory, RAM), and may also include non-volatile memory (Non-volatile Memory, NVM), such as at least one disk memory.

[0224] The bus may be an Industry Standard Architecture (ISA) bus, a Peripheral Component Interconnect (PCI) bus, an Extended Industry Standard Architecture (EISA) bus, etc. The bus can be divided into an address bus, a data bus, a control bus, etc. For the convenience of representation, the bus in the drawings of this application is not limited to only one bus or one type of bus.

[0225] This application also provides a computer program product, including a computer program, which when executed by a processor implements the above-mentioned training method of the SIM card detection model or the test case generation method.

[0226] This application also provides a computer-readable storage medium, in which computer-executable instructions are stored. When the processor executes the computer-executable instructions, the above-mentioned training method of the SIM card detection model or the test case generation method is implemented.

[0227] The above-mentioned readable storage medium can be implemented by any type of volatile or non-volatile storage device or a combination thereof, such as static random access memory (SRAM), electrically erasable programmable read-only memory (EEPROM), erasable programmable read-only memory (EPROM), programmable read-only memory (PROM), read-only memory (ROM), magnetic memory, flash memory, a disk or an optical disc. The readable storage medium can be any available medium that can be accessed by a general-purpose or special-purpose computer.

[0228] An exemplary readable storage medium is coupled to a processor, enabling the processor to read information from and write information to the readable storage medium. Of course, the readable storage medium can also be a component of the processor. The processor and the readable storage medium can be located in an Application Specific Integrated Circuit (ASIC). Of course, the processor and the readable storage medium can also exist as discrete components in a device.

[0229] The division of units is only a logical function division. In actual implementation, there can be other division methods. For example, multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Additionally, the couplings or direct couplings or communication connections shown or discussed between each other can be indirect couplings or communication connections through some interfaces, and the couplings or communication connections of devices or units can be in electrical, mechanical, or other forms.

[0230] The units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they can be located in one place or distributed to multiple network units. Some or all of the units can be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0231] In addition, in each embodiment of the present invention, the functional units can be integrated in a processing unit, or each unit can exist physically alone, or two or more units can be integrated in one unit.

[0232] If the function is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention, in essence, or the part that contributes to the prior art, or a part of this technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions for causing a computer device (which can be a personal computer, a server, or a network device, etc.) to execute all or part of the steps of the methods in each embodiment of the present invention. The aforementioned storage medium includes: USB flash drives, mobile hard disks, read-only memories (ROMs), random access memories (RAMs), magnetic disks, or optical discs, etc., which are various media that can store program codes.

[0233] Those of ordinary skill in the art will understand that all or part of the steps of implementing the above method embodiments can be completed by hardware related to program instructions. The aforementioned program can be stored in a computer-readable storage medium. When the program is executed, it performs the steps including those of the above method embodiments; and the aforementioned storage medium includes: various media such as ROM, RAM, magnetic disks, or optical discs that can store program codes.

[0234] Finally, it should be noted that those skilled in the art will readily conceive of other embodiments of the present invention after considering the specification and practicing the invention disclosed herein. The present invention is intended to cover any variations, uses, or adaptations of the present invention, which follow the general principles of the present invention and include common general knowledge or conventional technical means in the technical field not disclosed by the present invention. It is not limited to the exact structures described above and shown in the drawings, and various modifications and changes can be made without departing from its scope. The scope of the present invention is only limited by the appended claims.

Claims

1. A training method for a number card detection model, characterized in that: include: Based on the first data set in the sample number card data set, the initial model is trained to obtain a model to be determined; Calculating the recall value of the to-be-determined model based on a second data set in the sample number card data set; When the recall value is less than a preset recall threshold, updating the to-be-determined model based on a preset parameter update rule until the recall value of the to-be-determined model is greater than or equal to the preset recall threshold; When the recall value is greater than or equal to the preset recall threshold, determining the model to be determined as the target number card detection model; The recall value is used to indicate the consistency of the recall ranking of the to-be-determined model on the second data set.

2. The method according to claim 1, characterized in that The updating of the to-be-determined model based on a preset parameter updating rule includes: Based on a preset parameter updating rule, the model parameters in the initial model are updated to obtain the initial model after the parameters are updated; The initial model after parameter update is trained based on the first data set to obtain an updated model to be determined.

3. The method according to any one of claims 1 to 2, characterized in that: The calculating the recall value of the to-be-determined model based on the second data set in the sample number card data set includes: Input the second data set into the model to be determined, and obtain the prediction result corresponding to each sample number card data in the second data set, wherein the model to be determined includes multiple classification tasks, and the prediction result includes the prediction category identifier of the sample number card data on the multiple classification tasks, and the prediction probability value corresponding to each prediction category identifier; Generate a third data set based on the prediction result of each sample number card data and the sample number card data; The similarity between the second data set and the third data set is calculated to obtain a recall value of the model to be determined.

4. The method according to claim 3, characterized in that The prediction result based on each piece of sample number card data and the sample number card data is used to generate a third data set, including: For each sample number card data, based on the prediction result of the sample number card data, obtain the prediction category identification and prediction probability value of the sample number card data on multiple classification tasks; Based on the predicted probability value of the sample number card data on each classification task, calculate the predicted probability mean corresponding to the sample number card data; Generate a data set to be determined based on each sample number card data and its corresponding multiple prediction category identifiers; Based on the predicted probability mean, the multiple sample number card data in the to-be-determined data set are sorted in ascending order or descending order to obtain the third data set.

5. The method according to claim 4, characterized in that The calculating the similarity between the second data set and the third data set to obtain the recall value of the to-be-determined model includes: Obtain the first N sample number card data of the second data set and their corresponding actual results, and determine them as an actual set; wherein the actual results include the actual category identification of the sample number card data for each classification task; Obtaining the first N sample number card data of the third data set and the corresponding multiple prediction category identifiers, and determining them as a prediction set; Based on the intersection of the actual set and the predicted set, calculating the similarity between the actual set and the predicted set, and determining the similarity as the recall value of the model to be determined; Wherein, N is a positive integer greater than 1.

6. The method according to claim 1, characterized in that Before training the initial model based on the first data set in the sample number card data set to obtain the model to be determined, the method further includes: Perform feature extraction on historical sample card data to obtain multiple initial data features; Combining features based on multiple initial data features to obtain multiple initial combination features; Perform feature screening on multiple initial combination features to obtain multiple target combination features; Screening and marking the historical sample number card data based on multiple target combination features to obtain the sample number card data set; Dividing the sample number card data set into a first data set and a second data set according to a preset ratio; The plurality of sample number card data in the first data set and the second data set are evenly distributed on the plurality of combination features.

7. A training device for a number card detection model, characterized in that: include: A first processing module is used to train the initial model based on the first data set in the sample number card data set to obtain a model to be determined; A second processing module, used for calculating the recall value of the to-be-determined model based on a second data set in the sample number card data set; a third processing module, configured to update the to-be-determined model based on a preset parameter updating rule when the recall value is less than a preset recall threshold, until the recall value of the to-be-determined model is greater than or equal to the preset recall threshold; A fourth processing module, configured to determine the to-be-determined model as a target number card detection model when the recall value is greater than or equal to the preset recall threshold; The recall value is used to indicate the consistency between the prediction result of the to-be-determined model and the actual result of the second data set.

8. An electronic device, characterized in that: include: Memory, processor; The memory stores computer-executable instructions; The processor executes the computer-executable instructions stored in the memory, so that the processor performs the method according to any one of claims 1 to 6.

9. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores computer-executable instructions, which are used to implement the method according to any one of claims 1 to 6 when executed by a processor.

10. A computer program product, characterized in that The invention comprises a computer program, which implements the method according to any one of claims 1 to 6 when being executed by a processor.

Images