Data Identification Model Stability via Segmented Logistic Regression and DNN Training
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current identification models for fake transactions in e-commerce platforms, primarily using logistic regression, struggle with stability and adaptability due to proportional sampling of training samples without distinguishing positive samples, leading to low TOP bad-capture rates and limited ability to handle increasing dimensions.
Innovation Solution
The method involves feature engineering preprocessing and screening, followed by logistic regression training, and subsequent Deep Neural Networks (DNN) training using selectively sampled positive and negative samples to improve model stability and accuracy, enhancing the TOP bad-capture rate.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If logistic regression training is used with proportional sampling of training samples, then the model training process is simple, but the model stability and TOP bad-capture rate are poor
Solution Approach 1:
The patent divides the training process into two distinct stages: first training a logistic regression model to obtain initial identification results, then using these results to selectively sample positive samples for training a DNN model. This segmentation allows each model to focus on specific aspects of the problem, improving overall stability without requiring complete model redesign
Solution Approach 2:
The logistic regression model serves as a preliminary filtering stage that identifies candidate positive samples before the main DNN training. This preliminary action prepares high-quality training data by pre-screening samples based on initial model predictions, thereby improving the quality of training data for the subsequent DNN model
2Adaptability or versatility
If proportional sampling of training samples is used without distinguishing positive samples, then the sampling process is simple, but the ability to handle increasing dimensions and improve TOP bad-capture rate is limited
Solution Approach 1:
The patent applies different sampling strategies to different types of samples: positive samples are selectively sampled based on identification results from the logistic regression model, while negative samples are sampled proportionally. This local differentiation in sampling quality allows the model to focus computational resources on the most informative positive samples, improving adaptability to high-dimensional data
Solution Approach 2:
The logistic regression model acts as an intermediary that bridges the gap between simple proportional sampling and complex selective sampling. It processes all training samples first, then uses its predictions to guide the selective sampling of positive samples for DNN training, thereby enabling improved TOP bad-capture rate without requiring direct complex sampling of all samples
3Reliability
If DNN training is performed using all positive and negative samples, then comprehensive training is achieved, but the training efficiency and model stability are reduced
Solution Approach 1:
The patent extracts a subset of high-quality positive samples from the complete training set by using identification results from the logistic regression model. Instead of training DNN on all positive samples, only those selected based on the preliminary model's predictions are used, thereby improving training efficiency and model stability by focusing on the most informative samples
Data Source
AI summary
The present disclosure provides methods and an apparatuses for building a data identification model. One exemplary method for building a data identification model includes: performing logistic regression training using training samples to obtain a first model, the training samples comprising positive and negative samples; sampling the training samples proportionally to obtain a first training sample set; identifying the positive samples using the first model, and selecting a second training sample set from positive samples that have identification results after being identified using the first model; and performing Deep Neural Networks (DNN) training using the first training sample set and the second training sample set to obtain a final data identification model. The methods and the apparatuses of the present disclosure improve the stability of data identification models.