Data Classification Using Cross-Category Restriction Scoring
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing data classification methods fail to accurately distinguish between data categories with similar string patterns, especially when multiple values are available, leading to incorrect categorization and scoring, particularly in the presence of errors, which is a challenge in adhering to privacy regulations like GDPR.
Innovation Solution
The proposed method involves determining a loose string format and set of restrictions for data strings with the same string pattern, utilizing restriction information from other categories to classify data strings and adjusting the classification score based on the mean restriction matching proportion and expected mean, using techniques like Kullback-Leibler distance to determine the distribution of data strings and select the appropriate category.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Productivity
If a naive classifier is used to score data categories by checking the proportion of values that satisfy the category domain, then the classification process is simple and fast, but the classification accuracy deteriorates when values match multiple categories or contain errors
Solution Approach 1:
The patent changes the classification parameters from simple string pattern matching to incorporating domain restrictions and statistical properties. By evaluating multiple parameters including restriction satisfaction, distribution characteristics, and cross-category restriction analysis, the system achieves higher classification accuracy while maintaining automated processing efficiency
Solution Approach 2:
The patent adds another dimension to classification by introducing cross-category restriction analysis. Instead of evaluating each category in isolation, the system analyzes how restrictions in one category affect scoring in other categories, creating a multi-dimensional classification space that resolves ambiguities between competing categories
2Measurement precision
If manual prioritization of classifiers is used to improve classification accuracy, then classification precision may improve, but the ease of operation deteriorates due to required manual work
Solution Approach 1:
The system performs self-service by automatically learning and applying classification rules from the data itself. The automated classifier evaluates domain restrictions, calculates statistical distributions, and determines category priorities without manual intervention, while still achieving high accuracy through sophisticated analysis of cross-category restriction patterns
Solution Approach 2:
The system implements feedback mechanisms where classification results and restriction satisfaction patterns are used to refine scoring. The mean restriction matching proportion and expected mean calculations provide feedback that automatically adjusts category scores, eliminating the need for manual rule configuration
3Device complexity
If simple string pattern matching is used for classification, then the device complexity is low, but the measurement precision deteriorates when data includes errors or multiple matching categories
Solution Approach 1:
The system performs preliminary actions by pre-defining domain restrictions and validation rules for each category before classification. These pre-established constraints (such as Luhn checksum for country IDs, specific patterns for SSN) are applied automatically during classification, enabling accurate differentiation without complex real-time analysis
Solution Approach 2:
The classification approach combines multiple elements into a composite scoring system that integrates string pattern matching, domain restriction validation, statistical distribution analysis, and cross-category restriction evaluation. This composite methodology achieves high precision while keeping individual components relatively simple
Data Source
AI summary
Embodiments of the present systems and methods may provide techniques to distinguish between data categories. For example, a method implemented in a computer system may comprise obtaining, at the computer system, a plurality of data strings in different categories, each category having a same string pattern, determining a loose string format and a set of restrictions based on at least one string pattern, classifying the plurality of data strings to respective different categories based on a loose string format of the data strings and on the restrictions on the data strings of the different categories using a classification score indicating utilizing restriction information of other categories when determining the matching of a category, and decreasing the classification score if a mean restriction matching proportion is not part of a category or is a threshold amount above an expected mean restriction matching proportion.