Data Type Detection Using Existing Code Logic
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing systems are inefficient in identifying and detecting data types, particularly for rich semantic data like credit card numbers and ISBN numbers, due to their reliance on simple syntactic patterns rather than domain-specific semantics, leading to inaccurate detections and the need for manual, time-consuming code generation by programmers with limited domain knowledge.
Innovation Solution
The system utilizes existing open source code to identify relevant functions through crawling code sources, executes candidate functions with positive and negative values to distinguish between them, and provides function suggestions to users for accurate data type detection, facilitating the generation of synthesized functions for validation.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If pattern-based detection methods are used to identify data types, then the detection process is simple and fast, but the detection accuracy is low particularly for rich semantic data
Solution Approach 1:
The system pre-processes and stores detection functions, their parameters, and logic explanations in a structured format before actual data type detection occurs. This preliminary organization of detection resources enables accurate semantic analysis without requiring complex real-time processing, resolving the contradiction between accuracy and complexity.
Solution Approach 2:
The system introduces an intermediary layer that matches data values against pre-defined detection functions and their logic explanations. This intermediary matching process enables accurate detection of rich semantic data by comparing against established detection logic, while the system itself remains relatively simple in structure.
2Measurement precision
If manual code generation is performed to detect specific data types, then detection accuracy can be improved, but the time consumption increases significantly
Solution Approach 1:
Instead of generating custom detection code for each data type scenario, the system copies and reuses pre-defined detection functions and their logic explanations from a library. This copying approach maintains high detection accuracy by using proven detection logic while eliminating the time-consuming manual code generation process.
Solution Approach 2:
The system automatically performs data type detection by selecting and executing appropriate pre-defined functions based on the input data, without requiring manual intervention or custom code generation. This self-service capability achieves both high accuracy through proper function selection and fast processing by eliminating manual steps.
3Measurement precision
If domain-specific detection functions are created for each data type, then detection accuracy improves, but the system complexity and maintenance burden increase
Solution Approach 1:
The system creates a universal detection framework where pre-defined functions and their logic explanations can be applied to multiple different data types and scenarios. This multi-functional approach enables accurate domain-specific detection while maintaining system scalability, as the same framework handles various data types without requiring separate custom implementations for each.
Data Source
AI summary
Methods, computer systems, computer-storage media, and graphical user interfaces are provided for facilitating data type detection, according to embodiments of the present invention. In one embodiment, existing code is searched to identify a set of functions related to a target data type. Such functions can be executed using positive example values and negative example values. For each executed function, a logical explanation is generated that represents a distinction in execution of the positive example values from the negative example values. The executed functions can then be ranked based on the extent to which the corresponding logical explanations distinguish execution of the positive example values from the negative example values. A function suggestion corresponding with at least a highest ranked function can then be provided, for example to a user, to indicate a function for use in detecting the target data type.