Similarity-Guided Data Classification with Partial Memory Scanning
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
The K-nearest neighbor (KNN) algorithm for data classification involves scanning all data, leading to increased memory access and bottlenecks, necessitating a solution to reduce memory access while maintaining classification accuracy.
Innovation Solution
A system and method that determine a scan target based on group and data similarities, selecting data groups and pieces of data within those groups using reference values and similarity functions to output result data, thereby reducing memory access.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If the KNN algorithm scans all data to ensure classification accuracy, then the classification precision is improved, but the memory access increases and bottlenecks occur
Solution Approach 1:
The patent divides the entire data set into multiple data groups (clusters) and further segments them into sub-groups. This segmentation allows the system to scan only relevant portions of data rather than all data, reducing memory access while maintaining classification accuracy through hierarchical organization.
Solution Approach 2:
The patent implements partial scanning by selecting only certain data groups and sub-groups based on query similarity rather than scanning all data. The system determines a scan target including specific data groups and selects scan data from those groups, performing less than a complete scan while still achieving accurate classification results.
2Productivity
If the system scans partial data to reduce memory access, then the productivity is improved, but the complexity of determining scan targets increases
Solution Approach 1:
The patent performs preliminary actions by pre-calculating and storing group similarity information and data similarity information before actual queries. This preliminary computation of similarity metrics allows the system to quickly determine scan targets without complex real-time calculations, reducing the complexity during query processing.
Solution Approach 2:
The patent introduces group similarity and data similarity as intermediary metrics that mediate between the query and the data selection process. These similarity measures serve as intermediaries to simplify the scan target determination by providing a structured way to identify relevant data groups and sub-groups without direct complex comparisons.
3Productivity
If the system determines scan targets based on group similarities and data similarities, then the memory access is reduced, but the calculation complexity increases
Solution Approach 1:
The patent segments the similarity calculation into two distinct levels: group similarity calculation between data groups and query, and data similarity calculation within selected data groups. This segmentation of the calculation process reduces overall complexity by breaking down a single complex calculation into manageable hierarchical steps.
Solution Approach 2:
The patent applies partial calculation by computing group similarities for all data groups but only computing data similarities for selected data groups that form the scan target. This partial computation approach reduces the total calculation complexity compared to computing all possible similarities, while still achieving the desired memory access reduction.
Data Source
AI summary
A system for classifying data may include a memory, and a processor configured to determine a scan target including a data group selected from among data groups stored in the memory, based on a result of a comparison between first similarities of data groups stored in the memory and an externally received query, and a minimum value of second similarities of pieces of data included in a data group having a maximum value of the first similarities and the query, and to output, as result data responding to the query, scan data selected depending on a reference number of pieces of scan data from among pieces of scan data in the data group included in the scan target.