Code Snippet Fingerprinting With Winnowing for Large-Scale Similarity Search
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Identifying and reusing similar code snippets across large collections of programs and applications within organizations is difficult, leading to inefficiencies and increased code generation costs.
Innovation Solution
A system and method for fingerprinting code snippets using k-grams, normalizing code, and applying a winnowing process to generate fingerprints, which are then used to search and rank similar code snippets, excluding sanctioned software like library code and boilerplate, to facilitate efficient reuse.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Quantity of substance
If code snippets are stored in large data stores for organizational use, then code availability and resource sharing are improved, but code identification and retrieval become difficult
Solution Approach 1:
The code is segmented into k-grams (subsequences of k tokens), which are then hashed to create fingerprints. This segmentation transforms the large-scale code search problem into smaller, manageable units that can be efficiently compared and matched, resolving the difficulty of identifying code within large data stores.
Solution Approach 2:
Fingerprints serve as an intermediary representation between the original code and the search process. By converting code into fingerprint form through hashing and winnowing, the system enables efficient similarity search without directly comparing large volumes of raw code, thus improving retrieval efficiency in large code collections.
2Adaptability or versatility
If code search functionality is added to identify similar code snippets, then code reusability is improved, but system complexity increases
Solution Approach 1:
The system performs preliminary actions by pre-computing fingerprints for all code snippets in the collection and storing them in advance. When a search is needed, the system only needs to compare fingerprints rather than analyze full code, significantly reducing the complexity of the search operation while maintaining high code reusability.
Solution Approach 2:
Instead of storing and comparing entire code snippets, the system creates simplified copies in the form of fingerprints (hash values). These fingerprint copies retain the essential similarity information needed for code reuse identification while dramatically reducing the computational complexity of the search system.
3Adaptability or versatility
If multiple instances of similar code are written across different projects, then project-specific customization is improved, but code generation costs increase
Solution Approach 1:
The system provides feedback by identifying similar existing code through fingerprint matching, allowing developers to discover and reuse proven code patterns across projects. This feedback loop enables customization based on existing successful implementations, reducing redundant code generation while maintaining project-specific adaptations.
Data Source
AI summary
Systems and methods for identifying associations between a code snippet query and stored computer code stored. The method can receive a code query identifying a code snippet to search for, determine a fingerprint of the query code snippet, and search the stored software using the fingerprint to identify software results of code similar to the query code snippet. The fingerprint can be determined by generating k-grams of the code snippet. The k-grams used for the search can be down-selected based on a winnowing process. The method can remove from the software results code that is associated with sanctioned software. The method can include coalescing the software results to produce a subset of the software results, generating a code search user interface comprising information indicative of the subset of software results, and causing presentation of the code search user interface and displaying the subset of software results.