Source Code Duplicate Detection via Hash Encoding and Segmentation
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current methods for refactoring source code, particularly in large projects, are labor-intensive due to the need for manual extraction and correction of similar code blocks, and face challenges with high processing times (O(n^2)) for automatic extraction.
Innovation Solution
An information processing device that divides source code into units based on predetermined identifiers, encodes sub-blocks, and uses these encoded blocks as keys to detect duplicate content, thereby facilitating the identification and refactoring of similar code blocks.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If automatic extraction of duplicate code blocks is performed using traditional methods, then extraction accuracy is improved, but processing time increases to O(n^2) making it impractical for large projects
Solution Approach 1:
The source code is divided into blocks based on code block delimiters (such as curly braces), and each block is further segmented into sub-blocks with fixed sizes. This segmentation transforms the problem from comparing all possible pairs of code blocks (O(n^2)) to comparing segmented units, significantly reducing the search space and processing time while maintaining extraction accuracy.
Solution Approach 2:
The patent introduces a new dimension by encoding code blocks using hash functions and organizing them in a hierarchical structure (blocks → sub-blocks → encoded representations). This dimensional transformation allows duplicate detection to proceed through encoded comparisons rather than direct text comparisons, achieving O(n) or O(n log n) complexity while preserving extraction precision.
2Manufacturing precision
If manual extraction and correction of similar code blocks is performed, then refactoring precision is improved, but labor cost increases
Solution Approach 1:
The system performs self-service by automatically detecting, extracting, and organizing duplicate code blocks without requiring manual intervention. The automated extraction process identifies similar code blocks, generates refactoring candidates, and presents them to developers, thereby reducing labor costs while maintaining refactoring precision through algorithmic accuracy.
Solution Approach 2:
The patent replaces the mechanical manual process of code extraction and comparison with an automated computational system using hash encoding, block segmentation, and algorithmic detection. This substitution eliminates manual labor while preserving precision through systematic and consistent automated analysis of code structures.
3Productivity
If code blocks are divided into smaller units for detection, then detection speed is improved to O(n), but detection accuracy may be reduced due to loss of contextual information
Solution Approach 1:
The patent implements a nested hierarchical structure where source code is divided into blocks, which contain sub-blocks, which in turn contain encoded representations. This nested organization allows the system to work with smaller units for fast detection while maintaining the ability to reconstruct and evaluate complete code block contexts, thus preserving detection accuracy despite the segmented approach.
Solution Approach 2:
Hash encoding acts as an intermediary between the segmented code units and the detection process. The encoding preserves essential structural and semantic information in a compact form, enabling fast comparison of segmented units while maintaining the fidelity needed for accurate duplicate detection. The intermediary representation bridges the gap between speed-optimized segmentation and accuracy requirements.
Data Source
AI summary
An information processing device includes one or more processors. The one or more processors are configured to divide targets into a plurality of units for each predetermined identifier, define a set of a first predetermined number of said units as a sub-block from each of the units for the plurality of units; encode the sub-block by predetermined conversion, and obtain a code; and use the code as a key, and detect existence of another sub-block having identical content with the sub-block corresponding to the code when duplication of the key is detected.