Duplicate Code Detection Using Enhanced Generalized Suffix Arrays
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Large software projects often contain duplicate code sections, which lead to maintenance issues and potential copyright liabilities, and existing methods are inefficient in detecting and managing these duplicates across multiple files.
Innovation Solution
The method involves constructing and traversing an enhanced generalized suffix array (eGSA) to identify duplicate code sections by segmenting input files into statements based on instruction boundaries, using a hash or fingerprint for each statement, and comparing lexicographically ordered suffixes to detect maximal length duplicates.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If traditional duplicate detection methods are used, then duplicate code sections can be identified, but the processing time and memory resources required are excessive for large software projects
Solution Approach 1:
The source code files are segmented into individual statements based on instruction boundaries. Each statement is then processed independently to create suffix arrays, allowing the detection process to work on smaller, manageable units rather than entire files, significantly reducing processing time and memory requirements while maintaining detection accuracy
Solution Approach 2:
The patent transforms the duplicate detection problem from a traditional string comparison approach into a multi-dimensional data structure approach using enhanced generalized suffix arrays (eGSA). By organizing code statements into suffix arrays with additional metadata dimensions (statement type, file location, line numbers), the system achieves faster comparison and identification of duplicates across large codebases
2Measurement precision
If traditional duplicate detection methods are used, then duplicate code sections can be identified, but the memory resources required are excessive for large software projects
Solution Approach 1:
The codebase is divided into individual statements, and suffix arrays are constructed for each statement rather than for entire files. This segmentation reduces the memory footprint by processing and comparing smaller units independently, allowing efficient detection of duplicates in large software projects with limited memory resources
Solution Approach 2:
The patent creates compact representations of code statements through suffix arrays, which are efficient data structures that store only the necessary information for comparison. This copying approach uses minimal memory to represent and compare code segments, enabling duplicate detection in large projects without requiring excessive memory resources
3Productivity
If code duplication is not detected, then development continues quickly, but maintenance issues and copyright liabilities increase
Solution Approach 1:
The enhanced generalized suffix array construction and duplicate detection is performed early in the software development lifecycle, before code duplication causes maintenance problems. By establishing a comprehensive index of all code statements upfront, the system enables rapid identification of duplicates throughout subsequent development and maintenance phases, preventing accumulation of problematic duplicate code
Solution Approach 2:
The system provides continuous feedback about duplicate code sections by querying the pre-built eGSA structure. Developers receive immediate information about potential duplicates when writing or modifying code, allowing them to make informed decisions about code reuse versus duplication, thus maintaining code quality without significantly impacting development speed
Data Source
AI summary
Techniques for duplicate code section detection for source code are described herein. An aspect includes receiving a plurality of input files corresponding to a software project comprising source code written in a computer programming language. Another aspect includes segmenting each of the plurality of input files into a plurality of statements based on instruction boundaries corresponding to the computer programming language, wherein a respective statement start index is determined for each of the plurality of statements. Another aspect includes populating an enhanced generalized suffix array (eGSA) based on the determined statement start indices, wherein each statement start index corresponds to a respective suffix in a row in the eGSA, and wherein each row comprises a longest common prefix (LCP) field and a preceding statement value corresponding to the row's respective suffix. Another aspect includes identifying duplicate code sections in the plurality of input files based on the eGSA.