Source Code Classifier for Open Source Detection
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current methods for analyzing source code files to detect open source code and identify compliance and security risks are prone to false positives, are computationally intensive, and struggle with accurately distinguishing between source and non-source code components, leading to decreased accuracy and increased processing time.
Innovation Solution
A deep neural network classifier is trained to identify patterns in source code files, distinguishing between source code and non-source code elements such as license information, copyright information, and dependent library information, thereby reducing false positives and improving detection accuracy while reducing analysis time.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If traditional scanning methods are used to analyze source code files, then comprehensive detection of open source code can be performed, but false positive detections increase and processing time increases
Solution Approach 1:
The source code file is divided into multiple text lines, and the classifier processes each text line independently to determine whether it corresponds to code construct type information. This segmentation allows parallel processing and reduces the overall processing time while maintaining detection accuracy.
Solution Approach 2:
The classifier extracts and identifies specific text lines that correspond to code construct type information (such as license information) from the source code file. By extracting only the relevant portions, the system reduces the amount of data that needs to be processed further, thereby decreasing processing time without compromising detection accuracy.
2Reliability
If traditional scanning methods are used to analyze source code files, then open source code detection can be performed, but computational resources are excessively consumed
Solution Approach 1:
The classifier is pre-trained with sample source code files to learn the patterns and characteristics of code construct type information. This preliminary training enables the classifier to quickly and accurately identify relevant text lines during the actual analysis, reducing the computational resources required during the detection phase while maintaining high detection accuracy.
3Measurement precision
If the classifier identifies code construct type information in all text lines, then detection accuracy improves, but the subset generation process becomes more complex
Solution Approach 1:
The classifier automatically and independently evaluates each text line to determine whether it corresponds to code construct type information. This self-service approach allows the system to maintain high identification accuracy through intelligent classification while avoiding the need for complex manual processing or additional processing layers, thereby reducing overall processing complexity.
Data Source
AI summary
A method for processing a source code file comprises scanning the source code file to identify text lines, and analyzing, via one or more processors, the text lines with a classifier to identify one or more of the text lines that correspond to code construct type information. The code construct type information includes license information. The classifier is trained with sample source code files. The method further comprises generating a subset of the text lines that excludes the one or more of the text lines identified as corresponding to the code construct type information. Further, the method comprises determining first text lines within the subset that correspond to open source code by comparing the subset to a database. The database includes a plurality of text lines associated with open source code.