A multi-stage filtering source code data detection method and device

Abstract

The invention discloses a multistage-filtering source code data detection method and device. The device comprises a file type detection filtering module, a lexical analysis filtering module, a grammar analysis filtering module, a semantic analysis filtering module, a source code data containing file protection module, and a non-source code labelling module, wherein the file type detection filtering module is used for judging whether an input file is of an appointed file type or not; the lexical analysis filtering module extracts a lexical mark in the file, determines a corresponding weight, calculates a weighing score sum, and judges whether the weighing score sum exceeds an appointed threshold value or not; the grammar analysis filtering module intercepts a text of an appointed length as a suspected text from the file, extracts a grammar phrase and an expression contained in the suspected text, and judges the importance degrees of the phrase and the expression for a source code; the semantic analysis filtering module extracts the semantic features of the text, and carries out similarity analysis on the semantic features and the semantic features of an appointed core source code; the source code data containing file protection module carries out sensitive data protection on the file which contains source program data; and the non-source code labelling module carries out no source code labelling on the file. Through the above scheme, the accuracy of source code detection is improved, and the safety protection strength of the source code is reinforced.

Description

technical field [0001] The invention relates to the technical field of source code data detection, in particular to a multi-stage filtering source code data detection method and device. Background technique [0002] As an R&D and design enterprise, data such as design documents, drawings, and source codes are the core intellectual assets of the enterprise, as well as the core competitiveness of the enterprise. Effective management and control of these core data is the top priority of enterprise information security. Because the source code data exists in the form of text files or text fragments, it is more likely to be mixed or embedded in conventional text files, which may cause loss, leakage, or uncontrolled diffusion, which may endanger enterprise information security. Most of these sources of data loss are due to unintentional operations by internal personnel, and a few intentional leaks from internal personnel and malicious attacks from outside the enterprise. The occu...

AI Technical Summary

Problems solved by technology

[0010] (1) The method used in the above patent only intercepts network data streams, and is not suitable for detecting whether local storage files contain source code data, and cannot know the distribution of local source code

[0011] (2) The basis for judging whether a character stream contains source code in the above-mentioned patent is "preset detection strings and / or syntax analysis library functions corresponding to the programming language", and does not specify the file type, lexical token attributes and Semantic content is analyzed, and the accuracy of source code data detection is limited

[0012] (3) In enterprises with a low degree of standardization in the software development process, there are many development tools, different code styles, irregular identifier naming, and complex code labeling words

To detect these source codes, only relying on basic detection technologies such as file attribute detection, keyword matching, and regular expression matching will have the problem of inaccurate positioning (it is difficult to determine the location of source code data fragments in an ordinary document); Fingerprint, statistical learning and other methods also have the defect of insufficient versatility of detection methods due to incomplete sample acquisition.

[0013] (4) When the above-mentioned advanced detection technology of text data performs semantic analysis on a long text, it will calculate the full text of the text, which has a large amount of calculation and will cause a long time to output the detection result

Images