Method and system for detecting tampering of open source license and copyright information

Abstract

The present invention proposes a detection method and system for tampering of open source license and copyright information. The detection method includes: establishing a homology analysis feature knowledge base; matching the feature information of a file to be detected with the feature information in the homology analysis feature knowledge base, and judging whether the file to be detected has a homology relationship with the file in the feature information knowledge base; in the case of judging that there is a homology relationship file in the file to be detected, recording the homology relationship file, and matching and verifying the license and copyright information of the file to be detected with the license and copyright information of the homology relationship file, and judging whether the license and copyright information in the file to be detected has been tampered with, deleted, or added. The detection method for tampering of open source license and copyright information according to the present invention solves the problem of low efficiency and high false alarm rate in the related art of using manual processing for tampering detection of mixed-source licenses, and realizes automatic detection of tampering of open source license and copyright information, reduces cost and false alarm rate, and improves efficiency, detection rate, and detection quality.

Description

Technical Field

[0001] The present invention relates to the technical field of software development, and in particular to a method and system for detecting tampering of open source license and copyright information. Background Art

[0002] According to incomplete statistics, in software development at home and abroad, up to 99% of software projects use open source software to a greater or lesser extent, and the fields involved are also very wide, including the currently popular blockchain, artificial intelligence, big data, cloud computing, image recognition, industrial software and many other fields. At the same time, open source software authors are beginning to pay more and more attention to the protection of intellectual property rights. Software authors generally declare the corresponding open source software license for their software products to protect their own legal protection of open source software works and safeguard their own legitimate rights and interests.

[0003] When developers use open source software, they often use it in violation of regulations. For example, the famous open source software license Apache-2.0 clearly states that software developers need to retain the original license statement and copyright statement when using open source code. However, in practice, developers often intentionally or unintentionally delete relevant copyright statements and license statement information in order to claim purposes such as "self-research". This invisibly steals the fruits of the open source authors and infringes on their intellectual property rights.

[0004] Based on the above background, it is very meaningful to intelligently identify whether someone has cited other people's open source software works, but intentionally or unintentionally tampered with (including deletion and modification) the original author's relevant license and copyright statement. It can help software developers use open source software works reasonably and in compliance with regulations, and avoid potential legal risks and reputation losses. It also plays a great role in related fields such as intellectual property evaluation and intellectual property disputes.

[0005] At present, there is no intelligent detection method at home or abroad for detecting tampering of open source licenses of mixed-source software, and there are few related demand scenarios. When related things need to be done, they are generally identified and judged file by file through human eyes. Summary of the invention

[0006] The technical problem to be solved by the present invention is how to intelligently detect tampering of open source licenses and copyright information. The present invention provides a method and system for detecting tampering of open source licenses and copyright information.

[0007] The method for detecting tampering of open source license and copyright information according to an embodiment of the present invention includes:

[0008] Establish a knowledge base of homology analysis features;

[0009] Matching the feature information of the file to be detected with the feature information in the homology analysis feature knowledge base to determine whether the file to be detected has a homology relationship with the file in the feature information knowledge base;

[0010] When it is determined that there are homologous files in the file to be detected, the homologous files are recorded, and the license and copyright information of the file to be detected are matched and verified with the license and copyright information of the homologous files to determine whether the license and copyright information in the file to be detected has been tampered with, deleted, or added.

[0011] The method for detecting tampering of open source licenses and copyright information according to an embodiment of the present invention solves the problem of low efficiency and high false alarm rate in the related art of manually processing mixed-source license tampering detection, and realizes automatic detection of tampering of open source licenses and copyright information, reduces costs and false alarm rates, and improves efficiency, detection rate and detection quality.

[0012] According to some embodiments of the present invention, before performing a match check on the license and copyright information of the file to be detected, the method further includes:

[0013] Establish a license and copyright information knowledge base to store basic information and terms of open source licenses and copyrights.

[0014] In some embodiments of the present invention, when matching and verifying the license and copyright information of the file to be detected, the license and copyright information of the file to be detected is matched and verified with the basic information and terms information of the open source license and copyright in the license and copyright information knowledge base.

[0015] According to some embodiments of the present invention, the feature information in the homology analysis feature knowledge base includes: original file-level features, file-level features, and code snippet-level features, and matching the file to be detected with the feature information in the homology analysis feature knowledge base to determine whether the file to be detected is a file with a homology relationship includes:

[0016] Matching the file to be detected with the original file-level features in the homology analysis feature knowledge base to determine whether the file to be detected has a homology relationship file with the same source as the original file;

[0017] Matching the file to be detected with the file-level features in the homology analysis feature knowledge base to determine whether the file to be detected has a homology relationship file with file-level homology;

[0018] The file to be detected is matched with the code snippet-level features in the homology analysis feature knowledge base to determine whether the file to be detected has a homology relationship file with the same source at the code snippet level.

[0019] In some embodiments of the present invention, at least one of the original file-level features, the file-level features, and the code snippet-level features is a Hash feature.

[0020] The system for detecting tampering of open source license and copyright information according to an embodiment of the present invention includes:

[0021] A first knowledge base creation module is used to establish a homology analysis feature knowledge base;

[0022] A homology relationship judgment module, used to match the feature information of the file to be detected with the feature information in the homology analysis feature knowledge base, and judge whether the file to be detected has a homology relationship with the file in the feature information knowledge base;

[0023] The license and copyright verification module is used to record the homologous files when it is determined that the file to be detected has a homologous file, and to match and verify the license and copyright information of the file to be detected with the license and copyright information of the homologous file, and to determine whether the license and copyright information in the file to be detected has been tampered with, deleted or added.

[0024] The open source license and copyright information tampering detection system according to the embodiment of the present invention solves the problem of low efficiency and high false alarm rate in the related art of using manual processing for mixed-source license tampering detection, and realizes automatic detection of open source license and copyright information tampering, reduces costs and false alarm rates, and improves efficiency, detection rate and detection quality.

[0025] According to some embodiments of the present invention, the system further comprises:

[0026] The second knowledge base creation module is used to establish a license and copyright information knowledge base to store the basic information and terms information of the open source license and copyright.

[0027] In some embodiments of the present invention, the system also includes: an extraction module, which is used to match and verify the license and copyright information of the file to be detected with the basic information and terms information of the open source license and copyright in the license and copyright information knowledge base when matching and verifying the license and copyright information of the file to be detected.

[0028] According to some embodiments of the present invention, the feature information in the homology analysis feature knowledge base includes: original file-level features, file-level features and code snippet-level features, and the homology relationship judgment module includes:

[0029] A first judgment module is used to match the file to be detected with the original file-level features in the homology analysis feature knowledge base to determine whether the file to be detected has a homology relationship file with the same source as the original file level;

[0030] A second judgment module is used to match the file to be detected with the file-level features in the homology analysis feature knowledge base to determine whether the file to be detected is a homology relationship file with file-level homology;

[0031] The third judgment module is used to match the file to be detected with the code snippet-level features in the homology analysis feature knowledge base to determine whether the file to be detected is a homology relationship file with code snippet-level homology.

[0032] In some embodiments of the present invention, at least one of the original file-level features, the file-level features, and the code snippet-level features is a Hash feature. BRIEF DESCRIPTION OF THE DRAWINGS

[0033] Figure 1 A flow chart of a method for detecting tampering of open source license and copyright information according to an embodiment of the present invention;

[0034] Figure 2 A schematic diagram of a method for detecting tampering of open source license and copyright information according to an embodiment of the present invention;

[0035] Figure 3 is a flow chart of a method for determining homology relationships according to an embodiment of the present invention;

[0036] Figure 4 A flowchart of extracting file license and copyright information of a file to be detected according to an embodiment of the present invention;

[0037] Figure 5 Schematic diagram of a system for detecting tampering of open source license and copyright information according to an embodiment of the present invention.

[0038] Reference numerals:

[0039] Detection system 100,

[0040] A first knowledge base creation module 10, a homology relationship judgment module 20, and a license and copyright verification module 30. DETAILED DESCRIPTION

[0041] In order to further explain the technical means and effects adopted by the present invention to achieve the predetermined purpose, the present invention is described in detail below in conjunction with the accompanying drawings and preferred embodiments.

[0042] In the related art, the detection of tampering of open source licenses of mixed-source software is generally done by human eyes, identifying and judging each file one by one. There are the following defects:

[0043] High cost, low efficiency, high false alarm rate, low detection rate, and inconsistent quality of detection results due to uneven personnel quality.

[0044] In view of the above problems, the present invention proposes a method and system for detecting tampering of open source license and copyright information.

[0045] See also Figure 1 The method for detecting tampering of open source license and copyright information according to an embodiment of the present invention comprises the following steps:

[0046] S110, establishing a homology analysis feature knowledge base;

[0047] Specifically, we can rely on the collected and organized open source codes and use a lexical analyzer to establish a homology analysis feature knowledge base to support homology detection of files to be detected.

[0048] S120, matching the feature information of the file to be detected with the feature information in the homology analysis feature knowledge base, and determining whether the file to be detected has a homology relationship with the file in the feature information knowledge base;

[0049] S130, when it is determined that the file to be detected has a homologous file, the homologous file is recorded, and the license and copyright information of the file to be detected is matched and verified with the license and copyright information of the homologous file, to determine whether the license and copyright information in the file to be detected has been tampered with, deleted or added.

[0050] The method for detecting tampering of open source licenses and copyright information according to an embodiment of the present invention solves the problem of low efficiency and high false alarm rate in the related art of manually processing mixed-source license tampering detection, and realizes automatic detection of tampering of open source licenses and copyright information, reduces costs and false alarm rates, and improves efficiency, detection rate and detection quality.

[0051] According to some embodiments of the present invention, before performing a match check on the license and copyright information of the file to be detected, the method further includes:

[0052] Establish a license and copyright information knowledge base to store basic information and terms of open source licenses and copyrights.

[0053] It should be noted that when establishing a license and copyright information knowledge base, the basic information of the license and the corresponding terms information can be collected and organized to establish the license and copyright information knowledge base.

[0054] In some embodiments of the present invention, Figure 4 As shown, when matching and verifying the license and copyright information of the file to be detected, the license and copyright information of the file to be detected is matched and verified with the basic information and clause information of the open source license and copyright in the license and copyright information knowledge base.

[0055] According to some embodiments of the present invention, the feature information in the homology analysis feature knowledge base includes: original file level features, file level features and code snippet level features. Figure 3 As shown, the file to be detected is matched with the feature information in the homology analysis feature knowledge base to determine whether there is a homology relationship file in the file to be detected, including:

[0056] Match the file to be detected with the original file-level features in the homology analysis feature knowledge base to determine whether the file to be detected has a homology relationship file with the same source as the original file level;

[0057] Match the file to be detected with the file-level features in the homology analysis feature knowledge base to determine whether there is a homology relationship file with file-level homology in the file to be detected;

[0058] The file to be detected is matched with the code snippet-level features in the homology analysis feature knowledge base to determine whether there is a homology relationship file with code snippet-level homology in the file to be detected.

[0059] In some embodiments of the present invention, at least one of the original file level feature, the file level feature and the code snippet level feature is a Hash feature.

[0060] It should be noted that when extracting original file-level features, file-level features, and code segment-level features, a Hash algorithm can be used for processing. The original file Hash features, file-level Hash features, code segment-level Hash features, a sh features.

[0061] like Figure 5 As shown, the open source license and copyright information tampering detection system 100 according to an embodiment of the present invention includes: a first knowledge base creation module 10, a homology relationship judgment module 20 and a license and copyright verification module 30.

[0062] Among them, the first knowledge base creation module 10 is used to establish a homology analysis feature knowledge base;

[0063] Specifically, the first knowledge base creation module 10 can rely on the collected and organized open source codes and establish a homology analysis feature knowledge base through a lexical analyzer to support homology detection of files to be detected.

[0064] The homology relationship judgment module 20 is used to match the feature information of the file to be detected with the feature information in the homology analysis feature knowledge base, and judge whether the file to be detected has a homology relationship with the file in the feature information knowledge base;

[0065] The license and copyright verification module 30 is used to record the homologous files when it is determined that the file to be detected has a homologous file, and to match and verify the license and copyright information of the file to be detected with the license and copyright information of the homologous file, to determine whether the license and copyright information in the file to be detected has been tampered with, deleted or added.

[0066] The open source license and copyright information tampering detection system 100 according to the embodiment of the present invention solves the problem of low efficiency and high false alarm rate in the related art of using manual processing for mixed-source license tampering detection, and realizes automatic detection of open source license and copyright information tampering, reduces costs and false alarm rates, and improves efficiency, detection rate and detection quality.

[0067] According to some embodiments of the present invention, the system further comprises: a second knowledge base creation module, which is used to establish a license and copyright information knowledge base to store basic information and clause information of the open source license and copyright.

[0068] It should be noted that, when establishing the license and copyright information knowledge base, the second knowledge base creation module can collect and organize the basic information of the license and the corresponding terms information, and establish the license and copyright information knowledge base.

[0069] In some embodiments of the present invention, system 100 also includes: an extraction module, which is used to match and verify the license and copyright information of the file to be detected with the basic information and terms information of the open source license and copyright in the license and copyright information knowledge base when matching and verifying the license and copyright information of the file to be detected.

[0070] According to some embodiments of the present invention, feature information in the homology analysis feature knowledge base includes: original file-level features, file-level features, and code snippet-level features.

[0071] The homology relationship judgment module 20 includes: a first judgment module, a second judgment module and a third judgment module.

[0072] The first judgment module is used to match the file to be detected with the original file-level features in the homology analysis feature knowledge base, and determine whether the file to be detected has a homology relationship file with the same source as the original file level;

[0073] The second judgment module is used to match the file to be detected with the file-level features in the homology analysis feature knowledge base, and determine whether the file to be detected has a homology relationship file with file-level homology;

[0074] The third judgment module is used to match the file to be detected with the code snippet-level features in the homology analysis feature knowledge base, and determine whether there is a homology relationship file with code snippet-level homology in the file to be detected.

[0075] In some embodiments of the present invention, at least one of the original file level feature, the file level feature and the code snippet level feature is a Hash feature.

[0076] It should be noted that when extracting original file-level features, file-level features and code segment-level features, a Hash algorithm can be used to process original file Hash features, file-level Hash features and code segment-level Hash features.

[0077] The following describes in detail a method for detecting tampering of open source license and copyright information according to the present invention with reference to the accompanying drawings in a specific embodiment. It is worth noting that the following description is merely an exemplary description and should not be construed as a specific limitation on the present invention.

[0078] like Figure 2 As shown, the invention mainly includes five steps to realize intelligent detection of mixed-source software open source license and protocol tampering:

[0079] A100, establish a knowledge base of document homology analysis features;

[0080] When establishing the file homology analysis feature knowledge base, we mainly rely on the collected and organized open source code, and use the lexical analyzer to establish a source code feature knowledge base to support file homology detection. The original file Hash features, file-level Hash features, and code snippet-level Hash features will be extracted respectively.

[0081] A200, establish a knowledge base of basic license information and terms information;

[0082] When establishing a knowledge base of basic license information and terms information, the main task is to collect and organize the basic license information and corresponding terms information, and establish a corresponding knowledge base.

[0083] A300, using homology detection tools to determine the homology relationship of files;

[0084] Based on the homology detection tool and the file homology feature knowledge base, the file homology detection tool is used to identify whether the files in the mixed-source software have a homology relationship with the open source files, and the corresponding results are given, such as original level homology (exactly the same), file level homology (the overall file structure is consistent), and code snippet level homology (the file structure is consistent at the code snippet level). The specific process is as follows Figure 3 shown.

[0085] A400, extracts the open source license information and version information of the file to be detected and the same source file respectively;

[0086] Based on the results of the three tests in step A300, the files with homology relationship are identified, and the license information and copyright information of the files to be tested and the open source files are extracted respectively through the file license and copyright information extractor. The specific process is as follows: Figure 4 shown.

[0087] A500,compares license and copyright tampering relationships between files with homology relationships.

[0088] Through the license and copyright information extracted in step A400, corresponding comparison and analysis are performed to determine whether there are tampering, addition, deletion and other operations on the license and copyright information of the files and open source files in the mixed-source software.

[0089] In this step, the license and copyright information extracted in step 4 are used to identify whether there is any inconsistency between the file to be detected and the open source file with the same source relationship. The inconsistency is divided into four categories: tampering, deletion, addition, and unchanged. The corresponding situations and identification results are divided into the following 16 categories:

[0090] 1) New license, new copyright; 2) New license, unchanged copyright; 3) New license, modified copyright; 4) New license, deleted copyright; 5) Unchanged license, new copyright; 6) Unchanged license, unchanged copyright; 7) Unchanged license, modified copyright; 8) Unchanged license, deleted copyright; 9) Modified license, new copyright; 10) Modified license, unchanged copyright; 11) Modified license, modified copyright; 12) Modified license, deleted copyright; 13) Deleted license, new copyright; 14) Deleted license, unchanged copyright; 15) Deleted license, modified copyright; 16) Deleted license, deleted copyright.

[0091] In summary, the present invention relies on a homology feature knowledge base to automatically identify open source components in mixed-source software, identify files with homology relationships, intelligently extract license and copyright information from related files through a license and copyright information extractor, and respectively compare the license and copyright information in files with homology relationships to identify whether there are additions, deletions, tampering, etc. in the license and copyright information of the coded files in the mixed-source software and the open source files in the open source knowledge base. Except for the interpretation of the basic license information and terms information and the establishment of the knowledge base, other processes are all automated. Through the present invention, the detection of license and copyright information tampering in mixed-source software is intelligent, which will also bring the following benefits:

[0092] It realizes automatic detection of tampering of open source license and copyright information, reduces costs and false alarm rates, and improves efficiency, detection rate, and detection quality.

[0093] Through the description of the specific implementation methods, a deeper and more specific understanding of the technical means and effects adopted by the present invention to achieve the predetermined purpose should be obtained. However, the accompanying drawings are only for reference and illustration purposes and are not intended to limit the present invention.

Claims

1. A method for detecting tampering of open source license and copyright information, characterized in that: include: Establish a knowledge base of homology analysis features; Matching the feature information of the file to be detected with the feature information in the homology analysis feature knowledge base to determine whether the file to be detected has a homology relationship with the file in the feature information knowledge base; If it is determined that the file to be detected has a homologous file, the homologous file is recorded, and the license and copyright information of the file to be detected is matched and verified with the license and copyright information of the homologous file, to determine whether the license and copyright information in the file to be detected has been tampered with, deleted, or added; The feature information in the homology analysis feature knowledge base includes: original file level features, file level features and code snippet level features, and matching the file to be detected with the feature information in the homology analysis feature knowledge base to determine whether the file to be detected is a file with a homology relationship, including: Matching the file to be detected with the original file-level features in the homology analysis feature knowledge base to determine whether the file to be detected has a homology relationship file with the same source as the original file level; Matching the file to be detected with the file-level features in the homology analysis feature knowledge base to determine whether the file to be detected has a homology relationship file with file-level homology; Matching the file to be detected with the code snippet-level features in the homology analysis feature knowledge base to determine whether the file to be detected has a homology relationship file with the same source at the code snippet level; Specifically, extract the MD5 feature of the original file, perform clone collision retrieval, and determine whether the collision is successful. If successful, the file is characterized as being homologous to the original file; if unsuccessful, determine the programming language to which the file belongs by the file suffix, and call the corresponding lexical analyzer for structural preprocessing; extract the file-level feature MD5 of the preprocessed text, perform file-level clone detection, and continue to determine whether the collision is successful. If successful, the file is characterized as being homologous at the file level; if unsuccessful, perform code-level feature extraction through a code-level feature extraction algorithm, and perform code-level detection; continue to determine whether the collision is successful. If successful, the file is characterized as being homologous at the code snippet level; if unsuccessful, the file is characterized as being original; At least one of the original file-level feature, the file-level feature and the code snippet-level feature is a Hash feature; Before performing a match check on the license and copyright information of the file to be detected, the method further includes: Establish a license and copyright information knowledge base to store basic information and terms of open source licenses and copyrights.

2. The method for detecting tampering of open source license and copyright information according to claim 1, characterized in that: When matching and verifying the license and copyright information of the file to be detected, the license and copyright information of the file to be detected is matched and verified with the basic information and clause information of the open source license and copyright in the license and copyright information knowledge base.

3. A system for detecting tampering of open source license and copyright information, characterized in that: include: A first knowledge base creation module is used to establish a homology analysis feature knowledge base; A homology relationship judgment module, used to match the feature information of the file to be detected with the feature information in the homology analysis feature knowledge base, and judge whether the file to be detected has a homology relationship with the file in the feature information knowledge base; The license and copyright verification module is used to record the homologous files when it is determined that the file to be detected has homologous files, and to match and verify the license and copyright information of the file to be detected with the license and copyright information of the homologous files, and to determine whether the license and copyright information in the file to be detected has been tampered with, deleted or added; The feature information in the homology analysis feature knowledge base includes: original file level features, file level features and code snippet level features. The homology relationship judgment module includes: A first judgment module is used to match the file to be detected with the original file-level features in the homology analysis feature knowledge base to determine whether the file to be detected has a homology relationship file with the same source as the original file level; A second judgment module is used to match the file to be detected with the file-level features in the homology analysis feature knowledge base to determine whether the file to be detected is a homology relationship file with file-level homology; A third judgment module is used to match the file to be detected with the code snippet-level features in the homology analysis feature knowledge base to determine whether the file to be detected is a homology relationship file with code snippet-level homology; At least one of the original file-level feature, the file-level feature and the code snippet-level feature is a Hash feature; specifically, the MD5 feature of the original file is extracted, a clone collision search is performed, and it is determined whether the collision is successful. If successful, the file is characterized as being homologous to the original file; if unsuccessful, the programming language to which the file belongs is determined by the file suffix, and the corresponding lexical analyzer is called for structural preprocessing; the file-level feature MD5 is extracted from the preprocessed text, and a file-level clone detection is performed to continue to determine whether the collision is successful. If successful, the file is characterized as being homologous to the file level; if unsuccessful, code-level feature extraction is performed through a code-level feature extraction algorithm, and code-level detection is performed; it is continued to determine whether the collision is successful. If successful, the file is characterized as being homologous to the code snippet level; if unsuccessful, the file is characterized as being an original file; The system further comprises: The second knowledge base creation module is used to establish a license and copyright information knowledge base to store the basic information and terms information of the open source license and copyright.

4. The open source license and copyright information tampering detection system according to claim 3, characterized in that: The system also includes: an extraction module, which is used to match and verify the license and copyright information of the file to be detected with the basic information and terms information of the open source license and copyright in the license and copyright information knowledge base when matching and verifying the license and copyright information of the file to be detected.

Images