A cross-language and multi-scenario open source license compatibility detection system

Abstract

The present invention discloses a cross-language and multi-scenario open source license compatibility detection system. Through fine-grained license feature fusion and extraction, the license information can be parsed and represented more accurately, thereby improving the accuracy of subsequent compatibility detection. At the same time, license propagation calculation and conflict detection based on graph data structure effectively improve detection efficiency. Through the license reasoning module, the compatibility relationship of licenses is automatically compared and calculated, realizing license compatibility checking of complex software components and their interoperability in more complex, cross-language and multi-scenario large-scale open source or mixed source code projects.

Description

Technical Field

[0001] The present invention belongs to the technical field of open source compliance and trusted governance, and specifically relates to a cross-language and multi-scenario open source license compatibility detection system. Background Art

[0002] License compatibility checking is a crucial step in software development and project management. This process usually includes two core steps: first, identifying the various licenses introduced into the project; second, judging whether they can coexist harmoniously based on the compatibility between these licenses. Traditional checking methods mainly focus on a part of widely used licenses and rely on predefined license compatibility rules for verification. In such methods, the results of license identification are usually presented as a certain type of license ID.

[0003] However, with the increasing popularity of open source software and custom licenses, the limitations of this traditional approach have become increasingly apparent. In recent years, in order to adapt to a wider range of licenses, researchers have begun to explore new methods. One idea is to convert the license into a set of legal terms and dynamically determine the compatibility between licenses by automatically analyzing these terms. For example, some methods not only analyze the compatibility of licenses by automatically extracting legal terms, but also further consider the widespread problem of custom licenses. This method uses a license term recognition model to extract key legal terms, and based on these terms, it determines whether there are potential conflicts in the terms of the licenses in the project, which improves the flexibility and accuracy of license compatibility checks and provides a new solution for dealing with complex licensing environments.

[0004] Although these methods have made certain technical progress, there are still many challenges in practical applications in cross-language and multi-scenario situations, such as the diversity and complexity of custom licenses, and the differences in the rights and obligations requirements of legal terms in different usage scenarios. Specifically, the traditional method of structuring legal terms has significant limitations in its ability to express information, which makes it difficult for general license compatibility detection methods to fully and accurately capture the deep semantics of complex and ambiguous legal terms. This limitation has led to a series of problems: it is impossible to accurately analyze the compatibility status of license exceptions under different conditions, resulting in a large number of false alarms. In addition, license compatibility analysis in real scenarios is more complicated. Different licenses show very different compatibility characteristics in various combinations and dependencies. Past studies often explore license compatibility based on specific usage scenario assumptions. This method is incapable of dealing with the complex compatibility scenarios brought about by the interoperability of different programming languages. Especially in large open source or mixed source code projects, when many components using different programming languages ​​and interoperability methods are jointly built into a product, existing methods are difficult to cope with the differences in compatibility conclusions under different usage scenarios. Summary of the invention

[0005] In view of this, the present invention provides a cross-language and multi-scenario open source license compatibility detection system, which realizes the license compatibility detection in cross-language and multi-scenario.

[0006] The present invention provides a cross-language and multi-scenario open source license compatibility detection system, which includes a license compatibility data layer, a license scanning layer, and a license compatibility detection layer. The license compatibility data layer is used to store and maintain basic data required for license detection; the license scanning layer is used to scan all licenses in the project source code and parse software components and dependencies; the license compatibility detection layer is used to perform compatibility detection;

[0007] The license compatibility data layer includes a license compatibility knowledge graph module, a fine-grained license feature fusion module, a fine-grained license feature extraction module, and a license reasoning module; the license scanning layer includes a license scanning module and a source code interoperability information parsing module; the license compatibility detection layer includes an interoperability interaction graph construction module, a license propagation calculation module, and a license compatibility analysis module;

[0008] During operation, the source code interoperability information parsing module obtains initial interoperability interaction information from the source code to be analyzed, and the license scanning module obtains the first interoperability information and the first license information from the initial interoperability interaction information; the license compatibility knowledge graph module determines whether the first license information already exists in the license knowledge graph. If not, the fine-grained structured extraction module extracts the structured features of the first license information, and then the knowledge graph feature fusion module updates the feature semantic relationship in the first license information. Finally, the license reasoning module infers the first license compatibility relationship between the first license information and other licenses in the knowledge graph based on the updated feature semantic relationship; if it exists, the license ID is returned. The interoperability interaction graph construction module constructs a first interoperability interaction graph based on the first interoperability information and the first license information; the license propagation calculation module calculates the license propagation result based on the interaction path of the first interoperability interaction graph, and the license compatibility analysis module calculates the conflicting nodes in the first interoperability interaction graph based on the first license compatibility relationship.

[0009] Furthermore, the license compatibility knowledge graph module creates a knowledge graph based on predefined clause semantic concept triples, uses a large language model combined with manual auditing to determine the relationship between the newly added semantic concepts and the semantic concepts corresponding to each entity in the existing knowledge graph, and updates the feature knowledge graph.

[0010] Furthermore, the clause semantic concept triple is composed of authorized work, authorized source code, and the relationship between the authorized work and the authorized source code.

[0011] Furthermore, the relationship between the authorized work and the authorized source code includes equivalence, inclusion and mutual exclusion.

[0012] Furthermore, the fine-grained license feature extraction module constructs predefined clause phrases to refer to clauses of different licenses with different grammars but the same semantics according to the characteristics of the license, and then extracts and models the effective scope of the clause statements.

[0013] Furthermore, the source code interoperability information parsing module also includes a compile and build configuration parsing module, which is used to obtain the compile and build dependency information of the source code to be analyzed that has a compile and build process.

[0014] Beneficial effects:

[0015] The present invention can parse and represent license information more accurately through fine-grained license feature fusion and extraction, thereby improving the accuracy of subsequent compatibility detection. At the same time, the license propagation calculation and conflict detection based on the graph data structure effectively improve the detection efficiency. The license reasoning module automatically compares and calculates the compatibility relationship of licenses, thereby realizing license compatibility checks on complex software components and their interoperability in more complex, cross-language and multi-scenario large-scale open source or mixed source code projects. BRIEF DESCRIPTION OF THE DRAWINGS

[0016] Figure 1 A schematic diagram of the system architecture of a cross-language and multi-scenario open source license compatibility detection system provided by the present invention.

[0017] Figure 2 A schematic diagram of the processing flow of a cross-language and multi-scenario open source license compatibility detection system provided by the present invention. DETAILED DESCRIPTION

[0018] The present invention is described in detail with reference to the following embodiments.

[0019] The present invention provides a cross-language and multi-scenario open source license compatibility detection system, the system architecture is as follows: Figure 1 As shown, it includes a license compatibility data layer, a license scanning layer and a license compatibility detection layer.

[0020] The license compatibility data layer is used to store and maintain the basic data required for license detection, including the license compatibility knowledge graph module, the fine-grained license feature fusion module, the fine-grained license feature extraction module and the license reasoning module. The license compatibility knowledge graph module is used to store license structured information and license compatibility knowledge, the fine-grained license feature fusion module is used to store the same entity concepts referred to in different legal clauses in a unified form, the fine-grained license feature extraction module is used to extract structured clauses T and the scope of effectiveness of the clauses S(T), and the license reasoning module automatically compares the extracted license structured information with the structured information of other licenses to calculate the compatibility relationship between the two.

[0021] The license compatibility knowledge graph module uses a set of legal terms, rights, obligations, and restrictions to represent the license. For example, a license authorizes the user to use the related work commercially, which is recorded as T = (authorization, commercial use). On this basis, the effective scope of each legal term element T is recorded as Where A(T) is a set, which indicates the scope of application of the clauses explicitly mentioned in the original text of the legal clause; O(T) is a set, which indicates the scope of inapplicability of the clauses explicitly mentioned in the original text of the legal clause, and * is the Cartesian product. In addition, for the license compatibility knowledge graph module, the present invention models license compatibility through three relationships: unconditional compatibility, conditional compatibility, and incompatibility.

[0022] The fine-grained license feature fusion module mainly enables the concepts involved in different license terms to maintain the same semantic mapping relationship, thereby ensuring that the extraction of the fine-grained feature extraction module is meaningful. For example, the noun "work" is used in license A to refer to the authorized work, while license B uses "source code" to refer to the authorized source code. The inclusion relationship between "work" and "source code" can be stored in the knowledge graph in the form of a triple, that is, (work, inclusion, source code). The present invention organizes and stores the equivalence, inclusion, and mutual exclusion relationships that may exist between the semantic concepts referred to by legal terms in the form of a knowledge graph, and the fine-grained license feature fusion module maintains the semantic relationship between these concepts on this basis. Specifically, given a set of semantic concept triplets of terms predefined by experts in the field of open source licenses, a knowledge graph is created, and the relationship between the newly added semantic concepts and the corresponding semantic concepts of each entity in the existing knowledge graph is judged by a large language model combined with a manual audit method to update the feature knowledge graph, thereby completing the license feature fusion.

[0023] The fine-grained license feature extraction module constructs some predefined and determined clause phrases according to the characteristics of the license to refer to the clauses of different licenses with different grammars but the same semantics. For example, although the specific declaration statements of License A and License B are different, they both essentially indicate that the authorized work is allowed to be used for commercial activities, which is recorded as T1 = (authorization, commercial use); or there is a legal clause in License A requiring the user to open source related work, which is recorded as T2 = (must, open source). In this paper, T i It is called structured clause. On this basis, the present invention further iThe extraction modeling is carried out within what scope. For example, the LGPLv2.1 license requires that all derivative works based on its authorized work must be authorized with the same license in most cases, which is recorded as T = (must, same license). However, when its authorized work is used as a library to form a larger work in a dynamic linking manner, it is not necessary to follow the constraint of using the same license. At this time, the scope of effectiveness of the same license clause of LGPLv2.1 is represented by the scope of effectiveness expression method proposed by the present invention, S(T) = A(T)*O(T), where A(T) = {U}, U is the full set, indicating that the scope of effectiveness is effective under any conditions / usage scenarios; O(T) = {"dynamic link"} indicates that the scope of effectiveness is not effective under the "dynamic link" condition.

[0024] The license reasoning module automatically compares the extracted license structured information with the structured information of other licenses to calculate the compatibility relationship between the two.

[0025] The license scanning layer is responsible for scanning all licenses in the project source code and parsing the software components and dependencies of the project, including the license scanning module and the source code interoperability information parsing module. Among them, the license scanning module is used to identify the license, and the source code interoperability information parsing module includes parsing the software components and dependencies of the project. In the language with a compilation and build process, the source code interoperability information parsing module also includes parsing the compilation and build configuration module, which is used to obtain the compilation and build dependency information.

[0026] The license compatibility detection layer is used to execute the compatibility detection process, including an interoperability interaction graph construction module, a license propagation calculation module and a license compatibility analysis module.

[0027] The interoperability interaction graph construction module constructs the source code interoperability information and license information obtained by the license scanning layer into a graph data structure, denoted as G = (V, E), where V represents the set consisting of nodes in the graph, and E represents the set consisting of edges between nodes in the graph; the nodes and edges in the interoperability interaction graph of different programming languages ​​will also vary. For example, in the analysis of C / C++ projects, V represents the source code files in the project, and E represents the compilation and construction relationship between files; in Python-like scripting language projects, V represents dependent packages, and E represents the dependency relationship between packages.

[0028] The license propagation calculation module calculates the license propagation results on the interoperability interaction graph. i ∈V originally has a license statement, such as v i It is a C language source code file which declares the license A. The license is packaged to the upstream node v through the compilation and construction edge. j, then license A is obtained from v through interoperability i Spread to v j This is a promotion of the license contagion of the widely existing Copyleft licenses in the real world. License contagion means that in the process of copying, modifying, processing, reprinting, and displaying open source software, the effectiveness of the open source license is often continuous, and the authorization and restrictions of the license can be vertically transmitted to the derivative works developed based on itself, its own modified versions, and even horizontally to other parts of the software developed based on it. The present invention calculates the license propagation results from the bottom to the bottom along the path between the nodes of the interaction graph, and synchronously updates the licenses of the nodes in the interaction graph on the path.

[0029] The license propagation path in the present invention is: for a given interaction graph G (ensuring that graph G is a directed acyclic graph DAG), traverse the nodes according to the anti-topological sorting algorithm, and the traversal order is the propagation path order in this method.

[0030] The license compatibility analysis module calculates whether there is a license conflict based on the license propagation results of each node in the graph and the license data layer. The compatibility of open source licenses can be defined as follows: if license A is compatible with B, then B can combine or re-license its authorized works without violating the requirements of A. Otherwise, A and B are considered to be in conflict.

[0031] The present invention constructs a cross-language and multi-scenario open source license compatibility detection system, and the actual working process is as follows: Figure 2 As shown in the figure, after the system is started, the source code is first obtained, and then the initial interoperability information is obtained through the source code interoperability information parsing module of the system scanning layer, and then the license information is obtained through the license scanning module. In this process, each module of the license compatibility data layer will maintain the license compatibility data. Specifically, the system will check whether the scanning result of the license scanning module can match the license information already existing in the license knowledge graph. If the license does not exist, its structured features will be extracted through the fine-grained structured extraction module based on the knowledge graph, and then the feature semantic relationship will be updated through the knowledge graph feature fusion module. Finally, the license compatibility reasoning module obtains the compatibility relationship between the custom license and other licenses in the knowledge graph based on the updated feature semantic reasoning. The license compatibility detection layer completes the compatibility detection process of the project license. First, the interoperability interaction graph construction module constructs the interoperability interaction graph based on the interoperability information and license information obtained by the scanning layer. Secondly, the license propagation module calculates the process of license propagation on the constructed graph. Finally, the license analysis module combines the license compatibility relationship maintained by the license compatibility data layer to calculate whether the license conflict occurs at each node in the graph.

[0032] The core improvement of the present invention lies in its fine-grained license feature fusion and extraction technology, as well as license propagation calculation and conflict detection technology based on graph data structure. These technologies together ensure the efficiency and accuracy of the system and provide strong support for license management in the software development process.

[0033] In summary, the above are only preferred embodiments of the present invention and are not intended to limit the protection scope of the present invention. Any modifications, equivalent substitutions, improvements, etc. made within the spirit and principles of the present invention should be included in the protection scope of the present invention.

Claims

1. A cross-language and multi-scenario open source license compatibility detection system, characterized in that: It includes a license compatibility data layer, a license scanning layer and a license compatibility detection layer. The license compatibility data layer is used to store and maintain the basic data required for license detection; the license scanning layer is used to scan all licenses in the project source code and parse the software components and dependencies; the license compatibility detection layer is used to perform compatibility detection; The license compatibility data layer includes a license compatibility knowledge graph module, a fine-grained license feature fusion module, a fine-grained license feature extraction module, and a license reasoning module; the license scanning layer includes a license scanning module and a source code interoperability information parsing module; the license compatibility detection layer includes an interoperability interaction graph construction module, a license propagation calculation module, and a license compatibility analysis module; During operation, the source code interoperability information parsing module obtains initial interoperability interaction information from the source code to be analyzed, and the license scanning module obtains the first interoperability information and the first license information from the initial interoperability interaction information; the license compatibility knowledge graph module determines whether the first license information already exists in the license knowledge graph. If not, the fine-grained structured extraction module extracts the structured features of the first license information, and then the knowledge graph feature fusion module updates the feature semantic relationship in the first license information. Finally, the license reasoning module infers the first license compatibility relationship between the first license information and other licenses in the knowledge graph based on the updated feature semantic relationship; if it exists, the license ID is returned. The interoperability interaction graph construction module constructs a first interoperability interaction graph based on the first interoperability information and the first license information; the license propagation calculation module calculates the license propagation result based on the interaction path of the first interoperability interaction graph, and the license compatibility analysis module calculates the conflicting nodes in the first interoperability interaction graph based on the first license compatibility relationship.

2. The open source license compatibility detection system according to claim 1, characterized in that: The license compatibility knowledge graph module creates a knowledge graph based on predefined clause semantic concept triples, uses a large language model combined with manual auditing to determine the relationship between the newly added semantic concepts and the semantic concepts corresponding to each entity in the existing knowledge graph, and updates the feature knowledge graph.

3. The open source license compatibility detection system according to claim 2, characterized in that: The clause semantic concept triple is composed of authorized work, authorized source code and the relationship between the authorized work and the authorized source code.

4. The open source license compatibility detection system according to claim 3, characterized in that: The relationship between the authorized work and the authorized source code includes equivalence, inclusion, and mutual exclusion.

5. The open source license compatibility detection system according to claim 1, characterized in that: The fine-grained license feature extraction module constructs predefined clause phrases to refer to clauses of different licenses with different grammars but the same semantics according to the characteristics of the license, and then extracts and models the effective scope of the clause statements.

6. The open source license compatibility detection system according to claim 1, characterized in that: The source code interoperability information parsing module also includes a compile and build configuration parsing module, which is used to obtain the compile and build dependency information of the source code to be analyzed that has a compile and build process.

Images