A Method for Detecting Open Source Software License Terms Based on Graph Neural Networks

Abstract

This invention discloses a method for detecting open-source software license terms based on graph neural networks. The method includes the following steps: S1. Preprocessing the given license document to be analyzed to obtain a simplified list of sentence lexical units; S2. Initializing and constructing a structure graph corresponding to each sentence to obtain a list of structure graphs; S3. Initializing and constructing a content graph corresponding to each sentence to obtain a list of content graphs; S4. Connecting the structure graph and content graph corresponding to each sentence by adding a common root node to create a combination graph corresponding to that sentence, ultimately obtaining a list of combination graphs; S5. Traversing the list of combination graphs, constructing a graph neural network, calculating the attention coefficient between any two nodes in each combination graph, and finally obtaining and outputting a result list. The method of this invention can automatically, efficiently, and accurately identify and extract sentences related to specific terms in a given open-source software license document.

Description

Technical Field

[0001] This invention relates to the field of software engineering technology, and in particular to an open-source software license detection method based on graph neural networks, which can be used to identify and extract sentences related to the license terms in open-source software license documents. Background Technology

[0002] Open source software has become the cornerstone of modern software development, with over 90% of commercial products relying on open source software components. Licenses are the legal framework governing the use, modification, distribution, and copying of open source software. They define key license terms such as copyright, patent rights, distribution rules, and disclaimers, balancing the rights of users and developers while protecting the intellectual property rights of authors. Each license term can vary significantly between different licenses, ranging from the MIT license, which allows widespread use with minimal obligations, to the GNU GPL, which requires derivative works to share the same terms. Therefore, to avoid legal disputes, the most crucial step before using any open source software is to extract and understand the details of these license terms from the license document.

[0003] However, mature commercial software often relies on more than one open-source software component, and each license document contains thousands of sentences, not to mention that only a portion of them may relate to license terms. This complexity makes manual analysis impractical. Furthermore, licenses may be customized or modified by developers, further complicating the analysis process. Effectively extracting and verifying the various clauses concerning rights and obligations in a license requires significant human and material resources, and accuracy cannot be guaranteed. Therefore, as the open-source ecosystem grows, developing an automated method to automatically identify and extract all sentences related to license terms from license documents has become a pressing need.

[0004] However, existing technologies still have many shortcomings. First, some existing methods approach the problem from a pattern-based or similarity-based analysis perspective, using predefined keyword patterns and regular expressions to identify terms in segmented license sentences, or first using word embedding or document embedding techniques to convert sentences or documents into vectors, and then identifying sentences related to license terms by comparing the similarity between the labeled vectors. For example, in the existing technology "An Automatic Extraction Method for Open Source Software License Terms" (application number CN202110619980.3), the LDA model is first used to map terms to topics, then sentences are clustered by topic to form topic documents, and then document embedding technology (Doc2Vec) is used to vectorize the documents and compare similarity to determine the similarity between new input terms and historical topics. These methods cannot deeply analyze the structure and semantics of sentences in the license, so the accuracy of the identified license terms cannot be guaranteed in principle.

[0005] Secondly, to address the shortcomings of the aforementioned methods, some researchers have proposed ontology-based approaches, aiming to model licenses in a structured and semantically rich manner. Ontology-based methods typically extract structured tuples (such as subjects, behaviors, and objects) from the license text, providing a formal framework to represent the relationships between entities, thereby capturing the complexity of license terms and their interactions as much as possible. For example, the existing technology "License Selection Method and System Based on Open Source Software Graph" (application number CN202010983623.0) treats the software project to be licensed as an entity and license-related information as attributes, constructs triples, and stores them in the open source software graph. Based on this, license selection is performed using similarity, license-related attributes from the open source software graph, and software constraints. However, such methods often rely on manually extracting tuples from the license text, which is time-consuming, labor-intensive, and prone to human error, introducing unnecessary risks.

[0006] Furthermore, existing methods primarily focus on a small number of commonly used licenses during model training, making it difficult to effectively extend their methodology to less common or emerging licenses. For example, in the existing technology "An Automatic Extraction Method for Open Source Software License Terms" (application number CN202110619980.3), the dataset used for model training consists of 419 publicly available licenses, which do not contain non-English text. When the input license document type has not appeared in the dataset, the effectiveness cannot be guaranteed. Summary of the Invention

[0007] To address the problems existing in the prior art, the present invention aims to provide an open-source software license detection method based on graph neural networks. In order to efficiently and accurately detect the license terms of open-source projects, ontology-based methods rely on manual predefinition or extraction of tuple data, which is time-consuming, labor-intensive, and prone to human error. The present invention introduces and designs graph neural networks to establish a relationship model between license document sentences and license terms, and proposes an automated generation method for structural graphs (focusing on understanding the syntactic structure of sentences) and content graphs for graph neural networks (focusing on extracting important words and phrases that convey the core meaning of license terms).

[0008] To achieve the above objectives, this invention provides a method for detecting open-source software license terms based on graph neural networks, the method comprising the following steps:

[0009] S1. The user inputs the license document to be analyzed from the client. The server calls the text preprocessing module to perform text preprocessing on the given license document to be analyzed, and obtains a simplified list of sentence morphemes.

[0010] S2. Call the structure graph generation module, traverse the simplified sentence word list, initialize and construct the structure graph corresponding to each sentence, which is used to assist the graph neural network in capturing and analyzing the structural information of the license document statements, and finally obtain the list of structure graphs.

[0011] S3. Traverse the simplified sentence word list, initialize and construct the content graph corresponding to each sentence, which is used to assist the graph neural network in capturing and analyzing the content information of the license document statements, and finally obtain the content graph list;

[0012] S4. Traverse the list of structure diagrams and the list of content diagrams. By adding a common root node, connect the structure diagram and content diagram corresponding to each sentence to create the combined diagram corresponding to the sentence, and finally obtain the list of combined diagrams.

[0013] S5. Call the module for building graph neural networks and graph-level representation vectors, traverse the list of combined graphs, build a graph neural network, calculate the attention coefficient between any two nodes in each combined graph, and finally generate a 24-dimensional graph-level representation vector for each input combined graph to represent the predicted probability of the sentence corresponding to the graph being associated with or not associated with any of the 24 predefined specific license terms. Finally, obtain the result list and output it to the client.

[0014] Furthermore, in step S1, the text preprocessing includes segmentation, lexicalization, and simplification.

[0015] Furthermore, in step S2, the structural graph focuses on understanding the syntactic structure of a sentence and identifies its grammatical framework by applying graph embedding technology.

[0016] Furthermore, in step S3, the content graph focuses on extracting words and phrases that convey the core meaning of the license terms.

[0017] Furthermore, in step S5, the result list is a list of whether each sentence in the given license document to be analyzed corresponds to and which predefined license clause it corresponds to.

[0018] Furthermore, step S1 specifically includes the following steps:

[0019] S1-1. The text preprocessing module uses CoreNLP to segment all the text contained in the license document to be analyzed into several sentences, which are then stored in a list to obtain a list of sentence texts.

[0020] S1-2. Traverse the list of text sentences, perform word segmentation on each sentence, convert the words in the sentence into lexical units, and obtain the sentence lexical list corresponding to each sentence;

[0021] S1-3. Traverse the sentence lexicon list. For each list element, use Porter's stemming method to remove all stop words from the lexicon list. Then, convert each remaining lexicon into its corresponding root form to obtain the simplified sentence lexicon list.

[0022] Furthermore, step S3 specifically includes the following steps:

[0023] S3-1. Select the license terms and divide them into two categories: rights and obligations;

[0024] S3-2. Based on the user's actual needs, mark all sentences in any given historical license documents that are related to the main license terms defined in step S3-1. The marking content includes whether it is related to the license terms and which license terms it is related to, and obtain a list of marked documents containing several marked documents;

[0025] S3-3. Use It refers to all sentences in the documents contained in the marked document list that are related to a specific license term, and sets three indicators: average frequency of occurrence, specificity, and breadth;

[0026] S3-4. Traverse the sentence word list, iterating through each word in a sentence, and calculating the importance value of each word according to the three set indicators.

[0027] S3-5. Traverse the sentence word list. For each sentence, based on the importance value of each word obtained in step S3-4, construct its corresponding content graph on a sentence-by-sentence basis. For each sentence, define and calculate the following three types of feature values: important words, important phrases, and sentence length.

[0028] S3-6. For each sentence in the license document to be analyzed, treat its three types of features—important words, important phrases, and sentence length—as separate nodes and connect them sequentially with edges. Initialize the weight of each edge to 1 to obtain the corresponding content graph. Finally, obtain the list of content graphs corresponding to each sentence.

[0029] Furthermore, step S5 specifically includes the following steps:

[0030] S5-1. Traverse the list of combinatorial graphs and construct a graph neural network, which consists of three parts: encoder, propagation layer and aggregator; first, the encoder uses a multilayer perceptron to convert the initial values ​​of nodes and edges in the combinatorial graph into initial vectors for each input.

[0031] S5-2. The propagation layer updates the node vector by mapping the old vector to the new vector; by stacking multiple propagation layers, each node can accumulate knowledge from its local neighborhood.

[0032] S5-3. The aggregator takes all the previously obtained updated node vectors as input to generate a graph-level representation vector;

[0033] S5-4. The graph neural network generates a 24-dimensional graph-level representation vector corresponding to the input composite graph, which represents the predicted probability of the sentence corresponding to the graph being associated with a specific license clause.

[0034] Furthermore, in addition to the 23 predefined license terms, the 24-dimensional structure includes an extra dimension to indicate whether the sentence is irrelevant to any of them.

[0035] Furthermore, the 23 predefined license terms are as follows, where rights are shown in R + number and obligations are shown in O + number:

[0036] R1) Distribute

[0037] R2) Modify

[0038] R3) Commercial Use

[0039] R4 Relicense

[0040] R5) Hold Liable

[0041] R6) Use Patent Claims

[0042] R7) Sublicense

[0043] R8)Statically Link

[0044] R9) Private Use

[0045] R10) Use Trademark

[0046] R11) Place Warranty

[0047] O1) Include Copyright

[0048] O2) Include License

[0049] O3) Include Notice

[0050] O4) Disclose Source

[0051] O5) State Changes

[0052] O6) Include Original

[0053] O7) Give Credit

[0054] O8) Rename

[0055] O9) Contact Author

[0056] O10) Include Install instructions

[0057] O11) Compensate Damages

[0058] O12) Pay Above Threshold.

[0059] The beneficial effects of this invention are as follows:

[0060] The method of this invention can automatically, efficiently, and accurately identify and extract sentences related to specific clauses in a given open-source software license document.

[0061] This invention establishes a relationship model between license document sentences and license terms by introducing and designing graph neural networks, and proposes an automatic generation method for structure graphs and content graphs for graph neural networks. This solves the problem that ontology-based methods require manual predefinition or extraction of tuple data, which is time-consuming, labor-intensive, and prone to human error.

[0062] This invention addresses the problem that pattern- or similarity-based methods cannot deeply analyze the sentence structure and semantics of licenses. It further integrates structure graphs and content graphs through a combined graph, ensuring that the graph neural network can simultaneously learn the structural knowledge (syntactic composition of sentences) and content knowledge (important words and phrases that convey the core meaning of license terms) corresponding to each sentence, thereby guaranteeing the accuracy of recognition and extraction results.

[0063] Regarding the generation of structural graphs, this invention also innovatively designs a dual-mechanism graph embedding technology customized for natural language. Specifically, it obtains local information of the sentence through a window capture mechanism, focusing on extracting information only within the local area (i.e., inside the window) each time; and further ensures the gradual propagation and aggregation of relationships between lexical units (i.e., graph nodes) through a sliding window mechanism, thereby enabling the structural graph to reflect the overall syntactic framework of the sentence and achieving a stronger ability to capture sentence structural knowledge.

[0064] Meanwhile, this invention has good scalability and versatility. The method framework is applicable to both uncommon and common licenses. Users only need to provide different historical license tag documents according to their actual needs to achieve compatibility with uncommon licenses. Attached Figure Description

[0065] Figure 1 This is a flowchart illustrating the open-source software license term detection method based on graph neural networks of the present invention.

[0066] Figure 2 This is a flowchart of the text preprocessing module in the open-source software license terms detection method based on graph neural networks of this invention;

[0067] Figure 3 This is a flowchart of the structure graph generation module in the open-source software license terms detection method based on graph neural networks of this invention;

[0068] Figure 4 This is a flowchart of the content graph generation module in the open-source software license terms detection method based on graph neural networks of the present invention;

[0069] Figure 5 This is a flowchart of the module for constructing a graph neural network and generating a graph-level representation vector in the open-source software license term detection method based on graph neural networks of this invention. Detailed Implementation

[0070] The technical solution of the present invention will now be clearly and completely described with reference to the accompanying drawings. Obviously, the described embodiments are only some, not all, of the embodiments of the present invention. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0071] In the description of this invention, it should be noted that the terms "center," "upper," "lower," "left," "right," "vertical," "horizontal," "inner," and "outer," etc., indicate the orientation or positional relationship based on the orientation or positional relationship shown in the accompanying drawings. They are used only for the convenience of describing the invention and for simplifying the description, and do not indicate or imply that the device or element referred to must have a specific orientation, or be constructed and operated in a specific orientation. Therefore, they should not be construed as limitations on the invention. Furthermore, the terms "first," "second," and "third" are used for descriptive purposes only and should not be construed as indicating or implying relative importance.

[0072] In the description of this invention, it should be noted that, unless otherwise explicitly specified and limited, the terms "installation," "connection," and "linking" should be interpreted broadly. For example, they can refer to a fixed connection, a detachable connection, or an integral connection; they can refer to a mechanical connection or an electrical connection; they can refer to a direct connection or an indirect connection through an intermediate medium; and they can refer to the internal connection of two components. Those skilled in the art can understand the specific meaning of the above terms in this invention according to the specific circumstances.

[0073] The following combination Figures 1-5 Specific embodiments of the present invention will be described in detail below. It should be understood that the specific embodiments described herein are for illustrative and explanatory purposes only and are not intended to limit the present invention.

[0074] Graph Neural Networks (GNNs) are deep learning models used to process graph-structured data. They capture dependencies in a graph and update node state representations through message passing between nodes. The core idea is to update node values ​​via message passing. Each node's value depends not only on its own features but also on the features of its neighboring nodes. In recent years, with the rapid development of artificial intelligence technology, GNNs have been widely applied in many fields such as social networks and e-commerce due to their excellent performance.

[0075] The technical problem to be solved by this invention is to identify and extract sentences related to the license terms in open source software license documents based on graph neural networks.

[0076] The technical solution of the present invention is as follows:

[0077] A method for detecting open-source software license terms based on graph neural networks, the specific steps of which are as follows:

[0078] Step S1. The user inputs the license document D to be analyzed from the client. license (PDF, TXT, or DOCX text formats are all acceptable.) A computer with general computing capabilities, pre-installed with a Python runtime environment and relevant third-party libraries, will be used as the server to receive data and perform all subsequent calculations and processing. First, the text preprocessing module will be called to analyze the license document D. license This module performs text preprocessing operations, including segmentation, lexicalization, and simplification, each implemented using corresponding functions from third-party libraries. The final result is a simplified sentence lexical list L. sen_token_simplify ;

[0079] Step S2. Call the structure graph generation module. This module's sub-functions include graph node initialization and node edge initialization. Traverse the simplified sentence lexical list L. sen_token_simplify Initialize and construct the structure graph G corresponding to each sentence. struct This is used to assist graph neural networks in capturing and analyzing the structural information of sentences in license documents. The structural graph focuses on understanding the syntactic structure of sentences and identifies their grammatical framework by applying graph embedding techniques. The final result is a list G of structural graphs. struct_list ;

[0080] Step S3. Traverse the simplified sentence word list L sen_token_simplifyThe content graph generation module is invoked to initialize and construct the content graph G corresponding to each sentence. content This is used to assist graph neural networks in capturing and analyzing the content information of license document statements. The content graph focuses on extracting key words and phrases that convey the core meaning of the license terms. The final result is a list of content graphs G. content_list ;

[0081] Step S4. Traverse the structure graph list G struct_list and content image list G content_list By adding a common root node, the structure graph G corresponding to each sentence is... struct and content image G content Connect them to create a combinatorial graph G corresponding to the sentence. mix This approach ensures that the graph neural network can simultaneously learn both the structural and content knowledge corresponding to each sentence. The final result is a combined graph list G. mix_list .

[0082] Step S5. Call the module for constructing a graph neural network and generating graph-level representation vectors, and traverse the list of combined graphs G. mix_list A graph neural network is constructed, consisting of three sub-functional parts: an encoder, a propagation layer, and an aggregator. The computation of each combined graph G... mix The attention coefficients between any two nodes in the graph are finally used to construct the combined graph G for each input. mix Generate a 24-dimensional graph-level representation vector (Vector) for each. graph_level This represents the predicted probability of the sentence corresponding to the graph being associated with any of the 23 predefined specific license terms, or not associated with any of them. The final result is a given license document D to be analyzed. license Output a list of results showing whether each sentence corresponds to a predefined license clause and which clause it corresponds to.

[0083] Furthermore, step S1 specifically includes the following steps:

[0084] S1-1. The text preprocessing module can be implemented by calling the Python third-party library CoreNLP. The corresponding functions provided in this library can be used to perform segmentation, lexicalization, and simplification sub-functions. First, the text segmentation function provided by the library is used to analyze the license document D. license The entire text is divided into several sentences and stored in a list, resulting in a sentence text list L. sen_text .

[0085] S1-2. Traverse the list of text sentences L text_senUsing the word segmentation and lexicalization functions provided in the library, each sentence is segmented and its words are lexicalized. Sentence-by-sentence lexical lists L are obtained for each sentence. sen_token .

[0086] S1-3. Traversing the sentence word list L sen_token For each element in the list, i.e., the list of lexical units corresponding to a single sentence, a simplification operation is performed using the Porter stemming function provided in the library. This involves removing all stop words and converting each remaining lexical unit into its corresponding root form. The final result is the simplified sentence lexical list L. sen_token_simplify .

[0087] Furthermore, step S2 specifically includes the following steps:

[0088] S2-1. Call the structure diagram generation module to generate the structure diagram G for each sentence. struct This module's sub-functions include graph node initialization and node edge initialization. First, it iterates through the simplified sentence lexical list L. sen_token_simplify Each element, i.e., a single sentence, is used to construct a corresponding structure graph G. struct Specifically, G struct = {V, E}. Where V contains all the nodes in the graph, representing the set of words in the sentence, with each word corresponding to a single node in the graph; E contains all the edges in the graph, representing the relationships between these words in the sentence.

[0089] S2-2. Call the graph node initialization sub-function to initialize the structure graph G corresponding to each sentence. struct The V (i.e., graph nodes) contained in the graph are initialized. To facilitate further in-depth analysis, a word embedding method is used to assign initial values ​​to each node. For the word corresponding to each node, a search is performed in the pre-trained Glove3 embedding dictionary. If a word is found, the word embedding vector (300 dimensions) is used as the initial value for that node. If no word is found, a random vector of the same 300 dimensions is generated as the initial value for that node, ensuring that the value of each dimension of the vector is within the range of [-0.01, 0.01].

[0090] S2-3. Call the node edge initialization sub-function to calculate the structure graph G corresponding to each sentence. structThe weights of each edge of E contained in the graph are given. Considering the characteristics of natural language in license documents, this method designs a dual-mechanism graph embedding technique customized for natural language. Specifically, to capture local information of sentences, a window capture mechanism is set up, focusing only on extracting information within a local area (i.e., inside the window) at a time; furthermore, through a sliding window mechanism, the relationships between lexical units (i.e., nodes) gradually propagate and aggregate, thereby making the structural graph G... struct It can reflect the overall syntactic framework of a sentence. This dual mechanism helps preserve both local and global structural information of the sentence. The sliding window size is set to n, meaning the window captures n consecutive tokens at a time. The corresponding tokens of nodes i and j in the current window W are recorded. n The co-occurrence frequency in the matrix, and the initial edge weights between i and j. The total number of occurrences of the two lexical units across all sliding windows is determined by the formula shown in formula (1), where Given the length of the word sequence to be processed, the total number of sliding windows can be calculated as follows: , Let be the co-occurrence frequency function, representing the node and In the window The co-occurrence of elements is calculated using formula (2):

[0091] (1)

[0092] (2)

[0093] Furthermore, step S3 specifically includes the following steps:

[0094] S3-1. Based on historical experience, 23 key license terms that are highly relevant to the legality and compliance of open-source software use have been selected and divided into two categories: Rights (R) and Obligations (O). Rights define the permissions one has when using open-source software, such as distributing the software, modifying it, or using it commercially; obligations specify the requirements that must be met when using open-source software, such as including copyright, disclosing the source code, or explaining changes. The 23 selected key license terms are as follows, where rights are shown as R + numbered numbers, and obligations are shown as O + numbered numbers:

[0095] R1) Distribute

[0096] R2) Modify

[0097] R3) Commercial Use

[0098] R4 Relicense

[0099] R5) Hold Liable

[0100] R6) Use Patent Claims

[0101] R7) Sublicense

[0102] R8)Statically Link

[0103] R9) Private Use

[0104] R10) Use Trademark

[0105] R11) Place Warranty

[0106] O1) Include Copyright

[0107] O2) Include License

[0108] O3) Include Notice

[0109] O4) Disclose Source

[0110] O5) State Changes

[0111] O6) Include Original

[0112] O7) Give Credit

[0113] O8) Rename

[0114] O9) Contact Author

[0115] O10) Include Install instructions

[0116] O11) Compensate Damages

[0117] O12) Pay Above Threshold

[0118] S3-2. Based on the user's actual needs, arbitrarily provide several historical license documents D history_license All sentences in these historical documents that relate to the main license terms defined in step S3-1 must be marked with corresponding tags that can be automatically identified. The tags include whether the sentence is related to a license term (yes or no), and which license term it relates to (e.g., R1, 01, etc.). The final result is a list D of tagged documents containing several tagged documents. L ;

[0119] S3-3. Use Refers to the list of marked documents D L All sentences in the included documents related to a specific license clause t are analyzed using three metrics: average frequency, specificity, and breadth, to subsequently assess the importance of the lexical unit w. It is worth noting that, due to the processing in step S1, the simplified sentence lexical list L... sen_token_simplify The lexical units in the list are all in root form, so the list of marked documents D is also calculated first. L The words in each sentence of the contained document are converted into word roots and then compared. The definitions and calculation methods of the three indicators are as follows:

[0120] a) Average frequency of occurrence of word w : Reflects the appearance of the lexical unit w in the tagged document list D L The average frequency of sentences related to a specific license term t in the included documents. The calculation method is shown in formula (3), where n is... The total number of sentences in the text. For the word 'w' in the sentence The number of times it appears in This represents the total number of words in the current sentence, excluding punctuation and spaces.

[0121] (3)

[0122] b) Specificity of the lexical w : Reflects whether the word w is unique to the sentence related to a specific license clause t. Its calculation method is shown in formula (4), where, yes The number of sentences containing the lexical w. It is a list of marked documents D L Total number of sentences containing the term w in the included documents:

[0123] (4)

[0124] c) The breadth of the lexical w : Reflects whether the term w is in the tagged document list D L In each document contained therein, it appears in a sentence related to a specific license term t. Its calculation method is shown in formula (5), where, It is the number of tagged documents containing sentences that include the word 'w' and are related to specific license terms 't'. It is the total number of tagged documents containing sentences related to a specific license term t.

[0125] (5)

[0126] S3-4, based on the three indicators defined in step S3-3, iterates through the simplified sentence word list L. sen_token_simplify Iterate through each word in a sentence and calculate the importance of each word. The value is the relevance between the term w and a specific license term t. It is obvious that if the term w is in... If a clause has never appeared in any sentence related to a specific license clause in all marked documents, then its importance is... The value should be 0; conversely, if the lexical w is If a word is the only word that appears in the text and has not appeared anywhere else, then its importance should be the maximum value of 1. Therefore, its calculation method is as shown in formula (6):

[0127] (6)

[0128] S3-5 Traverses the simplified sentence word list L sen_token_simplify For each element contained in a sentence, based on the importance value of each word obtained in steps S3-4, a corresponding content graph G is constructed on a sentence-by-sentence basis. content For each sentence, define and calculate the following three types of feature values:

[0129] a) Important lexical units. Based on the simplified sentence lexical unit list L sen_token_simplify The importance of all lexical units contained in the sentence is ranked, and the top 10 most important lexical units are selected from highest to lowest importance. A 10-dimensional important lexical unit feature vector is defined, logically corresponding one-to-one with each of these units. This process iterates through these highly important lexical units. For each sentence, if its lexical unit list does not contain the lexical unit at the corresponding ranking position, then the value of its important lexical unit feature vector at that position is 0; otherwise, it equals the importance of that lexical unit. Finally, a 10-dimensional important lexical unit feature vector is obtained for each sentence as its feature value. This type of feature helps determine the strength of the connection between the sentence and a specific license clause t.

[0130] b) Important Phrases. Pair two adjacent words in each sentence to form a phrase, and obtain the simplified sentence word list L. sen_token_simplify For all corresponding phrase combinations, the importance of a phrase is the sum of the importance of its two corresponding lexical units. The top 10 most important phrases are selected in descending order, and a 10-dimensional important phrase feature vector is defined to correspond one-to-one with each of them. The value of this vector is determined similarly to that of important lexical units. If the phrase combination corresponding to the sentence's lexical unit list does not contain the phrase at the corresponding ranking position, then the value of its important phrase feature vector at that position is 0; otherwise, it equals the importance of that phrase. Finally, a 10-dimensional important phrase feature vector is obtained for each sentence as its feature value.

[0131] c) Sentence length. This feature value corresponds to the number of tokens in each sentence. Introducing this type of feature helps identify the type of license terms associated with a given sentence, considering that some license terms are often expressed in shorter sentences.

[0132] S3-6. For the license document D to be analyzed license For each sentence in the graph, its three typological features—important words, important phrases, and sentence length—are treated as separate nodes and connected sequentially by edges. The weight of each edge is initialized to 1, resulting in the corresponding content graph G. content Finally, a list G of content graphs corresponding to each sentence is obtained. content_list .

[0133] Furthermore, step S5 specifically includes the following steps:

[0134] S5-1. Traversing the Combinatorial Graph List G mix_list A graph neural network is constructed, consisting of three sub-functional parts: an encoder, a propagation layer, and an aggregator. The encoder, first, uses a multilayer perceptron (MLP) to combine the graphs G from each input. mix The initial values ​​of nodes and edges are converted into initial vectors, as shown in formulas (7) and (8), where The nodes contained in the graph, Let be the edges contained in the graph. Represents a node initial value, Using MLP mapping Post-encoded node vectors Represents a node and The initial value of the edge between them, and It is a mapping Encoded side vectors after MLP:

[0135] (7)

[0136] (8)

[0137] S5-2. Next is the propagation layer, which maps the old vector... (which includes both structural and content information) to a new vector To update the node vector, Represents a node The vector corresponding to the update t times. By stacking multiple propagation layers, each node can accumulate knowledge from its local neighborhood. The graph neural network will calculate the attention coefficient between neighboring nodes. This is used to reflect the importance of the relationship between structural and content information. (Note: The coefficient is not mentioned in the original text.) The calculation method is shown in formula (9), where It is a multilayer perceptron applied to serial input. Let be the number of edges contained in the graph, and t be the number of updates. For nodes Edge weights between:

[0138] (9)

[0139] S5-3. Finally, there's the aggregator, which takes all the previously obtained updated node vectors (i.e., the node vector set) as input to generate a graph-level representation vector (Vector). graph_level The aggregation operation is executed as shown in formula (10), which uses a gating vector. A weighted summation mechanism is used to aggregate information across nodes and filter out irrelevant details; among which... This represents the set of all gated multilayer perceptrons (MLPs). This represents the sigmoid activation function used to compress input values ​​into the (0,1) interval. Represents a node The vector after t updates:

[0140] (10)

[0141] S5-4. Finally, the graph neural network will target the input combined graph G. mix Generate a corresponding 24-dimensional graph-level representation vector. graph_level This is used to represent the predicted probability of the sentence corresponding to the graph being associated with a specific license term. Essentially, this method utilizes a graph neural network to perform a classification task, with the classification result represented by a graph-level representation vector. graph_level The highest value in the 24 dimensions is determined, so in addition to the 23 predefined license terms, there is an additional dimension to indicate whether the sentence is irrelevant to any of them.

[0142] The method of the present invention will be described in detail below with reference to a specific embodiment.

[0143] like Figure 1 As shown, the open-source software license term detection method based on graph neural networks of the present invention proceeds according to the following process: The user inputs the license document D to be analyzed from the client. license(PDF, TXT, or DOCX text formats are all acceptable.) A computer with general computing capabilities, pre-installed with a Python runtime environment and relevant third-party libraries, will be used as the server to receive data and perform all subsequent computational processing. Given a license document D to be analyzed... license The text preprocessing module is invoked to perform text preprocessing, namely segmentation, lexicalization, and simplification, to obtain a simplified sentence lexical list L. sen_token_simplify Then, based on this, the structure diagram generation module and the content diagram generation module are called respectively to initialize and construct the corresponding structure diagram list G. struct_list and content image list G content_list By adding a common root node, the structure graph G corresponding to each sentence is... struct and content image G content Connecting them together, we finally obtain a list of combination diagrams G. mix_list The module for constructing a graph neural network and generating graph-level representation vectors is invoked, and the list of combined graphs G is traversed. mix_list A graph neural network is constructed, consisting of three sub-functional parts: an encoder, a propagator, and an aggregator. The computation of each combined graph G is performed. mix The attention coefficients between any two nodes in the graph are finally used to construct the combined graph G for each input. mix Generate a 24-dimensional graph-level representation vector (Vector) for each. graph_level This represents the predicted probability of the sentence corresponding to the graph being associated with any of the 23 predefined specific license terms, or not associated with any of them. The final result is a given license document D to be analyzed. license The system outputs a list of results indicating whether each sentence corresponds to a predefined license clause. An example of this process is given below.

[0144] The specific implementation method is as follows:

[0145] S1. Perform text preprocessing, such as... Figure 2 As shown, in a specific embodiment, the license document D to be analyzed is given. licenseThe content is as follows: "This software is provided 'as is', without any warranty of any kind. You may modify and distribute the software under the terms of the MIT License. You must include the original copyright notice in all copies." The text preprocessing module is invoked to perform text preprocessing. This module's sub-functions include segmentation, lexicalization, and simplification, each implemented using corresponding functions from third-party libraries. The simplified sentence lexical list L is obtained. sen_token_simplify .

[0146] Specifically, it includes the following steps:

[0147] S1-1. The text preprocessing module can be implemented by calling the Python third-party library CoreNLP. Using the corresponding functions provided in this library, segmentation, lexicalization, and simplification sub-functions can be achieved. First, the text segmentation function provided by the library is used to analyze the license document D. license The entire text is divided into several sentences and stored in list form to obtain the sentence text list L. sen_text As shown below:

[0148] L sen_text = [

[0149] "This software is provided “as is”, without warranty of any kind.",

[0150] "You may modify and distribute the software under the terms of theMIT License.",

[0151] "You must include the original copyright notice in all copies." ];

[0153] S1-2, Traverse the text sentence list L text_sen Using the word segmentation and lexicalization functions provided in the library, each sentence is segmented and its words are lexicalized. Sentence-by-sentence lexical lists L are obtained for each sentence.sen_token As shown below:

[0154] L sen_token = [

[0155] ["This","software","is","provided","as","is",",","without","warranty","of","any","kind","."],

[0156] ["You","may","modify","and","distribute","the","software","under","the","terms","of","the","MIT","License","."],

[0157] ["You","must","include","the","original","copyright","notice","in","all","copies","."] ];

[0159] S1-3, Traversing the sentence word list L sen_token For each element in the list, i.e., the list of lexical units corresponding to a single sentence, a simplification operation is performed using the Porter stemming function provided in the library. This involves removing all stop words and converting each remaining lexical unit into its corresponding root form. The final result is the simplified sentence lexical list L. sen_token_simplify As shown below:

[0160] L sen_token_simplify = [

[0161] ["software","provide","warrant"],

[0162] ["modify","distribute","software","term","mit","license"],

[0163] ["must","include","origin","copyright","notice","copy"]

[0164] ].

[0165] S2. Call the structure graph generation module, whose sub-functions include graph node initialization and node edge initialization. Traverse the simplified sentence lexical list L.sen_token_simplify Initialize and construct the structure graph G corresponding to each sentence. struct This is used to assist graph neural networks in capturing and analyzing the structural information of sentences in license documents. The structural graph focuses on understanding the syntactic structure of sentences and identifies their grammatical framework by applying graph embedding techniques. The final result is a list G of structural graphs. struct_list .like Figure 3 As shown, it specifically includes:

[0166] S2-1. Call the structure diagram generation module to generate the structure diagram G corresponding to each sentence. struct This module's sub-functions include graph node initialization and node edge initialization. First, it iterates through the simplified sentence lexical list L. sen_token_simplify Each element, i.e., a single sentence, is used to construct a corresponding structure graph G. struct Specifically, G struct = {V, E}. Where V contains all the nodes in the graph, representing the set of words in the sentence, with each word corresponding to a single node in the graph; E contains all the edges in the graph, representing the relationships between these words in the sentence.

[0167] With L sen_token_simplify Taking the first sentence as an example, V = {"software","provide","warrant"}, E contains all the edges in the graph, that is, the connections between all the nodes in V. The structure of this graph is obvious, so no further diagram will be drawn here.

[0168] S2-2, Call the graph node initialization sub-function to initialize the structure graph G corresponding to each sentence. struct The V (i.e., graph nodes) contained in the graph are initialized. To facilitate further in-depth analysis, a word embedding method is used to assign initial values ​​to each node. For the word corresponding to each node, a search is performed in the pre-trained Glove3 embedding dictionary. If a word is found, the word embedding vector (300 dimensions) is used as the initial value for that node. If no word is found, a random vector of the same 300 dimensions is generated as the initial value for that node, ensuring that the value of each dimension of the vector is within the range of [-0.01, 0.01].

[0169] With L sen_token_simplify Taking the first sentence as an example, the word "warrant" does not exist in the Glove3 embedding dictionary, so its vector is a randomly generated 300-dimensional vector, ensuring that the value of each dimension of the vector is within the range of [-0.01, 0.01]. The other two words were successfully retrieved, so the existing word embedding vectors in the Glove3 embedding dictionary are used directly for representation. The initial values ​​of the generated nodes are shown in Table 1.

[0170] Table 1. Initial values ​​corresponding to the generated nodes

[0171] ;

[0172] S2-3. Call the node edge initialization sub-function to calculate the structure graph G corresponding to each sentence. struct The weights of each edge of E contained in the graph are given. Considering the characteristics of natural language in license documents, this method designs a dual-mechanism graph embedding technique customized for natural language. Specifically, to capture local information of sentences, a window capture mechanism is set up, focusing only on extracting information within a local area (i.e., inside the window) at a time; furthermore, through a sliding window mechanism, the relationships between lexical units (i.e., nodes) gradually propagate and aggregate, thereby making the structural graph G... struct It can reflect the overall syntactic framework of a sentence. This dual mechanism helps preserve both local and global structural information of the sentence. The sliding window size is set to n, meaning the window captures n consecutive tokens at a time. The corresponding tokens of nodes i and j in the current window W are recorded. n The co-occurrence frequency in the matrix, and the initial edge weights between i and j. The total number of occurrences of the two lexical units across all sliding windows is determined by the formula shown in formula (1), where Given the length of the word sequence to be processed, the total number of sliding windows can be calculated as follows: , Let be the co-occurrence frequency function, representing the node and In the window The co-occurrence of elements is calculated using formula (2):

[0173] (1)

[0174] (2)

[0175] Let the window size n=3, the total word length T=3, and the number of sliding windows 3–3+1=1. All three words appear simultaneously in the single sliding window, therefore x software_provide =x software_warrant =x provide_warrant =1.

[0176] In summary, with L sen_token_simplify Taking the first sentence as an example, the corresponding structure diagram G is... struct = {V, E}. Where V = {"software","provide","warrant"}, and the initial values ​​of the corresponding nodes are shown in Table 1; E includes all the pairwise connections between these three nodes, and the edge weight value is 1.

[0177] S3. Traverse the simplified sentence word list L sen_token_simplify The content graph generation module is invoked to initialize and construct the content graph G corresponding to each sentence. content This is used to assist graph neural networks in capturing and analyzing the content information of license document statements. The content graph focuses on extracting key words and phrases that convey the core meaning of the license terms. The final result is a list of content graphs G. content_list .like Figure 4 As shown, it includes:

[0178] S3-1. Based on historical experience, 23 key license terms that are highly relevant to the legality and compliance of open-source software use have been selected and divided into two categories: Rights (R) and Obligations (O). Rights define the permissions one has when using open-source software, such as distributing the software, modifying it, or using it commercially; obligations specify the requirements that must be met when using open-source software, such as including copyright, disclosing the source code, or explaining changes. The 23 selected key license terms are as follows, where rights are represented by R + number and obligations by O + number:

[0179] R1) Distribute

[0180] R2) Modify

[0181] R3) Commercial Use

[0182] R4 Relicense

[0183] R5) Hold Liable

[0184] R6) Use Patent Claims

[0185] R7) Sublicense

[0186] R8)Statically Link

[0187] R9) Private Use

[0188] R10) Use Trademark

[0189] R11) Place Warranty

[0190] O1) Include Copyright

[0191] O2) Include License

[0192] O3) Include Notice

[0193] O4) Disclose Source

[0194] O5) State Changes

[0195] O6) Include Original

[0196] O7) Give Credit

[0197] O8) Rename

[0198] O9) Contact Author

[0199] O10) Include Install instructions

[0200] O11) Compensate Damages

[0201] O12) Pay Above Threshold

[0202] S3-2. Based on the user's actual needs, arbitrarily provide several historical license documents D history_license All sentences in these historical documents that relate to the main license terms defined in step S3-1 must be marked with corresponding tags that can be automatically identified. The tags include whether the sentence is related to a license term (yes or no), and which license term it relates to (e.g., R1, O1, etc.). The final result is a list D of tagged documents containing several tagged documents. L ;

[0203] Mark a given number of historical license documents D history_license , mark the document list D L Due to its length, this example only shows a few sentences and their corresponding tags, as shown in Table 2 below:

[0204] Table 2 List of marked documents D L Several example sentences and their corresponding tags

[0205] ;

[0206] S3-3, Use Refers to the list of marked documents D L All sentences in the included documents related to a specific license clause t are analyzed using three metrics: average frequency, specificity, and breadth, to subsequently assess the importance of the lexical unit w. It is worth noting that, due to the processing in step S1, the simplified sentence lexical list L... sen_token_simplify The lexical units in the list are all in root form, so the list of marked documents D is also calculated first. LThe words in each sentence of the contained document are converted into word roots and then compared. The definitions and calculation methods of the three indicators are as follows:

[0207] a) Average frequency of occurrence of word w : Reflects the appearance of the lexical unit w in the tagged document list D L The average frequency of sentences related to a specific license term t in the included documents. The calculation method is shown in formula (3), where n is... The total number of sentences in the text. For the word 'w' in the sentence The number of times it appears in This represents the total number of words in the current sentence, excluding punctuation and spaces.

[0208] (3)

[0209] Taking specific license term R3 as an example, in this embodiment, n=2, for the word "software", its corresponding average frequency of occurrence .

[0210] b) Specificity of the lexical w : Reflects whether the word w is unique to the sentence related to a specific license clause t. Its calculation method is shown in formula (4), where, yes The number of sentences containing the lexical w. It is a list of marked documents D L Total number of sentences containing the term w in the included documents:

[0211] (4)

[0212] Regarding the word "software", (Sentences containing this term appeared in the annotated documents numbered 1, 2, and 11). Therefore, its specificity... .

[0213] c) The breadth of the lexical w : Reflects whether the term w is in the tagged document list D L In each document contained therein, it appears in a sentence related to a specific license term t. Its calculation method is shown in formula (5), where, It is the number of tagged documents containing sentences that include the word 'w' and are related to specific license terms 't'. It is the total number of tagged documents containing sentences related to a specific license term t.

[0214] (5)

[0215] Regarding the term "software," sentences containing this term appear in document 1, 2, and 11, but only in document 1 is the sentence related to the specific license clause R3. Therefore, it can be concluded that... Thus, widespread .

[0216] In summary, after steps S3-3, taking specific license terms R3 as an example, the simplified sentence lexical list L is... sen_token_simplify The average frequency, specificity, and universality values ​​of all lexical terms are shown in Table 3.

[0217] Table 3. Simplified Sentence Lexical List L sen_token_simplify Average frequency, specificity, and universality values ​​of all lexical terms

[0218] ;

[0219] S3-4. Based on the three indicators defined in step S3-3, traverse the simplified sentence word list L. sen_token_simplify Iterate through each word in a sentence and calculate the importance of each word. The value is the relevance between the term w and a specific license term t. It is obvious that if the term w is in... If a clause has never appeared in any sentence related to a specific license clause in all marked documents, then its importance is... The value should be 0; conversely, if the lexical w is If a word is the only word that appears in the text and has not appeared anywhere else, then its importance should be the maximum value of 1. Therefore, its calculation method is as shown in formula (6):

[0220] (6)

[0221] In this embodiment, based on the foregoing, it can be obtained that Therefore, we can know the word " The importance of " Taking specific license terms R3 as an example, the simplified sentence lexical list L... sen_token_simplify The importance values ​​of all lexical units are shown in Table 4.

[0222] Table 4. Simplified Sentence Lexical List L sen_token_simplify Importance values ​​of all lexical units

[0223] ;

[0224] S3-5. Traverse the simplified sentence word list L sen_token_simplifyFor each element contained in a sentence, based on the importance value of each word obtained in steps S3-4, a corresponding content graph G is constructed on a sentence-by-sentence basis. content For each sentence, define and calculate the following three types of feature values:

[0225] a) Important lexical units. Based on the simplified sentence lexical unit list L sen_token_simplify The importance of all lexical units contained in the sentence is ranked, and the top 10 most important lexical units are selected from highest to lowest importance. A 10-dimensional important lexical unit feature vector is defined to logically correspond to each of these units. This process iterates through these highly important lexical units. For each sentence, if its lexical unit list does not contain the lexical unit at the corresponding ranking position, then the value of its important lexical unit feature vector at that position is 0; otherwise, it equals the importance of that lexical unit. Finally, a 10-dimensional important lexical unit feature vector is obtained for each sentence as its feature value. This type of feature helps determine the strength of the connection between the sentence and a specific license clause t.

[0226] In this embodiment, the list of important keywords obtained through filtering is as follows: ["software", "mit", "include", "modify", "license", "provide", "warrant", "distribute", "term", "must"]

[0227] Based on this, the simplified sentence lexical list L is obtained. sen_token_simplify The feature values ​​of each sentence are shown in Table 5:

[0228] Table 5. Simplified Sentence Lexical List L sen_token_simplify Feature values ​​of each sentence

[0229] ;

[0230] b) Important Phrases. Pair two adjacent words in each sentence to form a phrase, and obtain the simplified sentence word list L. sen_token_simplify For all corresponding phrase combinations, the importance of a phrase is the sum of the importance of its two corresponding lexical units. The top 10 phrases, selected from highest to lowest importance, are called high-importance phrases. A 10-dimensional important phrase feature vector is defined to correspond one-to-one with each of these high-importance phrases, and its value is determined similarly to that of important lexical units. If the phrase combination corresponding to the sentence's lexical unit list does not contain the phrase at the corresponding ranking position, then the value of its important phrase feature vector at that position is 0; otherwise, it equals the importance of that phrase. Finally, a 10-dimensional important phrase feature vector is obtained for each sentence as its feature value.

[0231] In this embodiment, the simplified sentence lexical list L sen_token_simplifyThe corresponding phrase combinations and their importance are shown in Table 6:

[0232] Table 6. Simplified Sentence Lexical List L sen_token_simplify All corresponding phrase combinations and their corresponding importance values

[0233] ;

[0234] c) Sentence length. This feature value corresponds to the number of tokens in each sentence. Introducing this type of feature helps identify the type of license terms associated with a given sentence, considering that some license terms are often expressed in shorter sentences.

[0235] Simplified sentence lexical list L sen_token_simplify The sentence length values ​​for each sentence are shown in Table 7 below:

[0236] Table 7. Simplified Sentence Lexical List L sen_token_simplify The sentence length value of each sentence contained

[0237] ;

[0238] S3-6, For the license document D to be analyzed license For each sentence in the graph, its three typological features—important words, important phrases, and sentence length—are treated as separate nodes and connected sequentially by edges. The weight of each edge is initialized to 1, resulting in the corresponding content graph G. content Finally, a list G of content graphs corresponding to each sentence is obtained. content_list .

[0239] In this embodiment, L sen_token_simplify Taking the first sentence as an example, its corresponding content diagram can be obtained by combining the aforementioned products according to the above steps.

[0240] S4. Traversing the list of structure graphs G struct_list and content image list G content_list By adding a common root node, the structure graph G corresponding to each sentence is... struct and content image G content Connect them to create a combinatorial graph G corresponding to the sentence. mix This approach ensures that the graph neural network can simultaneously learn both the structural and content knowledge corresponding to each sentence. The final result is a combined graph list G. mix_list .

[0241] S5. Call the module for constructing graph neural networks and graph-level representation vectors, and traverse the combined graph list G. mix_listA graph neural network is constructed, consisting of three sub-functional parts: an encoder, a propagation layer, and an aggregator. Each combined graph G is computed. mix The attention coefficients between any two nodes in the graph are finally used to construct the combined graph G for each input. mix Generate a 24-dimensional graph-level representation vector (Vector) for each. graph_level This represents the predicted probability of the sentence corresponding to the graph being associated with any of the 23 predefined specific license terms, or not associated with any of them. The final result is a given license document D to be analyzed. license Output a list of results showing whether each sentence corresponds to a predefined license clause and which clause it corresponds to.

[0242] like Figure 5 As shown, it specifically includes:

[0243] S5-1, Traversing the Combinatorial Graph List G mix_list A graph neural network is constructed, consisting of three sub-functional parts: an encoder, a propagation layer, and an aggregator. The encoder, first, uses a multilayer perceptron (MLP) to combine the graphs G from each input. mix The initial values ​​of nodes and edges are converted into initial vectors, as shown in formulas (7) and (8), where The nodes contained in the graph, Let be the edges contained in the graph. Represents a node initial value, Using MLP mapping Post-encoded node vectors Represents a node and The initial value of the edge between them, and It is a mapping Encoded side vectors after MLP:

[0244] (7)

[0245] (8)

[0246] S5-2, The propagation layer maps the old vector. (which includes both structural and content information) to a new vector This updates the node vectors. By stacking multiple propagation layers, each node can accumulate knowledge from its local neighborhood. The graph neural network calculates the attention coefficients between neighboring nodes. This is used to reflect the importance of the relationship between structural and content information. (Note: The coefficient is not mentioned in the original text.) The calculation method is shown in formula (9), where It is a multilayer perceptron applied to serial input. Let be the number of edges contained in the graph, and t be the number of updates. For nodes Edge weights between:

[0247] (9)

[0248] S5-3, The aggregator takes all updated node vectors (i.e., the set of node vectors) as input to generate a graph-level representation vector. graph_level The aggregation operation is executed as shown in formula (10), which uses a gating vector. A weighted summation mechanism is used to aggregate information across nodes and filter out irrelevant details; among which... This represents the set of all gated multilayer perceptrons (MLPs). This represents the sigmoid activation function used to compress input values ​​into the (0,1) interval. Represents a node The vector after t updates:

[0249] (10)

[0250] S5-4, The graph neural network will generate a corresponding input combination graph G. mix 24-dimensional graph-level representation of vector graph_level This is used to represent the predicted probability of the sentence corresponding to the graph being associated with a specific license term. Essentially, this method utilizes a graph neural network to perform a classification task, with the classification result represented by a graph-level representation vector. graph_level The highest value in the 24 dimensions is determined, so in addition to the 23 predefined license terms, there is an additional dimension to indicate whether the sentence is irrelevant to any of them.

[0251] In this embodiment, L sen_token_simplify Taking the first sentence as an example, we can ultimately obtain a corresponding 24-dimensional graph-level representation vector Vector. graph_level Since the highest value corresponds to the position of the predefined license clause R11, the output recognition result is: "Yes, associated clause: R11 (Place Warranty)". This output result is the recognition result returned by the server for this sentence, and finally L sen_token_simplify Each sentence will generate a corresponding recognition result, which will be returned as a result list and printed to the client.

[0252] Compared with the prior art, the present invention has the following advantages and effects:

[0253] This invention provides a method that can automatically, efficiently, and accurately identify and extract sentences related to specific terms in a given open-source software license document. By introducing and designing a graph neural network to establish a relationship model between license document sentences and license terms, this invention proposes an automatic generation method for the structure graph and content graph of the graph neural network. This solves the problem that ontology-based methods require manual pre-definition or extraction of tuple data, which is time-consuming, labor-intensive, and prone to human error. Furthermore, by integrating the structure graph and content graph through a combined graph, this invention ensures that the graph neural network can simultaneously learn the structural knowledge (syntactic composition of the sentence) and content knowledge (important words and phrases conveying the core meaning of the license terms) corresponding to each sentence. This addresses the problem that pattern- or similarity-based methods cannot deeply analyze the sentence structure and semantics in the license, thus guaranteeing the accuracy of the identification and extraction results. To address the generation of structural graphs, this invention innovatively designs a dual-mechanism graph embedding technique tailored for natural language processing. This involves using a window-capturing mechanism to acquire local sentence information, focusing on extracting information only within a local area (i.e., inside the window) at a time; and a sliding window mechanism to further ensure the gradual propagation and aggregation of relationships between lexical units (i.e., graph nodes), thereby enabling the structural graph to reflect the overall syntactic framework of the sentence and achieving a stronger ability to capture sentence structural knowledge. Furthermore, the invention exhibits good scalability and versatility; the method framework is applicable to both less common and common licenses. Users only need to provide different historical license tag documents according to their actual needs to achieve compatibility with less common licenses.

Claims

1. A method for detecting open source software license terms based on graph neural networks, characterized in that, The method comprises the following steps: S1. The user transmits a license document to be analyzed from the client, and the server calls a text preprocessing module to preprocess the text of the license document to be analyzed, and obtains a simplified sentence token list; S2. A structure diagram generation module is called, the simplified sentence token list is traversed, and a structure diagram corresponding to each sentence in the simplified sentence token list is initialized and constructed, which is used to assist the graph neural network in capturing and analyzing the structural information of the license document sentence, and finally a structure diagram list is obtained; S3. The simplified sentence token list is traversed, and a content diagram corresponding to each sentence in the simplified sentence token list is initialized and constructed, which is used to assist the graph neural network in capturing and analyzing the content information of the license document sentence, and finally a content diagram list is obtained; S4. The structure diagram list and the content diagram list are traversed, a common root node is added, and the structure diagram and the content diagram corresponding to each sentence are connected respectively, so as to create a combined diagram corresponding to the sentence, and finally a combined diagram list is obtained; S5. A graph neural network and a graph-level representation vector module are called, the combined diagram list is traversed, the graph neural network is constructed, the attention coefficient between any two nodes in each combined diagram is calculated, and finally a 24-dimensional graph-level representation vector is generated for each input combined diagram, which is used to represent the prediction probability result of 24 kinds of situations that the combined diagram corresponding sentence is respectively associated with a specific pre-defined license clause or is not associated with any license clause, and finally a prediction probability result list is obtained and output to the client; In step S3, the content diagram focuses on extracting words and phrases conveying the core meaning of the license clause; Step S3 specifically comprises the following steps: S3-1. Select the license clause and divide it into two categories: rights and obligations; S3-2. According to the actual needs of the user, mark all the sentences related to the main license clause defined in step S3-1 in any given number of historical license documents, and the marking content includes whether the sentence is related to the license clause and which license clause it is related to, and obtain a marked document list containing a number of marked documents; S3-3. Use Three indicators are set: average frequency of occurrence, specificity, and generality, referring to all the sentences in the list of marked documents that are related to a certain specific license clause. S3-4. Traverse the sentence token list, and traverse each token in the sentence token list in units of sentences. The importance value corresponding to each token is calculated according to the three indicators set; S3-5. Traverse the sentence token list, and for each sentence in the sentence token list, construct a corresponding content diagram based on the importance value of each token obtained in step S3-4 in units of sentences. For each sentence, three types of feature values are defined and calculated: important tokens, important phrases, and sentence length; S3-6. For each sentence in the license document to be analyzed, the three types of features corresponding to the important tokens, important phrases, and sentence length are regarded as separate nodes and connected by edges in turn, and the weight of each edge is initialized to 1, to obtain the corresponding content diagram, and finally obtain a content diagram list corresponding to each sentence.

2. The graph neural network-based open source software license clause detection method according to claim 1, characterized in that, In step S1, the text preprocessing includes segmentation, tokenization, and simplification.

3. The graph neural network-based open source software license clause detection method according to claim 1, characterized in that, In step S2, the structure diagram focuses on understanding the syntactic structure of the sentence, and identifies the syntactic framework by applying graph embedding technology.

4. The graph neural network-based open source software license clause detection method according to claim 1, characterized in that, In step S5, the result list indicates whether each sentence in the given license document corresponds to a pre-defined license clause and which pre-defined license clause it corresponds to.

5. The graph neural network-based open source software license clause detection method according to claim 2, characterized in that, Step S1 specifically includes the following steps: S1-1. The text preprocessing module uses CoreNLP to divide all the text contained in the license document to be analyzed into several sentences and stores them in a list form to obtain a sentence text list; S1-2. Traverse the text sentence list and perform word segmentation operation on each sentence in it respectively, and convert the words in the sentence into word units, obtain the sentence word unit list corresponding to each sentence in units of sentences; S1-3. Traverse the sentence word unit list, and for each word unit list element, use the Porter stemming method to delete all stop words in it, and convert each word unit remaining after deletion into its corresponding root form, and finally obtain a simplified sentence word unit list.

6. The graph neural network-based open source software license clause detection method according to claim 1, characterized in that, Step S5 specifically includes the following steps: S5-1. Traverse the combination graph list to build a graph neural network, which consists of three parts: encoder, propagation layer and aggregator; first, the encoder uses a multi-layer perceptron to convert the initial values of the nodes and edges in each input combination graph into an initial vector; S5-2. The propagation layer updates the node vector by mapping the old vector to the new vector; by stacking multiple propagation layers, each node can accumulate knowledge from its local neighborhood; S5-3. The aggregator takes all the updated node vectors obtained previously as input to generate a graph-level representation vector; S5-4. The graph neural network generates a 24-dimensional graph-level representation vector corresponding to the input combination graph, which is used to represent the predicted probability result of the association between the sentence corresponding to the combination graph and a specific license clause.

7. The graph neural network-based open source software license clause detection method according to claim 6, characterized in that, In addition to the 23 pre-defined license clauses, the 24 dimensions include an additional dimension to indicate whether the sentence is irrelevant to any of them.

8. The graph neural network-based open source software license clause detection method according to claim 7, characterized in that, The 23 pre-defined license clauses are as follows, where rights are denoted by R+number and obligations are denoted by O+number: R1) Distribute R2) Modify R3) Commercial Use R4) Relicense R5) Hold Liable R6) Use Patent Claims R7) Sublicense R8) Statically Link R9) Private Use R10) Use Trademark R11) Place Warranty O1) Include Copyright O2) Include License O3) Include Notice O4) Disclose Source O5) State Changes O6) Include Original O7) Give Credit O8) Rename O9) Contact Author O10) Include Install instructions O11) Compensate Damages O12) Pay Above Threshold.

Images