Dynamic prompt-based api entity-relation joint extraction method and system

Abstract

The application relates to the field of artificial intelligence, and provides an API entity-relation joint extraction method and system based on dynamic prompting, which comprises the following steps: obtaining and marking a small amount of API text to obtain marked data; based on API entity and relation characteristics, a dynamic prompt capable of jointly extracting API entity-relation is constructed; based on the dynamic prompt, a structured extraction language is constructed; based on the structured extraction language, a training set and a test set are constructed; based on the training set and the test set, a large pre-training language model is fine-tuned to output a structured extraction language corresponding to API text, and joint extraction of API entity-relation is constructed. The application reduces the extraction difficulty of API entity-relation through dynamic prompting, and improves the extraction accuracy; only a small amount of data needs to be marked, and a large amount of API knowledge stored in the large pre-training language can be activated, so that the excessive manual work of the existing method is relieved, the extraction cost is reduced, and the extraction efficiency is improved.

Description

TECHNICAL FIELD

[0001] The present application relates to the field of artificial intelligence, in particular to an API entity-relation joint extraction method and system based on dynamic prompting. BACKGROUND

[0002] Application Programming Interface (API) is an important software engineering component, which can be often found in a wide range of natural language texts, such as official API documents, informal online forums. At the same time, API relationships are also embedded in these texts. For example, the text “To manipulate data you actually need executeUpdate() rather than executeQuery()” in the well-known forum StackOverflow describes the functional replacement relationship between executeUpdate() and executeQuery(). This API relationship indicates that we should use “executeUpdate()” instead of “executeQuery()” to achieve data manipulation. Extracting API entities and relationships from unstructured texts is a fundamental work, which is a prerequisite for effectively accessing and applying API knowledge to numerous software engineering tasks. Once these API knowledge is extracted, these entities and relationships can be organized into structured knowledge (such as knowledge graph) to support various software engineering tasks, such as API linking, API recommendation, API comparison. Taking StackOverflow as an example, as of May 2020, there are 35 million post replies in this forum, and there are rich API entities and relationships in these post replies. However, due to the unstructured nature of the text, it is very difficult to extract API entities and relationships from these texts.

[0003] Currently, there are two mainstream methods to extract API entities from unstructured text. The first one is rule-based method, such as regular expression and heuristic rule matching based on language convention. Because it is impossible to manually exhaust all the rules applicable to all texts, it has rule design overhead. The second one is sequence tagging-based method, such as training CRF and Bi-LSTM-CRF models. Because it is impossible to manually tag entities as training data for a large amount of text, it has data tagging overhead. Compared with API entity extraction, extracting relations from software texts is quite primitive, which relies on API syntax (e.g., declaration statements), specially tagged relations (e.g., See also keywords and hyperlinks), or some special relation phrases (e.g., "different from" and "similar to"). Like rule-based entity extraction, these API relation extraction methods also have rule design overhead. We collectively refer to rule design overhead and data tagging overhead as artificial overhead. In addition, the existence of three ambiguity phenomena in the text exacerbates this artificial overhead, which requires more artificial design of rules or more artificial tagging of data to distinguish ambiguous texts. The first one is API entity morphological ambiguity, including abbreviations, synonyms and spelling errors. For example, people will write java.util.iterator.remove() as remove(); the second one is API entity general word ambiguity, such as print can refer to the action "print", or refer to the API java.io.printwriter.print(); the third one is API relation expression ambiguity, such as "API_1 and API_2 are different" and "API_1 is (adjective), API_2 is not". The existence of these three ambiguity phenomena makes it more difficult to extract API entities and relations. More, existing methods treat API entity extraction and relation extraction as two independent tasks, making relation extraction heavily dependent on entity extraction results, thus leading to error propagation. SUMMARY

[0004] In view of the above problems in the prior art, the present application provides an API entity-relation joint extraction method and system based on dynamic prompting, which reduces the difficulty of API entity-relation extraction, improves the extraction accuracy, reduces the extraction cost, and improves the extraction efficiency.

[0005] The API entity-relation joint extraction method and system based on dynamic prompting.

[0006] The technical scheme adopted by the present application is:

[0007] In the first aspect, an API entity-relation joint extraction method based on dynamic prompting is provided, comprising the following steps:

[0008] Obtaining and tagging a small amount of API text to obtain tagged data;

[0009] Based on the API entity and relationship characteristics, a dynamic prompt capable of jointly extracting API entity-relationship is constructed;

[0010] Based on the dynamic prompt, a structured extraction language is constructed;

[0011] Based on the structured extraction language, a training set and a test set are constructed;

[0012] Based on the training set, a large pre-training language model is fine-tuned by inputting the training set into the model and calculating the loss function, and then performing back propagation and parameter update to optimize the parameters of the model; based on the test set, the fine-tuned large pre-training language model is fine-tuned, and the performance of the fine-tuned model is evaluated based on the test set; the structured extraction language corresponding to the API text is output by the fine-tuned model, and the joint extraction of API entity-relationship is realized.

[0013] In a second aspect, a dynamic prompt-based API entity-relationship joint extraction method is provided, and the step of obtaining labeled data includes:

[0014] Some API texts are crawled from SO forums, and the API texts are segmented and tokenized;

[0015] The segmented and tokenized text is manually annotated, and the API entities in the text and the API relationships between the entities are annotated to obtain labeled data.

[0016] In a third aspect, a dynamic prompt-based API entity-relationship joint extraction method is provided, and the step of constructing a dynamic prompt capable of jointly extracting API entity-relationship includes:

[0017] The format of the dynamic prompt is constructed based on the large pre-training language model to be selected;

[0018] Based on the prompt and API entity and relationship categories, the content of the dynamic prompt is constructed.

[0019] In a fourth aspect, a dynamic prompt-based API entity-relationship joint extraction method is provided, and the step of constructing a structured extraction language includes:

[0020] A unified hierarchical positioning association structure is constructed to encode different information extraction structures;

[0021] Through this hierarchical positioning association structure, the text is converted into structured data.

[0022] In a fifth aspect, a dynamic prompt-based API entity-relationship joint extraction method is provided, characterized in that:

[0023] The step of constructing a training set and a test set includes:

[0024] transforming the labeled data into the format of the structured extraction language;

[0025] The constructed data set includes API text and the structured extraction language format of the labeled data.

[0026] In a sixth aspect, an API entity-relation joint extraction method based on dynamic prompts is provided.

[0027] The steps of constructing the API entity-relation joint extraction include:

[0028] Based on the API text and the characteristics of the structured extraction language data format, a suitable pre-trained language model is selected to classify the API text and generate dynamic prompts.

[0029] A suitable pre-trained language model is selected to generate API entities and relations based on the API text and dynamic prompts, thereby realizing joint extraction of API entities and relations.

[0030] In a seventh aspect, an API entity-relation joint extraction system based on dynamic prompts is provided, which includes a dynamic prompt generator module and an API entity-relation joint extractor module.

[0031] The dynamic prompt generator module is used to generate dynamic prompts for input API text.

[0032] The API entity-relation joint extractor module extracts API entities and relations in the API text based on the dynamic prompts.

[0033] In an eighth aspect, an API entity-relation joint extraction system based on dynamic prompts is provided, and the dynamic prompt generator module includes an API relation classification module and a prompt generation module.

[0034] The API relation classification module is used to generate a small number of potential API relations contained in the input API text.

[0035] The prompt generation module converts the potential API relations into dynamic prompts based on the dynamic prompt format.

[0036] In a ninth aspect, an API entity-relation joint extraction system based on dynamic prompts is provided, and the API entity-relation joint extractor module includes a joint extraction module and a format conversion module.

[0037] The joint extraction module is used to generate corresponding structured extraction language for the input text.

[0038] The format conversion module is used to convert the generated structured extraction language into the form of API entity-relation triples.

[0039] The advantages of this invention over the prior art are:

[0040] This invention relates to a method and system for joint extraction of API entities and relationships based on dynamic prompts. By using a structured extraction language, it can unify different information extraction structures and achieve joint extraction of API entity relationships. Dynamic prompts reduce the difficulty of extracting API entities and relationships and improve extraction accuracy. Only a small amount of labeled data is needed to activate the massive amount of API knowledge stored in a large pre-trained language, alleviating the excessive manual intervention of existing methods, reducing extraction costs, and improving extraction efficiency. Attached Figure Description

[0041] Figure 1 This is a schematic diagram of the API entity-relationship joint extraction method based on dynamic prompts in an example of the present invention;

[0042] Figure 2 This is a schematic diagram of the dynamic prompt format in an example of the present invention;

[0043] Figure 3 This is a schematic diagram of the structured language extraction in an example of the present invention;

[0044] Figure 4 This is a schematic diagram of the API entity-relationship joint extraction system based on dynamic prompts in an example of the present invention. Detailed Implementation

[0045] The present invention will now be described in detail with reference to the accompanying drawings and embodiments:

[0046] Example 1:

[0047] like Figure 1 As shown, this embodiment provides a method for joint extraction of API entities and relationships based on dynamic prompts, including:

[0048] Retrieve API text data and tag API entities and relationships within the data;

[0049] Based on data characteristics, design dynamic prompts;

[0050] Construct a structured extraction language;

[0051] Based on the structured extraction language, the labeled data is transformed to construct training and test sets; the pre-trained language model is fine-tuned to achieve joint extraction of API entities and relations.

[0052] In this embodiment, the process of marking data includes: segmenting and tokenizing API texts in StackOverflow; manually marking API entities and API relationships of the API texts. In this embodiment, one API entity (i.e., API) and seven API relationships (i.e., functional similarity, behavioral difference, functional substitution, efficiency comparison, logical constraint, type conversion, and functional collaboration) are considered.

[0053] In this embodiment, T5 is selected as the pre-trained language model for entity and relationship extraction. Based on the model, the dynamic prompt format in this embodiment is as shown in Figure 2 [Entity type], [relationship type], and [text] are prefixes in the dynamic prompt, which are followed by different types of specific content. It should be noted that the number of relationship types in the dynamic prompt in this embodiment is 3, and therefore Figure 2 the number of relationship types shown in is 3.

[0054] Based on the dynamic prompt format, the relationship type-1, relationship type-2, and relationship type-3 in the dynamic prompt are the three API relationship types with the highest probability after 7 classification of the API texts. In this embodiment, Bert is used as the pre-trained language model for text classification.

[0055] In this embodiment, the process of designing the structured extraction language includes:

[0056] A hierarchical localization association structure is designed to encode different information extraction structures. Localization refers to identifying a specific type of target, i.e., an API entity in the text. Association refers to connecting different targets and assigning them semantic roles according to predefined patterns, i.e., API relationships between entities. Through this localization association structure, the text can be converted into structured data.

[0057] Based on the hierarchical localization association structure, the structured extraction language format in this embodiment is as shown in Figure 3 Through the structured extraction language, API entities and relationships can be extracted simultaneously, and the structure of both is unified.

[0058] The process of constructing the training set and the test set includes: based on the format of the structured extraction language, the manually marked data is converted to obtain the structured extraction language corresponding to the API texts. Then, the data is divided and the final training set and test set are obtained through a data augmentation strategy.

[0059] The process of fine-tuning the pre-trained language model includes: fine-tuning the classification model with the final training set to realize the classification of API text and obtain dynamic prompts; fine-tuning the joint extraction model with the final training set to realize the joint extraction of API entities and relations. In this embodiment, since the pre-training task of BERT (i.e., predicting the next sentence) is similar to the task of API relation classification, which is a classification task, the classification model of the present application adopts BERT. Since the pre-training task of T5 (sequence-to-sequence generation) corresponds to the generation task of the structured extraction language proposed by the present application, the joint extraction model of the present application adopts the T5 model.

[0060] Embodiment 2

[0061] The present application provides a system for jointly extracting API entities and relations, which comprises: Figure 4 The API relation classification system comprises a prompt generator module and a joint extractor module.

[0062] The dynamic prompt generator module is configured to generate a dynamic prompt for the input API text. Specifically, the module comprises an API relation classification module and a prompt generation module.

[0063] Specifically, the API relation classification module is configured to generate three API relations for the input API text. In this example, a classifier based on BERT is used to classify each input sentence into seven relation types, and then three relation types with the highest probability are selected to generate a dynamic prompt. In this example, the training set is used to fine-tune BERT. In backpropagation, the cross-entropy loss is used to calculate the loss of the classifier and adjust the parameters of BERT. The calculation method of the cross-entropy loss is as follows:

[0064]

[0065] wherein z represents the relation type predicted by the model, c represents the true relation type, j represents a certain relation type, and c-1 represents a total of c relation types.

[0066] The prompt generation module generates a dynamic prompt based on the dynamic prompt format, and the API relations in the dynamic prompt are the three API relations with higher probability output by the classifier.

[0067] The joint extractor module is configured to jointly extract API entities and relations in the API text based on the dynamic prompt. Specifically, the module comprises a joint extraction module and a format conversion module.

[0068] Specifically, the joint extraction module is configured to generate a corresponding structured extraction language for the input text. The input of the module is the API text and the dynamic prompt, and the output is the structured extraction language. The generation process is as follows:

[0069] [y1, ...y n ] = JE([P1, ... P n ])(Formula 2)

[0070] In this example, JE is a T5-based joint extractor, [y1, ... y n [P1, ... P] is a structured extraction language. n [] This is a dynamic suggestion, which can be represented in the following form:

[0071]

[0072] Where [spot] is the identifier for the entity type, [asso] is the identifier for the relation type, and x1, ... x n This represents the input API text.

[0073] Given dynamic cues, JE computes the hidden layer representation for each word using the following method, where Encoder represents the T5 encoder, [P1, ... P... n [This is a dynamic prompt:]

[0074] H = Encoder([P1, ... P1)) n ])(Formula 4)

[0075] Then, JE decodes the dynamic cue into a structured extraction language sequence using autoregression. In decoding step i, the decoder generates the i-th token y in the structured extraction language sequence using the following computation method. i The hidden state of the decoder at the current moment

[0076]

[0077] Where H represents the encoder output, For decoders.

[0078] In this example, cross-loss entropy is used to fine-tune T5 on the training set (D), with a learning rate of 1e-4, 50 epochs, and a batch size of 100. The cross-loss entropy is calculated as follows:

[0079] Loss=∑ (p，y)∈D -logP(y|p;θ e θ d )(Formula 6)

[0080] Where y represents the structured extraction language, p represents dynamic prompts, and θ represents the dynamic prompts. e θ d These are the parameters for the encoder and decoder, respectively.

[0081] Specifically, the format conversion unit converts the structured extraction language into the form of API entity-relation triplets (API entity-1, API relation, API entity-2), to obtain a final joint extraction result

[0082] The API entity-relation joint extraction method and system based on dynamic prompting can unify different information extraction structures and realize joint extraction of API entity relations through structured extraction language. The extraction difficulty of API entity-relation is reduced and the extraction accuracy is improved through dynamic prompting. Only a small amount of data needs to be labeled to activate the massive API knowledge stored in a large pre-training language, thereby alleviating the excessive manual work of existing methods, reducing the extraction cost, and improving the extraction efficiency.

[0083] The above is only a preferred embodiment of the present application, and does not limit the structure of the present application in any form. Any simple modification, equivalent change and modification of the above embodiment according to the technical essence of the present application are within the scope of the technical solution of the present application.

Claims

1. A method for joint extraction of API entities and relationships based on dynamic prompts, characterized in that, Includes the following steps: Obtain and tag a small amount of API text to obtain tagged data; Based on the characteristics of API entities and relationships, a dynamic prompt that can jointly extract API entities and relationships is constructed. Based on the aforementioned dynamic prompts, a structured extraction language is constructed; Based on the structured extraction language, a training set and a test set are constructed; Fine-tuning of a large pre-trained language model based on the training set involves inputting the training set into the model and calculating the loss function, followed by backpropagation and parameter updates to optimize the model's parameters. Fine-tuning a large pre-trained language model based on the test set, and evaluating the performance of the fine-tuned model using the test set; By leveraging the structured extraction language corresponding to the API text output by the fine-tuned model, the joint extraction of API entities and relationships can be achieved. The steps to build a structured extraction language include: Construct a unified hierarchical positioning association structure to encode different information extraction structures; By using this hierarchical positioning and related structure, text can be transformed into structured data; The steps for constructing an API entity-relationship joint extraction include: Based on the characteristics of the API text and structured extracted language data, a suitable pre-trained language model is selected to classify the API text and generate dynamic prompts. By selecting a suitable pre-trained language model, API entities and relationships are generated based on the API text and dynamic prompts, thereby achieving joint extraction of API entities and relationships.

2. The API entity-relationship joint extraction method based on dynamic prompts according to claim 1, characterized in that: The steps to obtain labeled data include: We crawled some API text from the SO forum and performed sentence and word segmentation on the API text. The text after sentence and word segmentation is manually annotated to identify API entities in the text and the API relationships between entities, thus obtaining labeled data.

3. The API entity-relationship joint extraction method based on dynamic prompts according to claim 1, characterized in that: The steps to build dynamic hints that can jointly extract API entity-relationship information include: The format for constructing dynamic prompts based on large pre-trained language models to be selected; Based on the aforementioned prompts, API entities, and relationship categories, construct the content of the dynamic prompts.

4. The API entity-relationship joint extraction method based on dynamic prompts according to claim 1, characterized in that: The steps for constructing training and test sets include: Transform the labeled data into the format of the structured extraction language; The constructed dataset includes API text and a structured extraction language format for the labeled data.

5. A dynamic prompt-based API entity-relationship joint extraction system, the system being used to implement the method as described in any one of claims 1 to 4, characterized in that, include: Dynamic suggestion generator module; API entity-relationship joint extractor module; The Dynamic Prompt Generator module is used to generate dynamic prompts for the input API text; The API entity-relationship joint extractor module extracts API entities and relationships from API text based on the dynamic prompts.

6. The API entity-relationship joint extraction system based on dynamic prompts according to claim 5, characterized in that: The dynamic suggestion generator module includes: an API relationship classification module and a suggestion generation module; The API relationship classification module is used to generate a small number of potential API relationships contained in the input API text; The prompt generation module converts potential API relationships into dynamic prompts based on the dynamic prompt format.

7. The API entity-relationship joint extraction system based on dynamic prompts according to claim 6, characterized in that: The API entity-relationship concatenation extractor module includes: a concatenation extraction module and a format conversion module; The joint extraction module is used to generate the corresponding structured extraction language for the input text; The format conversion module is used to convert the generated structured extraction language into API entity-relation triples.

Images