Data augmentation method for training traceability link recovery model based on unlabeled data

Abstract

The present invention discloses a method for enhancing training data of a traceability link recovery model based on unlabeled data, comprising the following steps: (1) predicting the similarity between unlabeled and labeled software products; (2) link-marking the unlabeled software products; and (3) supplementing the newly-marked link data to the training data set of the traceability link recovery model. The present invention uses a comparative learning similarity prediction method to predict the similarity relationship between unlabeled and labeled software products, and then generates a similarity list. By selecting product pairs with high similarity, links are created between the link objects of the unlabeled products and the labeled products, and the links are added to the training data set of the traceability link recovery model, thereby increasing the number of training data sets and improving the performance of the traceability link recovery model.

Description

Technical Field

[0001] The invention belongs to the field of traceability link recovery in software engineering, and in particular relates to a data enhancement method for training a traceability link recovery model based on unlabeled data. Background Art

[0002] The Center of Excellence for Software Traceability (CoEST) defines software traceability as "the ability to relate any uniquely identifiable software engineering artifact to other artifacts, maintain these links over time, and use the resulting network to answer questions about the software product and its development process". Traceability link recovery is a software engineering task to recover links between different types of software artifacts, such as requirements, source code, bug reports, test cases, and user documentation. Traceability plays an important role in software development and maintenance, providing useful support for various software activities, such as program comprehension, compliance verification, change impact analysis, and regression analysis of test cases.

[0003] However, as software scale and complexity increase, creating and maintaining accurate traceability links is challenging. In most cases, developers need to manually recover traceability links from software artifacts, which is time-consuming, labor-intensive and error-prone. Therefore, researchers have proposed an automated traceability link recovery method to reduce the burden on developers.

[0004] Since software artifacts are mainly composed of text data, most of the initial traceability link recovery methods are based on information retrieval technology. Information retrieval believes that software artifacts with high text similarity are more likely to have links. When the documents of software artifacts share a large number of terms, the recovery performance of information retrieval is better. However, information retrieval does not consider the semantic information between terms, and there are problems with synonyms and polysemy. Recently, deep learning has been widely used in the field of traceability link recovery, and its performance has been greatly improved compared with traditional methods based on information retrieval.

[0005] In order to achieve good performance, deep learning-based traceability link recovery methods require a large amount of labeled data for training classifiers. However, in actual projects, such data is manually labeled by experts with relevant domain knowledge, or heuristic methods (such as regular expressions) are used to label links. Manual labeling is expensive, and heuristic methods can only label a small part of the links, and a large number of links are still missed. These data are called unlabeled data.

[0006] Usually, these unlabeled data are excluded from the training data set and cannot be used to train the model. If we can use rich unlabeled data sources for model training, it is expected to improve the model performance. In order to improve the performance of the recovery method by using unlabeled data, the present invention proposes a data enhancement method for training a traceability link recovery model based on unlabeled data. Summary of the invention

[0007] Purpose of the invention: The present invention proposes a data enhancement method for training a traceability link recovery model based on unlabeled data, which can make full use of the similarity between unlabeled software artifacts and labeled software artifacts, create links between the link objects of labeled artifacts and unlabeled artifacts, increase the amount of training set data, train better classifiers, and improve recovery performance.

[0008] Technical solution: The present invention provides a data enhancement method for training a traceability link recovery model based on unlabeled data, comprising the following steps:

[0009] (1) using a similarity prediction method to predict the similarity between unlabeled and labeled software products and obtain a similarity list;

[0010] (2) link marking the unmarked software products according to the similarity list obtained in step (1);

[0011] (3) Supplement the newly labeled link data obtained in step (2) to the training data set of the traceability link recovery model.

[0012] Furthermore, the step (1) comprises the following steps:

[0013] (11) Performing text preprocessing on the natural language text of the software product, including word segmentation, conversion to lowercase, punctuation removal, and stop word removal;

[0014] (12) Using contrastive learning to predict the similarity between unlabeled and labeled software artifacts: The training data for contrastive learning is generated by linking labeled software artifacts, where source artifacts linked to the same target artifact are considered similar samples, i.e., positive samples; and source artifacts not linked to the same target artifact are considered dissimilar samples, i.e., negative samples;

[0015] Traverse the labeled links, group the source artifacts that are linked to the same target artifact into a group, the number of groups is equal to the total number of target artifacts, and perform Cartesian product operations on all source artifacts; then traverse each source artifact pair, if the two source artifacts appear in any source artifact group at the same time, they are considered as positive samples; if the two source artifacts do not appear in any source artifact group at the same time, they are considered as negative samples, thereby generating contrastive learning training data;

[0016] Each word in the software product document is assigned a unique index. The software product text is represented as a sequence of word indexes. Each word is encoded as a vector using the pre-trained GloVe word embedding model and filled into each row of the matrix in index order to form a word embedding matrix.

[0017] Establish a Siamese LSTM network. The Siamese network consists of two identical LSTM networks. The word embedding matrix is ​​input into the embedding layer of the Siamese network as the weight. Each sentence can find the corresponding vector representation in the word embedding matrix according to its word index.

[0018] The text of the software artifact is input into the Siamese network and represented as a vector a = {w1, w2, …, w n}, where w i is the embedding representation of the i-th word in this text, with a vector length of n, and enters the long short-term memory network w at time t i Represented as x t , the output h of the long short-term memory network at the current moment t Depends on the current input x t , the output h at the previous moment t-1 and the network’s long-term memory unit C;

[0019] The Siamese network outputs the Manhattan distance between software products and trains the model through the contrast loss function to make similar products closer and dissimilar products farther; a list of similarities of software products is obtained and sorted in descending order of similarity.

[0020] Furthermore, the step (2) comprises the following steps:

[0021] (21) defining a ratio of the number of new labeled links to be generated according to the total number of training data sets of the traceability link recovery model, thereby determining the number of new labeled links to be added;

[0022] (22) Loop through the similarity lists of the source product and the target product, starting with index 0, extract the source product pair and the target product pair according to the index, and select the product pair with higher similarity to generate a new link;

[0023] (23) If the software artifact pair obtained in step (21) is a source artifact pair, the source artifact index is incremented by 1; if a link exists between the marked source artifact and the target artifact, and the marked source artifact is similar to the unmarked source artifact, then the unmarked source artifact and the target artifact are marked as having a link;

[0024] (24) If the software artifact pair obtained in step (21) is a target artifact pair, the target artifact index is incremented by 1; if the marked target artifact is linked to the source artifact, and the marked target artifact is similar to the unmarked target artifact, then the unmarked target artifact is marked as being linked to the source artifact;

[0025] (25) If the similarity between the source product pair and the target product pair in step (21) is equal, the source product index and the target product index are both increased by 1, and a new link is generated for the source product pair and the target product pair;

[0026] (26) Continue looping through the similarity lists of the source artifact and the target artifact until a sufficient number of new links are generated and the loop stops.

[0027] Furthermore, the implementation process of step (3) is as follows:

[0028] The data set of the traceability link recovery model is divided into training, validation and test sets, and the new links obtained in step (2) are added to the training set for the traceability link recovery model training.

[0029] Furthermore, the long short-term memory in step (12) enables the network to use a series of "gates" to control how information in the data sequence enters, is stored, and leaves the network;

[0030] Forget Gate forgot Decide what information to forget:

[0031] g forgot (t) = σ(W f × t +U f ×h t-1 )

[0032] According to the short-term memory unit h at the previous moment t-1 and the current input x t , generating a memory unit Input gate g in Select the information to be remembered and add it to the last long-term memory cell C filtered by the forget gate t-1 and updated to a new long-term memory unit C t :

[0033] g in (t) = σ(W i × t +U i ×h t-1 )

[0034]

[0035]

[0036] Output gate g out Select the information related to the current task at the current moment and generate the output h t :

[0037] g out (t) = σ(W o × t +U o ×h t-1 )

[0038] h t =g out ×yanh(C t )

[0039] Among them, W and U are weight matrices, σ is the sigmoid function, and tanh is the activation function;

[0040] The Siamese network finally outputs an n-dimensional vector h, and the Manhattan distance is used to measure the similarity between two vectors, which is defined as:

[0041] Similarity(h1,h2)=exp(-||h1-h2||1)

[0042] Where h1 and h2 are vectors of two software artifacts, and exp(.) is used to normalize the Manhattan distance between 0 and 1;

[0043] Contrastive loss is the goal of model training, making similar samples closer and different samples farther apart, and is defined as:

[0044]

[0045] Among them, m is a hyperparameter, which is the lower limit of the distance between different samples; y and A pair of two products i and a j The true and predicted labels of ; if the two products are not similar, that is, y = 0, then the equation minimizes their predicted values Otherwise, it will minimize That is, maximize the predicted value

[0046] Beneficial effects: Compared with the prior art, the present invention has the following beneficial effects: the present invention makes full use of the abundant unlabeled data in the data set, learns the similarity relationship between unlabeled and labeled software products, thereby marking the unlabeled data and adding it to the training data set of the traceability link recovery model, thereby increasing the number of training data sets; compared with not using unlabeled data, the present invention improves the performance of the traceability link recovery model. BRIEF DESCRIPTION OF THE DRAWINGS

[0047] Figure 1 is a flow chart of the present invention;

[0048] Figure 2 The comparative learning flow chart proposed by the present invention;

[0049] Figure 3 This is a schematic diagram of the long short-term memory network structure proposed in the present invention. DETAILED DESCRIPTION

[0050] The present invention is further described in detail below with reference to the accompanying drawings.

[0051] The present invention proposes a data enhancement method for training a traceability link recovery model based on unlabeled data, such as Figure 1 As shown, the following steps are included:

[0052] Step 1: Use a similarity prediction method to predict the similarity of unlabeled and labeled software products. The similarity prediction method used in the present invention includes a vector space model and contrastive learning.

[0053] First, the natural language text of the software artifact is preprocessed, including word segmentation, conversion to lowercase, punctuation removal, and stop word removal.

[0054] Secondly, the vector space model and contrastive learning method are used to predict the similarity between unlabeled and labeled software artifacts.

[0055] The first similarity prediction method is the vector space model. The vector space model is an information retrieval model. Documents and terms are mapped to vectors. The similarity between two documents can be obtained by calculating the cosine or inner product between the corresponding vectors. A higher similarity indicates that the two documents share more common terms. The commonly used weight measurement in the vector space model is the term frequency inverse document frequency method. The term frequency rewards words that appear frequently in documents, while the inverse document frequency penalizes words that appear in too many documents, that is, non-discriminatory words. If a term appears multiple times in a document and is included in a few documents, the term will be given a relatively high weight, that is, it is considered to be related to the product content. The term frequency inverse document frequency method pays more attention to words that appear frequently in documents and appear in a few documents.

[0056] The document space is established using the traceability link recovery model training data. The document space is represented by an m×n term-document matrix, where m represents the number of terms, n represents the number of documents, and the matrix element d i，j represents the weight of the i-th term in the j-th document, expressed as the inverse document frequency of the term frequency:

[0057] d i，j =tf i，j ×idf i

[0058] Among them, tf i，j is the frequency of the i-th word in the j-th document, idf i Defined as:

[0059]

[0060] Where |D| is the total number of documents, |{d:t i ∈d}| means that it contains the term t i The number of documents.

[0061] The vector space model takes all documents as corpus V and a new document as query Q, represented as vectors in the same way. j The similarity between them is calculated as follows:

[0062]

[0063] Among them, d i，j It is document D i A one-dimensional vector, q i is a one-dimensional vector of query Q. The vector space model represents documents as vectors, and the cosine distance between vectors represents the similarity between documents.

[0064] The second similarity prediction method is contrastive learning: Contrastive learning is an unsupervised learning method used to learn the representation function of unlabeled data, with the goal of making similar samples closer and different samples farther apart. The main processes of contrastive learning include data enhancement, encoding, and contrastive loss. Data enhancement means that the data to be classified in contrastive learning is unlabeled, and pseudo labels need to be generated based on some attributes of the data as training data, in order to let the model know which data samples are similar and which data samples are different. Encoding refers to converting data into a form that the model can accept, such as encoding text into a vector. After the data is input into the model, it is necessary to minimize the loss through the contrastive loss function to make similar samples closer and different samples farther apart.

[0065] Typically, there is a many-to-one relationship between software artifacts, for example, there are traceability links between multiple source artifacts and the same target artifact. Taking software problem reports and code submissions as an example, during the software development process, multiple users may submit different problem reports for the same software problem. These problem reports use similar text to describe the same problem, and there are links between these problem reports and the fix submissions for the problem. If a new problem report is raised and it also describes the software problem, there should also be a link between this problem report and the fix submission for the problem. The present invention links unlabeled artifacts to link objects of similar labeled artifacts, thereby labeling the unlabeled data.

[0066] like Figure 2 As shown in Figure 1, the process of contrastive learning includes data augmentation, encoding, and contrastive loss. Data augmentation is to generate training data, use labeled software artifact links, traverse the labeled links, and divide the source artifacts linked to the same target artifact into a group. The number of groups is equal to the total number of target artifacts. Cartesian product operations are performed on all source artifacts. Each source artifact pair is traversed. If the two source artifacts appear in any source artifact group at the same time, they are regarded as positive samples. If the two source artifacts do not appear in any source artifact group at the same time, they are regarded as negative samples, thereby generating contrastive learning training data.

[0067] Then the text information of the software product is encoded into a vector. Each word in the document of the software product is assigned a unique index. The software product text is represented as a sequence of word indexes. Each word is encoded into a vector by the pre-trained GloVe word embedding model and filled into each row of the matrix in index order to form a word embedding matrix.

[0068] Next, a Siamese LSTM network is established. The Siamese network consists of two identical LSTM networks. The word embedding matrix is ​​input into the embedding layer of the Siamese network as the weight. Each sentence can find the corresponding vector representation in the word embedding matrix according to its word index.

[0069] The text of the software artifact is input into the Siamese network and represented as a vector a = {w1, w2, ..., w n}, where w i is the embedding representation of the i-th word in this text, and the vector length is n, such as Figure 3 As shown, w entering the long short-term memory network at time t i Represented as x t , the output of long short-term memory at the current moment h t Depends on the current input x t , the output h at the previous moment t-1 and the network's long-term memory unit C, which uses a series of "gates" to control how information in a data sequence enters, is stored, and leaves the network.

[0070] Forget Gate forgot Decide what information to forget:

[0071] g forgot (t) = σ(W f × t +U f ×h t-1 )

[0072] Then, according to the short-term memory unit h at the previous moment t-1 and the current input xt , generating a memory unit Input gate g in Select the information to be remembered and add it to the last long-term memory cell C filtered by the forget gate t-1 and updated to a new long-term memory unit C t :

[0073] g in (t) = σ(W i × t +U i ×h t-1 )

[0074]

[0075]

[0076] Finally, the output gate g out Select the information related to the current task at the current moment and generate the output h t :

[0077] g out (t) = σ(W o × t +U o ×h t-1 )

[0078] h t =g out ×yanh(C t )

[0079] Among them, W and U are weight matrices, σ is the sigmoid function, and tanh is the activation function.

[0080] The Siamese network finally outputs an n-dimensional vector h, and the Manhattan distance is used to measure the similarity between two vectors, which is defined as:

[0081] Similarity(h1,h2)=exp(-||h1-h2||1)

[0082] Here, h1 and h2 are vectors of two software artifacts, and exp(.) is used to normalize the Manhattan distance between 0 and 1.

[0083] Contrastive loss is the goal of model training, making similar samples closer and different samples farther apart, and is defined as:

[0084]

[0085] Wherein, m is a hyperparameter used to define the lower limit of the distance between different samples. In the present invention, m is set to 1. y and A pair of two products i and a j The true and predicted labels (i.e., Manhattan distance) of . If the two products are not similar (i.e., y = 0), then the equation minimizes their predicted values Otherwise, it will minimize That is, maximize their predicted values

[0086] The similarity prediction method is used to predict the similarity of unlabeled and labeled software products, and a similarity list of software products is obtained, which is sorted in descending order of similarity.

[0087] Step 2: According to the similarity list obtained in step 1, link tags to unlabeled software products.

[0088] First, according to the total number of training data of the traceability link recovery model, a certain ratio is set to determine the number N of new links that need to be added.

[0089] Then, loop through the similarity lists of source artifacts and target artifacts, starting with index 0, take out the source artifact pair and the target artifact pair according to the index, compare the similarity, and select the artifact pair with higher similarity. In a high-similarity software artifact pair, the unmarked artifact is similar to the marked artifact, and a new link is generated between the link objects of the unmarked artifact and the marked artifact.

[0090] If the source artifact pair has a higher similarity, the source artifact index is incremented by 1, and a new link is generated from the source artifact pair. If the target artifact has a higher similarity, the target artifact index is incremented by 1, and a new link is generated from the target artifact pair. If the source and target artifacts have the same similarity, both the source and target artifact indexes are incremented by 1, and a new link is generated for both the source and target artifact pairs. Continue looping through the similarity lists of the source and target artifacts until the number of new links generated is greater than or equal to N, and the loop stops.

[0091] Step 3: Add the newly labeled link data obtained in step 2 to the traceability link recovery model training set.

[0092] The traceability link recovery model dataset is divided into training, validation and test sets. New links are added to the traceability link recovery model training set for model training. The performance changes of the traceability link recovery model before and after adding new links are evaluated.

[0093] The above content has described in detail the method for enhancing the training data of the traceability link recovery model based on unlabeled data of the present invention, but it is obvious that the specific implementation form of the present invention is not limited thereto. For those skilled in the art, various obvious changes made to it without departing from the spirit and scope of the present invention are within the protection scope of the present invention.

Claims

1. A data enhancement method for training a traceability link recovery model based on unlabeled data, characterized in that: The following steps are involved: (1) using a similarity prediction method to predict the similarity between unlabeled and labeled software products and obtain a similarity list; (2) link marking the unmarked software products according to the similarity list obtained in step (1); (3) Supplementing the newly labeled link data obtained in step (2) to the training data set of the traceability link recovery model; The step (1) comprises the following steps: (11) Performing text preprocessing on the natural language text of the software product, including word segmentation, conversion to lowercase, punctuation removal, and stop word removal; (12) Using contrastive learning to predict the similarity between unlabeled and labeled software artifacts: The training data for contrastive learning is generated by linking labeled software artifacts, where source artifacts linked to the same target artifact are considered similar samples, i.e., positive samples; and source artifacts not linked to the same target artifact are considered dissimilar samples, i.e., negative samples. Traverse the labeled links, group the source artifacts that are linked to the same target artifact into a group, the number of groups is equal to the total number of target artifacts, and perform Cartesian product operations on all source artifacts; then traverse each source artifact pair, if the two source artifacts appear in any source artifact group at the same time, they are considered as positive samples; if the two source artifacts do not appear in any source artifact group at the same time, they are considered as negative samples, thereby generating contrastive learning training data; Each word in the software product document is assigned a unique index. The software product text is represented as a sequence of word indexes. Each word is encoded as a vector using the pre-trained GloVe word embedding model and filled into each row of the matrix in index order to form a word embedding matrix. Establish a Siamese LSTM network. The Siamese network consists of two identical LSTM networks. The word embedding matrix is ​​input into the embedding layer of the Siamese network as the weight. Each sentence can find the corresponding vector representation in the word embedding matrix according to its word index. The text of the software artifact is input into the Siamese network and represented as a vector a = {w1, w2, …, w n }, where w i is the embedding representation of the i-th word in this text, with a vector length of n, and enters the long short-term memory network w at time t i It is represented by x t , the output h of the long short-term memory network at the current moment t Depends on the current input x t , the output h at the previous moment t-1 and the network’s long-term memory unit C; The Siamese network outputs the Manhattan distance between software products, and trains the model through the contrast loss function to make similar products closer and dissimilar products farther; obtains a list of software product similarities and sorts them in descending order of similarity; The LSTM network described in step (12) uses a series of "gates" to control how information in the data sequence enters, is stored, and leaves the network; Forget Gate forgot Decide what information to forget: g forgot (t)=σ(W f ×x t +U f ×h t-1 ) According to the short-term memory unit h at the previous moment t-1 and the current input x t , generate a memory unit Input gate g in Select the information to be remembered and add it to the last long-term memory cell C filtered by the forget gate t-1 and updated to a new long-term memory unit C t : g in (t)=σ(W i ×x t +U i ×h t-1 ) Output gate g out Select the information related to the current task at the current moment and generate the output h t : g out (t)=σ(W o ×x t +U o ×h t-1 ) h t =g out ×tanh(C t ) Among them, W and U are weight matrices, σ is the sigmoid function, and tanh is the activation function; The Siamese network finally outputs an n-dimensional vector h, and the Manhattan distance is used to measure the similarity between two vectors, which is defined as: Similarity(h1,h2)=exp(-||h1-h2||1) Where h1 and h2 are vectors of two software artifacts, and exp(.) is used to normalize the Manhattan distance between 0 and 1; Contrastive loss is the objective of model training and is defined as: Among them, m is a hyperparameter, which is the lower limit of the distance between different samples; y and A pair of two products i and a j The true and predicted labels of ; if the two products are not similar, that is, y = 0, then the equation minimizes their predicted values Otherwise, it will minimize That is, maximize the predicted value 2. The method for enhancing training data of a traceability link recovery model based on unlabeled data according to claim 1, characterized in that: The step (2) comprises the following steps: (21) defining a ratio of the number of new labeled links to be generated according to the total number of training data sets of the traceability link recovery model, thereby determining the number of new labeled links to be added; (22) Loop through the similarity lists of the source product and the target product, starting with index 0, extract the source product pair and the target product pair according to the index, and select the product pair with higher similarity to generate a new link; (23) If the software artifact pair obtained in step (21) is a source artifact pair, the source artifact index is incremented by 1; if a link exists between the marked source artifact and the target artifact, and the marked source artifact is similar to the unmarked source artifact, then the unmarked source artifact and the target artifact are marked as having a link; (24) If the software artifact pair obtained in step (21) is a target artifact pair, the target artifact index is incremented by 1; if the marked target artifact is linked to the source artifact, and the marked target artifact is similar to the unmarked target artifact, then the unmarked target artifact is marked as being linked to the source artifact; (25) If the similarity between the source product pair and the target product pair in step (21) is equal, the source product index and the target product index are both increased by 1, and a new link is generated for the source product pair and the target product pair; (26) Continue looping through the similarity lists of the source artifact and the target artifact until a sufficient number of new links are generated and the loop stops.

3. The method for enhancing training data of a traceability link recovery model based on unlabeled data according to claim 1, characterized in that: The implementation process of step (3) is as follows: The data set of the traceability link recovery model is divided into training, validation and test sets, and the new links obtained in step (2) are added to the training set for the training of the traceability link recovery model.

Images