Text Retrieval Method Based on Adaptive Triplet Update

Abstract

The present invention discloses a text retrieval method based on adaptive triple update. The main problem to be solved is to automatically identify duplicate software defect reports before they are submitted to software development and maintenance personnel, and no longer manually classify the defect reports, thus saving the labor cost of software maintenance. The present invention mainly includes two points: one is to solve the problem of difficult extraction of the deep representation of software defect reports. A deep learning model is used to extract the deep representation of software defect reports, and the classification information such as the version information and priority of the software defect reports is obtained in the form of vectors. After extracting the deep information, it is fused with the deep representations of the short description and long description; the other is to alleviate the problem that it is difficult to use triples to train software duplicate defect reports. An adaptive triple update method is proposed to select triples, and the selected triples are used for training.

Description

Technical Field

[0001] The present invention belongs to the field of software duplicate defect report detection. In this paper, deep features of software defect reports are extracted through deep learning methods, and an adaptive triple update method is proposed to obtain a retrieval model that can detect duplicates of the latest software defect reports in the software report library. Background Art

[0002] Due to the continuous optimization and update of software, software projects have become increasingly large and complex, making it more difficult to detect software defects. Although there are already excellent software testing teams that conduct sufficient tests on the software before it goes live, various problems still occur after the software is officially launched. The common practice adopted by software companies is to collect various problems encountered by users during the use of the software, organize them in the form of software defect reports, and send them to software maintenance personnel for software correction. Currently, there is a large amount of research work on duplicate defect detection at home and abroad, which can be divided into three aspects: methods based on text similarity, methods based on topic models, and methods based on deep learning. Jalbert et al. proposed a classifier for predicting duplicate defect reports based on text surface features and text similarity in the literature "N. Jalbert and W. Weimer, "Automated Duplicate Detection for Bug Tracking Systems," in Proc. IEEE International Conference on Dependable Systems and Networks, 2008, pp. 52-61." By directly counting the number of occurrences of words, the vector form of each defect description is obtained, and then cosine similarity is used to sort the defect reports, ultimately ensuring that at least one real defect reaches the software maintenance personnel. This method can filter out 8% of the defect report copies. With the development of topic models, Patel et al. proposed an efficient two-layer defect report copy detection system based on latent Dirichlet distribution (LDA) topic modeling and embedded feature fusion in the literature "N. Patel, A. Thangarajah, D. Shah, and R. Mehta, "Fast Detection of Duplicate Bug Report Using LDA-Based Topic Modeling and Clustering," in Proc. IEEE International Conference on Systems, Man, and Cybernetics, 2020, pp. 1622-1629." First, the traditional LDA model is used to assign the topic with the highest probability to each defect report, then FastText and Word2vec are combined to extract features from the obtained topics, and finally, cosine similarity is used for sorting after the extracted features are obtained.With the development of deep learning, Rodrigues et al. proposed adding an attention mechanism to the neural network in the literature "I. Rodrigues, D. Aloise, E. Fernandes, and M. Dagenais, 'ASoftAlignment Model for Bug Deduplication,' in Proc. Intemational Conference onMining Software Repositories, 2020, pp. 43-53." to obtain the SABD model. First, the Glove model is used to convert the text into vectors, then the attention mechanism is added to the defect report vectors, and then the fully connected layer is input to extract the deep representation. In addition, the classification information is encoded as a binary vector according to similarities and differences and is also input to the fully connected layer, and the obtained outputs are concatenated, and finally, the Sigmoid classifier is used for classification. Summary of the Invention

[0003] The present invention discloses a text retrieval method based on adaptive triple update, and the main problem to be solved is to automatically identify duplicate software defect reports before the software defect reports are submitted to software development and maintenance personnel, without manually classifying the defect reports, thus saving the labor cost of software maintenance. The present invention mainly includes two points: one is to solve the problem of difficult extraction of the deep representation of software defect reports, using a deep learning model to extract the deep representation of software defect reports, and obtaining the vector forms of classification information such as the version information and priority of software defect reports. After extracting the deep information, it is fused with the deep representations of the short description and long description; the other is to alleviate the problem that it is difficult to use triples to train software duplicate defect reports, and a method of adaptive triple update is proposed to select triples and use the selected triples for training. Specifically, the purpose of the present invention is to improve the following two aspects.

[0004] 1. Software duplicate defect reports contain short descriptions, long descriptions, and various classification information, which are pure text data. After the present invention combines the short description and the long description, the BERT model is used to obtain its deep representation. The various classification information is encoded using one-hot vectors, and the fully connected layer is used to extract features, and then it is concatenated with the deep representations of the short description and the long description, and finally the deep representation of the software defect report is obtained. This method makes full use of the classification information of the defect report and obtains a more complete deep representation of the defect report and better results compared with only using the short description and the long description.

[0005] 2. The present invention proposes a method for adaptive triple update, which adaptively selects triples during the training process of the BERT model, and tries to ensure that each triple found can well assist the training of the BERT model. Since randomly selecting triples is faster, the random method is used to select triples at the initial stage of training. When the validation loss no longer decreases, it indicates that the model learning has encountered a bottleneck. At this time, by setting a dynamic threshold, the triples that are most beneficial to the current model training are selected. This method can not only improve the training speed but also enhance the ability of the model to extract the deep representation of software defect reports.

[0006] The present invention proposes a new method for extracting the deep representation of software defect reports, which introduces an adaptive triple update method. The technical solution is as follows:

[0007] 1. Extraction of deep representations of short and long descriptions. The short description and long description of the software defect report are concatenated to obtain the total description, and the BERT model is used to extract the deep representation of the concatenated total description.

[0008] 2. Feature extraction of classification information and training of the network model. All classification information of the software is counted to obtain a classification information list, and each type of classification information is represented using a ONE-HOT vector. The fully connected layer is used to extract the features of the classification information, and the extracted features are concatenated with the deep representation of the total description to obtain the deep representation of the software defect report. During the training process, the optimal triples are reselected for training in each round.

[0009] 3. Obtaining the duplicate defect reports of the software defect report. The deep representations of the short description and long description are concatenated with the classification information to obtain the deep representation information of the software defect report. The Euclidean distance is calculated between it and all the reports in the software defect library, sorted from small to large, and the top 20 software defect reports are taken to generate a report list. The duplicate defect reports of the current software defect report are in this list.

[0010] Generally speaking, the present invention makes full use of the short description, long description, and classification information provided by the software defect report to obtain the deep representation of the software defect report. During training, the optimal triples are selected for training each time. Finally, the model is used to extract the deep representation of each software defect report, calculate the Euclidean distance between each pair, sort them from small to large, and the defect reports with close clustering are the duplicate defect reports.

[0011] The present invention has the following advantages over the existing software duplicate defect report detection methods:

[0012] 1. Obtain a deep representation that can fully express the information in software defect reports. Use the BERT model for the short description and long description of software defect reports, use ONE-HOT vectors for classification information, and use a fully connected layer to extract features, and then splice the two, making full use of all the information in software defect reports.

[0013] 2. Improve the detection effect of duplicate defect reports. Propose a method of using adaptive triple update. During training, adaptively select the optimal triples for training, improving the detection effect of duplicate defect reports.

[0014] A text retrieval method based on adaptive triple update disclosed by the present invention specifically includes the following steps:

[0015] Step 1: Clean the software defect report data, and convert the short description, long description, and component information of the software defect report into a dictionary form with positive values.

[0016] Step 2: Splice the short description and long description of the software defect report to obtain a total description, and use the BERT model to extract the deep representation of the spliced total description; count the component information and obtain its corresponding ONE-HOT vector to obtain the component information feature.

[0017] Step 3: Iteratively train the network model, splice the deep representation of the total description and the component information feature to obtain the deep representation of the software defect report. Among them, during the training process, the optimal triples are reselected for training in each round.

[0018] Step 4: Calculate the Euclidean distance between the deep representation information of the software defect report and all the defect reports in the existing software defect library, sort them from small to large to obtain a top list, and search for duplicate reports of the software defect report in the top list.

[0019] In one embodiment, in step 1,

[0020] For the cleaning of software defect report data, perform operations of removing stop words and special characters on the read short description and long description of the defect report to obtain the cleaned short description and long description.

[0021] Add the string "[CLS]" before the short description, and add the string "[SEP]" after the short description and long description to truncate the short description and long description, and splice the two to obtain the total description.

[0022] When using BERT to convert it into an embedding representation, each defect report needs to be aligned. In the present invention, the length of the short description is set to 30, and the length of the long description is set to 300. The part exceeding this length is truncated from the head, and the part less than this length is padded with "[PAD]" to the specified length. The set classification information includes "severity", "product", "status", "component" and "priority" dictionaries, and the text value of each category corresponds to a unique integer. Then, the short description, long description and component information of the software defect report are extracted.

[0023] In one embodiment, in step 2, the short description and long description of the software defect report are concatenated to obtain the total description, and the BERT model is used to extract the deep representation of the concatenated total description, including

[0024] The total description is processed by BERT tokenization, and the corresponding dictionary positions are obtained through the tokenization dictionary;

[0025] A segment vector is set, and the length of this vector is equal to the length of the total description. The numbers 2, 1, and 0 are used to correspond to the short description, long description, and the part padded with "[PAD]" in the total description respectively;

[0026] The BERT encoding is used to obtain the deep representation matrix of the total description. Each token is converted into a vector with a dimension of 768. The first row of this deep representation matrix is taken out, and this vector is used as the deep representation of the total description;

[0027] The component information is counted and its corresponding ONE-HOT vector is obtained to get the component information feature;

[0028] The integers corresponding to the above obtained classification information are converted into binary by the ONE-HOT method;

[0029] The ONE-HOT vectors of the five classification information are concatenated to obtain the embedding representation of the classification information;

[0030] The fully connected layer is used to extract the features of the embedding representation of the classification information to obtain the final embedding representation of the classification information.

[0031] In one embodiment, in step 3, the iterative training network model concatenates the deep representation of the total description and the component information feature to obtain the deep representation of the software defect report. Among them, in each round of training, the optimal triple is reselected for training:

[0032] According to the true labels of the software defect reports, the defect reports of the same category are paired in pairs to obtain positive sample pairs;

[0033] Traverse all positive sample pairs and randomly select negative samples from the software defect report dataset, where the negative samples should satisfy:

[0034]

[0035] Among them represents the selected anchor sample, represents the positive sample of the same category as the anchor, represents the negative sample of a different category from the anchor. f(·) represents the mapping of the sample in the deep representation space, represents all possible triples in the sample, where each triple includes three defect reports. The first defect report is of the same category as the second defect report, and the third defect report is of a different category from the first defect report. Set the increment d m , whose maximum value is α, where the step size s of each increase in d is:

[0036]

[0037] Among them, α is a hyperparameter, indicating the minimum distance between positive and negative samples in the ideal state of training. d0 is the initial value of d, and N represents the maximum number of times to search for negative samples;

[0038] The loss function of this model is:

[0039]

[0040] Among them, [x] + means that when the value of x is less than 0, it is assigned a value of 0;

[0041] Fine-tune the parameters of the BERT model and the fully connected layer in the training classification information to obtain a model for well extracting the deep representation of software defect reports.

[0042] In one of the embodiments, in step 4, calculate the Euclidean distance between the deep representation information of the software defect report and all the defect reports in the existing software defect library, sort them from small to large, take out the top 20 software defect reports and generate a list. The 20 software defect reports in this list are the ones most similar to the current software defect report, and the duplicate defect reports are in this list. This list is also called the top list, and search for the duplicate reports of the software defect report in the top list. Brief Description of the Drawings

[0043] Figure 1 is a flowchart of a text retrieval method based on adaptive triple update of the present invention. Detailed Embodiments

[0044] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the following will further clarify the present invention in combination with the specific embodiments in the accompanying drawings of the embodiments of the present invention. It should be understood that these embodiments are only used to illustrate the present invention and not to limit the scope of the present invention. After reading the present invention, all other embodiments obtained by those skilled in the art without creative labor belong to the scope of protection of the present invention.

[0045] The present invention proposes a text retrieval method based on adaptive triple update. Referring to Figure 1 , the implementation steps of the present invention are as follows:

[0046] Step 1: Clean the software defect report data. Perform the operation of removing stop words and special characters on the short description and long description of the read defect report to obtain the cleaned short description and long description. Then add the string "[CLS]" before the short description and the string "[SEP]" after the short description and long description to truncate the short description and long description, and then splice the two to obtain the total description. Since the length of each defect report is different, when using BERT to convert it into an embedding representation, alignment processing needs to be performed on each defect report. The present invention sets the length of the short description to 30 and the length of the long description to 300. The part exceeding this length is truncated from the head, and the part less than this length is padded with "[PAD]" to the specified length. Set the classification information to include dictionaries of "severity", "product", "status", "component", and "priority", and the text value of each category corresponds to a unique integer. Furthermore, extract the short description, long description, and component information of the software defect report;

[0047] Step 2: Splice the short description and long description of the software defect report to obtain the total description, and use the BERT model to extract the deep representation of the spliced total description; specifically, first perform BERT tokenization on the total description, and obtain the corresponding dictionary positions through the tokenization dictionary. Then set the segment vector, the length of which is equal to the length of the total description, and use the numbers 2, 1, and 0 to correspond to the short description, long description, and the part padded with "[PAD]" for alignment in the total description respectively. Then use BERT encoding to obtain the deep representation matrix of the total description. Each token is converted into a vector with a dimension of 768. Take out the first row of the deep representation matrix and use this vector as the deep representation of the total description. Count the component information and obtain its corresponding ONE-HOT vector to obtain the component information feature. Convert the integer corresponding to the above classification information into binary using the ONE-HOT method, then splice the ONE-HOT vectors of the five classification information to obtain the embedding representation of the classification information, and then use a fully connected layer to extract the feature of the embedding representation of the classification information to obtain the final embedding representation of the classification information;

[0048] Step 3: Iteratively train the network model, concatenate the total description depth representation and the component information features to obtain the software defect report depth representation. Specifically, during each round of training, the optimal triplets are reselected for training. Specifically, according to the true labels of the software defect reports, pair the defect reports of the same type in pairs to obtain positive sample pairs. Then traverse all the positive sample pairs and randomly select negative samples from the software defect report dataset. The negative samples should satisfy:

[0049]

[0050] where represents the selected anchor sample, represents the positive sample of the same type as the anchor, represents the negative sample of a different type from the anchor. f(·) represents the mapping of the sample in the depth representation space, represents all possible triplets in the sample. Each triplet includes three defect reports. The first defect report is of the same type as the second defect report, and the third defect report is of a different type from the first defect report. Set the self-incrementing d m , whose maximum value is α. The step size s for each increase of d is:

[0051]

[0052] where α is a hyperparameter indicating the minimum distance between the positive sample and the negative sample in the ideal state of training. In the present invention, it is set to 1. d0 is the initial value of d, which is set to 0.1 in the present invention, and N represents the maximum number of times to search for negative samples. When the training effect of the model is good enough, the distance between the anchor sample and the positive sample is very small, and the distances between the anchor sample and all negative samples are very large. Then the above formula is always not established, that is, it is difficult to select triplets. Therefore, it is necessary to set the maximum number of times N to select negative samples to handle the situation where no triplets need to be selected for training.

[0053] The loss function of this model is:

[0054]

[0055] where [x] + means that when the value of x is less than 0, it is assigned 0.

[0056] Through the above training method, fine-tune the parameters of the BERT model and the fully connected layer in the training classification information to obtain a model for well extracting the software defect report depth representation.

[0057] Step 4: Calculate the Euclidean distance between the deep characterization information of the software defect report and all the defect reports in the existing software defect library, sort them from smallest to largest, take out the top 20 software defect reports and generate a list. The 20 software defect reports in this list are the ones most similar to the current software defect report, and the duplicate defect reports are in this list. This list is also called the top list. Search for the duplicate report of the software defect report in the top list.

[0058] The effects of the present invention can be further illustrated by the following experiments.

[0059] 1. Simulation conditions

[0060] The present invention is based on a central processing unit of i7-10700F 2.90GHz CPU, 16G of memory, and the WINDOWS 10 operating system. Experiments are conducted using the python language.

[0061] 2. Simulation content

[0062] In this paper, classical text retrieval metrics are used to evaluate the present invention, including Recall@K and Mean Average Precision (MAP) metrics. Here, K is set to 20, indicating that the top 20 reports most similar to the current defect report are selected. As long as the duplicate defect report of the current defect report is in this list, it is considered that the duplicate defect report has been found. During the experiment, timestamps of 1 year and 3 years are set respectively, representing searching only the report library within 1 year before the creation time of the current defect report and the report library within the previous three years. To prove the effectiveness of the present invention, three open-source large software defect report datasets, namely OpenOffice, Eclipse, and Netbeans, are used. And comparisons are made with another 5 methods, namely DWEN, Siamese Triplet, Siamese Pair, BM25Fext, and REP. Among them, the first three are deep learning methods, and the last two are traditional methods. The experimental results are as follows:

[0063] Table 1 Recall@20 and MAP metrics

[0064]

[0065] As can be seen from Table 1, the Recall@20 and MAP metrics of the present invention are better than other methods in both the OpenOffice and Netbeans datasets, and are second only to the REP model in the Eclipse dataset, being better than the other four models, which demonstrates the effectiveness of the present invention. Among them, OUR-RT represents randomly selecting triples, and OUR-AT represents adaptively selecting triples. The effect of adaptively selecting triples is better than the random effect, which demonstrates the effectiveness of adaptively selecting triples.

[0066] The above embodiments are preferred embodiments of this patent. However, the embodiments of the present invention are not limited by the above embodiments. Any other changes, modifications, substitutions, combinations, and simplifications made without departing from the spirit and principle of the present invention shall be equivalent replacement methods and are all included in the protection scope of the present invention.

Claims

1. A text retrieval method based on adaptive triple update, the specific steps are as follows: Step 1: Clean the software defect report data, and convert the short description, long description and component information of the software defect report into a dictionary form with positive values; Step 2: Concatenate the short description and long description of the software defect report to obtain the total description, and use the BERT model to extract the deep representation of the concatenated total description; count the component information and obtain its corresponding ONE-HOT vector to get the component information feature; Step 3: Iteratively train the network model, and splice the total description depth representation and the component information feature to obtain a software defect report depth representation, where During the training process, the optimal triples are reselected for training in each round. Among them, each triple includes three defect reports. The first defect report is of the same category as the second defect report, and the third defect report is of a different category from the first defect report; Step 4: Calculate the Euclidean distance between the deep representation information of the software defect report and all the defect reports in the existing software defect library, sort them from small to large to obtain the top list, and search for duplicate reports of the software defect report in the top list.

2. The method according to claim 1, wherein: In the above Step 1, For the cleaning of the software defect report data, the operations of removing stop words and special characters from the read short description and long description of the defect report are performed to obtain the cleaned short description and long description; Add the string "[CLS]" before the short description, and add the string "[SEP]" after the short description and long description to truncate the short description and long description, and concatenate the two to obtain the total description; When using BERT to convert it into an embedded representation, alignment processing needs to be performed on each defect report. In the present invention, the length of the short description is set to 30, and the length of the long description is set to 300. The part exceeding this length is truncated from the head, and the part less than this length is padded with "[PAD]" to the specified length. Among them, the classification information is set to include dictionaries of "severity", "product", "status", "component" and "priority". The text value of each category corresponds to a unique integer, and then the short description, long description and component information of the software defect report are extracted.

3. The method according to claim 1, wherein: In the above Step 2, concatenating the short description and long description of the software defect report to obtain the total description and using the BERT model to extract the deep representation of the concatenated total description includes, Performing BERT tokenization on the total description and obtaining the corresponding dictionary positions through the tokenization dictionary; Set the segment vector, the length of this vector is equal to the length of the total description, and use the numbers 2, 1, 0 to correspond to the short description, long description and the part padded with "[PAD]" for alignment in the total description respectively; Using BERT encoding to obtain the deep representation matrix of the total description. Each token is converted into a vector with a dimension of 768. Take out the first row of this deep representation matrix and use this vector as the deep representation of the total description; Count the component information and obtain its corresponding ONE-HOT vector to get the component information feature; Convert the integer corresponding to the above obtained classification information into binary using the ONE-HOT method; Concatenate the ONE-HOT vectors of the five classification information to obtain the embedded representation of the classification information; Use a fully connected layer to extract the features of the classification information embedding representation and obtain the embedding representation of the final classification information.

4. The method according to claim 1, wherein: In step 3, the iterative training network model concatenates the total description depth representation and the component information features to obtain the software defect report depth representation. During the training process, the optimal triplet is reselected for training in each round: According to the true labels of software defect reports, pair up the defect reports of the same type in pairs to obtain positive sample pairs; Traverse all positive sample pairs and randomly select negative samples from the software defect report dataset. The negative samples should satisfy: Among them represents the selected anchor samples represents the positive samples of the same category as the anchor represents the negative samples of different categories from the anchor, f(·) represents the mapping of the samples in the deep representation space, τ represents all possible triplets in the samples, and set to increment d m , whose maximum value is α, where the step size s for each increase of d is: where α is a hyperparameter indicating the minimum distance between positive and negative samples in the ideal training state, d0 is the initial value of d, and N represents the maximum number of times to search for negative samples; the loss function of this model is: where [x] + means that when the value of x is less than 0, it is assigned a value of 0; Fine-tune the parameters of the BERT model and the fully connected layer in the training classification information to obtain a model for well extracting the software defect report depth representation.

5. The method according to claim 1, characterized in that: In step 4, calculate the Euclidean distance between the depth representation information of the software defect report and all the defect reports in the existing software defect library, sort them from smallest to largest, take out the top 20 software defect reports and generate a list. The 20 software defect reports in this list are the most similar to the current software defect report, and the duplicate defect reports are in this list. This list is also called the top list, and search for the duplicate report of the software defect report in the top list.

Images