SATD identification method based on query strategy combining representativeness and uncertainty

Abstract

The invention is suitable for the technical field of computers, and provides an SATD identification method based on a query strategy combining representativeness and uncertainty, and the method comprises the following steps: data preprocessing and representation; constructing an initial training set; data selection based on a query strategy; stopping training; and analyzing a result. According to the method, the SATD recognition model trained on the basis of the BERT model is superior to an original method in F1 score index; a representative and uncertainty combined query strategy and an active learning method are adopted, a model with excellent performance can be obtained only by training part of data rich in information amount, and the manpower labeling cost is greatly reduced; the specific value of the number of all marked SATD annotations to the number of SATDs in all training sets serves as a stop threshold value, and flexible strategies are provided for different application scenes so as to obtain different SATD recognition models. According to the method, the SATD in the code annotation can be accurately and efficiently identified under the condition of using less data and label cost.

Description

Technical Field

[0001] The present invention belongs to the field of computer technology, and particularly relates to a SATD recognition method based on a combined representative and uncertainty query strategy. Background Art

[0002] In the process of software development and subsequent iterations, due to various factors, developers are prone to produce low-quality code that cannot meet the actual requirements or will affect the stability of the software. We usually metaphor this as technical debt. Self-Admitted Technical Debt (SATD) is a variant of it, which is actively introduced by developers through code comments during the software development process. It increases the software maintenance cost and urgently needs to be discovered in a timely manner.

[0003] Currently, many methods for automatically identifying SATD have emerged in the research, including unsupervised models based on natural language processing, traditional machine learning, deep learning, pattern matching, etc. Although deep learning can process a large amount of training data and has good performance, it relies on a large amount of labeled data, and collecting and annotating it is time-consuming and costly; active learning can select a small number of samples from unlabeled data for labeling and training to reduce the labeling cost. Therefore, combining active learning with deep learning has become a feasible solution to maximize the model performance under limited label costs / budgets in recent years. BERT in the Transformer architecture of deep learning is a pre-trained language model based on the Transformer encoder. With its bidirectional attention mechanism, it can efficiently model language context dependencies, be able to learn bidirectional context information, and is widely used in natural language processing tasks. Its multi-head self-attention and deep stacking structure make it an important model architecture in the field of natural language processing. There is a very important application in the field of natural language processing, that is, text classification, that is, based on the content of the text, predict the category to which the text belongs. This binary classification method can also be used in the SATD recognition task based on code comments.

[0004] The Yuan Meng team at Jilin University in China uses the BERT model based on the Transformer architecture to train the SATD recognition model, proposes an improved loss function for the class imbalance problem of self-admitted technical debt, and also proposes an interpretation scheme based on mathematical statistics. Combining existing deep learning interpretation tools, it analyzes the experimental result outliers from multiple angles and discovers the essential differences between different detection models. However, this research has the following disadvantages:

[0005] (1) High cost of manual annotation: This research does not study reducing the cost of manual annotation, which means that a large amount of manpower is required to annotate data in actual applications, increasing the time and economic costs.

[0006] (2) Lack of a flexible stopping strategy: The study did not provide a flexible stopping method for balancing performance and cost, and was unable to flexibly adjust the balance between labor cost and model performance according to the requirements of different application scenarios.

[0007] (3) Failure to fully utilize more valuable data: The study did not introduce an active learning strategy and was unable to select more informative data for annotation. It only used deep learning methods and lacked a process for selecting more valuable data. All data had the same weight, and more valuable data could not be fully utilized.

[0008] (4) Limited improvement in the imbalance problem: Although the study proposed an improved loss function to handle the imbalance problem of the SATD category, this method may not fully solve the problems brought by the imbalanced dataset in some cases. In contrast, active learning and uncertainty query strategies may be more effective in solving this problem to a certain extent.

[0009] Based on the problems existing in the above prior art, the present invention proposes a SATD recognition method based on combining representative and uncertainty query strategies. Summary of the Invention

[0010] The object of the present invention is to provide a SATD recognition method based on combining representative and uncertainty query strategies, aiming to solve the problems proposed in the above background technology.

[0011] The object of the present invention is achieved through the following technical solutions:

[0012] A SATD recognition method based on combining representative and uncertainty query strategies, comprising the following steps:

[0013] Step 1: Data preprocessing and representation;

[0014] First, clean the code comment text data, then perform word segmentation using a BERT tokenizer or word embedding processing using a TF-IDF vectorizer according to the data usage, and finally use a BERT model to convert the input data into embedding vectors, and extract the embedding vector of the [CLS] token as the encoder of the data to express the overall information of the data.

[0015] Step 2: Construct an initial training set;

[0016] In the initial stage of active learning, use a representative-based method to select informative instances from the training set to construct an initial training set;

[0017] Step 3: Data selection based on query strategy;

[0018] Select the next batch of code comments to be labeled and used for training from the unlabeled data according to the training results of the model and the representativeness and uncertainty of the data;

[0019] Step 4: Stop training;

[0020] After continuously repeating Step 3, decide whether to stop training according to the stopping strategy;

[0021] Step 5: Result analysis;

[0022] Use the F1 score evaluation method based on the confusion matrix to measure the effectiveness, and use cost to measure the efficiency.

[0023] Furthermore, in the step of using the BERT tokenizer for word segmentation or the TF-IDF vectorizer for word embedding processing according to the data usage, for the data to be used for model training, use the pre-trained BERT tokenizer for word segmentation, and add "[CLS]" before each piece of data; subsequently, for each segmented word, select the corresponding index in the pre-trained BERT vocabulary; set the data length to 70, and fill any value less than 70 with 0, and set the length of the mask value to 70; for the data used to construct the initial training set, use the TF-IDF vectorizer as the encoder to perform word embedding on it.

[0024] Furthermore, the specific process of Step 2 is as follows:

[0025] First, use the TF-IDF vectorizer to perform word embedding representation on the data, then use the KNN model to query the similar examples of each instance and obtain the Euclidean distance, take the square of the reciprocal of the distance mean as the data density and define it as the representativeness score, and finally select the batch of instances with the lowest representativeness score from the instances containing keywords as the initial training set;

[0026] The keywords are obtained by the unsupervised method MAT by matching the four task annotation tags of "TODO", "FIXME", "XXX", and "HACK".

[0027] Furthermore, the specific process of Step 3 is as follows:

[0028] Obtain the uncertainty score: Adopt the uncertainty sampling strategy, calculate the uncertainty score of the data according to the posterior probability of the data being SATD or non-SATD, obtain the highest value in the binary classification output probability of the current training model for each unlabeled data, calculate the difference between it and 0.5 and use the difference as the uncertainty score;

[0029] Obtain representative scores: The data is converted into embedding vectors by a pre-trained BERT model. The embedding vector of the [CLS] token is extracted from them as the encoder of the data to represent the overall information of the data. The KNN model is trained with all the training set data. Similar examples are queried for each unlabeled data and the Euclidean distance is obtained. The square of the reciprocal of the distance mean is used as the data density and defined as the representative score;

[0030] Comprehensively select instances: Multiply the representative score and the uncertainty score to obtain the final score, and select a batch of instances with the lowest scores for subsequent training.

[0031] Further, in step 4, the stopping strategy is to determine the stopping time by comparing the ratio of the number of all labeled code comments to the number of codes in all training sets.

[0032] Compared with the prior art, the beneficial effects of the present invention are:

[0033] 1. Strong professionalism: The present invention trains a model that can identify SATDs in code comments for the SATD classification task and performs excellently in the field of SATD recognition.

[0034] 2. High recognition performance: The SATD recognition model trained based on the BERT model in the present invention is superior to the original method in terms of the F1 score index.

[0035] 3. Reduce labor costs: The present invention adopts a query strategy combining representativeness and uncertainty and an active learning method, fully considering the general features of data in the form of embedding vectors as text representations, calculating its representative score and uncertainty score, and only requiring partial information-rich data for training to obtain an excellent model, greatly reducing the labor annotation cost.

[0036] 4. Flexibly balance performance and cost: The present invention uses the ratio of the number of all labeled SATD comments to the number of SATDs in all training sets as the stopping threshold, providing a flexible strategy for different application scenarios to obtain different SATD recognition models.

[0037] 5. Accurately and efficiently identify: The present invention can accurately and efficiently identify SATDs in code comments with less data and label costs.

[0038] 6. Innovative cost-benefit optimization: The present invention optimizes the cost-benefit for the SATD recognition task, improves effectiveness, and provides an innovative method for the identification and research of technical debt in software development. BRIEF DESCRIPTION OF THE DRAWINGS

[0039] Figure 1 It is a flowchart of the method of the present invention. DETAILED DESCRIPTION OF THE INVENTION

[0040] For a clearer understanding of the technical features, objectives, and beneficial effects of the present invention, the technical solution of the present invention will be described in detail below, but it should not be construed as a limitation on the scope of implementation of the present invention.

[0041] An embodiment of the present invention provides a SATD recognition method based on a combined representative and uncertainty query strategy, and its flowchart is as Figure 1 shown, and the method includes the following steps:

[0042] Step 1: Data preprocessing and representation;

[0043] In order to enable existing data to act on model training and allow the query strategy to calculate the representative score and uncertainty score of the existing data through its special text representation, it is necessary to perform multi-step processing on the text data of code comments and represent it in different ways at different stages of the method, so as to provide better support for model training.

[0044] Use common data preprocessing methods in the SATD recognition field to clean the text data of code comments. First, delete all characters other than letters, numbers, parentheses, commas, plus signs, exclamation marks, question marks, single quotes, and backticks; secondly, split the words, such as separating all text abbreviations like ('s, 'll, 'd), etc.; finally, convert all letters to lowercase letters. This is a common data cleaning method, and using this method can improve data consistency, reduce noise interference, improve word segmentation, reduce feature dimensions, and enhance model performance.

[0045] For the cleaned data, it is processed in two ways respectively: (1) For the data to be used for model training, use a pre-trained BERT tokenizer for word segmentation. In order to make the data meet the data format requirements of the BERT model, it is necessary to add "[CLS]" before each piece of data; subsequently, for each segmented word, select the corresponding index in the pre-trained BERT vocabulary; since the BERT model has strict requirements on the length of the input data, the data length is set to 70 in this method, and any value less than 70 is filled with 0; in addition, for the BERT model, the present invention needs to calculate the mask value, and here the length of the mask value is also set to 70. (2) For the data used to construct the initial training set, use a TF-IDF vectorizer as an encoder to perform word embedding on it. This method converts text data into numerical vectors by calculating word frequency and inverse document frequency, improving the accuracy of text analysis.

[0046] The input data is transformed into a set of embedding vectors through the hidden layer (a multi-layer stacked Transformer encoder module) in the pre-trained BERT model. The embedding vector of the [CLS] token is extracted from them as the encoder of the data to represent the overall information of the data, because it is used to represent the global semantic information of the entire input sequence. This data representation method is beneficial for the query strategy to calculate the representative score and the uncertainty score for the data, so as to select the most informative instances for training according to the two scores.

[0047] In the BERT model based on the Transformer architecture, it should sequentially include a text input embedding layer (including word embedding, position embedding, and segment embedding), a multi-layer stacked Transformer encoder module (composed of a multi-head self-attention mechanism and a feed-forward neural network), an output layer for global feature extraction, and a task-specific classification layer.

[0048] Step 2: Construct the initial training set;

[0049] At the initial stage of the active learning process, it is crucial to select seed data to start the whole process, and the quality of the selection will greatly affect the performance in the initial stage of active learning. In the present invention, a representative-based method is used to select a small number of the most informative instances from the training set as the initial training set, thus solving the problem of how to start the active learning process.

[0050] Specifically, the KNN model is used to query several similar examples for each instance in the training set and obtain the corresponding Euclidean distances. It should be noted here that the TF-IDF vectorizer is used as the encoder for the data to perform word embedding as the data representation form. After obtaining the distance data, the square of the reciprocal of the distance mean is used as the data density and defined as the representative score. A batch of instances with the lowest representative scores are selected from the instances with keywords as the initial training set. The keywords come from the unsupervised method MAT, which is a simple heuristic method to identify SATD annotations by matching four representative task annotation labels of "TODO", "FIXME", "XXX", and "HACK". We believe that the instances with such keywords are more likely to be SATD annotations and will have richer information for the class-imbalanced task of SATD recognition.

[0051] Step 3: Data selection based on the query strategy;

[0052] According to the training results of the model and the representativeness and uncertainty of the data, the next batch of code annotations to be labeled and used for training are selected from the unlabeled data. The specific process is as follows:

[0053] Obtaining Uncertainty Scores: Uncertainty sampling is the simplest and most commonly used query strategy, which tends to select the instances that are judged to be the most uncertain according to the model output. Intuitively, these are the instances closer to the decision boundary. In the present invention, the uncertainty score of the data is calculated based on the posterior probability of whether the data is SATD or non - SATD. Specifically, the higher value in the binary classification output probabilities of the currently trained model for each unlabeled data is obtained, and the difference between this higher value and 0.5 is calculated. This is because the higher value of the output probability of an instance closer to the decision boundary will be closer to 0.5, which means it has higher uncertainty. Therefore, through such a difference calculation, the smaller the difference of an instance, the higher its uncertainty, and this difference is regarded as the uncertainty score of the data.

[0054] Obtaining Representative Scores: The first step of the query strategy in the present invention is to obtain the distance between each unlabeled data and its similar data in the model feature space, and calculate the representative score of the data based on the distance. The specific implementation process is as follows: When the input text data passes through the hidden layer in the pre - trained BERT model, it will be transformed into a set of embedding vectors. The embedding vector of the [CLS] token is extracted from them as the encoder of the data to represent the overall information of the data, because it is used to represent the global semantic information of the entire input sequence. Immediately afterwards, a KNN model is trained with all the data in the training set, and the trained model is used to query several similar examples for each unlabeled data and obtain the corresponding Euclidean distance. In this way, the distance between the data and its similar data is obtained. The square of the reciprocal of the distance mean is used as the data density and defined as the representative score.

[0055] Comprehensively Selecting Instances: Apply the above two steps to all candidate instances, obtain their representative scores and uncertainty scores, and multiply the representative score and the uncertainty score to calculate the final score. The instances with lower final scores have more informative content, and a batch of instances with the lowest scores are selected for the subsequent training.

[0056] The pseudo - code of the query strategy is as follows:

[0057]

[0058] The relevant information is as follows:

[0059] Input: Training set C t , Unlabeled annotations C u , Labeled annotations C l , Query number a, Model M, Neighbor number n, Encoder Ф(.);

[0060] Output: Sampled annotations X;

[0061] Use the labeled annotations C l To train the model M;

[0062] For each data c in the training set C t perform the following operations: p Obtain distances and neighbors according to the formula

[0063] where is one of the n neighbors of a single training set data c and p represents the Euclidean distance between a single training set data c and the neighbor p , and Φ(C ) represents the encoding of all data in the training set. t

[0064] Calculate the representative score according to the formula where s r is the representative score, is the square of the average of the sum of the distances between data c p and its n neighbors.

[0065] Calculate the uncertainty score according to the formula p(y|c p ) ← M(c p ) and s u = p(y|c p ) - 0.5; where M(c p ) is the predicted output of the model for c p , p(y|c p ) is the probability of the larger class in the binary classification of c p , and s u is the uncertainty score.

[0066] Calculate the final score according to the formula s cp = s u * s r ; where s cp is the final score.

[0067] End the above calculation operations for each data c in the training set Ct p .

[0068] Select a batch of data according to the formula |X| = a; where X is the selected data, |X| is the number of X, and a is the number to be obtained in the word iteration, and u represents sorting each c in all unlabeled data C p in ascending order of the final score.

[0069] Return the selected data X.

[0070] ​Step 4: Stop training;

[0071] After continuously repeating Step 3, a certain stopping strategy will be used to determine whether to stop training. To find a balance between model performance and cost, and ensure that better performance can be obtained with as little labeled data as possible, thereby improving the efficiency of active learning, the present invention determines when to stop training by comparing the ratio of the number of all labeled code comments to the number of codes in all training sets. For example, when the preset stopping threshold is 0.5, it means that when the number of labeled code comments reaches half of the number of code comments in all training sets, training is immediately stopped to avoid unnecessary resource waste and ensure model performance.

[0072] Step 5: Result analysis:

[0073] After training stops and the final model for the SATD identification task is obtained, the entire method needs to be evaluated to test the performance of the model and the effectiveness of the entire method in terms of the cost of manual annotation. To evaluate the performance of this method, the F1 score based on the confusion matrix is used as the effectiveness evaluation index, and cost is used as the efficiency evaluation index.

[0074] For a typical binary classification problem of classifying code comments as SATD and non - SATD, the predictions of the classifier on the results in the experiment can be divided into four cases: FP (False Positive), that is, the number of non - SATD that are mispredicted as SATD; FN (False Negative), that is, the number of SATD that are mispredicted as non - SATD; TP (True Positive), that is, the number of SATD that are correctly predicted as SATD; TN (True Negative), that is, the number of non - SATD that are correctly predicted as non - SATD.

[0075] Since the present invention pays more attention to the correct classification of SATD, SATD is regarded as the positive sample. Under this setting, Precision represents the ratio of correctly classified among all comments classified as SATD; Recall represents the ratio of SATD comments among the correctly classified comments. And the F1 score, as the harmonic mean of precision and recall, can combine the above two indicators and more comprehensively reflect the performance of the model. Because in many cases, an increase in Precision often comes with a decrease in Recall, simply using these two statistical data to evaluate the performance of the model is not comprehensive enough, so the F1 score is used as the evaluation index.

[0076] In addition, "cost" is used to measure the efficiency of this method. Specifically, it is used to measure the proportion of code comments marked when this method achieves the best performance and other expected performances on the test set in the entire training set. For example, when a certain project reaches the best performance and the ratio of the labeled data to all data is 0.5, then it is considered that the cost for it to reach the best performance is 0.5. Obviously, the lower the value of this metric, the higher the efficiency of this method, and better model performance can be achieved with less annotation cost.

[0077] The following describes the specific implementation of the present invention in detail with specific embodiments.

[0078] Embodiment 1: This method can provide a high-performance model for the SATD recognition task. In this embodiment, through a commonly used dataset Groovy in the SATD field, the main operations and effects of this method are briefly described. The Groovy dataset contains code comments about the project.

[0079] (1) Data representation;

[0080] For the code comments in the dataset, according to the above processing method, general tools are used to complete text cleaning, and the TF-IDF vectorizer is used to complete word embedding representation, and the BERT model is used to generate the embedding vector of the [CLS] token. Table 1 shows the representation form of the data with one piece of data in this dataset as an example.

[0081] Table 1 Content and encoding of Groovy example code comments

[0082]

[0083] (2) Construct the initial training set;

[0084] Using the word embedding representation as shown in Table 1, the word embedding representations of all data are given to KNN for processing, and a corresponding density is generated for each piece of data. Thus, data suitable for constructing the initial training set is selected according to the density and the presence of keywords. Table 2 shows some code comments in this dataset and their densities. We will sort according to the density and select a part of the code comments with the largest density among the code comments with keywords as the data in the initial training set.

[0085] Table 2 Content and density of Groovy example code comments

[0086]

[0087]

[0088] (3) Select data;

[0089] Use a query strategy that combines uncertainty and representativeness to calculate the representativeness score and uncertainty score for each piece of data, and select the lowest final score to determine the data to be selected. Table 3 shows the token embeddings, representativeness scores, uncertainty scores, and final scores for 5 example pieces of data.

[0090] Table 3 Embedding Vectors and Corresponding Scores of Groovy Example Code Annotations

[0091]

[0092]

[0093]

[0094] (4) Stop training;

[0095] Only need to decide whether to stop training according to the ratio of the labeled data to all the data. The ratio can be determined according to the requirements for performance and labor cost. For example, when there is no need to save labor cost at all, all the data can be put into training, and the ratio will be 1 at this time.

[0096] (5) Result analysis;

[0097] The present invention uses an active learning method based on a query strategy of representativeness and uncertainty to train the BERT pre-training model. In order to verify the effectiveness of this method compared with the existing SATD recognition methods, a comparative experiment of in-project prediction is carried out, which is divided into three groups, using the method of the present invention, the CNN method, and the BERT method respectively. The specific experimental results are shown in Table 4:

[0098] Table 4 Comparative Results

[0099]

[0100]

[0101] As can be seen from Table 4, generally speaking, in the scenario of ten-fold cross-validation within the project, the method of the present invention demonstrates significant advantages. Compared with the CNN method, except for the Zookeeper project where the gap is tiny, only 0.032, with a 5.498% improvement, the method of the present invention leads significantly in the remaining projects. For example, in the Groovy project, the method of the present invention exceeds the CNN method by 0.260, with a 47.71% improvement. In terms of the average F1 score of the ten projects, the method of the present invention is 0.116 higher than the CNN method, with an 18.1% improvement. Compared with the BERT method, although the gap between the method of the present invention and it is not as large as that with the CNN method, the method of the present invention still outperforms the BERT method in overall performance. It is only lower than the BERT method in the two projects of Gradle and Tomcat, and is also 0.013 higher than the BERT method in the worst-performing project of Zookeeper. The gap between the two in the remaining projects is between 0.014 and 0.049, corresponding to an improvement of 1.915% to 7.101%. In terms of the average F1 score of the ten projects, the method of the present invention is 0.018 higher than the BERT method, with a 2.436% improvement.

[0102] In addition, when the method of the present invention achieves the best results in the ten projects, the set stopping thresholds are 0.7, 1.0, 0.7, 0.8, 0.7, 0.9, 0.5, 0.4, 0.8, 1.0 respectively. Since the BERT method uses all the data and manually annotates all of them, and the stopping threshold is equal to the ratio of the used data to the data used, it can be concluded that when the method of the present invention achieves the best results, the amount of data used and the amount of manual labels are 70%, 100%, 70%, 80%, 70%, 90%, 50%, 40%, 80%, 100% of the BERT method respectively. This indicates that the method of the present invention has achieved significant improvement in saving label costs and data volume.

[0103] The above is only the preferred implementation manner of the present invention. It should be noted that for those skilled in the art, without departing from the concept of the present invention, several deformations and improvements can still be made, which should also be regarded as the protection scope of the present invention, and these will not affect the implementation effect of the present invention and the practicality of the patent.

Claims

1. A SATD recognition method based on combining representativeness and uncertainty query strategy, characterized in that: The following steps are involved: Step 1: Data preprocessing and representation; First, the code comment text data is cleaned, and then the BERT tagger or TF-IDF vectorizer is used for word embedding according to the data purpose. Finally, the BERT model is used to convert the input data into an embedding vector, from which the embedding vector of the [CLS] token is extracted as the encoder of the data to express the overall information of the data. Step 2: Construct the initial training set; At the beginning of active learning, a representativeness-based approach is used to select information-rich instances from the training set to construct the initial training set; Step 3: Data selection based on query strategy; According to the training results of the model and the representativeness and uncertainty of the data, the next batch of code comments to be labeled and used for training are selected from the unlabeled data; Step 4: Stop training; After repeating step 3, decide whether to stop training based on the stopping strategy; Step 5: Result analysis; The F1 score based on the confusion matrix is ​​used to evaluate the effectiveness of the method, and the cost is used to measure the efficiency.

2. The SATD recognition method based on combining representativeness and uncertainty query strategy according to claim 1 is characterized in that: In the steps of using the BERT tagger for word segmentation or the TF-IDF vectorizer for word embedding according to the data usage, for the data to be used for model training, the pre-trained BERT tagger is used for word segmentation, and "[CLS]" is added before each data; then for each word after segmentation, the corresponding index is selected in the pre-trained BERT vocabulary; the data length is set to 70, any value less than 70 is padded with 0, and the length of the mask value is set to 70; for the data used to construct the initial training set, the TF-IDF vectorizer is used as the encoder for word embedding.

3. The SATD recognition method based on combining representativeness and uncertainty query strategy according to claim 1 is characterized in that: The specific process of step 2 is as follows: First, use the TF-IDF vectorizer to embed the data into words, then use the KNN model to query similar examples of each instance and obtain the Euclidean distance. The square of the inverse of the mean distance is used as the data density and defined as the representative score. Finally, select the batch of instances with the lowest representative scores from the instances containing keywords as the initial training set. Keywords are obtained by the unsupervised method MAT by matching the four task annotation labels "TODO", "FIXME", "XXX" and "HACK".

4. The SATD recognition method based on combining representativeness and uncertainty query strategy according to claim 1 is characterized in that: The specific process of step 3 is as follows: Obtaining uncertainty scores: Using an uncertain sampling strategy, the uncertainty scores of the data are calculated based on the posterior probability of whether the data is SATD or non-SATD. The highest value of the binary classification output probability of each unlabeled data of the current training model is obtained, and the difference between it and 0.5 is calculated and used as the uncertainty score. Get representative scores: The data is converted into embedded vectors by the pre-trained BERT model, and the embedded vectors of the [CLS] tokens are extracted as the encoder of the data to express the overall information of the data. The KNN model is trained with all the training set data. Similar examples are queried for each unlabeled data and the Euclidean distance is obtained. The square of the inverse of the mean distance is used as the data density and defined as the representative score. Comprehensive instance selection: Multiply the representative score and the uncertainty score to get the final score, and select a batch of instances with the lowest scores for subsequent training.

5. The SATD recognition method based on combining representativeness and uncertainty query strategy according to claim 1 is characterized in that: In step 4, the stopping strategy is to determine the stopping time by comparing the ratio of the number of all marked code annotations to the number of codes in all training sets.

Images