Title generation method based on large model

Abstract

The invention relates to a title generation method based on a large model, and provides a document component analyzer (DCA) and a title generation model (TGM) to automatically generate a high-quality title for a defect report. Specifically, a document component analyzer (DCA) is applied to convert different styles of defect reports into templated defect reports, and then a templated defect report title generation model (TGM) is used to automatically generate a high-quality title. The method demonstrates its robustness and effectiveness in generating high quality titles for various defect report formats.

Description

Technical Field

[0001] The present invention relates to a title generation method, and in particular to a title generation method for templated defect reports and non-templated defect reports. Background Art

[0002] Defect reports are essential artifacts in the software development lifecycle and play a vital role in maintaining software quality and usability. These reports enable users to communicate the problems they encounter when using software systems, helping developers identify and resolve defects. Given the central role of defect reports in software development, the quality of their titles becomes particularly important. The title of a defect report is the first entry point for developers, and concise and descriptive titles can speed up the defect classification and resolution process. On the contrary, low-quality titles and reports often increase the cognitive burden on developers and hinder effective defect resolution. Therefore, it is of great significance to automatically generate high-quality titles for defect reports.

[0003] Chen et al. proposed an automated approach called iTAPE, which focuses on generating accurate titles for bug reports on platforms such as GitHub. The core idea of ​​iTAPE is to treat bug report title generation as a one-sentence summarization task. In this approach, the content of the bug report is provided as input, and iTAPE adopts a sequence-to-sequence (seq2seq) model to automatically generate the title. iTAPE leverages advanced natural language processing techniques such as attention and encoder-decoder architectures that are commonly used in machine translation tasks. This advanced architecture enables iTAPE to achieve promising initial results. Despite these advances, our analysis of iTAPE found two major performance bottlenecks that hinder further progress.

[0004] The first limitation is that the different components of defect reports are not fully considered. iTAPE directly regards the entire defect report as raw input and generates the title directly without distinguishing the different components of the defect report, but in fact each component has a unique contribution to the description of the defect.

[0005] The second limitation is that it ignores the diversity of bug report styles. Li et al. found that as of 2020, only about 30% of the bug reports submitted on GitHub were templated. Therefore, a large proportion of bug reports are non-templated, with inconsistent or disorganized information, making them more difficult to handle. However, iTAPE directly handles templated and non-templated bug reports in a unified manner, without structuring the messy information in non-templated bug reports, which ultimately leads to poor title generation results. Summary of the invention

[0006] In view of the above problems existing in the prior art, the technical problem to be solved by the present invention is: how to automatically generate high-quality titles.

[0007] To solve the above technical problems, the present invention adopts the following technical solution: a title generation method based on a large model, including an offline training stage and an online reasoning stage. The offline training stage is divided into training a document component analyzer DCA and constructing a title generation model TGM.

[0008] Data is collected from public data and a data set is constructed. Each piece of data in the data set is a complete defect report. The defect report contains four fields: description, reproduction, expected behavior, and others. If the formats of these four fields are the same, the data is considered to be templated defect report data, otherwise it is non-templated defect report data.

[0009] In the offline training phase, the document component analyzer DCA is constructed and trained. The DCA structure includes an embedding layer, an encoding layer, a classification layer, and an output layer, including the following steps:

[0010] S101: Acquire templated defect report data, where each piece of data consists of text and a title.

[0011] S102: Each paragraph of text is divided into a series of individual sentences, each of which is labeled with a true category label, which is description, reproduction, expected behavior, and others.

[0012] S103: Use the pre-trained BERT model to initialize the parameters of DCA [that is, the parameters initialized by DCA use the parameters of the pre-trained BERT model], and use the pre-trained BERT model to convert the sentence S into a tag sequence T.

[0013] S104: Through the embedding layer, the labeled sequence T is converted into an embedded sequence X(T).

[0014] S105: For X(T), input it into the encoding layer to obtain the label vector h [CLS] , then, h [CLS] It is passed to the classification layer, and the softmax function is used to predict the predicted category of the sentence. The loss between the predicted category and the true category label of the sentence is calculated, and the parameters of DCA are updated according to the loss. In each round of training, all data in the templated defect report data are input into DCA for training. After the set rounds of training are completed, the trained DCA is obtained.

[0015] Construct a title generation model TGM. The structure of TGM includes an embedding layer, an encoder layer, a decoder layer, and an output layer, including the following steps:

[0016] S201: Acquire non-templated defect report data, and input the non-templated defect report data into a trained DCA to convert it into templated defect report data.

[0017] Each piece of data in the converted templated defect report data consists of text and title; each sentence in the text is labeled with a true category label, which is description, reproduction, expected behavior, and others.

[0018] S202: Design four input slots, corresponding to description, reproduction, expected behavior, and others in each converted templated defect report data, and embed each converted templated defect report data into the input slot to generate an input sequence f Input , f Input =DES:[X]:REP:[Y]:EXP:[Z]:OTH:[V], where [X], [Y], [Z] and [V] are texts of different real category labels in the converted templated defect report data; DES, REP, EXP and OTH are placeholders for different real category labels.

[0019] S203: Use the pre-trained CodeT5 to initialize the parameters of TGM [that is, the parameters of TGM initialization use the parameters of pre-trained CodeT5], and use the self-attention mechanism and the encoder-decoder attention mechanism to initialize the parameters of f Input After encoding, it is represented as

[0020] S204: Multiple Transformer encoders are passed in and self-attention mechanism, feedforward network and layer normalization operations are applied to obtain the context representation vector R.

[0021] S205: Input R into TGM to generate a predicted title, calculate the loss between the predicted title and the true title, and update the TGM parameters based on the loss. In each round of training, all data in the non-templated defect report data will be input into TGM for training. After the set rounds of training are completed, a trained TGM is obtained.

[0022] The online reasoning phase includes the following steps:

[0023] S300: for a defect report, determine whether the defect report is templated. If the defect report contains four fields, namely, description, reproduction, expected behavior, and others, and the formats of the four fields are the same, the defect report is considered to be a templated defect report; otherwise, it is a non-templated defect report.

[0024] If it is a templated defect report, the classified sentences are directly extracted; the classified sentences are embedded into the input slot using the method of S202 to generate f′ Input , and then use the method of S203 to convert f′ Input Convert to Using the S204 method Convert it into R', input R' into the trained TGM, and the output is the predicted title of the defect report;

[0025] If it is a non-templated defect report, the trained DCA is used to convert the defect report into a templated defect report; the defect report converted into a template is embedded into the input slot using the method of S202 to generate f′ Input , and then use the method of S203 to convert f′ Input Convert to Using the S204 method Convert it into R', input R' into the trained TGM, and the output is the predicted title of the defect report.

[0026] Furthermore, the process of converting the sentence S into the tag sequence T in S103 is:

[0027] Let S = [w1, w2, ..., w m ], where ω m Represents the mth word in S, uses the pre-trained BERT model to segment S, and converts S into a token sequence T = [[CLS], t1, t2, …, t n , [SEP]], where [CLS] is a special token representing the entire sentence, [SEP] is a delimiter token marking the end of the sentence, t i Represents each token in the sentence.

[0028] Specifically, the step of converting the tag sequence T into the embedded sequence X in S104 is:

[0029] The embedding layer of DCA includes word embedding layer and position embedding layer. For the token sequence T, word embedding is expressed as E(T) = [E([CLS]), E(t1), E(t2), ..., E(t n), E([SEP])], where E(T) is the word embedding matrix of T; position embedding can be expressed as, where P(T) is the position embedding matrix of T, and the final embedding sequence is X(T),

[0030] Specifically, the calculation process of the predicted category is:

[0031] Calculate h by linear transformation [CLS] Output score vector z, z = W·h [CLS] +b, where W is the weight matrix of the classification layer, b is the bias term, and z is a 4-dimensional vector representing the scores of the four categories, denoted as z = [z1, z2, z3, z4];

[0032] Apply the Softmax function to transform this score vector z into a probability distribution y = [y1, y2, y3, y4], where each y i Represents the probability of category i, the formula of the Softmax function is:

[0033] Select the category with the highest probability as the predicted category. Here, arg max() means taking out the maximum value.

[0034] Further, in S203, The process is:

[0035] The input is all the sentences in the defect report. Each sentence is embedded in the input slot according to its true category label to obtain f Input , for f Input Use a subword tagger to tokenize and decompose complex identifiers into subtokens to help alleviate out-of-vocabulary issues, and get a token sequence T input =[[CLS],sentence1,[SEP],sentence2,[SEP],...,sentence m , [SEP]], where [CLS] is a special tag representing an entire paragraph, and [SEP] is a delimiter tag marking the end of a sentence. i Represents each sentence in the paragraph, sentence i =[t1, t2, .., t n ], t i Represents each token in the sentence.

[0036] The tokenized input sequence is passed through the embedding layer of TGM, where each token is mapped to an embedding vector The embedding layer includes word embedding, position embedding and segment embedding. i , word embedding is represented by eword (t i )=E word [t i ], where E word Represents the embedding matrix. For position i, the position embedding is represented by e pos (i) = E pos [i], where E pos Represents the position embedding matrix, for the label t i , word embedding can be expressed as e seg (t i )=e seg [j],j∈{0,1,...,m-1}and t i ∈sentence j , e seg [j] is the segment embedding matrix vector, based on word embedding, position embedding and segment embedding, we get t i The final embedding of From this we can get sentence i The embedding sequence is So, the final embedding sequence for

[0037] Furthermore, the process of obtaining the context representation vector R in S204 is as follows:

[0038] After being processed by multiple stacked Transformer blocks, each Transformer block contains a multi-head attention mechanism, a feedforward network, and layer normalization. The single-head attention mechanism is defined as:

[0039]

[0040] Where Q is the query matrix, K is the key matrix, V is the value matrix, and d k is the dimension of the key vector.

[0041] Based on the single-head attention mechanism, the multi-head attention mechanism is defined as:

[0042] MultiHead=Concat(head1, head2,..., head h )W O

[0043] Among them head i is the attention calculation of the i-th head, i.e. Attention(Q, K, V) i , W O is the output projection matrix, h is the number of heads of multi-head attention, and Concat means concatenating the output heads of all heads iStitch by dimension.

[0044] Define the feedforward network FFN as:

[0045] FFN(x)=max(0,xW1+b1)W2+b2

[0046] Where W1 is the first layer weight matrix, W2 is the second layer weight matrix, b1 is the first layer bias vector, b2 is the second layer bias vector, and max(0, ) is the ReLU activation function.

[0047] Define layer normalization LayerNorm as:

[0048]

[0049] Where μ is the mean of the input vector, σ 2 is the variance of the input vector, ∈ is a smoothing term to prevent the denominator from being zero, and γ and β are learnable parameters.

[0050] right Apply Multihead to get

[0051]

[0052] right Apply the feedforward network and layer normalization operation to get the output X of the i-th transformer layer i :

[0053]

[0054] After being processed by l Transformer layers, the input sequence f Input Encoded as a sequence of contextual embeddings That is, the input sequence f Input The context representation vector R .

[0055] Furthermore, the step of inputting R into TGM to generate a predicted title in S205 is:

[0056] TGM generates words of the title one by one through the decoder. Each generated word depends on the previously generated content and the currently input defect report data. The generation probability formula is: P(y t |y ＜t , f input )=Softmax(W o LayerNorm(Multihead(y ＜t )+Multihead(y ＜t, R)+FFN))

[0057] Where W o is the linear transformation matrix, y t Indicates the word currently being generated, y ＜t Represents the previously generated content. According to the generation probability, the decoder will select the word with the highest probability as the generated word, and finally get the predicted title by connecting all the generated words.

[0058] Compared with the prior art, the present invention has at least the following advantages:

[0059] 1. The present invention trains a title generation model, which makes full use of different categories of data in the templated defect report, more accurately captures the semantic and structural information in the defect report, and provides strong support for automatically generating high-quality titles.

[0060] 2. This paper solves the limitation of existing methods in generating titles for non-templated defect reports. By training a document component analyzer, non-templated defect reports are effectively converted into templated defect reports. Then, the title generation model is used to greatly improve the quality of automatically generating titles for non-templated defect reports.

[0061] 3. It provides a unified method for processing templated and non-templated defect reports. The proposed method outperforms the existing best methods in terms of ROUGE-L, METEOR and chrF indicators, and manual evaluation results show that the titles generated by our method have good similarity, naturalness and information content. BRIEF DESCRIPTION OF THE DRAWINGS

[0062] Figure 1 It is the overall framework of the present invention.

[0063] Figure 2 This is a comparison of experimental results (automatic indicator evaluation results - for templated defect reports).

[0064] Figure 3 For comparison of experimental results (automatic indicator evaluation results - for non-templated defect reports).

[0065] Figure 4 This is a comparison of experimental results (manual evaluation results). DETAILED DESCRIPTION

[0066] The present invention is described in further detail below.

[0067] See also Figure 1, the present invention discloses a title generation method based on a large model, so as to automatically generate high-quality titles for defect reports of different styles. It includes the following steps: in the document component analyzer (DCA) training stage, templated defect report data is obtained; each text is divided into a series of single sentences, each sentence is divided into one of four categories, namely description, reproduction, expected behavior, and others; the pre-trained BERT model is used for initialization, and the sentence is converted into a tag sequence T based on the input sentence S; the tag sequence T is converted into an embedded sequence X through an embedding layer; based on the embedded sequence X, the sentence category y is predicted through a softmax function. In the title generation model (TGM) construction stage, software defect report data is obtained and processed, and each report contains four categories, namely description (description), reproduction steps (reproduction), expected behavior (expected behavior) and other fields (others); four input slots are designed, corresponding to the description (DES), reproduction steps (REP), expected behavior (EXP) and others (OTH) in the defect report, and the original input is embedded in the input slot to generate f Input ; Use the pre-trained CodeT5 model for initialization, use the self-attention mechanism and encoder-decoder attention mechanism to encode the input defect report component and represent it as Will Multiple Transformer encoders are passed in, and self-attention mechanism, feedforward network and layer normalization operations are applied to obtain the context representation vector R; based on the context representation vector R, TGM generates the words of the title one by one through the decoder. In the online reasoning stage, first determine whether the defect report is templated; if the defect report is templated, directly extract the different components of the defect report from it; if the defect report is non-templated, use the document component analyzer (DCA) to identify and mark the different components of the non-templated defect report. Through this operation, the non-templated defect report can be converted into a templated defect report; the different components of the defect report are combined with the prompt mark and input into the title generation model (TGM), which automatically generates a high-quality title for the defect report.

[0068] A title generation method based on a large model includes an offline training phase and an online reasoning phase. The offline training phase is divided into training a document component analyzer DCA and building a title generation model TGM.

[0069] Data is collected from public data and a data set is constructed. Each piece of data in the data set is a complete defect report. The defect report contains four fields: description, reproduction, expected behavior, and others. If the formats of these four fields are the same, the data is considered to be templated defect report data, otherwise it is non-templated defect report data.

[0070] Templated and non-templated defect reports are analyzed; the present invention collects data from GitHub and constructs a data set, analyzes templated and non-templated defect reports based on the general method TAB, and automatically generates high-quality titles for them.

[0071] In the offline training phase, we train the key components of the TAB method, including the following stages:

[0072] Construct and train the document component analyzer DCA. The DCA structure includes an embedding layer, an encoding layer, a classification layer, and an output layer. The steps are as follows:

[0073] S101: Acquire templated defect report data, where each piece of data consists of text and a title.

[0074] S102: Each paragraph of text is divided into a series of individual sentences, each of which is labeled with a true category label, which is description, reproduction, expected behavior, and others.

[0075] S103: Use the pre-trained BERT model to initialize the parameters of DCA [that is, the parameters initialized by DCA use the parameters of the pre-trained BERT model], and use the pre-trained BERT model to convert the sentence S into a tag sequence T.

[0076] S104: Through the embedding layer, the labeled sequence T is converted into an embedded sequence X(T).

[0077] S105: For X(T), input it into the encoding layer to obtain the label vector h [CLS] , then, h [CLS] It is passed to the classification layer, and the softmax function is used to predict the predicted category of the sentence. The loss between the predicted category and the true category label of the sentence is calculated, and the parameters of DCA are updated according to the loss. In each round of training, all data in the templated defect report data are input into DCA for training. After the set rounds of training are completed, the trained DCA is obtained.

[0078] Construct a title generation model TGM. The structure of TGM includes an embedding layer, an encoder layer, a decoder layer, and an output layer, including the following steps:

[0079] S201: Acquire non-templated defect report data, and input the non-templated defect report data into a trained DCA to convert it into templated defect report data.

[0080] Each piece of data in the converted templated defect report data consists of text and title; each sentence in the text is labeled with a true category label, which is description, reproduction, expected behavior, and others.

[0081] S202: Design four input slots, corresponding to description, reproduction, expected behavior, and others in each converted templated defect report data, and embed each converted templated defect report data into the input slot to generate an input sequence f Input , f Input =DES:[X]:REP:[Y]:EXP:[Z]:OTH:[V], where [X], [Y], [Z] and [V] are texts of different real category labels in the converted templated defect report data; DES, REP, EXP and OTH are placeholders for different real category labels.

[0082] S203: Use the pre-trained CodeT5 to initialize the parameters of TGM [that is, the parameters of TGM initialization use the parameters of pre-trained CodeT5], and use the self-attention mechanism and the encoder-decoder attention mechanism to initialize the parameters of f Input After encoding, it is represented as

[0083] S204: Multiple Transformer encoders are passed in and self-attention mechanism, feedforward network and layer normalization operations are applied to obtain the context representation vector R.

[0084] S205: Input R into TGM to generate a predicted title, calculate the loss between the predicted title and the true title, and update the TGM parameters based on the loss. In each round of training, all data in the non-templated defect report data will be input into TGM for training. After the set rounds of training are completed, a trained TGM is obtained.

[0085] In the online reasoning phase, our method automatically generates high-quality titles for bug reports, including the following steps:

[0086] S300: for a defect report, determine whether the defect report is templated. If the defect report contains four fields, namely, description, reproduction, expected behavior, and others, and the formats of the four fields are the same, the defect report is considered to be a templated defect report; otherwise, it is a non-templated defect report.

[0087] If it is a templated defect report, the classified sentences are directly extracted; the classified sentences (classified sentences indicate that we have obtained the true category label of each sentence) are embedded into the input slot using the method of S202 to generate f′ Input , and then use the method of S203 to convert f′ Input Convert to Using the S204 method Convert it into R', input R' into the trained TGM, and the output is the predicted title of the defect report;

[0088] If it is a non-templated defect report, the trained DCA is used to convert the defect report into a templated defect report; the defect report converted into a template is embedded into the input slot using the method of S202 to generate f′ Input , and then use the method of S203 to convert f′ Input Convert to Using the S204 method Convert it into R', input R' into the trained TGM, and the output is the predicted title of the defect report.

[0089] Specifically, the process of converting the sentence S into the tag sequence T in S103 is:

[0090] Let S = [w1, w2, ..., w m ], where ω m Represents the mth word in S, uses the pre-trained BERT model to segment S, and converts S into a token sequence T = [[CLS], t1, t2, …, t n , [SEP]], where [CLS] is a special token representing the entire sentence, [SEP] is a delimiter token marking the end of the sentence, t i Represents each token in the sentence.

[0091] Specifically, the step of converting the tag sequence T into the embedded sequence X in S104 is:

[0092] The embedding layer of DCA includes word embedding layer and position embedding layer. For the token sequence T, word embedding is expressed as E(T) = [E([CLS]), E(t1), E(t2), ..., E(t n), E([SEP])], where E(T) is the word embedding matrix of T; position embedding can be expressed as, where P(T) is the position embedding matrix of T, and the final embedding sequence is X(T),

[0093] Specifically, the calculation process of the predicted category is:

[0094] Calculate h by linear transformation [CLS] Output score vector z, z = W·h [CLS] +b, where W is the weight matrix of the classification layer, b is the bias term, and z is a 4-dimensional vector representing the scores of the four categories, denoted as z = [z1, z2, z3, z4];

[0095] Apply the Softmax function to transform this score vector z into a probability distribution y = [y1, y2, y3, y4], where each y i Represents the probability of category i, the formula of the Softmax function is:

[0096] Select the category with the highest probability as the predicted category. Here, arg max() means taking out the maximum value.

[0097] Specifically, in S203, The process is:

[0098] The input is all the sentences in the defect report. Each sentence is embedded in the input slot according to its true category label to obtain f Input , for f Input Use a subword tagger to tag and decompose complex identifiers (including long compound words, proper nouns, and irregularly spelled words) into sub-tokens to help alleviate out-of-vocabulary problems, and get the token sequence T input =[[CLS],sentence1,[SEP],sentence2,[SEP],...,sentence m , [SEP]], where [CLS] is a special tag representing an entire paragraph, and [SEP] is a delimiter tag marking the end of a sentence. i Represents each sentence in a paragraph. Each piece of data is a bug report. A paragraph is all the sentences in a bug report (because a bug report contains many sentences). i =[t1, t2, ..., t n ], t i Represents each token in the sentence.

[0099] The tokenized input sequence is passed through the embedding layer of TGM, where each token is mapped to an embedding vector The embedding layer includes word embedding, position embedding and segment embedding. i , word embedding is represented by e word (t i )=E word [t i ], where E word Represents the embedding matrix. For position i, the position embedding is represented by e pos (i) = E pos [i], where E pos Represents the position embedding matrix, for the label t i , word embedding can be expressed as e seg (t i )=e seg [j],j∈{0,1,...,m-1}and t i ∈sentence j , e seg [j] is the segment embedding matrix vector, based on word embedding, position embedding and segment embedding, we get t i The final embedding of From this we can get sentence i The embedding sequence is So, the final embedding sequence for

[0100] Specifically, the process of obtaining the context representation vector R in S204 is as follows:

[0101] After being processed by multiple stacked Transformer blocks, each Transformer block contains a multi-head attention mechanism, a feedforward network, and layer normalization. The single-head attention mechanism is defined as:

[0102]

[0103] Where Q is the query matrix, K is the key matrix, V is the value matrix, and d k is the dimension of the key vector.

[0104] Based on the single-head attention mechanism, the multi-head attention mechanism is defined as:

[0105] MultiHead=Concat(head1, head2,..., head h )W O

[0106] Among them head iis the attention calculation of the i-th head, i.e. Attention(Q, K, V) i , W O is the output projection matrix, h is the number of heads of multi-head attention, and Concat means concatenating the output heads of all heads i Stitch by dimension.

[0107] Define the feedforward network FFN as:

[0108] FFN(x)=max(0,xW1+b1)W2+b2

[0109] Where W1 is the first layer weight matrix, W2 is the second layer weight matrix, b1 is the first layer bias vector, b2 is the second layer bias vector, and max(0, ) is the ReLU activation function.

[0110] Define layer normalization LayerNorm as:

[0111]

[0112] Where μ is the mean of the input vector, μ2 is the variance of the input vector, ∈ is a smoothing term to prevent the denominator from being zero, and γ and β are learnable parameters.

[0113] right Apply Multihead to get

[0114]

[0115] right Apply the feedforward network and layer normalization operation to get the output X of the i-th transformer layer i :

[0116]

[0117] After being processed by l Transformer layers, the input sequence f Input Encoded as a sequence of contextual embeddings That is, the input sequence f Input The context representation vector R of .

[0118] Specifically, the step of inputting R into TGM to generate a predicted title in S205 is as follows:

[0119] Based on the different categories of the input defect report data (description, reproduction steps, expected behavior, and other fields), TGM generates the words of the title one by one through the decoder. Each generated word depends on the previously generated content and the current input defect report data. The generation probability formula is: P(y t |y ＜t , f input )=Softmax(W o LayerNorm(Multihead(y ＜t )+Multihead(y ＜t , R)+FFN))

[0120] Where W o is the linear transformation matrix, y t Indicates the word currently being generated, y ＜t Represents the previously generated content. According to the generation probability, the decoder will select the word with the highest probability as the generated word, and finally get the predicted title by connecting all the generated words.

[0121] The dataset selected by the invention comes from the website Github. We collected a total of 28,273 templated bug reports from the top 1,000 star-rated repositories and a total of 333,563 non-templated bug reports from the top 200 star-rated repositories.

[0122] We sort the bug reports in order of their creation time from small to large, put the first 80% of the bug reports into the training set, and shuffle the remaining 20% ​​of the bug reports and divide them equally into the test set and validation set. In this way, we ensure that earlier issues are used for training and later issues are used for validation and testing. The present invention filters out sentences containing 1) URLs, 2) @name mentions, and 3) markdown titles, because these elements do not directly affect the core bug description and may introduce noise.

[0123] The evaluation metrics are ROUGE-L, METEOR, and chrF. ROUGE-L is a metric that measures the overlap between the generated text and the reference text, with a special focus on the longest common subsequence (LCS).

[0124] The calculation of ROUGE-L is based on the following formula:

[0125]

[0126] Among them, y is the generated text, y refis the reference text, LCS(y,y ref ) is the length of the longest common subsequence between the generated text and the reference text, |y ref | is the length of the reference text.

[0127] METEOR is an indicator used to evaluate the quality of generated text, taking into account precision, recall, and synonym matching. The calculation formula of METEOR is:

[0128]

[0129] Precision is the proportion of words in the generated text that match the reference text. Recall is the proportion of words in the reference text that match the generated text. α is a weight factor used to balance precision and recall. SynonymMatching is the synonym matching degree, which reflects the degree of overlap between the generated text and the reference text at the synonym level.

[0130] chrF is a character-based F1 score that is particularly suitable for text generation evaluation in low-resource languages. The calculation formula of chrF is:

[0131]

[0132] Precision is the matching ratio between the generated text and the reference text at the character level, and Recall is the matching ratio between the reference text and the generated text at the character level.

[0133] In order to further verify the effectiveness of the present invention, it is compared with popular mainstream title generation algorithms, which are as follows:

[0134] iTAPE: The first automated method for bug report title generation, and also a sequence-to-sequence text summarization method that formulates the title generation task as a single-sentence summary generation task.

[0135] NNGen: A state-of-the-art approach to generating commit messages that uses a nearest neighbor algorithm to retrieve the most similar content from historical commit records.

[0136] The title generation method based on a large model proposed in the present invention is called TAB.

[0137] The experimental results show that, see Figure 2 , for templated defect reporting, TAB outperforms iTAPE and NNGen in all metrics; see Figure 3,For non-templated bug reports, TAB consistently outperforms iTAPE, achieving significant improvements in all metrics, which highlights the robustness and scalability of TAB in automatically generating bug report titles, and TAB also shows strong adaptability and effectiveness, making it a powerful solution for generating high-quality titles even in less structured scenarios; see Figure 4 ,The manual evaluation results show that the titles generated by method TAB are ,better than the existing method iTAPE in terms of similarity, naturalness and ,information criteria.

[0138] In short, this paper proposes a large model-based title generation method, TAB, which aims to generate accurate and meaningful titles for defect reports. TAB is particularly effective for templated and non-templated defect reports. For templated reports, TAB directly generates titles by leveraging the structured information in the report. For non-templated reports, TAB first applies a classification step to segment the report into meaningful components, and then generates titles based on these classifications. We evaluate TAB on two datasets using three automatic metrics - one containing templated defect reports and the other containing non-templated defect reports. The results show that TAB consistently outperforms existing methods, demonstrating its robustness and effectiveness in generating high-quality titles for a variety of defect report formats. This makes TAB a valuable tool for improving the accuracy and efficiency of defect resolution in software development.

[0139] Finally, it should be noted that the above embodiments are only used to illustrate the technical solution of the present invention rather than to limit it. Although the present invention has been described in detail with reference to the preferred embodiments, those skilled in the art should understand that the technical solution of the present invention can be modified or replaced by equivalents without departing from the purpose and scope of the technical solution of the present invention, which should be included in the scope of the claims of the present invention.

Claims

1. A title generation method based on a large model, characterized in that: It includes an offline training phase and an online reasoning phase. The offline training phase is divided into training the document component analyzer DCA and building the title generation model TGM. Collect data from public data and build a data set. Each data in the data set is a complete defect report. The defect report contains four fields: description, reproduction, expected behavior, and others. If the formats of these four fields are the same, the data is considered to be templated defect report data, otherwise it is non-templated defect report data. The offline training phase includes the following stages: Construct and train the document component analyzer DCA. The DCA structure includes an embedding layer, an encoding layer, a classification layer, and an output layer, including the following steps: S101: Acquire templated defect report data, where each piece of data consists of text and a title; S102: Segment each text into a series of individual sentences, each of which is labeled with a true category label, which is description, reproduction, expected behavior, and others. S103: Initialize the parameters of DCA using the pre-trained BERT model, and convert the sentence S into a token sequence T using the pre-trained BERT model; S104: Convert the labeled sequence t into an embedded sequence X(T) through an embedding layer; S105: For X(T), input it into the encoding layer to obtain the label vector h [CLS] , then, h [CLS] It is passed to the classification layer, and the softmax function is used to predict the predicted category of the sentence. The loss between the predicted category and the true category label of the sentence is calculated, and the parameters of DCA are updated according to the loss. In each round of training, all data in the templated defect report data are input into DCA for training. After the set rounds of training are completed, the trained DCA is obtained. Construct a title generation model TGM. The structure of TGM includes an embedding layer, an encoder layer, a decoder layer, and an output layer, including the following steps: S201: Acquire non-templated defect report data, and input the non-templated defect report data into a trained DCA to convert it into templated defect report data; Each piece of data in the converted templated defect report data consists of text and title. Each sentence in the text is labeled with a true category label, which is description, reproduction, expected behavior, and others. S202: Design four input slots, corresponding to description, reproduction, expected behavior, and others in each converted templated defect report data, and embed each converted templated defect report data into the input slot to generate an input sequence f Input , f Input =DES:[X]:REP:[Y]:EXP:[Z]:OTH:[V], where [X], [Y], [Z] and [V] are texts of different real category labels in the converted templated defect report data; DES, REP, EXP and OTH are placeholders for different real category labels; S203: Use the pre-trained CodeT5 to initialize the parameters of TGM, and use the self-attention mechanism and encoder-decoder attention mechanism to initialize the parameters of f Input After encoding, it is represented as S204: Pass multiple Transformer encoders and apply self-attention mechanism, feedforward network and layer normalization operation to obtain the context representation vector R; S205: Input R into TGM to generate a predicted title, calculate the loss between the predicted title and the true title, and update the TGM parameters according to the loss. In each round of training, all data in the non-templated defect report data will be input into TGM for training. After the set rounds of training are completed, a trained TGM is obtained; The online reasoning phase includes the following steps: S300: for a defect report, determine whether the defect report is templated. If the defect report contains four fields, namely, description, reproduction, expected behavior, and others, and the formats of the four fields are the same, the defect report is considered to be a templated defect report; otherwise, it is a non-templated defect report. If it is a templated defect report, the classified sentences are directly extracted; the classified sentences are embedded into the input slot using the method of S202 to generate f′ Input , and then use the method of S203 to convert f′ Input Convert to Using the S204 method Convert it into R', input R' into the trained TGM, and the output is the predicted title of the defect report; If it is a non-templated defect report, the trained DCA is used to convert the defect report into a templated defect report; the defect report converted into a template is embedded into the input slot using the method of S202 to generate f′ Input , and then use the method of S203 to convert f′ Input Convert to Using the S204 method Convert it into R', input R' into the trained TGM, and the output is the predicted title of the defect report.

2. A method for generating a title based on a large model as claimed in claim 1, characterized in that: The process of converting the sentence S into the tag sequence T in S103 is: Let S = [w1, w2, ..., w m ], where ω m Represents the mth word in S, uses the pre-trained BERT model to segment S, and converts S into a token sequence T = [[CLS], t1, t2, …, t n , [SEP]], where [CLS] is a special token representing the entire sentence, [SEP] is a delimiter token marking the end of the sentence, t i Represents each token in the sentence.

3. A method for generating a title based on a large model as claimed in claim 2, characterized in that: The step of converting the tag sequence T into the embedded sequence X in S104 is: The embedding layer of DCA includes word embedding layer and position embedding layer. For the token sequence T, word embedding is represented as E(T)=[E(CLS]), E(t1), E(t2), ..., E(t n ), E([SEP])], where E(T) is the word embedding matrix of T; Position embedding can be expressed as, where P(T) is the position embedding matrix of T, and the final embedding sequence is X(T), 4. A method for generating a title based on a large model as claimed in claim 3, characterized in that: The calculation process of the predicted category is: Calculate h by linear transformation [CLS] Output score vector z, z = W·h [CLS] +b, where W is the weight matrix of the classification layer, b is the bias term, and z is a 4-dimensional vector representing the scores of the four categories, denoted as z = [z1, z2, z3, z4]; Apply the Softmax function to transform this score vector z into a probability distribution y = [y1, y2, y3, y4], where each y i Represents the probability of category i, the formula of the Softmax function is: Select the category with the highest probability as the predicted category. Here, arg max() means taking out the maximum value.

5. A method for generating a title based on a large model as claimed in claim 4, characterized in that: The S203 is obtained The process is: The input is all the sentences in the defect report. Each sentence is embedded in the input slot according to its true category label to obtain f Input , for f Input Use a subword tagger to tag and decompose complex identifiers (including long compound words, proper nouns, and irregularly spelled words) into sub-tokens to help alleviate out-of-vocabulary problems, and get the token sequence T input =[[CLS],sentence1,[SEP],sentence2,[SEP],...,sentence m , [SEP]], where [CLS] is a special tag representing an entire paragraph, and [SEP] is a delimiter tag marking the end of a sentence. i Represents each sentence in a paragraph. Each piece of data is a bug report. A paragraph is all the sentences in a bug report (because a bug report contains many sentences). i =[t1, t2, .., t n ], t i Represents each token in the sentence; The tokenized input sequence is passed through the embedding layer of TGM, where each token is mapped to an embedding vector The embedding layer includes word embedding, position embedding and segment embedding. i , word embedding is represented by e word (t i )=E word [t i ], where E word represents the embedding matrix. For position i, the position embedding is represented by e pos (i) = E pos [i], where E pos Represents the position embedding matrix, for the label t i , word embedding can be expressed as e seg (t i )=e seg [j],j∈{0,1,...,m-1}and t i ∈sentence j , e seg [j] is the segment embedding matrix vector, based on word embedding, position embedding and segment embedding, we get t i The final embedding of From this we can get sentence i The embedding sequence is So, the final embedding sequence for 6. A method for generating a title based on a large model as claimed in claim 5, characterized in that: The process of obtaining the context representation vector R in S204 is as follows: After being processed by multiple stacked Transformer blocks, each Transformer block contains a multi-head attention mechanism, a feedforward network, and layer normalization. The single-head attention mechanism is defined as: Where Q is the query matrix, k is the key matrix, V is the value matrix, and d k is the dimension of the key vector; Based on the single-head attention mechanism, the multi-head attention mechanism is defined as: MultiHead=Concat(head1,head2,…,head h )W O Among them head i is the attention calculation of the i-th head, i.e. Attention(Q, K, V) i , W O is the output projection matrix, h is the number of heads of multi-head attention, and Concat means concatenating the output heads of all heads i Splice by dimension; Define the feedforward network FFN as: FFN(x)=max(0,xW1+b1)W2+b2 Where W1 is the weight matrix of the first layer, W2 is the weight matrix of the second layer, b1 is the bias vector of the first layer, b2 is the bias vector of the second layer, and max(0, ) is the ReLU activation function; Define layer normalization LayerNorm as: Where μ is the mean of the input vector, σ 2 is the variance of the input vector, ∈ is a smoothing term to prevent the denominator from being zero, and γ and β are learnable parameters; right Apply Multihead to get right Apply the feedforward network and layer normalization operation to get the output X of the i-th transformer layer i : After being processed by l Transformer layers, the input sequence f Input Encoded as a sequence of contextual embeddings That is, the input sequence f Input The context representation vector R of .

7. A method for generating a title based on a large model as claimed in claim 6, characterized in that: The step of inputting R into TGM to generate a predicted title in S205 is: TGM generates words of the title one by one through the decoder. Each generated word depends on the previously generated content and the currently input defect report data. The generation probability formula is: P(y t |y ＜t , f input )=Softmax(W o LayerNorm(Multihead(y ＜t )+Multihead(y ＜t , R)+FFN)) Where W o is the linear transformation matrix, y t Indicates the word currently being generated, y ＜t Represents previously generated content; According to the generation probability, the decoder will select the word with the highest probability as the generated word, and finally get the predicted title by connecting all the generated words.

Images