A title generation method based on a large model

Abstract

The application relates to a large model-based title generation method, which proposes a document component analyzer DCA and a title generation model TGM to automatically generate high-quality titles for defect reports. Specifically, the document component analyzer DCA is applied to convert defect reports of different styles into templated defect reports, and then the templated defect report title generation model TGM is used to automatically generate high-quality titles. The method shows its robustness and effectiveness in generating high-quality titles for various defect report formats.

Description

Technical Field

[0001] This invention relates to a title generation method, and more particularly to a title generation method for templated defect reports and non-templated defect reports. Background Technology

[0002] Defect reports are essential artifacts in the software development lifecycle, playing a crucial role in maintaining software quality and usability. These reports enable users to communicate problems they encounter while using the software system, helping developers identify and resolve defects. Given the central role of defect reports in software development, the quality of their titles becomes particularly important. The title of a defect report is the first point of entry for developers; a concise, descriptive title can accelerate the defect classification and resolution process. Conversely, low-quality titles and reports often increase the cognitive burden on developers and hinder effective defect resolution. Therefore, it is highly beneficial to automatically generate high-quality titles for defect reports.

[0003] Chen et al. proposed an automated method called iTAPE, which focuses on generating accurate titles for defect reports on platforms such as GitHub. The core idea of ​​iTAPE is to treat defect report title generation as a one-sentence summarization task. In this method, the content of the defect report is provided as input, and iTAPE automatically generates the title using a sequence-to-sequence (seq2seq) model. iTAPE leverages advanced natural language processing techniques, such as attention and encoder-decoder architectures commonly used in machine translation tasks. This advanced architecture has enabled iTAPE to achieve promising initial results. Despite these advances, our analysis of iTAPE reveals two main performance bottlenecks hindering further progress.

[0004] The first limitation is that it does not adequately consider the different components of the defect report. iTAPE treats the entire defect report as raw input and generates the title directly without distinguishing the different components of the defect report, even though each component actually makes a unique contribution to the description of the defect.

[0005] The second limitation is that it ignores the diversity of defect report styles. Li et al.'s research found that, as of 2020, only about 30% of defect reports submitted on GitHub were templated. Therefore, a large portion of defect reports are non-templated, with inconsistent or disorganized information, making them more difficult to process. However, iTAPE directly and uniformly processes both templated and non-templated defect reports without structuring the chaotic information in non-templated reports, ultimately resulting in poor title generation. Summary of the Invention

[0006] In view of the above-mentioned problems in the existing technology, the technical problem to be solved by the present invention is: how to automatically generate high-quality titles.

[0007] To solve the above-mentioned technical problems, the present invention adopts the following technical solution: a title generation method based on a large model, including an offline training stage and an online inference stage. The offline training stage is divided into training the document component analyzer DCA and constructing the title generation model TGM.

[0008] Data is collected from publicly available data and a dataset is constructed. Each data entry in the dataset is a complete defect report. The defect report contains four fields: description, reproduction, expected behavior, and others. If these four fields have the same format, the data is considered a templated defect report; otherwise, it is considered a non-templated defect report.

[0009] During the offline training phase, the document component analyzer DCA is built and trained. The DCA structure includes an embedding layer, an encoding layer, a classification layer, and an output layer, and includes the following steps:

[0010] S101: Obtain templated defect report data, where each data entry consists of text and a title.

[0011] S102: Divide each text segment into a series of individual sentences, and label each sentence with a real category label, which is description, reproduction, expected behavior, and others.

[0012] S103: Initialize the parameters of DCA using a pre-trained BERT model [i.e., the parameters of DCA are initialized using the parameters of the pre-trained BERT model], and convert the sentence S into a labeled sequence T using the pre-trained BERT model.

[0013] S104: Transform the labeled sequence T into the embedded sequence X(T) through the embedding layer.

[0014] S105: For X(T), input it into the coding layer to obtain the label vector h. [CLS] Then, h [CLS] The data is passed to the classification layer, where the softmax function is used to predict the predicted category of the sentence. The loss between the predicted category and the true category label of the sentence is calculated, and the parameters of DCA are updated based on the loss. In each training round, all data from the templated defect report data are input into DCA for training. After completing the set number of training rounds, the trained DCA is obtained.

[0015] Constructing a title generation model (TGM) involves the following steps: The TGM structure comprises an embedding layer, an encoder layer, a decoder layer, and an output layer.

[0016] S201: Obtain non-template-based defect report data, and input the non-template-based defect report data into the trained DCA to convert it into template-based defect report data.

[0017] Each data point in the converted templated defect report data consists of text and a title; each sentence in the text is labeled with a real category tag, which is description, reproduction, expected behavior, and others.

[0018] S202: Design four input slots, corresponding to the description, reproduction, expected behavior, and others in each converted templated defect report data. Embed each converted templated defect report data into the input slots to generate the input sequence f. Input f Input =DES:[X]:REP:[Y]:EXP:[Z]:OTH:[V], where [X], [Y], [Z] and [V] are the text of different real category labels in the converted templated defect report data; DES, REP, EXP and OTH are placeholders for different real category labels.

[0019] S203: Initialize the TGM parameters using pre-trained CodeT5 [i.e., the TGM initialization parameters use the pre-trained CodeT5 parameters], employing self-attention and encoder-decoder attention mechanisms for f. Input After encoding, it is represented as

[0020] S204: Will Multiple Transformer encoders are input, and self-attention mechanism, feedforward network and layer normalization operation are applied to obtain the context representation vector R.

[0021] S205: Input R into TGM to generate predicted titles, calculate the loss between predicted titles and real titles, and update TGM parameters based on the loss. In each training round, all data from the non-template-based defect report data will be input into TGM for training. After completing the set number of training rounds, the trained TGM will be obtained.

[0022] The online reasoning phase includes the following steps:

[0023] S300: For a defect report, determine whether the defect report is templated. If the defect report contains four fields: description, reproduction, expected behavior, and others, and these four fields have the same format, then the defect report is considered a templated defect report; otherwise, it is a non-templated defect report.

[0024] If it is a templated defect report, the categorized sentences are extracted directly; the S202 method is used to embed the categorized sentences into the input slots to generate f′. Input Then use the method of S203 to f′ Input Transform into Using method S204 Convert it to R', input R' into the trained TGM, and the output is the predicted title of the defect report;

[0025] If the defect report is not templated, the trained DCA is used to convert it into a templated defect report; the S202 method is then used to embed the converted templated defect report into the input slot to generate f′. Input Then use the method of S203 to f′ Input Transform into Using method S204 Convert it to R', input R' into the trained TGM, and the output is the predicted title of the defect report.

[0026] Furthermore, the process of converting sentence S into a marker sequence T in S103 is as follows:

[0027] Let S = [w1, w2, ..., w m ], where ω m Let S be the m-th word in S. Use a pre-trained BERT model to segment S, converting S into a labeled sequence T = [[CLS], t1, t2, ..., tm]. n [SEP], where [CLS] is a special marker representing the entire sentence, and [SEP] is a separator marker marking the end of the sentence. i This represents each marker in the sentence.

[0028] Specifically, step S104, which converts the marker sequence T into the embedding sequence X, is as follows:

[0029] The embedding layer of DCA includes a word embedding layer and a position embedding layer. For a labeled sequence T, the word embedding is represented as E(T) = [E([CLS]), E(t1), E(t2), ..., E(t3)]. n), E([SEP])], where E(T) is the word embedding matrix of T; positional embedding can be represented as, where P(T) is the positional embedding matrix of T, and the final embedding sequence is X(T).

[0030] Specifically, the calculation process for the predicted category is as follows:

[0031] h is calculated using linear transformation. [CLS] Output the score vector z, z = W·h [CLS] +b, where W is the weight matrix of the classification layer, b is the bias term, and z is a 4-dimensional vector representing the scores of the four categories, denoted as z = [z1, z2, z3, z4];

[0032] The Softmax function is applied to transform the score vector z into a probability distribution y = [y1, y2, y3, y4], where each y... i Let i represent the probability of category i. The formula for the Softmax function is:

[0033] Choose the category with the highest probability as the predicted category. Here, arg max() means extracting the maximum value.

[0034] Furthermore, in S203, the following is obtained: The process is as follows:

[0035] The input consists of all sentences from the defect report. Each sentence is embedded into the input slot according to its true category label to obtain f. Input , for f Input Subtotaling is performed using a subtotaling algorithm, and complex identifiers are broken down into subtotals to help alleviate extralexical problems, resulting in a subtotaling sequence T. input =[[CLS],sentence1,[SEP],sentence2,[SEP],...,sentence m [SEP], where [CLS] is a special marker representing the entire paragraph, and [SEP] is a separator marker marking the end of a sentence. i Represents each sentence in a paragraph. i = [t1, t2, ..., t n ], t i This represents each marker in the sentence.

[0036] The tokenized input sequence is passed through the embedding layer of the TGM, so that each token is mapped to an embedding vector. The embedding layer includes word embedding, positional embedding, and segment embedding, for the tag t i Word embedding is represented as eword (t i ) = E word [t i ], where E word Let e ​​represent the embedding matrix, where the position embedding for position i is denoted as e. pos (i)=E pos [i], where E pos The position embedding matrix represents the position for label t. i Word embedding can be represented as e seg (t i ) = e seg [j],j∈{0,1,...,m-1}and t i ∈sentence j e seg [j] represents the segment embedding matrix vector, from which t is obtained based on word embedding, position embedding, and segment embedding. i The final embedding is From this, we can obtain the sentence. i The embedded sequence is Therefore, the final embedded sequence for

[0037] Furthermore, the process of obtaining the context representation vector R in S204 is as follows:

[0038] The process involves multiple stacked Transformer blocks, each containing a multi-head attention mechanism, a feedforward network, and layer normalization. The single-head attention mechanism is defined as follows:

[0039]

[0040] Where Q is the query matrix, K is the key matrix, V is the value matrix, and d k is the dimension of the key vector.

[0041] Based on the single-head attention mechanism, the multi-head attention mechanism is defined as follows:

[0042] MultiHead=Concat(head1, head2,..., head h W O

[0043] head i This refers to the attention calculation for the i-th head, i.e., Attention(Q, K, V). i W O This is the output projection matrix, where h is the number of heads in the multi-head attention mechanism, and Concat represents the output of all heads. iComponents are stitched together by dimension.

[0044] Define the feedforward network (FFN) as follows:

[0045] FFN(x) = max(0, xW1+b1)W2+b2

[0046] Where W1 is the first layer weight matrix, W2 is the second layer weight matrix, b1 is the first layer bias vector, b2 is the second layer bias vector, and max(0,·) is the ReLU activation function.

[0047] Define the layer normalization layerNorm as:

[0048]

[0049] Where μ is the mean of the input vector, σ 2 is the variance of the input vector, ∈ is the smoothing term to prevent the denominator from being 0, and γ and β are learnable parameters.

[0050] right Apply Multihead to obtain

[0051]

[0052] right The output X of the i-th transformer layer is obtained by applying a feedforward network and layer normalization operations. i :

[0053]

[0054] After processing by l Transformer layers, the input sequence f Input Encoded as a context embedding sequence That is, the input sequence f Input Context representation vector R .

[0055] Furthermore, step S205, which involves inputting R into the TGM to generate the predicted title, is as follows:

[0056] TGM generates the title words one by one through a decoder. Each generated word depends on previously generated content and the currently input defect report data. The generation probability formula is: P(y t |y ＜t f input =Softmax(W) o ·LayerNorm(Multihead(y ＜t )+Multihead(y ＜t,R)+FFN))

[0057] Among them W o It is a linear transformation matrix, y t y represents the word that is currently being generated. ＜t This represents previously generated content. Based on the generation probability, the decoder selects the word with the highest probability as the generated word, and finally obtains the predicted title by concatenating all the generated words.

[0058] Compared with the prior art, the present invention has at least the following advantages:

[0059] 1. This invention trains a title generation model that makes full use of different categories of data in templated defect reports, more accurately captures the semantic and structural information in defect reports, and provides strong support for automatically generating high-quality titles.

[0060] 2. This method overcomes the limitations of existing methods in generating titles for non-template-based defect reports. By training a document component analyzer, non-template-based defect reports are effectively converted into template-based defect reports. Then, by utilizing a title generation model, the quality of automatically generated titles for non-template-based defect reports is greatly improved.

[0061] 3. This invention provides a unified method for processing templated and non-templated defect reports. The method of this invention outperforms the best existing methods in terms of ROUGE-L, METEOR, and chrF metrics, and human evaluation results show that the titles generated by our method have good similarity, naturalness, and information content. Attached Figure Description

[0062] Figure 1 This forms the overall framework of the present invention.

[0063] Figure 2 For comparison of experimental results (automatic indicator evaluation results - targeting template-based defect reports).

[0064] Figure 3 For comparison of experimental results (automatic indicator evaluation results - for non-template-based defect reports).

[0065] Figure 4 For comparison of experimental results (human evaluation results). Detailed Implementation

[0066] The present invention will now be described in further detail.

[0067] See Figure 1This invention discloses a title generation method based on a large model, which automatically generates high-quality titles for defect reports of different styles. The method includes the following steps: In the Document Component Analyzer (DCA) training phase, templated defect report data is acquired; each text segment is divided into a series of individual sentences, each sentence belonging to one of four categories: description, reproduction, expected behavior, and others; a pre-trained BERT model is used for initialization, and based on the input sentence S, the sentence is converted into a labeled sequence T; through an embedding layer, the labeled sequence T is converted into an embedding sequence X; based on the embedding sequence X, the sentence category y is predicted using a softmax function. In the Title Generation Model (TGM) construction phase, software defect report data is acquired and processed. Each report contains four categories: description, reproduction, expected behavior, and others; four input slots are designed, corresponding to the description (DES), reproduction (REP), expected behavior (EXP), and others (OTH) in the defect report, respectively; the original input is embedded into the input slots to generate f. Input Initialization was performed using a pre-trained CodeT5 model. Self-attention and encoder-decoder attention mechanisms were used to encode and represent the input defect report component. Will Multiple Transformer encoders are input, and a self-attention mechanism, a feedforward network, and layer normalization are applied to obtain a context representation vector R. Based on the context representation vector R, the Title Generator (TGM) generates the words of the title one by one through the decoder. During the online inference phase, it is first determined whether the defect report is templated. If the defect report is templated, its different components are directly extracted. If the defect report is non-templated, the Document Component Analyzer (DCA) is used to identify and label the different components of the non-templated defect report, thus converting it into a templated defect report. The different components of the defect report are combined with the cue tags and input into the Title Generator (TGM), which automatically generates a high-quality title for the defect report.

[0068] A large-model-based title generation method includes an offline training phase and an online inference phase. The offline training phase is divided into training the document component analyzer (DCA) and building the title generation model (TGM).

[0069] Data is collected from publicly available data and a dataset is constructed. Each data entry in the dataset is a complete defect report. The defect report contains four fields: description, reproduction, expected behavior, and others. If these four fields have the same format, the data is considered a templated defect report; otherwise, it is considered a non-templated defect report.

[0070] This invention analyzes both templated and non-templated defect reports. It collects data from GitHub and constructs a dataset, using the general method TAB to analyze both templated and non-templated defect reports and automatically generate high-quality titles for them.

[0071] During the offline training phase, we train the key components in the method TAB, including the following stages:

[0072] The document component analyzer DCA is constructed and trained. The DCA structure includes an embedding layer, an encoding layer, a classification layer, and an output layer, and includes the following steps:

[0073] S101: Obtain templated defect report data, where each data entry consists of text and a title.

[0074] S102: Divide each text segment into a series of individual sentences, and label each sentence with a real category label, which is description, reproduction, expected behavior, and others.

[0075] S103: Initialize the parameters of DCA using a pre-trained BERT model [i.e., the parameters of DCA are initialized using the parameters of the pre-trained BERT model], and convert the sentence S into a labeled sequence T using the pre-trained BERT model.

[0076] S104: Transform the labeled sequence T into the embedded sequence X(T) through the embedding layer.

[0077] S105: For X(T), input it into the coding layer to obtain the label vector h. [CLS] Then, h [CLS] The data is passed to the classification layer, where the softmax function is used to predict the predicted category of the sentence. The loss between the predicted category and the true category label of the sentence is calculated, and the parameters of DCA are updated based on the loss. In each training round, all data from the templated defect report data are input into DCA for training. After completing the set number of training rounds, the trained DCA is obtained.

[0078] Constructing a title generation model (TGM) involves the following steps: The TGM structure comprises an embedding layer, an encoder layer, a decoder layer, and an output layer.

[0079] S201: Obtain non-template-based defect report data, and input the non-template-based defect report data into the trained DCA to convert it into template-based defect report data.

[0080] Each data point in the converted templated defect report data consists of text and a title; each sentence in the text is labeled with a real category tag, which is description, reproduction, expected behavior, and others.

[0081] S202: Design four input slots, corresponding to the description, reproduction, expected behavior, and others in each converted templated defect report data. Embed each converted templated defect report data into the input slots to generate the input sequence f. Input f Input =DES:[X]:REP:[Y]:EXP:[Z]:OTH:[V], where [X], [Y], [Z] and [V] are the text of different real category labels in the converted templated defect report data; DES, REP, EXP and OTH are placeholders for different real category labels.

[0082] S203: Initialize the TGM parameters using pre-trained CodeT5 [i.e., the TGM initialization parameters use the pre-trained CodeT5 parameters], employing self-attention and encoder-decoder attention mechanisms for f. Input After encoding, it is represented as

[0083] S204: Will Multiple Transformer encoders are input, and self-attention mechanism, feedforward network and layer normalization operation are applied to obtain the context representation vector R.

[0084] S205: Input R into TGM to generate predicted titles, calculate the loss between predicted titles and real titles, and update TGM parameters based on the loss. In each training round, all data from the non-template-based defect report data will be input into TGM for training. After completing the set number of training rounds, the trained TGM will be obtained.

[0085] During the online inference phase, our method automatically generates high-quality titles for defect reports, including the following steps:

[0086] S300: For a defect report, determine whether the defect report is templated. If the defect report contains four fields: description, reproduction, expected behavior, and others, and these four fields have the same format, then the defect report is considered a templated defect report; otherwise, it is a non-templated defect report.

[0087] If it is a templated defect report, the categorized sentences are extracted directly; the S202 method is used to embed the categorized sentences (the categorized sentences indicate that we have obtained the true category label of each sentence) into the input slots to generate f′. Input Then use the method of S203 to f′ Input Transform into Using method S204 Convert it to R', input R' into the trained TGM, and the output is the predicted title of the defect report;

[0088] If the defect report is not templated, the trained DCA is used to convert it into a templated defect report; the S202 method is then used to embed the converted templated defect report into the input slot to generate f′. Input Then use the method of S203 to f′ Input Transform into Using method S204 Convert it to R', input R' into the trained TGM, and the output is the predicted title of the defect report.

[0089] Specifically, the process of converting sentence S into a marker sequence T in S103 is as follows:

[0090] Let S = [w1, w2, ..., w m ], where ω m Let S be the m-th word in S. Use a pre-trained BERT model to segment S, converting S into a labeled sequence T = [[CLS], t1, t2, ..., tm]. n [SEP], where [CLS] is a special marker representing the entire sentence, and [SEP] is a separator marker marking the end of the sentence. i This represents each marker in the sentence.

[0091] Specifically, step S104, which converts the marker sequence T into the embedding sequence X, is as follows:

[0092] The embedding layer of DCA includes a word embedding layer and a position embedding layer. For a labeled sequence T, the word embedding is represented as E(T) = [E([CLS]), E(t1), E(t2), ..., E(t3)]. n), E([SEP])], where E(T) is the word embedding matrix of T; positional embedding can be represented as, where P(T) is the positional embedding matrix of T, and the final embedding sequence is X(T).

[0093] Specifically, the calculation process for the predicted category is as follows:

[0094] h is calculated using linear transformation. [CLS] Output the score vector z, z = W·h [CLS] +b, where W is the weight matrix of the classification layer, b is the bias term, and z is a 4-dimensional vector representing the scores of the four categories, denoted as z = [z1, z2, z3, z4];

[0095] The Softmax function is applied to transform the score vector z into a probability distribution y = [y1, y2, y3, y4], where each y... i Let i represent the probability of category i. The formula for the Softmax function is:

[0096] Choose the category with the highest probability as the predicted category. Here, arg max() means extracting the maximum value.

[0097] Specifically, in S203, the following is obtained: The process is as follows:

[0098] The input consists of all sentences from the defect report. Each sentence is embedded into the input slot according to its true category label to obtain f. Input , for f Input Sub-word taggers are used for tagging, and complex identifiers (including long compound words, proper nouns, and irregularly spelled words) are broken down into sub-tags to help alleviate extralexical problems, resulting in a tag sequence T. input =[[CLS],sentence1,[SEP],sentence2,[SEP],...,sentence m [SEP], where [CLS] is a special marker representing the entire paragraph, and [SEP] is a separator marker marking the end of a sentence. i Each sentence in the paragraph represents a defect report, and each data point represents a defect report. A paragraph contains all the sentences in a defect report (because a defect report contains many sentences). i = [t1, t2, ..., t n ], t i This represents each marker in the sentence.

[0099] The tokenized input sequence is passed through the embedding layer of the TGM, so that each token is mapped to an embedding vector. The embedding layer includes word embedding, positional embedding, and segment embedding, for the tag t i Word embedding is represented as e word (t i ) = E word [t i ], where E word Let e ​​represent the embedding matrix, where the position embedding for position i is denoted as e. pos (i)=E pos [i], where E pos The position embedding matrix represents the position for label t. i Word embedding can be represented as e seg (t i ) = e seg [j],j∈{0,1,...,m-1}and t i ∈sentence j e seg [j] represents the segment embedding matrix vector, from which t is obtained based on word embedding, position embedding, and segment embedding. i The final embedding is From this, we can obtain the sentence. i The embedded sequence is Therefore, the final embedded sequence for

[0100] Specifically, the process of obtaining the context representation vector R in S204 is as follows:

[0101] The process involves multiple stacked Transformer blocks, each containing a multi-head attention mechanism, a feedforward network, and layer normalization. The single-head attention mechanism is defined as follows:

[0102]

[0103] Where Q is the query matrix, K is the key matrix, V is the value matrix, and d k is the dimension of the key vector.

[0104] Based on the single-head attention mechanism, the multi-head attention mechanism is defined as follows:

[0105] MultiHead=Concat(head1, head2,..., head h W O

[0106] head iThis refers to the attention calculation for the i-th head, i.e., Attention(Q, K, V). i W O This is the output projection matrix, where h is the number of heads in the multi-head attention mechanism, and Concat represents the output of all heads. i Components are stitched together by dimension.

[0107] Define the feedforward network (FFN) as follows:

[0108] FFN(x) = max(0, xW1+b1)W2+b2

[0109] Where W1 is the first layer weight matrix, W2 is the second layer weight matrix, b1 is the first layer bias vector, b2 is the second layer bias vector, and max(0,·) is the ReLU activation function.

[0110] Define the layer normalization layerNorm as:

[0111]

[0112] Where μ is the mean of the input vector, μ2 is the variance of the input vector, ∈ is a smoothing term to prevent the denominator from being 0, and γ and β are learnable parameters.

[0113] right Apply Multihead to obtain

[0114]

[0115] right The output X of the i-th transformer layer is obtained by applying a feedforward network and layer normalization operations. i :

[0116]

[0117] After processing by l Transformer layers, the input sequence f Input Encoded as a context embedding sequence That is, the input sequence f Input The context representation vector R.

[0118] Specifically, step S205, which involves inputting R into TGM to generate a predicted title, is as follows:

[0119] Based on the different categories of the input defect report data (description, reproduction steps, expected behavior, and others), TGM generates title words one by one through a decoder. Each generated word depends on the previously generated content and the currently input defect report data. The generation probability formula is: P(y t |y ＜t f input =Softmax(W) o ·LayerNorm(Multihead(y ＜t )+Multihead(y ＜t ,R)+FFN))

[0120] Among them W o It is a linear transformation matrix, y t y represents the word that is currently being generated. ＜t This represents previously generated content. Based on the generation probability, the decoder selects the word with the highest probability as the generated word, and finally obtains the predicted title by concatenating all the generated words.

[0121] The dataset selected for the invention comes from the website Github. We collected a total of 28,273 templated defect reports from the top 1,000 highest-rated repositories and a total of 333,563 non-templated defect reports from the top 200 highest-rated repositories.

[0122] We sorted the defect reports in ascending order of their creation time, placing the top 80% into the training set. The remaining 20% ​​were shuffled and then equally divided between the test and validation sets. This ensures that earlier issues are used for training, and later issues are used for validation and testing. This invention filters out sentences containing 1) URLs, 2) @name mentions, and 3) Markdown headings, as these elements do not directly affect the core defect description and may introduce noise.

[0123] The evaluation metrics are ROUGE-L, METEOR, and chrF. ROUGE-L is a metric that measures the overlap between the generated text and the reference text, with a particular focus on the longest common subsequence (LCS).

[0124] ROUGE-L is calculated based on the following formula:

[0125]

[0126] Where y is the generated text, y refFor reference text, LCS(y, y) ref ) represents the length of the longest common subsequence between the generated text and the reference text, |y ref | represents the length of the reference text.

[0127] METEOR is a metric used to evaluate the quality of generated text, taking into account precision, recall, and synonym matching. The formula for calculating METEOR is:

[0128]

[0129] Precision is the proportion of words in the generated text that match the reference text. Recall is the proportion of words in the reference text that match the generated text. α is a weighting factor used to balance precision and recall. SynonymMatching reflects the degree of overlap between the generated and reference texts at the synonym level.

[0130] chrF is a character-based F1 score, particularly suitable for evaluating text generation in low-resource languages. The formula for calculating chrF is:

[0131]

[0132] Precision represents the character-level matching ratio between the generated and reference texts. Recall represents the character-level matching ratio between the reference and generated texts.

[0133] To further verify the effectiveness of this invention, it was compared with popular and mainstream title generation algorithms, as follows:

[0134] iTAPE: The first automated method for generating defect report titles, and also a sequence-to-sequence text summarization method. This method formulates the title generation task into a single-sentence summary generation task.

[0135] NNGen: A state-of-the-art method for generating commit messages, which uses the nearest neighbor algorithm to retrieve the most similar content from historical commit records.

[0136] The title generation method based on a large model proposed in this invention is called TAB.

[0137] Experimental results show that, see Figure 2 For templated defect reports, TAB outperforms iTAPE and NNGen across all metrics; see [link / reference]. Figure 3For non-template-based defect reports, TAB consistently outperforms iTAPE, achieving significant improvements across all metrics. This highlights TAB's robustness and scalability in automatically generating error report headers. Furthermore, TAB demonstrates strong adaptability and effectiveness, making it a powerful solution for generating high-quality headers even in less structured scenarios. See also Figure 4 Human evaluation results show that the titles generated by the TAB method are superior to the existing method iTAPE in terms of similarity, naturalness, and information content.

[0138] In summary, this invention proposes a large-model-based title generation method, TAB, designed to generate accurate and meaningful titles for defect reports. TAB is particularly effective for both templated and non-templated defect reports. For templated reports, TAB directly generates titles by utilizing the structured information within the report. For non-templated reports, TAB first applies a classification step to segment the report into meaningful components, and then generates titles based on these classifications. We evaluate TAB on two datasets using three automated metrics—one containing templated defect reports and the other containing non-templated defect reports. The results show that TAB consistently outperforms existing methods, demonstrating its robustness and effectiveness in generating high-quality titles for various defect report formats. This makes TAB a valuable tool for improving the accuracy and efficiency of defect resolution in software development.

[0139] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention and are not intended to limit it. Although the present invention has been described in detail with reference to preferred embodiments, those skilled in the art should understand that modifications or equivalent substitutions can be made to the technical solutions of the present invention without departing from the spirit and scope of the technical solutions of the present invention, and all such modifications or substitutions should be covered within the scope of the claims of the present invention.

Claims

1. A title generation method based on a large model, characterized in that, It includes an offline training phase and an online inference phase. The offline training phase is divided into training the Document Component Analyzer (DCA) and building the Title Generation Model (TGM). Data is collected from publicly available data and a dataset is constructed. Each data entry in the dataset is a complete defect report. The defect report contains four fields: description, reproduction, expected behavior, and others. If these four fields have the same format, the data is considered a templated defect report; otherwise, it is considered a non-templated defect report. The offline training phase includes the following stages: The document component analyzer DCA is constructed and trained. The DCA structure includes an embedding layer, an encoding layer, a classification layer, and an output layer, and includes the following steps: S101: Obtain templated defect report data, where each data entry consists of text and a title; S102: Divide each text segment into a series of individual sentences, and label each sentence with a real category label, which is description, reproduction, expected behavior, and others; S103: Initialize the parameters of DCA using a pre-trained BERT model, and convert sentence S into a labeled sequence T using a pre-trained BERT model; S104: Transform the labeled sequence T into the embedded sequence X(T) through the embedding layer; S105: For X(T), input it into the coding layer to obtain the label vector h. [CLS] Then, h [CLS] The data is passed to the classification layer, where the softmax function is used to predict the predicted category of the sentence. The loss between the predicted category and the true category label of the sentence is calculated, and the parameters of DCA are updated based on the loss. In each training round, all data from the templated defect report data are input into DCA for training. After completing the set number of training rounds, the trained DCA is obtained. Constructing a title generation model (TGM) involves the following steps: The TGM structure comprises an embedding layer, an encoder layer, a decoder layer, and an output layer. S201: Obtain non-template-based defect report data, and input the non-template-based defect report data into the trained DCA to convert it into template-based defect report data; Each data point in the converted templated defect report data consists of text and a title; each sentence in the text is labeled with a real category tag, which is description, reproduction, expected behavior, and others; S202: Design four input slots, corresponding to the description, reproduction, expected behavior, and others in each converted templated defect report data. Embed each converted templated defect report data into the input slots to generate the input sequence f. Input f Input =DES:[X]:REP:[Y]:EXP:[Z]:OTH:[V], where [X], [Y], [Z], and [V] are the text of different real category labels in the converted templated defect report data; DES, REP, EXP, and OTH are placeholders for different real category labels; S203: Initialize the parameters of the TGM using pre-trained CodeT5, and employ self-attention and encoder-decoder attention mechanisms to manage f. Input After encoding, it is represented as S204: Will Multiple Transformer encoders are input, and self-attention mechanism, feedforward network and layer normalization operation are applied to obtain the context representation vector R; S205: Input R into TGM to generate predicted titles, calculate the loss between predicted titles and real titles, and update TGM parameters based on the loss. In each training round, all data in the non-template-based defect report data will be input into TGM for training. After completing the set number of training rounds, the trained TGM will be obtained. The online reasoning phase includes the following steps: S300: For a defect report, determine whether the defect report is templated. If the defect report contains the four fields description, reproduction, expected behavior, and others, and these four fields have the same format, then the defect report is considered to be a templated defect report; otherwise, it is a non-templated defect report. If it is a templated defect report, the categorized sentences are extracted directly; the S202 method is used to embed the categorized sentences into the input slots to generate f′. Input Then use the method of S203 to f′ Input Transform into Using method S204 Convert it to R', input R' into the trained TGM, and the output is the predicted title of the defect report; If the defect report is not templated, the trained DCA is used to convert it into a templated defect report; the S202 method is then used to embed the converted templated defect report into the input slot to generate f′. Input Then use the method of S203 to f′ Input Transform into Using method S204 Convert it to R', input R' into the trained TGM, and the output is the predicted title of the defect report.

2. The title generation method based on a large model as described in claim 1, characterized in that: The process of converting sentence S into marker sequence T in S103 is as follows: Let S = [w1, w2, ..., w m ], where ω m Let S be the m-th word in S. Use a pre-trained BERT model to segment S, converting S into a labeled sequence T = [[CLS], t1, t2, ..., tm]. n [SEP], where [CLS] is a special marker representing the entire sentence, and [SEP] is a separator marker marking the end of the sentence. i This represents each marker in the sentence.

3. The title generation method based on a large model as described in claim 2, characterized in that: The step in S104 of converting the marker sequence T into the embedded sequence X is as follows: The embedding layer of DCA includes a word embedding layer and a position embedding layer. For a labeled sequence T, the word embedding is represented as E(T) = [E([CLS]), E(t1), E(t2), ..., E(t3)]. n ), E([SEP])], where E(T) is the word embedding matrix of T; Positional embedding can be represented as , where P(T) is the positional embedding matrix of T, and the final embedding sequence is X(T).

4. The title generation method based on a large model as described in claim 3, characterized in that: The calculation process for the predicted category is as follows: h is calculated using linear transformation. [CLS] Output the score vector z, z = W·h [CLS] +b, where W is the weight matrix of the classification layer, b is the bias term, and z is a 4-dimensional vector representing the scores of the four categories, denoted as z = [z1, z2, z3, z4]; The Softmax function is applied to transform the score vector z into a probability distribution y = [y1, y2, y3, y4], where each y... i Let i represent the probability of category i. The formula for the Softmax function is: Choose the category with the highest probability as the predicted category. Here, arg max() means extracting the maximum value.

5. The title generation method based on a large model as described in claim 4, characterized in that: The S203 obtained The process is as follows: The input consists of all sentences from the defect report. Each sentence is embedded into the input slot according to its true category label to obtain f. Input , for f Input Sub-word taggers are used for tagging, and complex identifiers are decomposed into sub-tags to help alleviate extralexical problems, resulting in a tag sequence T. input =[[CLS],sentence1,[SEP],sentence2,[SEP],...,sentence m [SEP], where [CLS] is a special marker representing the entire paragraph, and [SEP] is a separator marker marking the end of a sentence. i Each sentence in a paragraph represents a defect report, and each data point represents a defect report. A paragraph contains all the sentences in a defect report, as a defect report typically contains many sentences. i =[t1,t2,..,t n ], t i Each marker represents a character in the sentence; the complex identifiers include long compound words, proper nouns, and words with irregular spelling. The tokenized input sequence is passed through the embedding layer of the TGM, so that each token is mapped to an embedding vector. The embedding layer includes word embedding, positional embedding, and segment embedding, for the tag t i Word embedding is represented as e word (t i ) = E word [t i ], where E word Let e ​​represent the embedding matrix, where the position embedding for position i is denoted as e. pos (i)=E pos [i], where E pos The position embedding matrix represents the position for label t. i Word embedding can be represented as e seg (t i ) = e seg [j], j∈{0,1,...,m-1} and t i ∈sentence j e seg [j] represents the segment embedding matrix vector, from which t is obtained based on word embedding, position embedding, and segment embedding. i The final embedding is From this, we can obtain the sentence. i The embedded sequence is Therefore, the final embedded sequence for 6. The title generation method based on a large model as described in claim 5, characterized in that: The process of obtaining the context representation vector R in S204 is as follows: The process involves multiple stacked Transformer blocks, each containing a multi-head attention mechanism, a feedforward network, and layer normalization. The single-head attention mechanism is defined as follows: Where Q is the query matrix, K is the key matrix, V is the value matrix, and d k The dimension of the key vector; Based on the single-head attention mechanism, the multi-head attention mechanism is defined as follows: MultiHead=Concat(head1,head2,…,head h )W O head i This refers to the attention calculation for the i-th head, i.e., Attention(Q,K,V). i W O This is the output projection matrix, where h is the number of heads in the multi-head attention mechanism, and Concat represents the output of all heads. i Dimensionally stitched; Define the feedforward network (FFN) as follows: FFN(x) = max(0, xW1+b1)W2+b2 Where W1 is the first layer weight matrix, W2 is the second layer weight matrix, b1 is the first layer bias vector, b2 is the second layer bias vector, and max(0,·) is the ReLU activation function; Define the layer normalization layerNorm as: Where μ is the mean of the input vector, σ 2 is the variance of the input vector, ∈ is the smoothing term to prevent the denominator from being 0, and γ and β are learnable parameters; right Apply Multihead to obtain right The output X of the i-th transformer layer is obtained by applying a feedforward network and layer normalization operations. i : After processing by l Transformer layers, the input sequence f Input Encoded as a context embedding sequence That is, the input sequence f Input The context representation vector R.

7. The title generation method based on a large model as described in claim 6, characterized in that: The step S205, which inputs R into TGM to generate a predicted title, is as follows: TGM generates the title words one by one through a decoder. Each generated word depends on previously generated content and the currently input defect report data. The generation probability formula is: P(y t |y <t ,f input =Softmax(W) o ·LayerNorm(Multihead(y <t )+Multihead(y <t ,R)+FFN)) Among them W o It is a linear transformation matrix, y t y represents the word that is currently being generated. ＜t This indicates content that has already been generated; Based on the generation probability, the decoder selects the word with the highest probability as the generated word, and finally obtains the predicted title by concatenating all the generated words.

Images