A Deep Learning-Based Method for Industrial Software Fault Reporting

Abstract

This invention belongs to the field of software testing technology and relates to a technique for improving the efficiency of software fault report processing. Specifically, it is a deep learning-based method for identifying industrial software fault reports, including: collecting fault reports from specified projects on GitHub by filtering according to keywords, preprocessing the exported dataset, data augmentation, data cleaning, model training, and fault report prediction. This invention can be used for fault report type identification in different industrial software projects, such as open-source projects like Kratos and Dealii; it can also be adapted to the relevant steps of this invention based on the standard fault report information content of a specific software project, thereby identifying and classifying fault reports for that specific software project.

Description

Technical Field

[0001] This invention belongs to the field of software testing technology and relates to a technology for improving the efficiency of software fault report processing, specifically a deep learning-based industrial software fault report discrimination method. Background Technology

[0002] In modern industrial software development, software fault reports are a crucial source of information for development and maintenance teams. Fault reports typically include problems encountered by users during software use, abnormal behaviors, and related software environment information. These reports are invaluable feedback for software developers, helping to identify and resolve problems and improve software quality and stability. However, as software size and complexity increase, the number of fault reports grows exponentially. Processing a large volume of fault reports becomes extremely time-consuming and challenging, especially in complex systems such as industrial software. Furthermore, some fault reports are not actually genuine faults but rather errors caused by user actions, requiring manual verification by developers. A large number of "false" fault reports can severely impact the efficiency of error handling for developers.

[0003] To improve the efficiency and accuracy of software fault report processing, fault report discrimination methods based on deep learning have emerged. Deep learning technology, through learning from a large number of fault reports, can automatically learn patterns and rules, achieving intelligent classification and discrimination of reports. However, current deep learning research in fault report classification mostly focuses on deduplicating fault reports or using different deep learning methods to locate fault codes based on fault report information; research focusing on the discrimination of the fault reports themselves is relatively limited. Summary of the Invention

[0004] To address the aforementioned problems, this invention provides a deep learning-based method for identifying industrial software fault reports. This invention can be used to identify fault report types in different industrial software projects, such as open-source projects like Kratos and Dealii; it can also be adapted to the standard fault report information content of a specific software project, thereby classifying and identifying fault reports for that specific software project.

[0005] Technical solution of the present invention:

[0006] A deep learning-based method for identifying fault reports in industrial software includes the following steps:

[0007] Step 1: Collect failure reports from specified projects on GitHub by filtering them according to keywords.

[0008] By using a keyword filtering method, we collect fault reports that are currently closed using the project repository API provided by GitHub. These fault reports need to be related to common errors in industrial software, i.e., the keywords to be searched should appear in the title or tags. Then, we use code to store the extracted data in a structured format and export it to a CSV file.

[0009] Step 2: Preprocess the exported dataset

[0010] First, the fault reports collected in Step 1 are labeled and categorized, and the label name for each data entry is uniformly modified. The label name includes the error category name, symptom name, and tag name. After initial classification, it is determined whether the error is non-program error and then labeled accordingly. Log information is then extracted from the fault report content to determine the standard log output format for the project. Since the extracted data may contain noise, unstructured text, or other unexpected features, regular expressions are used to clean the data according to the determined standard log format. Finally, standard log information is obtained for each fault report, and the filtered data is output as a JSON file using the JSON file operation module.

[0011] Step 3: Data Augmentation

[0012] The JSON file obtained in step 2 is augmented with the following data: based on the filenames and error function names extracted from the log information, the specific implementations and comments of the corresponding functions are obtained. All project source files involved in the dataset are cloned from GitHub to the local machine. The file paths and extensions are obtained programmatically, and then the files in the project directory are traversed to search for the specified source file. If the target source file does not exist, it is skipped and the search continues. If it exists, regular expressions are used to parse the project source code, extracting comments, function names, and function implementations, which are then added to the JSON file.

[0013] Step 4: Data Cleaning

[0014] The data augmentation file obtained in step 3 is cleaned, and the output is the input dataset for model training. A stop word list is predefined, which contains common words that are usually ignored in text analysis, such as "the" and "and". Special symbols in the text, such as punctuation marks, parentheses, and quotation marks, are removed using regular expressions or other string processing techniques. The cleaned data is output as a CSV file and divided into two columns: a data column and a label information column.

[0015] Step 5: Model Training

[0016] a: We selected a pre-trained open-source BERT-BASE-UNCASED model for fine-tuning. BERT-BASE-UNCASED is a model pre-trained through large-scale unsupervised learning, which learns general language representations. By fine-tuning on a specific task, the model can transfer previously learned knowledge to new tasks, thus achieving better results even on relatively small datasets. Therefore, we first fine-tuned the model by adding two linear layers on top of BERT for classification purposes, setting dropout = 0.1 and ReLU as activation functions. Using the dataset output in step 4, we encoded the text data into an input format acceptable to the model using the `encode_plus` method through a tokenizer corresponding to the pre-trained model. The encoded data was returned as a dictionary, including token IDs, attention masks, token type IDs, and corresponding judgment labels.

[0017] b: To compensate for the potential shortage of data in the training dataset, a K-fold training method is used, dividing the dataset into K equal-sized subsets. Each subset serves as the validation set, while the remaining K-1 subsets are used as the training set. The model is trained, and its performance is evaluated on the validation set. This process is repeated K times, and the average of the K validation results is used as the final evaluation result. The model is then fine-tuned and saved, and a model training log is output.

[0018] Step 6: Fault Report Prediction

[0019] The BERT-BASE-UNCASED model, fine-tuned in step 5, is used to classify and predict fault reports. The information in the newly acquired fault report is input, and the model outputs a prediction of whether the report is a true error. The prediction result is either Y or N.

[0020] The beneficial effects of this invention are:

[0021] (1) Data Enlargement: The discrimination method of this invention not only relies on the log information provided by the fault report, but also uses the log information to expand the data by searching the source code of the error. Based on error clustering, that is, when an error is found, more related or similar error points may be found. The function information suggested in the log association is enriched, thereby enriching the training data and further improving the generalization ability and discrimination performance of the model.

[0022] (2) User-centric Judgment: Compared with other deep learning methods, the method of this invention focuses on judgment from the user's perspective. Current research on fault report judgment in deep learning mostly focuses on filtering duplicate fault reports and locating the erroneous lines of code based on the fault reports, without researching or inventing methods that focus on determining whether the fault report itself represents a real fault or a user's usage problem. With the judgment method of this invention, developers can process an error report more quickly, instead of rerunning the review process every time, saving time and improving error handling efficiency. Attached Figure Description

[0023] Figure 1 This is a schematic diagram of the workflow of the present invention. Detailed Implementation

[0024] The specific embodiments of the present invention will be further described below with reference to the accompanying drawings and technical solutions.

[0025] like Figure 1 As shown, the specific steps of the industrial software fault report discrimination method based on deep learning of the present invention are as follows:

[0026] Step 1: Collect failure reports from specified projects on GitHub by filtering them according to keywords.

[0027] Using a keyword filtering method, fault reports that are currently closed are collected using the project repository API provided by GitHub. These fault reports need to be related to common errors in industrial software. Based on experience and research statistics, the following keywords are selected in this embodiment.

[0028] Bugfix|error|fault|issue|wrong|incorrect|defect|mistake|flaw|nan|inf|fail|crash|exc eption|overflow|underflow|precision|unstable|instability|ringing|unbounded|roundoff|truncation|rounding|diverge|cancellation|cancel|accuracy|accurate

[0029] The extracted data is then stored in a structured format and exported to a CSV file.

[0030] Step 2: Preprocess the exported dataset

[0031] First, based on published papers, the collected fault reports are tagged and categorized, and the tag names for each data entry are standardized, including the error category name, symptom name, and label name. After initial categorization, it is determined whether the error is non-programmatic and categorized accordingly. Log information is then extracted from the fault report content to determine the project's standard log output format. For example, in a fault report log entry used in the dealii project, the extracted log information is: "An error occurred in line". <290> of file in function std::pair <DerivativeForm<1,dim,spacedim> ,Tensor<1,spacedim>>dealii::GridTools::affine_cell_approximation(constArrayView <const Point <spacedim>>&)[dim=2,spacedim=3]The violated condition was: ::dealii::deal_II_exceptions::internals::compare_for_equality(vertices.size(),GeometryInfo <dim>::vertices_per_cell) Additional information: Dimension 9 not equal to 4. Because the extracted data contains noise, unstructured text, and other unexpected features, this invention uses regular expressions in a programming environment and performs data cleaning according to a summarized standard log format. Finally, standard log information is obtained for each fault report. The cleaned data is as follows:

[0032] "file_path":" / var / tmp / portage / sci-libs / dealii-9999 / work / dealii-9999 / source / grid / grid_tools.cc",

[0033] "file_name":"grid_tools.cc",

[0034] "line_number":"290",

[0035] "in function":"std::pair <DerivativeForm<1,dim,spacedim> ,Tensor<1,spacedim>>dealii::GridTools::affine_cell_approximation(const ArrayView <constPoint <spacedim>>&)[dim=2,spacedim=3]",

[0036] "function_name":"affine_cell_approximation",

[0037] Output the filtered and sorted data as a JSON file.

[0038] Step 3: Data Augmentation

[0039] The JSON file obtained in step 2 is augmented. The augmentation goal is to extract the specific implementation and comments of the corresponding functions based on the filename (file_name) and the error function name (in function) from the extracted log information. All project source files involved in the dataset are cloned from GitHub to the local machine, and the file paths and extensions are obtained programmatically. Files in the project directory are traversed to find the specified source file. If the target source file does not exist, it is skipped and the search continues. If it exists, regular expressions are used to extract comments (r'\ / \ / [^\n]*|\ / \*[\s\S]*? \*\ / "). The project source code is parsed, and comments, function names, and function implementations are extracted and added to the JSON file. For example, the log data in step 2 after augmentation is:

[0040] "id":13734,

[0041] "title":"TransfiniteInterpolationManifold in co-dimension 1",

[0042] "file_path":" / var / tmp / portage / sci-libs / dealii-9999 / work / dealii-9999 / source / grid / grid_tools.cc",

[0043] "file_name":"grid_tools.cc",

[0044] "line_number":"290",

[0045] "in function":"std::pair<DerivativeForm<1,dim,spacedim>,Tensor<1,spacedim>>dealii::GridTools::affine_cell_approximation(const ArrayView<constPoint <spacedim>>&)[dim=2,spacedim=3]",

[0046] "function_name":"affine_cell_approximation",

[0047] "function_implementation":"\nAssertDimension(vertices.size(),GeometryInfo <dim>::vertices_per_cell);\n\n / / A=vertex*KA\nDerivativeForm<1,dim,spacedim>A;\n\n for(unsigned int d=0;d<spacedim;++d)\n for(unsigned intv=0;v<GeometryInfo <dim>::vertices_per_cell;++v)\n for(unsigned int e=0;e<dim;++e)\n A[d][e]+=vertices[v][d]*TransformR2UAffine <dim>::KA[v][e];\n\n / / b=vertex*Kb\n Tensor<1,spacedim>b;\n for(unsigned int v=0;v<GeometryInfo <dim>::vertices_per_cell;++v)\n b+=vertices[v]*TransformR2UAffine <dim>::Kb[v];\n\n return std::make_pair(A,b);\n". The function_implementation part is the data content expanded in this step.

[0048] Step 4: Data Cleaning

[0049] The data augmentation file obtained in step 3 is cleaned, and the output is the input dataset for model training. A predefined stop word list "a", "an", "the", "and", "in", "on", "is", "of", "with", "at", "for", "to", "by", "as", "but", "or", "if", "this", "that" is used to remove stop words from the text; the regular expression [^\u4e00-\u9fa5a-zA-Z0-9] is used to match all characters except Chinese characters, English letters, and numbers, and replace them with spaces.The cleaned data is output as a CSV file and divided into two columns: a data column and a label information column. The example data in step 3 after cleaning is as follows: TransfiniteInterpolationManifold in co-dimension 1gridtools cc affine cell approximation assertdimension vertices size geometryinfovertices per cell a vertex ka derivativeform a unsigned int d 0d spacedim dunsigned int v 0v vertices per cell v unsigned int e 0e dim eade verticesv d transformr2uaffine ka veb vertex kb tensor b unsigned int v 0v verticesper cell vb vertices v transformr2uaffine kb vreturn std make pair ab gridtools cc affine cell approximation assertdimension vertices size geometryinfovertices per cell a vertex ka derivativeform a unsigned int d 0d spacedim dunsigned int v 0v vertices per cell v unsigned int e 0e dim ead evertices vd transformr2uaffine ka veb vertex kb tensor b unsigned int v 0v verticesper cell vb vertices v transformr2uaffine kb v return std make pair ab. ;

[0050] Step 5: Model Training

[0051] a: We selected a pre-trained open-source BERT-BASE-UNCASED model for fine-tuning. BERT-BASE-UNCASED learns general language representations through large-scale unsupervised learning of pre-trained models. By fine-tuning on specific tasks, the model can transfer previously learned knowledge to new tasks, thus achieving better results even on relatively small datasets. Therefore, we first fine-tuned the model by adding two linear layers on top of BERT for classification purposes, setting dropout = 0.1 and ReLU as activation functions. Using the dataset output in step 4, we encoded the text data into an input format acceptable to the model using the `encode_plus` method through a tokenizer corresponding to the pre-trained model. The encoded data was returned as a dictionary, including token IDs, attention masks, token type IDs, and corresponding judgment labels.

[0052] b: To compensate for the potential shortage of data in the training dataset, a K-fold training method is used, dividing the dataset into K equal-sized subsets. Each subset serves as the validation set, while the remaining K-1 subsets are used as the training set. The model is trained, and its performance is evaluated on the validation set. This process is repeated K times, and the average of the K validation results is used as the final evaluation result. The model is then fine-tuned and saved, and a model training log is output.

[0053] Step 6: Fault Report Prediction

[0054] The BERT-BASE-UNCASED model trained in step 5 is used to classify and predict fault reports. The information in the newly acquired fault report is input, and the model outputs a prediction of whether the report is a real error. The prediction result is Y or N.< / dim> < / dim> < / dim> < / dim> < / dim> < / spacedim> < / spacedim> < / dim> < / spacedim>

Claims

1. A method for identifying industrial software fault reports based on deep learning, characterized in that, Includes the following steps: Step 1: Collect failure reports from specified projects on GitHub by filtering them according to keywords. By using keyword filtering, we collect fault reports that are currently closed using the project repository API provided by GitHub. These fault reports contain the keywords we need to search for in their titles or tags. Then, code is used to store the extracted data in a structured format and export it to a CSV file; Step 2: Preprocess the exported dataset First, the fault reports collected in step 1 are marked and classified, and the label name of each data is uniformly modified. The label name includes the error category name, symptom name and tag name. After initial classification, it is determined whether the error is not a program error and marked accordingly. Log information is then extracted from the fault report content to determine the standard log output format for the project. Regular expressions are used to clean the data according to the determined standard log format. Finally, the standard log information for each fault report is obtained, and the filtered data is output as a JSON file using the JSON file operation module. Step 3: Data Augmentation The JSON file obtained in step 2 is augmented with the following data: Based on the filenames and error function names in the extracted log information, the specific implementation and comments of the corresponding functions are obtained; all project source files involved in the dataset are cloned from GitHub to the local machine, the file paths and extensions are obtained through programming, and then the files in the project directory are traversed to find the specified source file. It is determined whether the target source file exists. If it does not exist, the data is skipped and the search continues. If it exists, the project source code is parsed using regular expressions, and the comments, function names and function implementations are extracted and written into the JSON file. Step 4: Data Cleaning The data augmentation file obtained in step 3 is cleaned, and the output is the input dataset for model training; a stop word list is predefined; special characters in the text, including punctuation marks, parentheses, and quotation marks, are removed using regular expressions or other string processing techniques; the cleaned data is output as a CSV file and divided into two columns to store the data column and the label information column. Step 5: Model Training a: Fine-tuning is performed using the pre-trained open-source BERT-BASE-UNCASED model. Specifically: First, the model is fine-tuned by adding two linear layers on top of BERT for classification purposes, and setting dropout = 0.1 and ReLU as activation functions. Using the dataset output in step 4, the text data is encoded into an input format that the model can accept through the tokenizer corresponding to the pre-trained model using the encode_plus method, and the encoded data is returned in dictionary form, including token IDs, attention masks, token type IDs, and corresponding judgment labels. b: Use the K-fold training method to divide the dataset into K subsets of equal size; for each subset, use it as the validation set, and use the remaining K-1 subsets as the training set. Train the model and evaluate its performance on the validation set. Repeat the above steps K times and calculate the average of the K validation results as the final evaluation result; perform model fine-tuning training and save it, and output the model training log at the same time. Step 6: Fault Report Prediction The BERT-BASE-UNCASED model, fine-tuned in step 5, is used to classify and predict fault reports. The information in the newly acquired fault report is input, and the model outputs a prediction of whether the report is a true error. The prediction result is either Y or N.

Images