Code automatic reconstruction method based on deep learning

Abstract

The invention relates to the technical field of code reconstruction, and discloses an automatic code reconstruction method based on deep learning, which comprises the following steps of: constructing a model library by using a deep learning algorithm, and then processing and analyzing an original source code to be reconstructed to obtain a data result; and selecting a corresponding reconstruction analysis model according to the data result, and obtaining a related reconstruction suggestion according to the reconstruction analysis model, wherein a user can complete a reconstruction process according to the reconstruction suggestion to obtain a target source code. The method has the beneficial effects that the intelligence of the code reconstruction process is improved, the code reconstruction workload and difficulty are reduced, and the code reconstruction efficiency is improved.

Description

Technical Field

[0001] The present invention relates to the technical field of code refactoring, and in particular to an automatic code refactoring method based on deep learning. Background Art

[0002] In the process of software system development, in order to expand new functions for the software system, it is necessary to continuously pile up the source code of the software system. If the source code is not maintained in time, the source code will evolve in an increasingly chaotic direction. Therefore, when developing new code for the software system, the source code needs to be reconstructed.

[0003] The existing source code refactoring method requires developers to manually download the latest source code from the software system configuration library and import it into the local development tool, where they need to strip the unnecessary code from the source code one by one and check whether there are syntax errors in the remaining source code. If there are syntax errors, they need to be further modified to obtain the refactored source code. It can be seen that the existing source code refactoring method requires developers to perform a lot of manual operations, which reduces the efficiency of source code refactoring. Summary of the invention

[0004] The purpose of the present invention is to solve the above problems and design a code automatic reconstruction method based on deep learning.

[0005] The technical solution of the present invention to achieve the above object is a method for automatic code reconstruction based on deep learning, which comprises the following steps:

[0006] Step 1: Build a model library. Based on the deep learning algorithm, build a refactoring analysis model suitable for different code analyses. Deploy the built multiple refactoring analysis models as a model library to the modeling development platform.

[0007] Step 2: triggering the source code reconstruction mechanism. The user obtains the original source code and loads the obtained original source code to the modeling development platform. Then, the source code reconstruction mechanism is triggered on the modeling development platform in an active or passive manner. The active triggering manner is a manner in which a part of the original source code is manually selected and the source code reconstruction mechanism is manually started. The passive triggering manner is a manner in which the modeling development platform automatically starts the source code reconstruction mechanism after identifying the reconstruction feature during the loading of the original source code.

[0008] Step 3: Processing and analyzing source code: Process and analyze the original source code to be reconstructed in the modeling development platform and obtain data results. Arrange and combine the elements in the data results to obtain different representations of the data results. The model library automatically selects relevant reconstruction analysis models according to the different representations of the data results. The relevant reconstruction analysis models identify and process the different representations of the data results and generate corresponding reconstruction suggestions. The modeling development platform presents the different representations of the data results and the corresponding generated reconstruction suggestions to the user.

[0009] Step 4: Implement refactoring. The user judges and screens the different representations of the data results and the corresponding generated refactoring suggestions, and then completes the refactoring operation according to the selected refactoring suggestions to finally obtain the target code.

[0010] The process of building the model library in step 1 is as follows:

[0011] a. Build a code base, collect and obtain code from open source projects or internal open source projects and use the obtained code to build a code base;

[0012] b. Feature extraction: extract features from the codes in the code base and obtain feature data corresponding to the codes, classify and adjust the feature data corresponding to the codes. The classification and adjustment rules are to classify the feature data corresponding to the codes according to the correlation between the feature data corresponding to the codes, and then arrange and combine the classified feature data corresponding to the codes to form multiple data patterns;

[0013] c. Construct a model library, use multiple deep learning models to process and analyze the classified and adjusted feature data, select a suitable deep learning model as the reconstruction analysis model corresponding to the feature data according to the processing and analysis results, and use the obtained multiple reconstruction analysis models to construct a model library;

[0014] d. Training and evaluation: training and evaluating multiple reconstruction analysis models in the model library and adjusting hyperparameters;

[0015] e. Deployment and iteration: deploy multiple reconstruction analysis models in the model library to the modeling development platform, collect user feedback on the reconstruction results, and then regularly update the reconstruction analysis model and adjust the model architecture and parameters based on the feedback results.

[0016] The codes in the code library need to be marked and preprocessed, and after being marked and preprocessed, the codes can meet the needs of feature extraction.

[0017] The feature extraction method includes: code embedding, abstract syntax tree (AST) and code context, and the feature data corresponding to the code is obtained by using code embedding, abstract syntax tree (AST) and code context.

[0018] After classification and adjustment, the feature data corresponding to the code needs to be processed by multiple deep learning models and the results obtained after processing are analyzed. According to the analysis results, a suitable deep learning model is selected as the reconstruction analysis model. The reconstruction analysis model needs to be trained, evaluated and hyperparameter adjusted before it can be deployed. There is a one-to-one mapping relationship or a one-to-many mapping relationship between the feature data and the reconstruction analysis model. The deep learning models include: recurrent neural network (RNN) model, long short-term memory artificial neural network (LSTM) model, deep learning framework (Transformer) or graph neural network model.

[0019] The processing process of the deep learning model on the feature data corresponding to the code is as follows:

[0020] Input layer: takes the feature data corresponding to the code as the input of the model;

[0021] Hidden layer: multiple hidden layers are used to capture the data pattern in the feature data corresponding to the code, that is, the data pattern formed after the feature data corresponding to the code is classified and adjusted;

[0022] Output layer: Outputs reconstruction suggestions based on the captured data patterns.

[0023] The training process of the reconstruction analysis model is to separate a part of the marked and preprocessed code in the code base as a training set, and use the data in the training set to train the reconstruction analysis model. The whole training process involves multiple training cycles and parameter adjustment cycles; the evaluation of the reconstruction analysis model is to separate a part of the marked and preprocessed code in the code base as a verification set, and use the verification set to evaluate the performance of the reconstruction analysis model. The evaluation indicators include but are not limited to accuracy, recall rate, and F1 score; the hyperparameter adjustment of the reconstruction analysis model is to adjust the hyperparameters of the model through grid search, random search or Bayesian optimization.

[0024] The channel for obtaining the original source code in step 2 is loading it from a code configuration library or from an open source project.

[0025] The method of performing data processing on the original source code to be reconstructed in the modeling development platform in step three includes but is not limited to: removing irrelevant information such as comments and spaces, and converting the code into a unified format and representation; the method of analyzing the original source code to be reconstructed in the modeling development platform includes but is not limited to: extracting key elements in the original source code, arranging and combining the extracted key elements to obtain a variety of different data result representations, and identifying potential problems in the original source code based on the above key elements; the reconstruction suggestions include but are not limited to: code structure optimization, algorithm optimization, and code style improvement.

[0026] The refactoring operation and refactoring process performed by the user in step 4 will be recorded by the modeling development platform as feedback of the refactoring results. The feedback of the refactoring results will be used to enrich the code base and iteratively optimize the refactoring analysis model.

[0027] Compared with the prior art, the present invention has the following beneficial effects:

[0028] 1. The present invention uses a deep learning model to automatically analyze the code and obtain a reconstruction analysis model, then processes and analyzes the original source code to obtain data results, and then intelligently analyzes the reconstruction process of the original source code based on the data results and the reconstruction analysis model and obtains reconstruction suggestions, thereby improving the user's reconstruction efficiency of the original source code and reducing the workload of reconstruction.

[0029] 2. The present invention uses a reconstruction analysis model to analyze the original source code according to the user's request and obtains corresponding reconstruction suggestions, and then automatically reconstructs the original source code through the reconstruction suggestions to obtain the target source code required by the user, thereby realizing intelligent code reconstruction and reducing the operational difficulty during the code reconstruction process. BRIEF DESCRIPTION OF THE DRAWINGS

[0030] Figure 1 It is a flowchart of a method for automatic code reconstruction based on deep learning described in the present invention. DETAILED DESCRIPTION

[0031] The present invention will be described in detail below in conjunction with the accompanying drawings. Figure 1 As shown;

[0032] A code automatic refactoring method based on deep learning, the method comprising the following steps:

[0033] Step 1: Build a model library. Based on the deep learning algorithm, build a refactoring analysis model suitable for different code analyses. Deploy the built multiple refactoring analysis models as a model library to the modeling development platform.

[0034] Step 2: trigger the source code refactoring mechanism. The user obtains the original source code and loads it to the modeling development platform. Then, the source code refactoring mechanism is triggered on the modeling development platform in an active or passive manner. The active triggering method is to manually select a part of the original source code and manually start the source code refactoring mechanism. The passive triggering method is that the modeling development platform automatically starts the source code refactoring mechanism after identifying the refactoring features during the loading of the original source code. The refactoring features include: code smell: identifying common code smells, such as duplicate code, too long methods, too large classes, too deep inheritance levels, data clumps, etc.; performance bottlenecks: analyzing performance bottlenecks and identifying which parts need to be optimized; test coverage: checking test coverage, especially those code areas that lack tests. These areas may be more prone to errors and are also good candidates for refactoring;

[0035] Step 3: Processing and analyzing source code: Analyze the original source code to be reconstructed in the modeling development platform and obtain data results. Process the data results, that is, arrange and combine the elements in the data results to obtain different forms of expression of the data results. The model library automatically selects and matches relevant reconstruction analysis models according to the different forms of expression of the data results. The relevant reconstruction analysis models identify and process the different forms of expression of the data results and generate corresponding reconstruction suggestions. The modeling development platform presents the different forms of expression of the data results and the corresponding generated reconstruction suggestions to the user.

[0036] Step 4: Implement refactoring. The user judges and screens the different representations of the data results and the corresponding generated refactoring suggestions, and then completes the refactoring operation according to the selected refactoring suggestions to finally obtain the target code.

[0037] It should be noted that the process of building the model library in this application is as follows:

[0038] a. Build a code base, collect and obtain code from open source projects or internal open source projects and use the obtained code to build a code base;

[0039] b. Feature extraction: extract features from the codes in the code base and obtain feature data corresponding to the codes, classify and adjust the feature data corresponding to the codes. The classification and adjustment rules are based on the correlation between the feature data corresponding to the codes, and then arrange and combine the classified feature data corresponding to the codes to form multiple data patterns;

[0040] c. Construct a model library, use multiple deep learning models to process and analyze the classified and adjusted feature data, select a suitable deep learning model as a reconstruction analysis model based on the processing and analysis results, and use the obtained multiple reconstruction analysis models to construct a model library;

[0041] d. Training and evaluation: training and evaluating multiple reconstruction analysis models in the model library and adjusting hyperparameters;

[0042] e. Deployment and iteration: deploy multiple reconstruction analysis models in the model library to the modeling development platform, collect user feedback on the reconstruction results, and then regularly update the reconstruction analysis model and adjust the model architecture and parameters based on the feedback results.

[0043] It should be noted that the code in the code base needs to be marked and preprocessed. After marking and preprocessing, the code can meet the needs of feature extraction. The marked content includes but is not limited to: the code version that needs to be refactored and the refactored code version. The preprocessing method is to convert the code into a format that can be processed by the reconstruction analysis model in the model library. These formats include but are not limited to: word embedding and abstract syntax tree (AST).

[0044] Among them, the feature extraction methods include: code embedding, abstract syntax tree (AST) and code context. Code embedding, abstract syntax tree (AST) and code context are used to obtain feature data corresponding to the code. Among them, code embedding is to use a pre-trained word embedding model (such as Word2Vec, BERT) to convert the code into a vector representation; the abstract syntax tree (AST) is to extract the abstract syntax tree (AST) features of the code, and capture the structural information of the code through the abstract syntax tree (AST) features; the code context is to select and identify the features of the context around the code, such as function name, class name, variable name, command and object, etc.

[0045] In the specific implementation process of this application, the feature data corresponding to the code needs to be processed by multiple deep learning models after classification and adjustment, and the results obtained after processing are analyzed. According to the results of the analysis, a suitable deep learning model is selected as the reconstruction analysis model. The reconstruction analysis model needs to be trained, evaluated and hyperparameter adjusted before it can be deployed. There is a one-to-one mapping relationship or a one-to-many mapping relationship between the feature data corresponding to the code and the reconstruction analysis model; the deep learning model includes but is not limited to: a recurrent neural network (RNN) model, a long short-term memory artificial neural network (LSTM) model, a deep learning framework (Transformer) or a graph neural network model.

[0046] Among them, the process of deep learning model processing feature data includes:

[0047] Input layer: takes the feature data corresponding to the code as the input of the model;

[0048] Hidden layer: multiple hidden layers are used to capture the data pattern in the feature data corresponding to the code, that is, to capture the data pattern after the feature data corresponding to the code is classified and adjusted;

[0049] Output layer: Outputs reconstruction suggestions based on the captured data patterns.

[0050] The training process of the reconstruction analysis model is to separate a part of the marked and preprocessed code in the code base as a training set, and use the data in the training set to train the model. The entire training process involves multiple training cycles and parameter adjustment cycles; the evaluation of the reconstruction analysis model is to separate a part of the marked and preprocessed code in the code base as a validation set, and use the validation set to evaluate the performance of the model. The evaluation indicators include but are not limited to accuracy, recall rate, and F1 score; the hyperparameter adjustment of the reconstruction analysis model is to adjust the hyperparameters of the model through grid search, random search, or Bayesian optimization. It should be noted that hyperparameters are parameters manually set before the algorithm runs to control the behavior and performance of the model; the choice of hyperparameters will affect the model's training speed, convergence, capacity, and generalization ability. Common hyperparameters include: learning rate, number of iterations, regularization parameter, number of neurons in the hidden layer, etc.

[0051] In the specific implementation process of this application, the original source code is obtained by loading it from the code configuration library or from the open source project; the data processing method of the original source code to be reconstructed in the modeling development platform includes but is not limited to: removing irrelevant information such as comments and spaces, and converting the code into a unified format and representation method; the analysis method of the original source code to be reconstructed in the modeling development platform includes but is not limited to: identifying and extracting elements in the original source code to form data results, and arranging and combining the elements in the data results, thereby obtaining a variety of different data result representations; the elements include: key information such as functions, commands, variables, objects, and control structures;

[0052] The reconstruction analysis model is automatically selected and matched according to the above data result representation form. The selection and matching rules are: use the element categories and the arrangement or combination of elements in the data result representation form to match with the data patterns in the model library. When a data result representation form is successfully matched with a data pattern, the data result representation form matches the reconstruction analysis model corresponding to the data pattern, and outputs reconstruction suggestions for potential problems in the original source code based on the reconstruction analysis model; these potential problems include: code redundancy, code duplication, performance bottlenecks, poor maintainability, etc.; reconstruction suggestions include but are not limited to: code structure optimization, algorithm optimization, code style improvement, etc.

[0053] It should be noted that the refactoring operations and refactoring processes implemented by the user will be recorded by the modeling development platform as feedback on the refactoring results. The feedback on the refactoring results will be used to enrich the code base and iteratively optimize the refactoring analysis model.

[0054] In addition, in the specific implementation process of this application, how to collect high-quality labeled data is very important, which directly affects the performance and effect of code refactoring. In addition, the diversity and complexity of code data may also have a negative impact on the performance of the model. Therefore, the reconstruction analysis model needs to ensure that it can run and output stably when deployed.

[0055] Example 1

[0056] Take a Python code as an example, and take this code as the original source code. This original source code is used to perform a common task: calculate and print the sum of all integers from 1 to 10 (including 1 and 10). The specific content is as follows:

[0057] "# Initialize a variable to store the sum

[0058] total_sum=0

[0059] #Use a for loop to traverse from 1 to 10 (including 10)

[0060] for number in range(1,11):

[0061] # Add the current number to the total

[0062] total_sum+=number

[0063] #Print the sum

[0064] print("The sum of integers from 1 to 10 is:",total_sum)";

[0065] The above original source code is processed, comments and spaces in the original source code are removed, and the following content is obtained:

[0066] "total_sum = 0

[0067] for number in range(1,11):

[0068] total_sum+=number

[0069] print;"

[0070] The processed original source code is analyzed, and the variable total_sum, commands for, range, and function print defined in the original source code are extracted. The original source code is analyzed based on the extracted variables, commands, and functions, and the types of problems that may exist in the original source code are considered. Then, the types of problems are recorded. For example, according to the command type and the number of code lines, it is determined whether the original source code has a problem of lengthy code, and according to the function type, it is determined whether there is a stability problem in the code operation; then the extracted variables, commands, and functions are arranged and combined to obtain multiple data result representations, and the data result representations are used to match the corresponding reconstruction analysis model in the model library, and refactoring suggestions are generated based on the matched reconstruction analysis model. Then, the generated refactoring suggestions and the recorded problem types are presented to the technicians. The technicians can complete the reconstruction process based on the reconstruction suggestions and the recorded problem types.

[0071] Example 2

[0072] Take a C++ code as an example, which creates a simple text editor framework, which contains different types of documents (such as text files and HTML files), which can be read, edited (only simulated here) and saved. The specific content is as follows:

[0073]

[0074]

[0075]

[0076]

[0077]

[0078] The above original source code is processed to remove comments and spaces in the original source code, and then the processed original source code is analyzed to extract the abstract base class Document, two derived classes: Text Document and HtmlDocument, class Editor, management command std::unique_ptr and object Document in the original source code, and the abstract syntax tree features in the original source code are extracted to capture the structural information of the code through the abstract syntax tree features. The original source code is analyzed based on the extracted structural information of the abstract base class, derived class, management command, object and code, and the types of problems that may exist in the original source code are recorded, for example: the maintainability and performance bottleneck of the code are analyzed through the classes and objects included in the code and the structural information of the code; then the extracted structural information of the abstract base class, derived class, management command, object and code is arranged and combined to obtain multiple data result representations, and the corresponding reconstruction analysis model is matched in the model library using the data result representation, and reconstruction suggestions are generated based on the matched reconstruction analysis model, and then the generated reconstruction suggestions and the recorded problem types are presented to the technicians together, and the technicians can complete the reconstruction process based on the reconstruction suggestions and the recorded problem types.

[0079] Example 3

[0080] Take a Python code as an example, and use it as the original source code. This code implements a simple command line to-do manager, which allows users to add, view, complete, and delete to-do items. Its content is as follows:

[0081]

[0082]

[0083]

[0084]

[0085] The above original source code is processed to remove comments and spaces in the original source code, and then the processed original source code is analyzed to extract classes, objects, code structures and command features in the original source code. Then, the original source code is analyzed based on the structural information of the extracted classes, objects, commands and codes, and the types of problems that may exist in the original source code are recorded. Subsequently, the structural information of the extracted classes, objects, commands and codes is arranged and combined to obtain multiple data result representations, and the corresponding reconstruction analysis models are matched in the model library using the data result representations, and reconstruction suggestions are generated based on the matched reconstruction analysis model. Then, the generated reconstruction suggestions and the recorded problem types are presented to the technicians, and the technicians can complete the reconstruction process based on the reconstruction suggestions and the recorded problem types.

[0086] It should be noted that, in this article, relational terms such as first and second, etc. are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Moreover, the terms "include", "comprise" or any other variants thereof are intended to cover non-exclusive inclusion, so that a process, method, article or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or device. In the absence of further restrictions. The sentence "includes an element defined by ... does not exclude the existence of other identical elements in the process, method, article or device including the element".

[0087] The above technical solutions only reflect the preferred technical solutions of the technical solutions of the present invention. Some changes that may be made to certain parts thereof by technicians in this technical field all reflect the principles of the present invention and fall within the protection scope of the present invention.

Claims

1. A code automatic refactoring method based on deep learning, characterized in that: The method comprises the following steps: Step 1: Build a model library. Based on the deep learning algorithm, build a refactoring analysis model suitable for different code analyses. Deploy the built multiple refactoring analysis models as a model library to the modeling development platform. Step 2: triggering the source code reconstruction mechanism, the user obtains the original source code and loads the obtained original source code to the modeling development platform, and then triggers the source code reconstruction mechanism on the modeling development platform in an active or passive manner; Step 3: Processing and analyzing source code: Processing and analyzing the original source code to be reconstructed in the modeling development platform to obtain data results, arranging and combining the elements in the data results to obtain multiple different data result representations. The model library automatically selects and matches the relevant reconstruction analysis model according to the data result representation. The relevant reconstruction analysis model identifies and processes the data result representation and generates corresponding reconstruction suggestions. The modeling development platform presents the data result representation and the corresponding generated reconstruction suggestions to the user. Step 4: Implement refactoring. The user judges and screens the data result representation and the corresponding generated refactoring suggestions, and then completes the refactoring operation according to the selected refactoring suggestions to finally obtain the target code.

2. The method for automatic code refactoring based on deep learning according to claim 1, characterized in that: The process of building the model library in step 1 is as follows: a. Build a code base, collect and obtain code and use the obtained code to build a code base; b. Feature extraction: extract features from the code in the code base and obtain feature data corresponding to the code, and classify and adjust the feature data corresponding to the code to form multiple data patterns; c. Build a model library, use multiple deep learning models to process and analyze the feature data corresponding to the code, select a suitable deep learning model as a reconstruction analysis model based on the processing and analysis results, and use the obtained multiple reconstruction analysis models to build a model library; d. Training and evaluation: training and evaluating multiple reconstruction analysis models in the model library and adjusting hyperparameters; e. Deployment and iteration: deploy the model library to the modeling development platform, collect user feedback on the reconstruction results, and regularly update the reconstruction analysis model, adjust the model architecture and parameters.

3. The method for automatic code refactoring based on deep learning according to claim 2, characterized in that: The codes in the code library need to be marked and preprocessed, and after being marked and preprocessed, the codes can meet the needs of feature extraction.

4. The method for automatic code refactoring based on deep learning according to claim 3, characterized in that: The feature extraction method includes: code embedding, abstract syntax tree and code context, and the feature data corresponding to the code is obtained by using the code embedding, abstract syntax tree and code context.

5. The method for automatic code refactoring based on deep learning according to claim 4, characterized in that: The feature data corresponding to the code selects a suitable deep learning model as a reconstruction analysis model based on the data pattern formed by the feature data.

6. The method for automatic code refactoring based on deep learning according to claim 5, characterized in that: The process of the deep learning model processing the feature data corresponding to the code is as follows: Input layer: takes the feature data corresponding to the code as the input of the model; Hidden layers: Multiple hidden layers are used to capture data patterns in the feature data corresponding to the code; Output layer: Outputs reconstruction suggestions based on the captured data patterns.

7. The method for automatic code refactoring based on deep learning according to claim 5, characterized in that: The training process of the reconstruction analysis model is to separate a part of the marked and preprocessed code in the code base as a training set, and use the data in the training set to train the model; The evaluation of the reconstruction analysis model is to separate a part of the marked and preprocessed code in the code base as a verification set, and use the verification set to evaluate the performance of the reconstruction analysis model; the hyperparameter adjustment of the reconstruction analysis model is to adjust the hyperparameters of the reconstruction analysis model through grid search, random search or Bayesian optimization.

8. The method for automatic code refactoring based on deep learning according to claim 1, characterized in that: The channel for obtaining the original source code in step 2 is loading it from a code configuration library or from an open source project.

9. The method for automatic code refactoring based on deep learning according to claim 1, characterized in that: The method of performing data processing on the original source code to be reconstructed in the modeling development platform in step 3 includes but is not limited to: removing irrelevant information and converting the code into a unified format and representation; Methods for analyzing the original source code to be reconstructed in the modeling development platform include but are not limited to: extracting key elements in the original source code, and arranging and combining the extracted key elements to obtain a variety of different forms of expression; the reconstructing suggestions in step three include but are not limited to: code structure optimization, algorithm optimization, and code style improvement.

10. The method for automatic code refactoring based on deep learning according to claim 1, characterized in that: The refactoring operation and refactoring process performed by the user in step 4 will be recorded by the modeling development platform as feedback of the refactoring results. The feedback of the refactoring results will be used to enrich the code base and iteratively optimize the refactoring analysis model.

Images