Software design pattern recommendation method based on dual-mode joint modeling

Abstract

This invention discloses a software design pattern recommendation method based on bimodal joint modeling. It fully utilizes deep language models to jointly model natural language and programming language to solve the task of recommending software design patterns to users. First, a corpus for software design pattern recommendation is constructed. Second, user requirement text and software source code are converted into real-valued feature vectors. Then, a bimodal joint modeling software design pattern recommendation model is constructed, and the feature vectors from the two different modalities are input for interactive matching. Finally, software design pattern recommendations are implemented based on user needs. This invention can provide users with a candidate list of software design patterns that meet their needs, and it has a more accurate recommendation effect compared to other models, demonstrating the practicality of the bimodal joint modeling-based software design pattern recommendation method.

Description

Technical Field

[0001] This invention belongs to the field of information retrieval and tag recommendation, and specifically relates to a technology that uses a deep language model to jointly model natural language and programming language, and to classify target tags. Background Technology

[0002] Software design patterns are a programming style and a design style. Software engineers face the challenge of choosing the right design patterns during the coding process of software projects. Over the long course of software development and design, and in response to different software design needs, excellent software developers have proposed a variety of mature design patterns. Good design patterns can match current software design requirements, better cover various software use cases, and improve the coding efficiency of developers during the software development process.

[0003] Software design requirements are diverse during software development. How to use appropriate design patterns to cover these requirements has traditionally relied on the personal experience of software engineers. Therefore, many methods for recommending software design patterns to developers have emerged. The general process for recommending software design patterns is as follows: first, understand the user requirements; then, find corresponding code examples and relevant design pattern tags to recommend software design patterns.

[0004] Existing design pattern recommendations suffer from difficulties in understanding requirements and code functionality through word matching retrieval methods. Research indicates that no relevant products or technologies on the market can address this deficiency. Therefore, this invention proposes an automated method to provide software development engineers with design patterns and related sample code that meet their design needs. Summary of the Invention

[0005] To overcome the shortcomings of existing technologies, this invention proposes a complete solution for the software source code design pattern recommendation problem for the first time, providing a software design pattern recommendation method based on bimodal joint modeling. By providing a method to recommend design patterns that meet the needs of software developers, it overcomes the shortcomings of existing software design pattern methods that rely too much on the personal experience of software development engineers and have a low degree of automation. Compared with machine learning methods and information retrieval methods, it has a higher accuracy advantage.

[0006] This invention employs a bimodal co-modeling software design pattern recommendation model. It fully utilizes software design requirement text and software source code to construct a bimodal co-modeling-based software design pattern recommendation method, solving the task of automatically providing software development engineers with design patterns that meet their design requirements, along with relevant sample code. The method first constructs a corpus for recommending software design patterns to users. Based on this, it proposes a bimodal co-modeling software design pattern recommendation model. Finally, based on this model and the corpus, and through understanding the user's input requirement text described in natural language, it provides users with appropriate design pattern labels and relevant sample software source code.

[0007] The technical solution of this invention is as follows:

[0008] The overall flowchart of the software design pattern recommendation method based on bimodal joint modeling is as follows: Figure 1 As shown, the steps are as follows:

[0009] Step 1: Construct a corpus for recommending software design patterns. The corpus includes software source code for different design patterns, software design requirement text, and design pattern tags shared by the software source code and software design requirement text.

[0010] Step 2: Apply the bimodal joint modeling software design pattern recommendation model. Based on the corpus constructed in Step 1, implement user-demand-oriented software design pattern recommendations, providing users with a list of candidate recommendations that meet their needs. The specific steps are as follows:

[0011] Step 2.1: Extract software design requirement features from the user requirement text, and then input it into the software design requirement understanding module to convert it into a user requirement feature vector, thereby realizing user requirement understanding;

[0012] Step 2.2: Extract multi-source features from the software source code in the corpus, then input them into the software design pattern understanding module, and finally transform them into real number feature vectors of software design patterns.

[0013] Step 2.3: Input the user demand feature vector output in Step 2.1 and the real number feature vector of the software design pattern output in Step 2.2 into the dual-modal joint modeling software design pattern recommendation model to perform vector interaction in sequence, realize software design pattern matching, obtain the relevance score between the two, and based on this, sort the software source code in the corpus in descending order according to the relevance score to obtain the candidate list of design pattern recommendation corpus, and output the design pattern recommendation result to the user.

[0014] Furthermore, the process of constructing the corpus for software design pattern recommendation in step 1 includes the collection, cleaning, storage, annotation, and merging / alignment of two types of data:

[0015] The software source code and software design requirements text share the same data collection, cleaning and storage process: First, the software source code and software design requirements text are collected through web crawlers; second, the useless data is removed through the data cleaning pipeline, and only the required data is retained; finally, the data is stored in the server for later use.

[0016] The data annotation specifically includes: annotating the design patterns used in the software source code, with only three design patterns retained as annotation labels when annotating the software source code; and annotating the design patterns involved in the software design requirement text, selecting appropriate design patterns as labels based on the specific design problems, design functions, and related indicators described in the software design requirement text, with a maximum of three design pattern labels for each software design requirement text.

[0017] The data merging and alignment process specifically includes: when software source code and software design requirement text share at least one design pattern label, they are paired as a single data pair, and each shared design pattern label and data pair constitutes a corpus entry. The basic data format of the corpus consists of code text at the method or file level corresponding to the software source code, pure natural language text descriptions of the software design requirements, and standard names of the labeled design patterns corresponding to the design pattern labels. The specific data format is shown in Table 1. This corpus was subsequently used for training a design pattern recommendation model and for user-oriented design pattern label recommendation.

[0018] Table 1 Corpus

[0019]

[0020]

[0021] Furthermore, the construction of the dual-modal joint modeling software design pattern recommendation model in step 2 specifically includes: firstly, extracting software design requirement text features from the software design requirement text in the corpus of step 1, and then converting them into real-valued feature vectors of software design requirements using the software design requirement understanding module; secondly, extracting multi-source features from the software source code in the corpus of step 1, and then converting them into real-valued feature vectors of software design patterns using the software design pattern understanding module; and finally, inputting the two real-valued feature vectors into the dual-modal joint modeling software design pattern recommendation model for model training, thereby completing the construction of the recommendation model.

[0022] Specifically, the software design requirement text feature extraction involves the following steps: First, based on the established corpus, the software design requirement text described in natural language is further cleaned. Non-natural language components (such as software source code and identifier names) are normalized, for example, by segmenting the identifier components to obtain software design requirement text that conforms to natural language expression standards. Then, the programming language features (such as Java, Python, PHP, and Javascript) mentioned in the software design requirement text are extracted. These features, along with the cleaned software design requirement text, constitute the software design requirement text feature data.

[0023] Specifically, the software design requirements understanding module includes a first natural language pre-training submodule, a project development programming language feature embedding submodule, and a first feature fusion submodule, such as... Figure 3 As shown, based on the feature extraction of the software design requirement text, the software design requirement text feature data is converted into a feature vector from the perspective of the requirement text through the first natural language pre-training submodule. Here, the maximum length of the input requirement text is limited to 512 words depending on the type of the pre-trained language model. The programming language type feature data is converted into a feature vector from the perspective of the programming language through the project development programming language feature embedding submodule. Here, if the corresponding programming language type record is missing in the embedding submodule, a special symbol representing "unknown" is used. <unk>The corresponding representation vectors are replaced, and the two feature vectors are simultaneously input into the first feature fusion module to fuse them into a real number feature vector for software design requirements.

[0024] Specifically, the multi-source feature extraction of software source code includes: first, extracting and dividing the collected raw data, removing configuration files, resource files, and files unrelated to the software source code; second, compiling and verifying code fragments for software source code or files in different programming languages ​​using a compiler to ensure code correctness, and converting them into corresponding abstract syntax trees; then, extracting three key features from the software source code, including software source code method names, software source code API names, and the software source code abstract syntax tree. The method names and API names in the software source code are stored as string sequences. The abstract syntax tree data corresponding to the software source code is combined with a random walk algorithm to sample each path of the syntax tree, obtaining the corresponding string sequences as a representation of the structure between nodes in the software source code, i.e., the sampling path of the sampling syntax tree. Finally, the three key features and the software source code constitute multi-source software source code feature data.

[0025] Specifically, the software design pattern understanding module includes a software source code pre-training submodule, a second natural language pre-training submodule, a software source code structural feature extraction submodule, and a second feature fusion submodule, such as... Figure 4 As shown, firstly, based on the feature data of multi-source software source code, the software source code is input into the software source code pre-training submodule to obtain the representation feature vector of the software source code. Here, depending on the type of pre-training model, the input software source code text is limited to a maximum of 512 words. Secondly, the method name and API name of the software source code are concatenated in their original order in the software source code and input into the second natural language pre-training submodule to obtain the deep semantic feature vector of the natural language components in the software source code. Then, the abstract syntax tree sampling path is input into the software source code structural feature extraction submodule to construct the structural feature vector of the software source code. Finally, the three feature vectors are simultaneously input into the second feature fusion submodule to fuse and form the real number feature vector of the software design pattern.

[0026] Specifically, the training process for the software design pattern recommendation model based on dual-modal joint modeling is as follows:

[0027] The model is based on the BERT Siamese network, and the model training process is as follows: Figure 5 As shown, firstly, the real feature vectors of software design requirements in the natural language modality and the real feature vectors of software design patterns in the programming language modality, output by the understanding module, are input into the feature fusion neural network of the feature interaction fusion module for interaction fusion to obtain a bimodal design pattern representation vector. A deep model for software design pattern recommendation is then constructed. Secondly, the design pattern labels in the corpus are used as training targets, and an output structure for multi-label classification, namely the design pattern prediction classification head, is designed. Then, the text classification loss function is used as the loss function for model training. The design pattern labels predicted by the design pattern prediction classification head are combined with the correct labels and the labels of negative samples to form triples. The text classification loss function is used to learn the sample difference between positive and negative examples. The gradient backpropagation algorithm is used during model training. Finally, the optimization methods commonly used in neural networks are used to find the convergence state of the bimodal joint modeling software design pattern recommendation model.

[0028] Furthermore, the candidate list data format of the design pattern recommendation corpus in step 2.3 is shown in Table 2, including: software source code, design pattern tags, a list of method names contained in the code, a list of APIs, and a relevance score as the sorting criterion.

[0029] Table 2. List of recommended candidate data for design patterns

[0030]

[0031] Furthermore, in step 2.3, the design pattern recommendation results are output to the user. Based on the candidate list sorted in descending order of relevance score, the code snippets in the candidate list are formatted according to different programming languages ​​to facilitate user reading. The specific code snippet formatting methods include: automatic indentation, automatic code layout, and code highlighting.

[0032] The beneficial effects of this invention are as follows: First, it proposes a corpus construction method for software design pattern recommendation. Based on this, it proposes a bimodal joint modeling method for software design pattern recommendation using a pre-trained language model, achieving a balance between deep understanding of the natural semantics of software design requirements text and deep understanding of design patterns in software source code. Finally, based on the corpus and the bimodal joint modeling software design pattern recommendation model, this invention proposes a user-demand-oriented software design pattern recommendation method that can better serve software development. Attached Figure Description

[0033] Figure 1 This is the overall flowchart of the technical solution of the present invention.

[0034] Figure 2 This is a flowchart of the data acquisition, cleaning, and storage process of the technical solution of this invention.

[0035] Figure 3 This is a schematic diagram of the software design requirements understanding module structure of the technical solution of this invention.

[0036] Figure 4 This is a schematic diagram of the software design pattern understanding module structure of the technical solution of this invention.

[0037] Figure 5 This is a flowchart of the software design pattern recommendation model training process of the technical solution of this invention.

[0038] Figure 6 This is a flowchart of a software design pattern recommendation model based on user development needs, which is the technical solution of this invention.

[0039] Figure 7 This is the StackOverflow data download and cleaning process in an embodiment of the technical solution of the present invention.

[0040] Figure 8 This is a recommended model diagram of a software design pattern for dual-modal joint modeling, which is an embodiment of the technical solution of the present invention. Detailed Implementation

[0041] To make the above-mentioned objects, features and advantages of the present invention more apparent and understandable, the present invention will be further described in detail below with reference to the accompanying drawings and specific embodiments.

[0042] Example 1

[0043] This invention builds a corpus based on the open-source question-answering community Stack Overflow, and utilizes the natural language pre-trained model Roberta and the software source code pre-trained model CodeBert, employing methods such as... Figure 1 As shown, the recommended method for constructing a dual-modal joint modeling software design pattern is implemented through the following steps:

[0044] S1: Using the Stack Overflow open-source software knowledge-sharing platform as the data source, web crawlers were used to automatically collect post data from the Stack Overflow platform. The Stack Overflow post data format is as follows: a post must contain at least the poster's question about a software development issue (title), a question description (post body), software source code results from other users' answers to the question, and post tags to distinguish the domain of the question. After cleaning the collected data, text content relevant to the software design pattern recommendation task was retained, including but not limited to: post title, post body, software source code within the post, text content of other users' replies and their software source code, and post tags. Stack Overflow posts contain software design requirement text and software source code examples, and the tags in the posts include design pattern tags related to the post; therefore, data annotation can be automated. The specific data collection and cleaning process is as follows: Figure 7 As shown.

[0045] First, this invention combines Wikipedia and relevant textbooks to compile commonly used design pattern tag data. Authoritative design pattern books categorize their lists of design patterns according to problem definitions and design specifications, describing them by a chapter or section. The descriptive corpus contains 431 documents from 13 design pattern books and 125 Wikipedia pages, covering 372 unique design patterns. A dictionary of design pattern tag names, V, is constructed. DP .

[0046] All data on the Stack Overflow site is publicly accessible and regularly updated in SQL database format. After downloading, the raw data is stored in a MySQL database. Before cleaning, SQL statements are written based on the post IDs related to the desired design patterns to extract key data items (ID, Title, Body, Tag) from the corresponding posts, resulting in a dataset of 180,000 questions and 300,000 answers. Source code blocks are extracted from the answers to obtain the software source code related to the post questions. Finally, for the data annotation, to obtain data with higher semantic relevance to the design patterns, the obtained V... DP The 372 design pattern names and aliases are used to filter posts containing the design pattern tag (DP_Tag), which is then combined with the tag section that is filtered simultaneously.

[0047] In the data cleaning section, this invention uses NLTK, a commonly used text analysis library for natural language processing, to clean the original text. The text portion of each document is preprocessed, and a compact vocabulary is constructed by filtering out irrelevant and redundant information: First, code-like tags in natural language sentences, such as function names, are segmented according to camelCase to ensure the semantic integrity of the sentences; then, each document is tokenized and lowercased, and English stop words, HTML tags, non-alphabetic characters, and words appearing no more than 5 times that have no value in reflecting the semantic relationship between natural language and design patterns are removed.

[0048] Finally, after completing the data collection and cleaning steps, the data was aligned and summarized according to Table 1, and the data was saved on the server in CSV format, thus completing the construction of the corpus.

[0049] S2: Based on corpus data and the natural language pre-trained model Roberta and the software source code pre-trained language model CodeBert, this paper presents the specific structure of a dual-modal joint modeling software design pattern recommendation model. The model structure is as follows: Figure 8 As shown, it includes a programming language word embedding matrix, a source code node embedding matrix, two Roebrta pre-trained models, a CodeBert pre-trained model, and a Transformer encoder structure.

[0050] Specifically, the software design requirements text is denoted as S. N , and have Where w i The input text represents natural language phrases. Following the input format of the Robeta pre-trained natural language model, the input software design requirements text needs to be formatted using a template. Let the processed software design requirements text be denoted as... The valid input format for software design requirements text is shown in formula (1):

[0051]

[0052] Among them, ` <cls>The backslash (`) is used to identify the beginning of the current sentence, while the backslash (`) <sep>The backslash (`) is used as a terminator. The software design requirement text, packaged using the above template, is input into the Roberta model to obtain its corresponding software design requirement text representation feature vector. The specific process is shown in formula (2):

[0053] The programming language platform term extracted from the software design requirements text corresponds to W. P This describes the programming language platforms users face when solving current software development problems. This invention compiles a list of mainstream programming languages, organizes them into a lexicon, and constructs a programming language word embedding matrix E to describe the programming language characteristics in software design requirements. P Each row of the matrix represents a word representing a programming language name (e.g., Java, Python, etc.). The dimension of the word vector is the same as the dimension of the feature vector representing the software design requirements text obtained by Roberta as described in formula (1) above, so as to facilitate feature fusion. Therefore, the feature vector from the programming language perspective... Then, as shown in formula (3):

[0054]

[0055] Finally, by and The real-valued feature vector V is obtained by fusing the feature vectors through feature vector summation. N As shown in formula (4):

[0056]

[0057] The software source code input is divided into three types: software source code text content S P Software source code method name and API name sequence and the abstract syntax tree sequence corresponding to the software source code. Similar to the processing of software requirements text, the CodeBert model also needs to wrap the text when obtaining the representation vector of the software source code text, denoted as... Then use the template to wrap the input text. As shown in formula (5):

[0058]

[0059] Then the characteristic vector representing the software source code It can be obtained through formula (6):

[0060]

[0061] Similarly, the sequence of method names and API names in the software source code of the Roberta model is used as input. The corresponding input text after packaging is shown in formula (7):

[0062]

[0063] The deep semantic feature vector of the software source code It can be obtained through formula (8):

[0064]

[0065] This place and In contrast, method names and API names in software source code are components of the code. Specifically, compared to other components of software source code, their natural language readability is stronger, so natural language models are used to extract their features.

[0066] Finally, for the abstract syntax tree sequence of software source code, it is first processed through an abstract syntax tree node embedding matrix E. AST The node representation vectors are obtained, and then passed through a 4-layer Transformer encoder to obtain the feature representation of the syntax tree sequence. The syntax tree sequence is denoted as... Embedsion vectors can be obtained from the word embedding matrix. As shown in formula (9):

[0067]

[0068] Structural feature vectors representing software source code It can be obtained from formula (10):

[0069]

[0070] Finally, by adding and fusing the three together using formula (11), we can represent the real-valued eigenvector of the software design pattern:

[0071]

[0072] From the above process, we can obtain the real-valued feature vector V of the software design requirements. N And the real eigenvector V of software design patterns P Since both vectors have the same dimension, feature interaction and fusion are performed through vector multiplication to obtain a vector V used to predict the design pattern label. pred As shown in formula (12):

[0073] V pred =matmul(V N V P (12)

[0074] Finally, a fully connected layer paired with a softmax activation function is used to classify the input to obtain y. pred As shown in formula (13), its classification space dimension is consistent with the design pattern label dimension obtained in the above process:

[0075] y pred =Softmax(MLP(V pred (13)

[0076] The Softmax function is shown in formula (14):

[0077]

[0078] For any real vector of length K, Softmax can compress it into a real vector of length K, with values ​​in the interval (0,1), and the sum of all elements in the vector is 1. It also has many applications in multivariate classification and neural networks. Softmax differs from the ordinary max function: the max function only outputs the largest value, while Softmax ensures that smaller values ​​also have a small probability and are not directly discarded.

[0079] In the actual training process, for a uniform batch containing N samples, each sample has M labels, and the loss is calculated as shown in the following formula (15):

[0080]

[0081] Where ln is the loss corresponding to the nth sample, and the calculation of ln is shown in formula (16):

[0082]

[0083] The calculation process is shown in formula (17):

[0084]

[0085] Among them, w i It is a hyperparameter used to handle the problem of sample imbalance between labels: For a batch of training sets, if a certain label appears less frequently, it should be given a higher weight when calculating the loss. The final form of L is shown in formula (18):

[0086]

[0087] Finally, during training, this invention employs an early stopping strategy to ensure model convergence while avoiding overfitting. The Adam optimizer is used, with the learning rate parameter set to 5e. -4 .

[0088] S3: Finally, based on the corpus and the bimodal joint modeling of the software design pattern recommendation model, implement a user-oriented software design pattern recommendation function. The specific flowchart is as follows: Figure 6 As shown: First, based on the model obtained in step S2, software design requirement features are extracted from the user-inputted user requirement text. Then, the user requirement feature vector is obtained using the software design requirement understanding module as the query vector. For the software source code data in the corpus, multi-source features are extracted from the software source code. The real-valued feature vectors of software design patterns are obtained through the software design pattern understanding module. Then, the Facebook vector retrieval library is used for storage and management to form a set of real-valued feature vectors of software design patterns for fast vector retrieval. Then, this invention sets a retrieval result threshold K=100 and uses the requirement feature vector as the query vector for relevance retrieval. Finally, the software source code results are beautified using the syntax tree analysis tool TreeSitter and the Beautify library to provide users with a better browsing experience.

[0089] S4: Select accuracy, precision, recall, F1 score, and Hamming score as evaluation metrics to assess the model. When calculating these metrics, the standard labels in the data are typically denoted as the true labels, and the predicted labels provided by the model are denoted as the predicted labels. The following convention applies between the prediction results and the standard labels:

[0090] True positive (TP): The data label is positive and the prediction result is also positive;

[0091] True negative (TN): The data label is negative, and the prediction result is also negative;

[0092] False positive (FP): The data label is negative, but the prediction result is positive;

[0093] False negative (FN): The data label is positive, and the prediction result is negative.

[0094] Accuracy, as the most common evaluation metric, represents the proportion of correctly predicted samples, expressed as the number of correctly predicted samples divided by the total number of samples. Generally speaking, the higher the accuracy, the better the classifier; however, in cases of imbalanced positive and negative samples, relying solely on accuracy to evaluate an algorithm model is not scientifically comprehensive enough. The process of calculating accuracy using the binary confusion matrix is ​​shown in formula (19), and the process of calculating accuracy using the multi-label classification model applied in this invention is shown in formula (20):

[0095]

[0096]

[0097] Where |D| represents the total number of samples, H represents the multi-label classifier, and Z... i =H(x) i ) indicates that H is based on x i The prediction result set.

[0098] Precision represents the proportion of samples that are actually classified as positive among those that are classified as positive. The process of calculating precision using the binary confusion matrix is ​​shown in formula (21), and the process of calculating precision using the multi-label classification model applied in this invention is shown in formula (22).

[0099]

[0100]

[0101] Precision and recall metrics sometimes contradict each other. Therefore, the F-score is usually used to calculate the weighted harmonic mean of precision and recall to account for the two metrics. The specific calculation is shown in formula (23). When the parameter α = 1, it is the most common F1 value, which is calculated as shown in formula (24). The calculation of the multi-label classification model used in this invention is expressed as formula (25):

[0102]

[0103]

[0104]

[0105] Hamming score, compared to accuracy, is a better indicator for evaluating multi-label classification models. Hamming loss is a specific evaluation metric for multi-label classification, representing the proportion of incorrect samples among all predictions. A smaller value indicates stronger classification ability. The specific calculation is shown in formula (26).

[0106]

[0107] Where |D| represents the total number of samples, |L| represents the total number of labels, and x i and y i These represent the predicted result and the true label, respectively, with xor representing the exclusive OR operation.

[0108] This invention sets the Hamming score based on the Hamming loss. To unify the evaluation process, the Hamming score is set to be positively correlated with the model performance. After binarizing the prediction result matrix, the specific calculation process is shown in formula (27):

[0109]

[0110] S5: Comparative Experiment Section. This embodiment selects common machine learning classification models and neural network classification models as comparison objects. This embodiment selects logistic regression, gradient boosting decision tree (GBDT / LightGBM algorithm used in this paper), and support vector machine (SVM) models as target machine learning models; and multilayer perceptron (MLP), neural bag-of-words (NBOW) model, and convolutional neural network (CNN) model as target neural network models.

[0111] First, the main model based on this invention is compared longitudinally with three traditional machine learning baseline models: logistic regression, LightGBM, and support vector machine. The resulting metrics are shown in Table 3. The model constructed in this invention significantly outperforms the three baseline models across all metrics. It improves accuracy and reduces classification error by approximately 82% compared to the best-performing SVM model. In the evaluation of multi-label classification tasks, it also improves the Hamming score by approximately 74% compared to the best-performing LightGBM model. This fully demonstrates the effectiveness and advancement of deep semantic understanding models in constructing text semantics compared to traditional machine learning models.

[0112] Table 3 Comparison of the main model with the machine learning baseline model

[0113]

[0114] This invention uses fixed statistical methods, such as TF-IDF, for feature selection and text encoding in machine learning models. Compared to deep learning models learning features from data themselves, this is not only time-consuming and labor-intensive, but also greatly limits the ideal performance of machine learning models, thus resulting in lower machine learning model metrics.

[0115] Secondly, the main model of this invention is compared horizontally with three deep learning baseline models based on NBOW, CNN and MLP, and all models are guaranteed to have the same dataset, model framework and linear classifier except for the encoder. The comparison of the indicators is shown in Table 4.

[0116] Table 4 Comparison of the main model with the deep learning baseline model

[0117]

[0118] As can be seen from the performance metrics on deep learning baseline models, the main model constructed in this invention, based on the model presented in this paper, still exhibits significant advantages to varying degrees. Regarding the slight imbalance of positive and negative samples in the dataset, the model of this invention achieves an accuracy of 95.46% in representing the proportion of samples classified as positive that are actually positive. In the evaluation of multi-label classification tasks, the model of this paper achieves a Hamming score of 93.43%.

[0119] Compared to the 42.21% classification accuracy of the NBOW-based deep learning model, the model of this invention achieved an accuracy of 91.55%, representing an improvement of 49.34% and a reduction of approximately 85% in classification error. Analyzing the essence of the NBOW model, this text representation method is based on the traditional bag-of-words model, which counts words as independent and does not consider their order or relationships when representing sentences, thus losing a significant amount of semantic information. Therefore, compared to the deep bidirectional language representation of this invention that integrates left and right context information, there is a significant difference in performance on classification tasks.

[0120] Compared to the 60.87% classification accuracy of the MLP-based deep learning model, the model of this invention improves the accuracy by 30.68% and reduces the classification error by approximately 78%. Because the MLP network consists of many hidden layers and uses fully connected layers, it involves a large number of parameters, which not only makes training difficult but also leads to overfitting, which is the main reason preventing MLP from achieving optimal performance.

[0121] Compared to the 74.39% classification accuracy of the CNN-based deep learning model, the model in this invention improves the accuracy by 17.16% and reduces the classification error by approximately 67%. The CNN model, benefiting from inter-layer local perception and parameter sharing mechanisms, demonstrates strong processing capabilities for the large datasets and high-dimensional data used in this invention, achieving the highest performance among baseline models. However, its advantage in extracting local features from short texts in the NLP field is not fully utilized in the context of long texts in this invention, which to some extent limits its performance.

[0122] Overall, the classification performance of the model in this invention, compared with the three deep learning baseline models, is roughly ranked as follows: model in this invention > CNN > MLP > NBOW. This also indirectly confirms the continuous development and improvement of neural network models. As a hybrid method based on attention mechanism modeling, the model in this invention obtains the most comprehensive local and global feature representations to date, and is well-suited to the task scenario of this study.

[0123] Finally, to verify the necessity of concatenating natural language and programming language for achieving better deep semantic understanding, we tested the impact of two training methods on the recommendation results during the model training phase: inputting only user request text and inputting only software source code text.

[0124] Table 5 shows the main model using only natural language or programming language.

[0125]

[0126] As shown in Table 5, the present invention model using only natural language as a feature achieves an accuracy of 84.73%, while the present invention model using only programming language as a feature achieves an accuracy of 77.45%. This represents a reduction of 6.82% and 14.1% respectively compared to the original present invention model. This demonstrates that the bimodal coding based on the present invention model is more advantageous than modeling using only one language, and that natural language is more important for semantic understanding.

[0127] In summary, the software design pattern recommendation method proposed in this invention has high advancement.

[0128] This document uses specific examples to illustrate the principles and implementation schemes of the present invention. The descriptions of the above embodiments are only for the purpose of helping to understand the method and core ideas of the present invention. Furthermore, those skilled in the art will recognize that, based on the ideas of the present invention, there will be changes in the specific implementation methods and application scope. Therefore, the content of this specification should not be construed as a limitation of the present invention.< / sep> < / cls> < / unk>

Claims

1. A software design pattern recommendation method based on dual-modal joint modeling, characterized in that, The steps include the following: Step 1: Construct a corpus for recommending software design patterns. The corpus includes software source code for different design patterns, software design requirement text, and design pattern tags shared by the software source code and software design requirement text. Step 2: Apply the bimodal joint modeling software design pattern recommendation model. Based on the corpus constructed in Step 1, implement user-demand-oriented software design pattern recommendations, providing users with a list of recommended candidates that meet their needs. The specific steps are as follows: Step 2.1: Extract software design requirement features from the user requirement text, and then input it into the software design requirement understanding module to convert it into a user requirement feature vector, thereby realizing user requirement understanding; the software design requirement understanding module includes a first natural language pre-training submodule, a project development programming language feature embedding submodule, and a first feature fusion submodule; Based on the feature extraction of software design requirements text, the feature data of software design requirements text is converted into a representation feature vector of software design requirements text through the first natural language pre-training submodule; the feature data of programming language type is converted into a feature vector from the perspective of programming language through the project development programming language feature embedding submodule. The two feature vectors are simultaneously input into the first feature fusion submodule and fused to form a real number feature vector of software design requirements. Step 2.2 involves extracting multi-source features from the software source code in the corpus and inputting them into the software design pattern understanding module, ultimately transforming them into real-valued feature vectors of software design patterns. The software design pattern understanding module includes a software source code pre-training submodule, a second natural language pre-training submodule, a software source code structural feature extraction submodule, and a second feature fusion submodule. First, based on the multi-source software source code feature data, the software source code is input into the software source code pre-training submodule to obtain the representation feature vector of the software source code. Second, the method names and API names of the software source code are concatenated in their original order in the software source code and input into the second natural language pre-training submodule to obtain the deep semantic feature vector of the software source code. Then, the abstract syntax tree sampling path is input into the software source code structural feature extraction submodule to construct the structural feature vector of the software source code. Finally, the three feature vectors are simultaneously input into the second feature fusion module to fuse them into a real-valued feature vector of the software design pattern. Step 2.3: Input the user demand feature vector output in Step 2.1 and the real number feature vector of software design pattern output in Step 2.2 into the dual-modal joint modeling software design pattern recommendation model for vector interaction to achieve software design pattern matching, obtain the relevance score between the two, and based on this, sort the software source code in the corpus in descending order according to the relevance score to obtain the candidate list of design pattern recommendation corpus, and output the design pattern recommendation result to the user; The training process for the software design pattern recommendation model for dual-modal joint modeling is as follows: The model, based on BERT's Siamese network, firstly inputs the real feature vectors of software design requirements from the natural language modality and the real feature vectors of software design patterns from the programming language modality, output by the understanding module, into the feature fusion neural network of the feature interaction fusion module for interaction fusion, obtaining a bimodal design pattern representation vector, and constructing a deep model for software design pattern recommendation. Secondly, the design pattern labels in the corpus are used as training targets, and a design pattern prediction classification head for multi-label classification is designed. Then, the text classification loss function is used as the loss function for model training. Triplets are formed between the design pattern labels predicted by the design pattern prediction classification head, the correct labels, and the negative sample labels. The text classification loss function is used to learn the sample differences between positive and negative examples. The gradient backpropagation algorithm is used during model training. Finally, common optimization methods for neural networks are used to find the convergence state of the bimodal joint modeling software design pattern recommendation model.

2. The software design pattern recommendation method based on bimodal joint modeling according to claim 1, characterized in that, The process of constructing the corpus for software design pattern recommendation in step 1 includes the collection, cleaning, storage, annotation, and merging / alignment of the software source code and software design requirement text. The software source code and software design requirements text share the same data collection, cleaning and storage process: First, the software source code and software design requirements text are collected through web crawlers; second, the useless data is removed through the data cleaning pipeline, and only the required data is retained; finally, the data is stored in the server for later use. The data annotation specifically includes: annotating the design patterns used in the software source code, with only three design patterns retained as annotation labels when annotating the software source code; and annotating the design patterns involved in the software design requirement text, selecting appropriate design patterns as labels based on the specific design problems, design functions, and related indicators described in the software design requirement text, with a maximum of three design pattern labels for each software design requirement text. The data merging and alignment specifically includes: when the software source code and the software design requirement text share at least one design pattern label, the two are paired as a data pair, and each shared design pattern label and data pair is a corpus; the basic data form of the corpus is the code text at the method level or file level corresponding to the software source code, the software design requirement text is the pure natural language text description content, and the design pattern label corresponds to the standard name of the labeled design pattern.

3. The software design pattern recommendation method based on bimodal joint modeling according to claim 1, characterized in that, The construction of the dual-modal joint modeling software design pattern recommendation model in step 2 specifically includes: firstly, extracting software design requirement text features from the software design requirement text in the corpus of step 1, and then converting them into real-valued feature vectors of software design requirements using the software design requirement understanding module; secondly, extracting multi-source features from the software source code in the corpus of step 1, and then converting them into real-valued feature vectors of software design patterns using the software design pattern understanding module; and finally, inputting the two real-valued feature vectors into the dual-modal joint modeling software design pattern recommendation model for model training, thereby completing the construction of the recommendation model.

4. The software design pattern recommendation method based on bimodal joint modeling according to claim 3, characterized in that, The software design requirement text feature extraction process is as follows: First, based on the established corpus, the software design requirement text described in natural language is further cleaned, and the non-natural language parts contained therein are normalized to obtain software design requirement text that meets the natural language expression standard. Then, the programming language features mentioned in the software design requirement text related to the current software design are extracted. These features, together with the cleaned software design requirement text, constitute the software design requirement text feature data.

5. The software design pattern recommendation method based on bimodal joint modeling according to claim 3, characterized in that, The specific process for multi-source feature extraction of software source code is as follows: First, the collected raw data is divided and files unrelated to the software source code are removed. Second, the code snippets of software source code or files in different programming languages ​​are compiled and verified using a compiler to ensure code correctness and convert them into corresponding abstract syntax trees. Then, three key features are extracted from the software source code, including the software source code method name, the software source code API name, and the software source code abstract syntax tree. The method name and API name in the software source code are stored as string sequences. The abstract syntax tree data corresponding to the software source code is combined with a random walk algorithm to sample each path of the syntax tree, and the corresponding string sequences are used as sampling paths for the sampling syntax tree. Finally, the three key features and the software source code constitute multi-source software source code feature data.

6. The software design pattern recommendation method based on bimodal joint modeling according to claim 1, characterized in that, The candidate list of design pattern recommendation corpus in step 2.3 includes: software source code, design pattern tags, a list of method names contained in the code, a list of APIs, and a relevance score as the sorting criterion.

7. The software design pattern recommendation method based on bimodal joint modeling according to claim 1, characterized in that, In step 2.3, the design pattern recommendation results are output to the user. Based on the candidate list sorted in descending order of relevance score, the software source code in the candidate list is formatted according to different programming languages. The specific formatting methods for the software source code include: automatic indentation, automatic code layout, and code highlighting.

Images