Web API recommendation method based on deep learning enhancement

Abstract

The invention belongs to the technical field of recommendation, and particularly relates to a Web API recommendation method based on deep learning enhancement, in particular to a Web API recommendation enhancement multi-task fusion model (EMTFM-BERT) based on BERT and oriented to mixed applications. According to the method, multi-model fusion and multi-task learning are combined with a BERT pre-training model and a fine tuning technology thereof. The proposed solution is divided into three key stages: 1, constructing a recommendation model; secondly, training the model by using the usage data of the Web API and the hybrid application collected from the service provider; and thirdly, processing natural language input submitted by a developer or a user. Based on these inputs, the system generates a set of recommended Web APIs and proposes relevant tags or categories that meet specified requirements.

Description

Technical Field

[0001] The present invention belongs to the field of recommendation technology, and in particular relates to a Web API recommendation method based on deep learning enhancement, which is a method for recommending a suitable Web API to a hybrid service based on hybrid requirements. Background Art

[0002] In recent years, web-based application programming interfaces (APIs) have become indispensable in software development. However, the proliferation of Web APIs has made it challenging for developers to select the most appropriate APIs from the large number of available APIs when creating new hybrid applications. Through analysis of the 2021 ProgrammableWeb database, we found that the number of APIs actually used by developers in hybrid applications is much lower than expected. Despite extensive efforts, the challenge of effectively applying APIs in hybrid applications remains. Therefore, various research activities have emerged in the field of service computing to recommend an appropriate set of Web APIs. There are various methods and research trends in this field, including collaborative filtering (CF), functional attribute-based, content-based, and deep learning-enhanced recommendation systems. For example, function-based APIs achieve recommendations by matching functional similarities between Web services and user queries, but they are often limited when building hybrid applications due to the short service descriptions. To overcome this problem, there have been some attempts to enrich these descriptions through multiple data sources. However, these methods often have difficulties in capturing key features or the most relevant functional terms, making their applicability in all scenarios uncertain. Recently, content-based recommendation methods combined with deep learning techniques have attracted great attention among researchers. These techniques are able to autonomously learn the features of items and thus improve the recommendation success rate. However, they often have difficulty identifying dependencies between elements and assigning appropriate weights to these items.

[0003] Developers encounter significant challenges when using Web APIs, mainly due to hidden dependencies between applications, the diversity of APIs across industries, and the lack of a unified management platform. This diversity complicates the effective tracking and use of APIs and even leads to incompatibility issues between APIs with similar functions. In addition, enterprises have full control over API changes, forcing users to constantly track and adapt to these changes, thereby increasing maintenance costs. In the absence of comprehensive operational data, it also becomes difficult to evaluate the popularity and effectiveness of specific Web APIs. Techniques such as collaborative filtering (CF) and content-based methods have been applied to improve API recommendations. CF-based methods use historical mixed data to recommend APIs, while content-based methods focus on mining the relationship between requirements and APIs. Despite some success, there are still challenges in feature fusion. CF usually has difficulty adapting to mixed requirements well, while content-based methods may not capture detailed requirements. To improve accuracy, natural language processing (NLP) can help optimize keyword selection and align freely described requirements with standard tags. In addition, API compatibility evaluation is often difficult due to the lack of semantic annotations, and further research is needed to better evaluate compatibility using historical mixed data.

[0004] Innovative approaches are necessary to overcome the limitations of traditional recommendation techniques such as collaborative filtering and content-based methods, which mainly focus on historical mixed data and text descriptions but often fail to capture the subtle differences in API functionality and compatibility. Summary of the invention

[0005] In order to make full use of the characteristics of Web APIs and hybrid applications to solve the complexity of using and managing diverse and incompatible Web APIs, as well as the limitations of existing recommendation methods (such as deficiencies in feature fusion and accurate evaluation of API compatibility of hybrid applications, mainly due to insufficient semantic annotations and hidden dependencies), this paper proposes a Web API recommendation method based on deep learning enhancement. This method integrates the latest technologies including multi-task learning and BERT models to better capture the complex relationship between APIs and hybrid applications, thereby achieving more effective decision-making and improving developer experience.

[0006] The technical solution of the present invention:

[0007] A Web API recommendation method based on deep learning enhancement is an Enhanced Multi-Task Fusion Model for Web API Recommendation for Hybrid Applications based on BERT (EMTFM-BERT). The method combines multi-model fusion and multi-task learning with the BERT pre-trained model and its fine-tuning technology. The proposed solution is divided into three key stages: first, building a recommendation model; second, training the model using usage data of Web APIs and hybrid applications collected from service providers; third, processing natural language inputs submitted by developers or users. Based on these inputs, the system generates a set of recommended Web APIs and proposes relevant tags or categories that meet the specified requirements.

[0008] The steps include:

[0009] Step 1: Prepare data by cleaning and standardizing API and hybrid app descriptions.

[0010] Step 2: Extract deep semantic features from API descriptions and hybrid application data.

[0011] Step 3: Reduce the continuous features extracted from the mixed application category to quantitative features.

[0012] Step 4: Use deep neural networks (DNNs) to learn complex relationships between semantic features and identify how different APIs and hybrid applications interact.

[0013] Step 5: Adjust BERT parameters for API recommendation so that the model better matches API descriptions and hybrid application requirements.

[0014] Step 6: Merge the outputs of BERT and other components into a unified model to form the EMTFM-BERT model, and apply multi-task learning to simultaneously predict related APIs and categories.

[0015] Step 7: The EMTFM-BERT model is used to learn to predict the most appropriate API and identify the appropriate category, ensuring that practical and targeted recommendations are provided to developers.

[0016] Beneficial effects of the present invention:

[0017] 1. Leveraging BERT’s advanced language modeling capabilities: The model captures complex semantic relationships in API descriptions and hybrid application requirements. BERT’s bidirectional context processing enables the model to understand the full meaning of words in a specific context, thereby improving its accuracy in matching APIs with hybrid application requirements. This leads to more effective API recommendations that better meet the functional and semantic needs of hybrid application developers.

[0018] 2. Adopting a multi-task learning strategy: The model simultaneously optimizes related tasks such as API recommendation and hybrid application category classification. This joint learning method enables the model to share valuable insights between tasks, thereby reducing training time and improving the performance of each task. In this way, the EMTFM-BERT model becomes more versatile and can handle different recommendation scenarios without the need to build a specific model for each task, thus simplifying the recommendation process.

[0019] 3. Combining matrix decomposition and deep neural network (DNN) to capture linear and nonlinear interactions of API features: This dual approach helps the model understand how different aspects of APIs and hybrid applications interact, so that it not only considers basic attributes, but also identifies complex feature relationships for recommendation. In addition, by integrating metadata such as tags, categories, and descriptions, the model is able to make personalized recommendations for specific user needs, which is particularly valuable for developers looking for APIs with specific functions or fields.

[0020] 4. Designed for large-scale environments: The EMTFM-BERT model uses quantization technology to reduce memory usage and increase processing speed, making it suitable for processing huge API databases without affecting performance. This computational efficiency ensures that the model can provide real-time recommendations even under resource constraints. In addition, accurate similarity measurements between APIs and hybrid applications achieved with advanced feature analysis ensure that the most relevant and applicable APIs are ranked higher in the recommendation list, thereby increasing the integration success rate and improving developer satisfaction. The combination of these features makes the model perform well and adaptably in a variety of use cases from small-scale projects to large dynamic API ecosystems. BRIEF DESCRIPTION OF THE DRAWINGS

[0021] Figure 1 It is an architecture diagram of the EMTFM-BERT of the present invention. DETAILED DESCRIPTION

[0022] The specific implementation of the present invention is further described below in conjunction with the accompanying drawings and technical solutions.

[0023] The Web API recommendation method based on deep learning enhancement of the present invention adopts the EMTFM-BERT model, such as Figure 1 As shown, the steps are as follows:

[0024] The specific process of step 1 is as follows:

[0025] Hybrid applications and Web API descriptions often contain unnecessary information and are not suitable for model training directly. To this end, various preprocessing steps are applied, including abbreviation expansion for consistency, text filtering to remove irrelevant elements (such as punctuation and stop words), and lemmatization to standardize words to their basic form for semantic analysis. Word vectorization creates a vocabulary by assigning unique indexes to words, using pre-trained Glove vectors for known words and randomly initializing new words. In addition, one-hot encoding is used to convert categorical data into binary vectors, with relevant categories marked as 1 and irrelevant labels as 0.

[0026] The specific process of step 2 is as follows:

[0027] The semantic component (SC) of the EMTFM-BERT model focuses on accurately representing and modeling text descriptions of Web APIs and hybrid applications to improve recommendation quality. Traditional methods such as the Bag of Words (BoW) model cannot capture word order and context, resulting in insufficient semantic representation. EMTFM-BERT addresses these limitations by leveraging advanced deep learning techniques, especially convolutional neural networks (CNNs), to extract meaningful features from word sequences. For example, suppose E m =lookup_table(m) represents the word embedding matrix of the hybrid application or requirement m, where the lookup table retrieves its embedding for each word. This matrix represents the semantics of the text captured based on a pre-trained model (such as GloVe or Word2Vec). In the convolutional layer, a kernel At size w j Slide on the word window and perform the convolution operation:

[0028] U MM =U SC .V T (1)

[0029] g i =ReLU(G i E m +b i ) (2)

[0030] Among them, U MM is the interaction matrix, V T is the transpose of the API embedding matrix V, b i represents the bias term, and ReLU is an activation function used to add nonlinearity, enabling the model to capture complex semantic features. By extracting local features from the word window, U SC The CNN component can effectively model text content, thereby gaining a deeper understanding of API functionality and hybrid application requirements.

[0031] The specific process of step 3 is as follows:

[0032] Quantization is used in the EMTFM-BERT model to optimize computational efficiency without significantly sacrificing model accuracy. By reducing the precision of the weights and activation values ​​of the neural network from formats such as 32-bit floating points to lower precision formats such as 8-bit integers, quantization enables the model to compute faster and take up less memory. This approach is particularly useful for large datasets or scenarios with limited computing resources because it speeds up data analysis while maintaining sufficient accuracy for accurate Web API recommendations. Its main goal is to balance computational speed with model performance, making this technology valuable in real-time applications or when scaling models to handle larger datasets.

[0033] The specific process of step 4 is as follows:

[0034] The Feature Interaction Component (FIC) in the EMTFM-BERT model is designed to capture linear and nonlinear interactions between hybrid application and Web API features. It combines matrix factorization (MF) with multi-layer perceptron (MLP) to more fully understand the relationship between different features.

[0035] This process captures common patterns in historical interaction data and helps identify relationships between hybrid apps and APIs based on past usage. To better capture complex relationships that may be missed by matrix factorization alone, the model also uses a concatenation approach to combine the embeddings of hybrid apps and APIs into a single vector U concat The concatenated vector is then passed to a multi-layer perceptron (MLP) to learn complex high-order interactions through multiple layers and activation functions. The MLP process is expressed as:

[0036] U MLP =δ(W z (…δ(W1U Concat +b1)…+b z ) (3)

[0037] Finally, the feature interaction component (FIC) merges the outputs of matrix factorization and multi-layer perceptron (MLP) into a unified representation.

[0038] U FIC =[U MM ; U MLP ] (4)

[0039] Among them, U FIC is the final feature interaction component, which combines the outputs of different interaction models. MLPis the output of the multilayer perceptron, capturing nonlinear interactions, and (;) represents the concatenation of two vectors. This combination enables the model to effectively capture simple linear relationships and more complex nonlinear interactions, thereby enhancing the overall understanding of the dependencies between hybrid applications and Web APIs.

[0040] The specific process of step 5 is as follows:

[0041] This step consists of two sub-steps: pre-training and fine-tuning. During pre-training, BERT is trained on large-scale unlabeled text datasets such as BooksCorpus and Wikipedia to help it learn general language representations. This stage focuses on understanding context through a bidirectional self-attention mechanism, allowing the model to grasp the complex relationships between words in the text. After pre-training, BERT is fine-tuned for specific tasks such as API recommendation by adjusting its parameters using labeled data from the target domain. Due to BERT's adaptable architecture, changes between the pre-training and downstream task stages are minimized, and the fine-tuning process is relatively simple. For example, the same BERT architecture for tasks such as question answering can be used by simply modifying the input and output layers. This adaptability ensures that BERT can focus on the details of API descriptions and hybrid application requirements. The fine-tuning stage enables BERT to adjust its understanding capabilities based on the characteristics of the API recommendation task, improving its ability to match API descriptions with user needs. The final model not only leverages its deep pre-trained language understanding, but also adapts to the specific context of API recommendation for hybrid applications.

[0042] The specific process of step 6 is as follows:

[0043] By combining features from different components and training the model to complete multiple related tasks. The model uses an early fusion strategy to integrate various feature representations, including features from semantic components and feature interaction components. The combined feature representation U MMF By splicing the semantic components U SC and feature interaction component U FIC The output is formed as:

[0044] U MMF =[U SC ; U FIC ] (5)

[0045] This fused representation is then used to generate the output of the downstream task. The output layer of each task uses the weight matrix W TL and the bias vector b TL To generate predictions:

[0046]

[0047] To prevent overfitting, the dropout technique is applied during training to randomly deactivate some neurons to help the model generalize better. In this context, multi-task learning (MTL) enables the model to exploit shared information between related tasks to improve training efficiency and prediction accuracy. Specifically, the main task (MT) is designed as a multi-label classification problem to predict the relevant APIs for a given hybrid application. Each API a in set A is treated as a separate label, and the binary cross entropy loss (BCELoss) is used to train the model:

[0048]

[0049] Among them, y m [α] represents the target value of API a, is the predicted value. In addition, an auxiliary task (AT) is added to classify the category of the hybrid application, which is constructed as a binary classification problem for each category c∈C. The loss function of this task also uses the binary cross entropy loss (BCELoss), expressed as:

[0050]

[0051] The overall objective function combines the loss functions of the two tasks and applies L2 regularization to reduce overfitting:

[0052]

[0053] in, Contains model parameters, λ controls the regularization strength. This task approach enables the EMTFM-BERT model to optimize both API recommendation and category classification, thereby improving the overall performance and accuracy of the model in generating context-relevant API recommendations.

[0054] The specific process of step 7 is as follows:

[0055] This step enhances the embedding of the Web API by leveraging additional metadata such as tags, categories, and descriptions. The text description of the Web API is first parsed and passed to a semantic component (usually a convolutional neural network CNN) to extract deep features, denoted as V2 α =CNN(θ2,α), where θ2 represents the parameters of the CNN model. In addition to text data, a specific embedding layer handles the labels or categories associated with the Web API, formalized as V2 α =lookup_table(T α ), where T α A label or category that represents a specific API.

[0056] These two feature vectors are concatenated to generate a comprehensive representation of the API V = [V1 α V2 α ]. This approach integrates semantic information and categorical data from descriptions, enabling the model to better understand the functionality and context of each Web API. By considering these diverse aspects, EMTFM-BERT effectively improves the quality and relevance of recommendations, tailoring selections to meet the specific needs of hybrid applications.

Claims

1. A Web API recommendation method based on deep learning enhancement, characterized in that: The steps include: Step 1: Prepare data by cleaning and standardizing API and hybrid application descriptions; Step 2: Extract deep semantic features from API descriptions and hybrid application data; Step 3: Simplify the continuous features extracted from the mixed application category into quantitative features; Step 4: Use deep neural networks (DNNs) to learn complex relationships between semantic features and identify how different APIs and hybrid applications interact. Step 5: Adjust BERT parameters for API recommendation so that the model better matches API descriptions and hybrid application requirements; Step 6: Merge the outputs of BERT and other components into a unified model to form the EMTFM-BERT model, and apply multi-task learning to simultaneously predict related APIs and categories; Step 7: Use EMTFM-BERT model learning to predict the most appropriate API and identify appropriate categories to ensure practical and targeted recommendations for developers; The specific process of step 1 is as follows: Various preprocessing steps are applied, including abbreviation expansion for consistency, text filtering to remove irrelevant elements, and lemmatization to normalize words to their basic form for semantic analysis. Word vectorization creates a vocabulary by assigning unique indices to words, using pre-trained Glove vectors for known words and random initialization for new words. In addition, one-hot encoding is used to convert categorical data into binary vectors, with relevant categories labeled as 1 and irrelevant labels as 0. The specific process of step 2 is as follows: The semantic component SC of the EMTFM-BERT model focuses on accurately representing and modeling text descriptions of Web APIs and hybrid applications; EMTFM-BERT addresses these limitations by leveraging advanced deep learning techniques to extract meaningful features from word sequences; m =lookup_table(m) represents the word embedding matrix of the hybrid application or requirement m, where the lookup table retrieves its embedding for each word; the matrix represents the semantics of the text captured based on the pre-trained model (such as GloVe or Word2Vec); in the convolutional layer, a kernel At size w j Slide on the word window and perform the convolution operation: U MM =U SC .V T (1) g i JReLU(G i E m +b i ) (2) Among them, U MM is the interaction matrix, V T is the transpose of the API embedding matrix V, b i ReLU is an activation function used to add nonlinearity, which enables the model to capture complex semantic features. By extracting local features from the word window, U SC The CNN component models the text content; The specific process of step 3 is as follows: Use quantization in the EMTFM-BERT model to optimize computational efficiency by reducing the precision of the neural network weights and activation values ​​from 32-bit floating point format to 8-bit integer format; The specific process of step 4 is as follows: The feature interaction component FIC in the EMTFM-BERT model is designed to capture the linear and nonlinear interactions between hybrid application and Web API features; it combines matrix factorization MF with multi-layer perceptron MLP to more comprehensively understand the relationship between different features; To better capture complex relationships that may be missed by matrix factorization alone, a concatenation approach is used to combine the embeddings of hybrid applications and APIs into a single vector U concat ; The concatenated vector is then passed to the multi-layer perceptron MLP, which learns complex high-order interactions through multiple layers and activation functions; the MLP process is expressed as: U MLP =δ(W z (…δ(W1U Concat +b1)…+b z ) (3) Finally, the feature interaction component FIC merges the outputs of the matrix factorization and the multi-layer perceptron MLP into a unified representation; IN FIC =[U MM ;IN MLP ] (4) Among them, U FIC is the final feature interaction component, which combines the outputs of different interaction models. MLP is the output of the multilayer perceptron, capturing nonlinear interactions, (;) represents the concatenation of two vectors; The specific process of step 5 is as follows: This step consists of two sub-steps: pre-training and fine-tuning. During pre-training, BERT is trained on a large-scale unlabeled text dataset to help it learn general language representations. This stage focuses on understanding context through a bidirectional self-attention mechanism, so that the model can grasp the complex relationships between words in the text. After pre-training, BERT is fine-tuned for specific tasks such as API recommendation by adjusting its parameters using labeled data from the target domain. The specific process of step 6 is as follows: By combining features from different components and training the model to complete multiple related tasks; the model uses an early fusion strategy to integrate various feature representations, including features from semantic components and feature interaction components; the combined feature representation U MMF By splicing the semantic components U SC and feature interaction component U FIC The output is formed by: IN MMF =[U SC ;IN FIC ] (5) This fused representation is then used to generate the output of the downstream task; the output layer of each task uses the weight matrix W TL and the bias vector b TL To generate predictions: To prevent overfitting, the dropout technique is applied during the training process to randomly deactivate some neurons to help the model generalize better; multi-task learning (MTL) enables the model to utilize shared information between related tasks to improve training efficiency and prediction accuracy; specifically, the main task MT is designed as a multi-label classification problem to predict the relevant APIs for a given hybrid application; each API a in set A is regarded as a separate label, and the binary cross entropy loss (BCELoss) is used to train the model: Among them, y m [α] represents the target value of API a, is the predicted value; in addition, an auxiliary task AT is added to classify the category of the hybrid application, which is constructed as a binary classification problem for each category c∈C; the loss function of this task also uses binary cross entropy loss, expressed as: The overall objective function combines the loss functions of the two tasks and applies L2 regularization to reduce overfitting: in, Contains model parameters, λ controls the regularization strength; The specific process of step 7 is as follows: This step enhances the embedding of the Web API by leveraging additional metadata; first, the text description of the Web API is parsed and passed to the semantic component to extract deep features, denoted as V2 α =CNN(θ2,α), where θ2 represents the parameters of the CNN model; in addition to text data, a specific embedding layer handles the labels or categories associated with the Web API, formalized as V2 α =lookup_table(T α ), where T α A label or category that represents a specific API; These two feature vectors are concatenated to generate a comprehensive representation of the API V = [V1 α V2 α ].

Images