Web API recommendation method based on comparative learning and joint training

Abstract

The invention discloses a WebAPI recommendation method based on comparative learning and joint training. The WebAPI recommendation method comprises the following steps: acquiring a first text description and a second text description; processing the first text description and the second text description through a BERT model to obtain a first text representation and a second text representation; obtaining an interactive network structure of Mashup and Web API, and processing the interactive network structure through a Light GCN model to obtain a first structural representation and a second structural representation; splicing the first text description, the second text description, the first structural representation and the second structural representation to generate overall embedding, and processing the overall embedding through an SANFM model to obtain a WebAPI recommendation result; wherein the BERT model, the LightGCN model and the SANFM model are obtained by performing optimization through comparative learning and joint training.

Description

Technical Field

[0001] The present invention belongs to the technical field of Web API recommendation, and in particular relates to a Web API recommendation method based on contrastive learning and joint training. Background Art

[0002] Web API (Web service) is a network application program interface. Through the API interface, network applications can implement storage services, message services, computing services and other capabilities. These capabilities can be used to develop powerful Mashup applications. Mashup combines multiple individual services into value-added composite services. In recent years, the popularity of Mashup has promoted the development of the service market. Developers can upload the developed Web API and Mashup to the Web API market platform for other developers to use. With the development of service-oriented computing technology, the number of Web APIs published to public platforms is increasing. Selecting a Web API that meets one or some specific requirements has become a major challenge. To alleviate this problem, Web API recommendation has become an effective way.

[0003] Existing Web API recommendation works can be roughly divided into three categories: content-based methods, graph-based methods, and hybrid methods. Content-based Web API recommendation methods mainly rely on service function description information and metadata, including service name, description, input and output parameters, category labels, etc. Content-based recommendation methods parse documents into structured information and convert services into feature vectors through feature extraction and representation learning techniques such as TF-IDF and Word2Vec. Then, the feature vectors of Mashup and candidate APIs are matched by calculating similarity, and APIs with high matching degree are recommended to users. Graph-based methods extract service function semantics from service networks and extract service function semantics from the relationships between services (such as composition relationships and label sharing relationships). The idea is that service function semantics are implicit in the service network structure. In this type of work, network representation methods are usually used, such as GNN, GCN, GAT, Node2vec, LightGCN, etc. Although users and services can be embedded by learning network representations from service networks, many works only focus on structural semantics and ignore text semantics. In order to make full use of both textual semantics and structural semantics, hybrid methods are used to extract textual semantics and structural semantics from functional description documents and service networks respectively to achieve service recommendation. This type of method has a significant improvement in service recommendation performance. Therefore, they are currently the most advanced methods in the field of Web API recommendation.

[0004] Although hybrid-based methods have made great progress in service recommendation, the existing work has been unable to fully extract rich semantics from service description documents and service networks, and associate and promote each other, resulting in limited recommendation performance. Therefore, how to simultaneously combine textual semantic representation and structural semantic representation to obtain appropriate embedding for effective service recommendation remains an unsolved problem. Summary of the invention

[0005] In order to solve the above technical problems, the present invention proposes a Web API recommendation method based on contrastive learning and joint training to solve the problems existing in the above prior art.

[0006] To achieve the above object, the present invention provides a Web API recommendation method based on contrastive learning and joint training, comprising:

[0007] Obtain a first text description and a second text description, wherein the first text description and the second text description are text descriptions of a mashup and a web API, respectively; process the first text description and the second text description through a BERT model to obtain a first text representation and a second text representation;

[0008] Obtain the interactive network structure of Mashup and Web API, process the interactive network structure through the LightGCN model, and obtain the first structural representation and the second structural representation;

[0009] The first text description, the second text description, the first structural representation and the second structural representation are concatenated to generate an overall embedding, and the overall embedding is processed by the SANFM model to obtain a Web API recommendation result;

[0010] Among them, the BERT model, LightGCN model and SANFM model are optimized through comparative learning and joint training.

[0011] Optionally, after obtaining the first text description and the second text description, the method further includes: preprocessing the first text description and the second text description by a natural language preprocessing method, wherein the preprocessing process includes: normalization, stem extraction, tokenization and stop word deletion.

[0012] Optionally, the interactive network structure is a bipartite graph, which includes a Mashup collection, a Web API collection, and an edge collection representing the interactive relationship between each Mashup and each Web API.

[0013] Optionally, after processing the first text description and the second text description through the BERT model, it also includes adjusting the dimension of the output result of the BERT model through a multi-layer perceptron to obtain the first text representation and the second text representation of the same dimension as the first structural representation and the second structural representation.

[0014] Optionally, the joint training process includes:

[0015] Construct the initial SANFM model, the initial BERT model and the initial LightGCN model and obtain the training set, perform comparative learning on the initial BERT model and the initial LightGCN model through the training set, and jointly train the initial BERT model, the initial LightGCN model and the initial SANFM model respectively. During the comparative learning process, calculate the comparative learning loss value, and calculate the joint training loss value based on the comparative learning loss value, optimize the initial BERT model, the initial LightGCN model and the initial SANFM model according to the joint training loss value, and use the optimized BERT model, LightGCN model and SANFM model to generate recommendation results.

[0016] Optionally, the calculation process of the joint training loss value includes:

[0017]

[0018] Among them, L represents the joint training loss value, γ is the weight parameter that controls the contrastive learning loss value, Θ is the trainable parameter of the entire model, λ is the L2 regularization weight to alleviate overfitting, and L BPR and L Rec They are the loss functions of the LightGCN model and the SANFM model, and Indicates the comparative learning loss values ​​corresponding to Mashup and Web API.

[0019] Optionally, the contrastive learning process includes:

[0020] The initial BERT model and the initial LightGCN model are compared through cross-view contrast learning, including:

[0021] Constructing an initial BERT model and an initial LightGCN model, wherein the training set includes a text training set and a structure training set, processing the text training set by the initial BERT model to obtain a predicted text representation of the Mashup and the Web API; processing the structure training set by the initial LightGCN model to obtain a predicted structure representation of the Mashup and the Web API;

[0022] Compute the contrastive learning loss based on the predicted text representations of Mashup and Web API and the predicted structural representations of Mashup and Web API.

[0023] Optionally, the calculation process of the contrastive learning loss value includes:

[0024]

[0025] in, Represents the contrastive learning loss value corresponding to Mashup, Indicates the contrastive learning loss value corresponding to the Web API, Indicates Mashup collection, M i represents the i-th Mashup, Represents a collection of Web APIs, A i Represents the i-th WebAPI, Represents the embedding vector corresponding to the i-th Mashup output by the BERT model, Represents the embedding vector corresponding to the i-th Mashup output by the LightGCN model, represents the embedding vector corresponding to the i-th Web API output by the BERT model, represents the embedding vector corresponding to the i-th Web API output by the LightGCN model, s(·,·) is used to calculate the similarity between two vectors, and τ is the temperature parameter.

[0026] Compared with the prior art, the present invention has the following advantages and technical effects:

[0027] The present invention proposes a Web API recommendation method based on contrastive learning and joint training, which considers both textual semantic information and structural semantic information. First, the BERT model and the LightGCN model are used to extract the textual semantics and structural semantics of Mashup and Web API respectively. Then, the text representation obtained by the BERT module is dimensionally reduced so that the textual semantic embedding dimension is the same as the structural semantic embedding from the LightGCN module. Through cross-view contrastive learning, the textual semantic representation is associated with the structural semantic representation, and the semantic information is supplemented with each other, so that the accuracy of the textual semantic representation and the structural semantic representation is improved at the same time. The downstream service recommendation task adopts the SANFM model, which is jointly trained with the previous textual semantic representation and structural semantic representation tasks, so that the service representation is more suitable for the service recommendation task, thereby improving the performance of service recommendation. To verify the effectiveness of the method, the present invention uses real data from the ProgrammableWeb platform to conduct experiments. The experimental results show that compared with the baseline method, the Web API recommendation method based on contrastive learning and joint training shows better performance in all evaluation indicators. BRIEF DESCRIPTION OF THE DRAWINGS

[0028] The drawings constituting a part of the present application are used to provide a further understanding of the present application. The illustrative embodiments and descriptions of the present application are used to explain the present application and do not constitute an improper limitation on the present application. In the drawings:

[0029] Figure 1 A Web API recommendation framework diagram based on contrastive learning and joint training according to an embodiment of the present invention;

[0030] Figure 2 An example diagram of the BERT model input representation according to an embodiment of the present invention;

[0031] Figure 3 This is a LightGCN framework diagram of an embodiment of the present invention;

[0032] Figure 4 A schematic diagram of a neural network architecture of SANFM according to an embodiment of the present invention;

[0033] Figure 5 is the hyperparameter d of the embodiment of the present invention l Impact curve diagram on the model;

[0034] Figure 6 is the hyperparameter d of the embodiment of the present invention s Impact curve diagram on the model;

[0035] Figure 7 This is a curve diagram showing the influence of the hyperparameter τ on the model according to an embodiment of the present invention;

[0036] Figure 8 Graph showing the influence of the hyperparameter γ on the model according to an embodiment of the present invention. DETAILED DESCRIPTION

[0037] It should be noted that, in the absence of conflict, the embodiments and features in the embodiments of the present application can be combined with each other. The present application will be described in detail below with reference to the accompanying drawings and in combination with the embodiments.

[0038] It should be noted that the steps shown in the flowcharts of the accompanying drawings can be executed in a computer system such as a set of computer executable instructions, and that, although a logical order is shown in the flowcharts, in some cases, the steps shown or described can be executed in an order different from that shown here.

[0039] In order to improve the recommendation performance of existing methods, this paper proposes a WebAPI recommendation method based on contrastive learning and joint training to solve the problem of how to combine text semantics and structural semantics to obtain appropriate embedding for effective service recommendation. First, the LightGCN and BERT models are used to obtain the node representation of the Mashup-API interaction graph, i.e., the structural representation, and the text representation of the service description document, respectively. Then, these two types of representations are contrastively learned, and then the two representations of Mashup and WebAPI are spliced ​​together as the input of the SANFM (Self-Attention Neural Factorization Machine) model to achieve the downstream service recommendation task.

[0040] The present invention provides a Web API recommendation method based on contrastive learning and joint training. First, the BERT model and the LightGCN model are used to extract text semantics, i.e., text representation, and structural semantics, i.e., structural representation, for Mashup and Web API, respectively. Then, the text representation output obtained by the BERT model is dimensionally reduced so that the text representation embedding dimension is the same as the structural semantic embedding from the LightGCN model. Through cross-view contrastive learning, the text representation is associated with the structural representation, and the semantic information is represented mutually, so that the accuracy of the text semantic representation and the structural semantic representation is improved at the same time. The downstream service recommendation task adopts the SANFM model, and is jointly trained with the previous text semantic representation and structural semantic representation tasks, so that the output service representation or recommendation result is more suitable for the service recommendation task, thereby improving the performance of the service recommendation. To verify the effectiveness of the method, the present invention uses real data from the ProgrammableWeb platform to conduct experiments. The experimental results show that compared with the baseline method, the Web API recommendation method based on contrastive learning and joint training shows better performance in all evaluation indicators.

[0041] The above technical solution is described in detail:

[0042] The Web API recommendation method framework based on contrastive learning and joint training proposed in this invention is as follows Figure 1 As shown, it is mainly divided into four steps:

[0043] First, based on the text descriptions of Mashup and Web API, the BERT (Bidirectional Encoder Representations from Transformers) model is used to obtain the text representation of Mashup, i.e., the first text representation, and the text representation of Web API, i.e., the second text representation.

[0044] Secondly, the Mashup-API interaction network is input into the LightGCN model to obtain the structural representation of the Mashup, i.e. the first structural representation, and the structural representation of the Web API, i.e. the second structural representation;

[0045] Then, cross-view contrastive learning is used to align the text representation from BERT and the structural representation from LightGCN, making the distance between positive samples closer and the distance between negative samples farther;

[0046] Finally, the SANFM (Self-Attention Neural Factorization Machine) model is used to deeply integrate the semantic representation information of Mashup and Web API for service recommendation. The service recommendation task is jointly trained with the previous text representation and structure representation tasks, so that the final service representation result, i.e., the service recommendation result, is more suitable for the service recommendation task.

[0047] The recommendation results of Web API are generated through the BERT model, LightGCN model and SANFM model optimized by contrastive learning and joint training.

[0048] The data preprocessing, model building and joint training of the present invention are described in detail below.

[0049] Data preprocessing:

[0050] Data preprocessing processes the crawled Mashup and Web API text information into a form that can be used by the model, and uses natural language preprocessing methods to preprocess the function description documents in the Mashup and Web API text description data set. The preprocessing includes: normalization, stem extraction, tokenization, and stop word removal.

[0051] The interaction record between Mashup and Web API is performed to generate a bipartite graph. The preprocessing process includes the following steps:

[0052] 1. Extract the Mashup name column, the called Web API column, the Mashup text description column, the Web API name column, and the Web API text description column of the ProgrammableWeb dataset;

[0053] 2. Delete the Mashup that uses the externally imported Web API;

[0054] 3. Delete the Web API that has not been called;

[0055] 4. Number them and build a Mashup-API bipartite graph based on the interaction records between Mashup and Web API.

[0056] Model building and joint training:

[0057] The overall process of the CLJT method consists of four main parts:

[0058] (1) Obtaining text representations of mashups and web APIs: Based on the text descriptions of mashups and web APIs, the BERT (Bidirectional Encoder Representations from Transformers) model is used to obtain the text representations of mashups and web APIs respectively.

[0059] (2) Input the Mashup-API interaction network into the LightGCN model to obtain the structural representation of the Mashup and the structural representation of the WebAPI;

[0060] (3) Cross-view contrastive learning is used to align the text representation from BERT and the structural representation from LightGCN, making the distance between positive samples closer and the distance between negative samples farther;

[0061] (4) The SANFM (Self-Attention Neural Factorization Machine) model is used to deeply integrate the semantic representation information of Mashup and Web API for service recommendation. The service recommendation task is jointly trained with the previous text representation and structure representation tasks, making the service representation results more suitable for the service recommendation task.

[0062] 1. Acquisition of Text Representation of Mashup and Web API

[0063] When a sentence is input to the BERT model, it uses a deep neural network to learn three embedding features: TokenEmbeddings, Segment Embeddings, and Position Embeddings. Figure 2 An example representation of a given sentence input is shown. First, the model performs word segmentation on the input sentence, dividing it into basic words that exist in the vocabulary, and then converts each word into a corresponding word vector. As shown in the figure, "playing" is not a basic word, so it is also split into "play" and "**ing". Token Embeddings are composed of converted word vectors, which imply the semantic information of the words in the sentence. It is worth noting that the beginning and end of each sentence will have unique markers that are not repeated with other text words, namely [CLS] and [SEP], which correspond to unique special word vectors that are not repeated with other word vectors. Segment Embeddings are mainly used to distinguish different sentences used in the pre-training process. The Segment Embedding of each sentence is different, so BERT can distinguish the semantics of different sentences. Since BERT is a Transformers-based model, it does not process position information in sequence. To solve this problem, BERT introduces Position Embeddings to represent the token position in the sentence. The dimension of Position Embeddings is [seq_length, width], where seq_length represents the sequence length and width represents the vector length corresponding to each token. From the implementation point of view, the position encoding is created as a tensorflow variable in BERT, and it is broadcast to the same dimension as the word embedding encoding and then added. The present invention directly inputs the text description of the Web API and the text description of the Mashup in the service library into the BERT model to obtain the Mashup vector representation and API vector representation composed of tag embedding, segment embedding and position embedding. Then, the final Mashup text embedding and API text embedding are obtained through calculation inside the model.

[0064] After obtaining the Mashup text embedding and API text embedding, they need to be fine-tuned. The specific operation is to add a multilayer perceptron part after the BERT output. The multilayer perceptron is a basic artificial neural network, usually composed of an input layer, a hidden layer, and an output layer. The multilayer perceptron can be used to adjust the dimension of the BERT model output vector to make it the same as the dimension of the LightGCN model output, which is convenient for comparative learning. The multilayer perceptron can be used for dimensional alignment, adjusting the dimension of the BERT model output vector to make it consistent with the dimension of the vector output by the LightGCN model, so as to achieve subsequent comparative learning.

[0065] The following is the specific formula of the multi-layer perceptron:

[0066] e B =σ2(W2(σ1(W1x+b1)+b2)) (1)

[0067] Where x is the input vector, W1 is the weight matrix of the first hidden layer, b1 is the bias vector of the first hidden layer, σ is the activation function, which is usually a nonlinear function, such as Sigmoid, ReLU or Tanh, e B is the text representation vector output after fine-tuning. i (Web API A i )’s final text embedding or text representation is composed of the vector express.

[0068] 2. Acquisition of structural representations of Mashup and Web API

[0069] The present invention constructs the interaction records of Mashup and Web API into a user-project bipartite graph (user is Mashup, project is Web API), which is expressed as in For Mashup collection, It is a collection of Web APIs. is a collection of edges, representing the interactive relationship between Mashup and Web API, and graph G is the input of LightGCN. The LightGCN model consists of three parts: embedding layer, embedding propagation layer and prediction layer. The process is as follows: First, randomly initialize the embedding vectors of Mashup and Web API; use the LightGCN model to convolve the constructed Mashup-API bipartite graph, and obtain the embedding vector representation of the next layer through neighborhood aggregation; after obtaining the vector representation of the Lth layer, perform weighted summation on the representation of each layer to obtain the final structural representation of Mashup and Web API. The framework of the model is as follows Figure 3 shown.

[0070] 2.1 Embedding Layer

[0071] The embedding layer randomly initializes the embedding of each Mashup and API, i.e., the structural representation, as shown in formula (2). i (Web API A i ) is embedded by the vector Represented by , where N is the number of Mashups and M is the number of Web APIs. These embedding vectors are optimized through propagation in the embedding propagation layer, and more accurate embeddings can be obtained through collaborative signal propagation.

[0072]

[0073] 2.2 Embedding Propagation Layer

[0074] The embedding propagation layer uses simplified graph convolution operations to propagate information by aggregating the embedding vectors of direct and indirect neighbors. Specifically, for each node, the embedding vectors of its neighbors are aggregated by weighted summation to obtain an embedding vector that reflects the interactive updates of its neighbors. Different strategies can be used for the aggregation function. However, in terms of the nature of the recommendation scenario, LightGCN considers a simple weighted aggregation function. The propagation of embedding can be divided into two parts: one is the embedding propagation of Mashup, as shown in formula (3); the other is the embedding propagation of API, as shown in formula (4).

[0075]

[0076] Among them, k represents the number of convolution layers. and represent the direct neighbors of MashupM and APIA, respectively, and the normalized coefficients The weight coefficients are the same as those designed for standard GCN, which can keep the value range of embedding in an appropriate range. Note: This part only aggregates connected neighbors, but does not include the target node itself (i.e. self-connection), which is different from the general convolution strategy.

[0077] 2.3 Weighted Combination Layer

[0078] The weighted combination operation is to add the embedding vectors of the Mashup and Web API of each layer after the graph convolution, and then divide them by the number of layers to obtain the final embedding representation, namely the first structure representation and the second structure representation, as shown in formulas (5) and (6).

[0079]

[0080] in, Represents the weight of the vector representation obtained in the kth layer when forming the final embedding.

[0081] Traditional GCN methods generally select the last layer as the final representation. LightGCN, on the other hand, combines the representations of each layer to obtain the final representation for three reasons: (1) As the number of layers increases, the embedding will become overly smooth, so simply using the last layer may not be sufficient; (2) The embeddings of different layers capture different semantics, and combining them can make the representation more comprehensive; (3) The weighted sum of the vector representations obtained from each layer of the convolution can achieve an effect equivalent to a self-connection operation, which is why the embedding propagation layer does not consider self-connection.

[0082] To obtain the trained LightGCN model, the BPR (Bayesian Personalized Ranking) loss is used so that the prediction of observed items is higher than the pairwise loss of unobserved items, which is defined as follows:

[0083]

[0084] in, represents paired data samples, is the observed interaction record (i.e., positive sample), It is an unobserved interaction record (i.e., negative sample).

[0085] 3. Contrastive Learning of Text Representation and Structural Representation

[0086] Contrastive learning encourages features from different angles to be closer in the embedding space, thereby achieving self-differentiated feature representation in an unsupervised manner and supplementing supervised tasks with unlabeled data. First, cross-view contrastive learning is used to align the text representation from BERT and the structural representation from LightGCN. The calculation of contrastive learning loss usually adopts InfoNCE Loss, defined as formula (8) and formula (9), to make the distance between positive samples as small as possible and the distance between negative samples as large as possible.

[0087]

[0088] in, Represents the contrastive learning loss value corresponding to Mashup, Indicates the contrastive learning loss value corresponding to the Web API, Indicates Mashup collection, M i represents the i-th Mashup, Represents a collection of Web APIs, A i Represents the i-th WebAPI, Represents the embedding vector corresponding to the i-th Mashup output by the BERT model, Represents the embedding vector corresponding to the i-th Mashup output by the LightGCN model, represents the embedding vector corresponding to the i-th Web API output by the BERT model, represents the embedding vector corresponding to the i-th Web API output by the LightGCN model, s(·,·) is used to calculate the similarity between two vectors, and τ is the temperature parameter.

[0089] The size of the temperature parameter will directly affect the effectiveness of contrastive learning, especially when the sample features are evenly distributed. Therefore, this parameter should be set appropriately according to the experimental situation. By performing contrastive learning of formula (8) and formula (9), the structural semantics and text semantics are combined to make the feature extraction more representative. Next, the text representation and structural representation of Mashup and Web API are combined. Splice them together to get the overall embedding e input , defined as formula (10), is used as the input of the downstream Web API recommendation task.

[0090]

[0091] 4. Web API Recommendations

[0092] In the final prediction part, the present invention uses SANFM to predict the Web API recommendation probability.

[0093] In the process of modeling and fusion embedding features, the SANFM model uses the linear part to model the first-order features, the interaction pooling layer to model the second-order interactions, and the self-attention mechanism and multi-layer perceptron to model the high-order interactions. Finally, the modeling of low-order and high-order interaction features is combined to recommend Web APIs. The service recommendation task is jointly trained with the previous text representation and structure representation tasks, making the service representation results more suitable for the service recommendation task. The expression definition of SANFM is shown in formula (11).

[0094]

[0095] in, represents the predicted value of the SANFM model, w0 represents the bias, n is the number of input features, and w i represents the linear coefficient, x i represents the feature value, h(x) represents the attention part and the modeling of high-order feature interactions.

[0096] The first two terms of the expression are the same as basicFM, representing the global bias and the linear part respectively, while the last term includes feature interaction, self-attention mechanism and deep neural network. h(x) is the core part of SANFM. Figure 4The figure below is the neural network structure diagram of SANFM. The following will explain it in sequence according to the hierarchical structure of the figure.

[0097] 4.1 Interaction Layer

[0098] The purpose of the interaction layer is to transform the sparse features of the input into a low-dimensional dense vector. It usually uses a fully connected neural network technique to capture the original embedding of users and items. Here, the vector is transformed into V x ={x1v1,…,x n v n}, where x={x1,…,x n} is the original eigenvector, v i is feature x i After that, the vector is sent to the interaction layer for feature interaction modeling, Z x ' represents the modeling result, as shown in formula (12), ⊙ represents the bitwise product operation of two vectors, that is, v i ⊙v j =v ik ⊙v jk .

[0099]

[0100] 4.2 Self-Attention Layer

[0101] Different feature interactions have different importance to the final prediction, so the self-attention mechanism is used to calculate the attention weight between each interaction feature and other interaction features, as shown in formula (13).

[0102]

[0103] Among them, Z x represents the feature interaction with self-attention weights, Q, K, and V represent the query matrix, key matrix, and value matrix respectively, and d k The query matrix, key matrix and value matrix are used to calculate the attention weights between features to facilitate interactive learning of features of different importance.

[0104] 4.3 Deep Neural Network Layers

[0105] In the original factorization machine model, the linear part cannot capture the high-order relationships between input features, while the deep neural network layer can learn these complex relationships through a combination of multiple layers of neurons and nonlinear activation functions. To model the high-order feature interaction relationship between Mashup and API, the vector is input into the multi-layer perceptron, as defined in formula (14).

[0106]

[0107] Among them, i represents the output of the i-th layer, σ i is an activation function that introduces nonlinear characteristics. i is the weight matrix of the i-th layer, which converts the output of the previous layer into the input of this layer. i is the bias of the i-th layer, which is used to adjust the input data of this layer. The output of the last hidden layer is converted into the prediction probability h(x), which is defined as shown in formula (15). Formula (16) is the expansion of formula (15), where L is the number of layers of the deep neural network layer and h is the neural network parameter weight in the prediction layer.

[0108] h(x)=h T o L (15)

[0109] h(x)=h T σ L (W L (…(W1Z x +b1)…)+b L ) (16)

[0110] 4.4 Prediction Layer

[0111] Considering that the task of the SANFM model is a Top-K recommendation problem, the predicted probability needs to be controlled within the range of [0,1]. The specific operation is shown in formula (17).

[0112]

[0113] in, Indicates mapping the predicted value obtained by the SANFM model to the predicted probability of [0,1], represents the predicted value calculated by the SANFN model,

[0114] Finally, the objective function of SANFM is calculated using the cross entropy loss, as shown in formula (18).

[0115]

[0116] Among them, L Rec represents the loss value of the SANFM model, N is the number of training samples, and y (i) is the actual value, and is the predicted value. The present invention uses the stochastic gradient descent algorithm to optimize the loss function. The final output consists of a list of Top-KWeb APIs, and the services in the list are sorted in descending order according to the predicted probability.

[0117] By combining the task loss functions of the above models and setting weight coefficients, a comprehensive joint training objective function can be obtained, as shown in formula (20). The joint training objective function aims to achieve an effective balance between the tasks of each model and maintain reasonable parameters during the model training process, thereby enhancing the robustness and generalization ability of the entire method.

[0118]

[0119] Among them, γ is the parameter that controls the contrast loss, Θ is the trainable parameter of the entire model, λ is the L2 regularization weight to alleviate overfitting, and L BPR and L rec They are the loss values ​​of the LightGCN model and the SANFM model respectively.

[0120] Experimental example:

[0121] In this embodiment, an experiment is conducted to verify the Web API recommendation method provided by the present invention. The data set and experimental settings, evaluation indicators, comparison methods and experimental results of this embodiment are described below.

[0122] Dataset and Experimental Settings:

[0123] The data used in the experiment comes from the ProgrammableWeb platform. As of February 2022, a total of 22,576 Web APIs and 6,454 Mashups have been crawled from the platform. For Mashups, the data used in this experiment are the name of the Mashup, the text description, and the interaction record with the Web API. For APIs, the data used in this experiment are the name and text description of the Web API. Filter out data items that lack metadata, and filter out APIs that have never been called. After processing, the available Mashups and APIs are 5,751 and 1,188, respectively, and the average number of Web APIs called by each Mashup is 1.86. In order to evaluate the performance of the proposed method, the experimental data set is randomly divided into 5 parts and marked with serial numbers. One part is selected in sequence as the test data set, and the remaining four parts are used as training data sets. Finally, the average of the 5 experimental results is taken. The specific description of the data set is shown in Table 1.

[0124] Table 1

[0125]

[0126] Evaluation indicators:

[0127] The experiment uses the following evaluation indicators: Precision, Recall, F1 Score, MAP and NDCG.

[0128] (1) Precision: Calculate the proportion of actual positive samples among the predicted positive samples. The definition of Precision is shown in formula (21).

[0129]

[0130] (2) Recall: The recall rate is also called the recall rate. It is calculated as the ratio of the actual number of positive samples in the positive samples to the total number of positive samples in the samples. It is defined as shown in formula (22).

[0131]

[0132] (3) F1 Score: This evaluation index is the weighted average of Precision and Recall. Precision reflects the model's ability to distinguish between positive and negative samples. The higher the Precision, the stronger the model's ability to distinguish between positive and negative samples. Recall reflects the model's ability to identify positive samples. The higher the Recall, the stronger the model's ability to identify positive samples. F1 Score combines the two to comprehensively evaluate the model. The higher the F1 Score, the more robust the model. The expression of F1 is shown in formula (23).

[0133]

[0134] (4) MAP: Average Precision is very useful for evaluating the effectiveness of recommendation methods. It provides a comprehensive measure of the accuracy of different angles in the recommendation list, as defined in Equations (24) and (25).

[0135]

[0136] (5) NDGC: Normalized Discounted Cumulative Gain is a widely used evaluation indicator. It is usually used to measure the ranking quality of the recommendation list, that is, whether the selected service is relatively high in the recommendation list. The calculation of NDCG includes two aspects: DCG and IDCG, as shown in formula (28), where the definitions of DCG and IDCG are shown in formulas (26) and (27), respectively. i is whether the user has used the i-th candidate Web API. If yes, it is marked as 1, and if no, it is marked as 0. s is the number of APIs actually used in the recommendation list.

[0137]

[0138] Comparison method:

[0139] The proposed method will be compared with the following methods: content-based methods (DeepFM and NAFM); graph-based methods (NGCF, BPRMF and LightGCN); and a hybrid method (SRCLML).

[0140] (1) DeepFM: The Deep Neural Network Factorization Machine can effectively capture low-order and high-order feature relationships, and can also learn complex nonlinear correlations in the text descriptions of Mashups and Web APIs, thereby achieving accurate service recommendations.

[0141] (2) NAFM: Neural and Attentional Factorization Machine integrates deep neural networks to model the interaction of nonlinear features, and uses the attention mechanism to identify the importance of interactions between different features.

[0142] (3) NGCF: Neural Graph Collaborative Filtering combines graph neural networks with collaborative filtering technology, using graph neural networks to represent mashup and Web API nodes and using collaborative filtering to capture their similarities, thereby achieving effective service recommendations.

[0143] (4) BPRMF: BPRMF refers to Bayesian Personalized Ranking Matrix Factorization, which combines Bayesian inference and personalized ranking and applies it to recommendation systems. The core idea is to learn the feature vectors of Mashup and Web API and use these representations to predict developers' preferences for APIs they have never used. Different from traditional methods based on rating prediction, it focuses more on personalized recommendations, that is, preference ranking.

[0144] (5) LightGCN: LightGCN is a lightweight graph convolutional neural network model that simplifies the classic graph convolutional network framework by eliminating feature transformation and nonlinear activation, and verifies that these two parts have no gain for graph convolution. The LightGCN model mainly uses graph convolution to represent Mashup and Web API nodes for service recommendation.

[0145] (6) SRCLML: The SRCLML method uses Transform to extract features from the text descriptions of Mashup and Web API, and then uses LightGCN to extract structural information from the Mashup-API interaction graph. After that, the information from the two perspectives is integrated for service recommendation.

[0146] Performance evaluation:

[0147] Table 2 gives the default parameters of this model. The present invention will conduct experimental analysis on the performance impact of the main hyperparameters in the future.

[0148] Table 2

[0149]

[0150] This section mainly compares the performance of the proposed method CLJT with the baseline method, as shown in Table 3. Service recommendation needs to predict the first K items in the list, and the present invention sets K to 3, 5, and 7. Since the interaction records of Mashup and Web API are relatively sparse, the overall performance of service recommendation is relatively low.

[0151] The performance comparison results of different Web API recommendation methods are shown in Table 3. It can be seen from Table 3 that methods such as DeepFM, NAFM and NGCF perform poorly on all evaluation indicators because DeepFM and NAFM mainly use text vectors obtained from the text descriptions of Mashups and Web APIs to capture potential information. Due to the low quality and lack of standardized formats of the description documents of Mashups and APIs, they may not provide enough details and features. Therefore, relying solely on text information for recommendation is limited. LightGCN performs better than BPRMF and NGCF because BPRMF is mainly based on matrix decomposition for recommendation. When dealing with large-scale data and complex relationships, it is difficult to capture high-order and hidden relationships between services, and the performance will drop rapidly. Although the core of NGCF and LightGCN are both based on graph convolutional neural networks, in contrast, LightGCN improves on the basis of classic graph convolutional neural networks and removes feature transformation and nonlinear activation parts, which makes the model's recommendation more accurate. SRCLML surpasses the above methods. It comprehensively considers the text information of Mashups and Web APIs and the structural information in the historical interaction graph, and combines multi-task learning for better optimization.

[0152] Nevertheless, compared with the above methods, the CLJT method has more significant advantages because it introduces cross-view contrastive learning, which enables CLJT to effectively integrate text information and structural information, and introduces joint training to further improve the performance of the model. Under the condition of Top-K=3, compared with the second best method SRCLML, the CLJT method improves Precision, Recall, F1 Score, MAP and NDCG by 76.58%, 1.39%, 48.12%, 17.07% and 24.41% respectively. It can be observed that the graph-based method performs better than the content-based method, and the hybrid method performs better than the first two methods. In summary, a large number of experimental analyses show the effectiveness of the proposed method CLJT.

[0153] Table 3

[0154]

[0155]

[0156] Ablation experiment:

[0157] In order to study the impact of each part of the proposed method on the overall model, in addition to the comparative experiments, ablation experiments were also conducted. The four main parts considered are BERT, LightGCN, contrastive learning (CL) and joint training (JT). By removing these parts, four variant methods can be derived.

[0158] (1) CLJT-BERT: This method is derived by removing the BERT-based text representation. Since contrastive learning requires comparing the output vectors of BERT and LightGCN, this part is not included after removing BERT.

[0159] (2) CLJT-LightGCN: This method is obtained by removing the structural representation of LightGCN. In this model, only text representation is used to explore the relationship between Mashup and Web API, so contrastive learning is also removed.

[0160] (3) CLJT-CL: This method is derived by removing the contrastive learning part between the text representation and structural representation of Mashup and Web API. That is, the text representation and vector representation are directly concatenated and used as the input of SANFM for recommendation.

[0161] (4) CLJT-JT: This method removes the joint training, trains the text representation, structure representation and recommendation model independently, and removes contrastive learning.

[0162] Table 4

[0163]

[0164] The results of the ablation experiment are shown in Table 4. It can be seen from the table that the contrastive learning part integrates the text information and the structural information of the network, and achieves better performance. At the same time, joint training helps to effectively balance the tasks of each part. In addition, the recommendation performance based only on text information or only on network structure information is not as good as the model that comprehensively considers the two, because the information possessed by recommendation from only one perspective is often insufficient. The experimental results also illustrate this point. CLJT-CL and CLJT-JT are better than CLJT-BERT and CLJT-LightGCN. In contrast, CLJT-JT performs slightly better than CLJT-CL, indicating that contrastive learning is more helpful to improve the service recommendation performance than the joint training mechanism in the method of the present invention. In short, among these methods, CLJT always has the best performance.

[0165] Hyperparameter Analysis:

[0166] In the method proposed in this paper, the embedding dimension d of the LightGCN part is l , the embedding dimension d of the SANFM part s , contrastive learning temperature parameter τ and contrastive learning loss weight γ in joint loss are the four key parameters of the whole method framework. Therefore, it is necessary to conduct hyperparameter experimental analysis on the dataset in order to study the performance of the proposed method under different parameters. Note that all the following experiments are conducted under the condition of K=5.

[0167] (1) LightGCN embedding dimension d l In order to study the impact of LightGCN embedding dimension on model performance, l ={16,32,…,256}, observe the performance of different values. Figure 5 is the performance curve when the model changes. As the dimension increases, the performance of the CLJT model gradually improves, and then l = 64. However, as the dimension continues to increase, the performance of the model begins to decline. This is because when the dimension is too small, the model will be limited from capturing potential structural features, resulting in poor performance; on the contrary, when the embedding dimension is too large, the complexity of the model will increase, which is prone to overfitting and the introduction of noise, all of which will cause the model performance to decline.

[0168] (2) SANFM embedding dimension d s The impact of: By changing the embedding dimension d of SANFM s ={16,32,…,256} to study the impact on model performance, Figure 6 is the model's d sPerformance curve when d changes. Since the SANFM model integrates self-attention layer and deep neural network layer, the smaller dimension limits the expressiveness of the model and makes it difficult to process complex data. This constraint may cause the model to be underfitting, that is, the model cannot fully capture the features. s =128, the model performance is the best. s As it continues to increase, the model performance decreases.

[0169] (3) The influence of temperature parameter τ: The temperature parameter is crucial in contrastive learning and will directly affect the effectiveness of contrastive learning, especially when the sample features are uniformly distributed. A smaller τ will enhance contrastive learning by reducing the distance between similar samples and increasing the distance between samples with large differences, thereby enhancing the recognition of sample similarities. However, if τ is set to an extreme value, it may destroy the uniformity of the sample feature distribution and affect the effectiveness of contrastive learning. In preliminary experiments, a series of different values ​​of τ were explored, ranging from large to small. Based on the observation of the experimental results, the range was finally concentrated on an interval that could be proven to have a greater positive impact on model performance. Therefore, the determined range is τ = {0.0001, 0.0005,…, 0.002}. As Figure 7 As shown in Figure 2, when τ is 0.0015, the performance is optimal. However, as the value of τ increases, the model's ability to detect negative samples weakens, resulting in a decrease in the effectiveness of contrastive learning.

[0170] (4) Comparison of learning loss weight γ: Adjust γ to {0.0001, 0.0003, …, 0.0009} to observe the performance of the model under different conditions. The results are as follows Figure 8 As shown in the figure, as γ increases, all indicators are improved, highlighting the effectiveness of contrast loss. The model reaches its peak when γ = 0.0005, and then the performance gradually decreases. The reason for this is that in the process of joint training, a large value of γ will overemphasize the contrast objective function, which may cause deviations in the representation learning of Mashup and Web API, resulting in a decrease in model performance.

[0171] This paper mainly conducts a series of in-depth explorations on the Web API recommendation problem created by Mashup. At present, several mainstream methods for Web API recommendation have their advantages and areas for improvement, but the methods that rely solely on text information or structural information for service recommendation have shortcomings and cannot fully capture the information that can be used for recommendation. Therefore, how to integrate text and structural semantics for recommendation is a difficult point. This paper proposes a Web API recommendation method CLJT based on contrastive learning and joint training. This method uses text semantic information and structural semantic information for recommendation. The related work is summarized as follows:

[0172] First, the BERT model and the LightGCN model are used to extract text semantics and structural semantics for Mashup and Web API respectively. Then, after the BERT model obtains the text representation, the output is reduced in dimension so that the text semantic embedding dimension is the same as the structural semantic embedding from the LightGCN model. Through cross-view contrastive learning, the text semantic representation and the structural semantic representation are associated to complement each other's semantic information, so that the accuracy of the text semantic representation and the structural semantic representation are improved at the same time. The downstream service recommendation task uses the SANFM model and is jointly trained with the previous text semantic representation and structural semantic representation tasks, so that the service representation is more suitable for the service recommendation task, thereby improving the performance of the service recommendation. To verify the effectiveness of the method, the present invention uses real data from the ProgrammableWeb platform to conduct experiments. The experimental results show that compared with the baseline method, the Web API recommendation based on contrastive learning and joint training shows better performance in all evaluation indicators.

[0173] The above are only preferred specific implementations of the present application, but the protection scope of the present application is not limited thereto. Any changes or substitutions that can be easily thought of by a person skilled in the art within the technical scope disclosed in the present application should be included in the protection scope of the present application. Therefore, the protection scope of the present application should be based on the protection scope of the claims.

Claims

1. A WebAPI recommendation method based on contrastive learning and joint training, characterized in that: include: Obtain a first text description and a second text description, wherein the first text description and the second text description are text descriptions of a mashup and a web API, respectively; process the first text description and the second text description through a BERT model to obtain a first text representation and a second text representation; Obtain the interactive network structure of Mashup and Web API, process the interactive network structure through the LightGCN model, and obtain the first structural representation and the second structural representation; The first text description, the second text description, the first structural representation and the second structural representation are concatenated to generate an overall embedding, and the overall embedding is processed by the SANFM model to obtain a Web API recommendation result; Among them, the BERT model, LightGCN model and SANFM model are optimized through comparative learning and joint training.

2. The method according to claim 1, characterized in that After obtaining the first text description and the second text description, the method further includes: preprocessing the first text description and the second text description by a natural language preprocessing method, wherein the preprocessing process includes: normalization, stem extraction, tokenization and stop word deletion.

3. The method according to claim 1, characterized in that The interactive network structure is a bipartite graph, which includes a Mashup collection, a Web API collection, and an edge collection representing the interactive relationship between each Mashup and each WebAPI.

4. The method according to claim 1, characterized in that: After the first text description and the second text description are processed by the BERT model, the method further includes adjusting the dimension of the output result of the BERT model by a multi-layer perceptron to obtain the first text representation and the second text representation of the same dimension as the first structural representation and the second structural representation.

5. The method according to claim 1, characterized in that: The joint training process includes: Construct the initial SANFM model, the initial BERT model and the initial LightGCN model and obtain the training set, perform comparative learning on the initial BERT model and the initial LightGCN model through the training set, and jointly train the initial BERT model, the initial LightGCN model and the initial SANFM model respectively. In the comparative learning process, calculate the comparative learning loss value, and calculate the joint training loss value based on the comparative learning loss value, optimize the initial BERT model, the initial LightGCN model and the initial SANFM model according to the joint training loss value, and use the optimized BERT model, LightGCN model and SANFM model to participate in the generation of recommendation results.

6. The method according to claim 5, characterized in that The calculation process of the joint training loss value includes: Among them, L represents the joint training loss value, γ is the weight parameter that controls the contrastive learning loss value, Θ is the trainable parameter of the entire model, λ is the L2 regularization weight to alleviate overfitting, and L BPR and L Rec They are the loss functions of the LightGCN model and the SANFM model, and Indicates the comparative learning loss values ​​corresponding to Mashup and Web API.

7. The method according to claim 5, characterized in that The process of contrastive learning includes: The initial BERT model and the initial LightGCN model are compared through cross-view contrast learning, including: Constructing an initial BERT model and an initial LightGCN model, wherein the training set includes a text training set and a structure training set, processing the text training set by the initial BERT model to obtain predicted text representations of Mashup and Web API; processing the structure training set by the initial LightGCN model to obtain predicted structure representations of Mashup and WebAPI; Compute the contrastive learning loss based on the predicted text representations of Mashup and Web API and the predicted structural representations of Mashup and Web API.

8. The method according to claim 5, characterized in that The calculation process of the contrastive learning loss value includes: in, Represents the contrastive learning loss value corresponding to Mashup, Indicates the contrastive learning loss value corresponding to the Web API, Indicates Mashup collection, M i represents the i-th Mashup, Represents a collection of Web APIs, A i Represents the i-th WebAPI, Represents the embedding vector corresponding to the i-th Mashup output by the BERT model, Represents the embedding vector corresponding to the i-th Mashup output by the LightGCN model, represents the embedding vector corresponding to the i-th Web API output by the BERT model, represents the embedding vector corresponding to the i-th Web API output by the LightGCN model, s(·,·) is used to calculate the similarity between two vectors, and τ is the temperature parameter.

Images