Web API recommendation method based on multi-task deep neural network

Abstract

The invention provides a Web API recommendation method based on a multi-task deep neural network. The Web API recommendation method comprises the following steps: S1, cleaning a Mashup description document and a Web API description document in a data set; s2, mapping character information in the cleaned data set into digital codes; s3, generating sample data based on digital coding; s4, constructing a Web API recommendation model based on the multi-task deep neural network; s5, inputting the sample data into the Web API recommendation model for pre-training to obtain a pre-trained Web API recommendation model; s6, performing fine adjustment on the pre-trained Web API recommendation model by using a Mashup and API matching score prediction task; s7, obtaining a matching score list of the Mashup demand and each Web API (Application Program Interface); and S8, sorting the matching score lists of the Mashup requirements and the Web APIs in a descending order according to the matching scores, and selecting the first K Web APIs to form a final recommendation list. According to the method, the problem that no-interaction or low-interaction Web APIs in historical data are difficult to recommend accurately is solved, and indexes such as the recall rate and the accuracy rate of recommendation are improved.

Description

Technical Field

[0001] The present invention relates to the technical field of Web service recommendation, and in particular to a Web API recommendation method based on a multi-task deep neural network. Background Art

[0002] With the rapid development of service oriented computing (SOC) technology, the number and diversity of available services have also increased dramatically, and services have become increasingly important in people's daily work and life. At the same time, in order to meet complex development requirements, service composition application technology led by Mashup (aggregate service) has also developed rapidly. By integrating data, content or functions from multiple application programming interfaces (Web API, Application Programming Interface), a new, integrated aggregate service is created to provide users with a richer and more convenient experience. In the process of Mashup development, it is crucial to choose the right Web API, because it directly affects the performance, function richness and user experience of the Mashup. The huge number of Web APIs has brought great challenges to developers to effectively select, use and manage services. Therefore, how to accurately and efficiently recommend candidate services for Mashup from massive Web APIs has become an important issue in the field of service computing, which has research and application value.

[0003] At present, the Web API recommendation methods for Mashup development mainly include collaborative filtering recommendation methods, content-based recommendation methods, and hybrid recommendation methods. Collaborative filtering recommendation methods use historical interaction relationships to predict recommendation results. Based on the historical call relationship between Mashup and Web API, they recommend Web APIs that have been called by Mashups similar to their requirements to new Mashup developers. However, this type of method only uses the similarity relationship between Mashups, but ignores the matching relationship between Mashup and WebAPI, such as the similarity of the functional description text and the consistency of the category, and has a serious cold start problem. Content-based recommendation methods use the content relevance between Mashup requirements and Web API description texts, calculate the similarity between the two, and recommend Web APIs that are more similar to Mashup requirements to users. However, this type of method focuses on the semantic features of Mashup and WebAPI description documents, while the Web service ecosystem contains a large amount of information that can be used to enhance content representation, such as the call relationship and category attributes between Mashup and Web API, so the quality of the obtained semantic features is low.

[0004] Multi-task learning learns multiple related tasks at the same time by sharing representation information, and has the characteristics of implicit data enhancement, attention focus, feature eavesdropping, inductive bias, and regularization. Some researchers have proposed MTFM recommendation methods based on multi-model fusion and multi-task learning. First, semantic features and interaction features are extracted respectively using semantic feature extraction components based on convolutional neural networks and feature interaction components based on matrix factorization (MF) and multi-layer perceptron (MLP). The two extracted features are then spliced ​​together. Finally, the two multi-label classification tasks of WebAPI recommendation and Mashup classification are used for training using the spliced ​​features to enable the model to have multi-task learning capabilities. Similarly, the patent "Web service package recommendation method, system, medium and device based on multi-task learning" builds reusable service packages and learns their combined semantics through historical Mashup and WebAPI call records, fuses content-based and historical call-based models through semantic feature extraction components and feature interaction components, and builds a multi-task learning framework for service package recommendation. However, this method only uses the Mashup classification task as an auxiliary task and improves the generalization ability of the model by inductive bias. It does not mine the more fine-grained category-level semantic association information in the semantic information, nor does it consider simultaneously learning the category-level semantic association information and sparse function-level semantic association information between Mashup and Web API, resulting in insufficient semantic information mining and low recommendation accuracy.

[0005] In summary, existing Web API recommendation methods still have the following problems: (1) Since Web API and Mashup requirement description documents are short texts with high noise and sparse features, it is difficult to obtain high-quality semantic feature representation by using only a single task to extract text information, resulting in low recommendation accuracy. (2) Existing Web API recommendation methods do not consider the mutual promotion of category-level semantic association information and function-level semantic information between Mashup and Web API. Separately learning the two cannot fully mine semantic information, making it difficult to accurately recommend Web APIs with no or low interactions in historical data, resulting in low recommendation accuracy. Summary of the invention

[0006] In view of this, the purpose of the present invention is to propose a Web API recommendation method based on a multi-task deep neural network, which uses the idea of ​​multi-task learning to construct a multi-task deep neural network, and uses the interactive information and category information of Mashup and API to design multiple tasks for pre-training, which plays a role in implicit data enhancement and expanding the number of samples, thereby improving the quality of text semantic features extracted by the model. At the same time, category-related auxiliary tasks are introduced, and the characteristics of multi-task learning attention focus and feature eavesdropping are used to enable the model to capture category-level semantic association information that is easy to classify, and at the same time learn and extract more comprehensive semantic features from the category level and function level perspectives, thereby solving the problem of difficulty in accurately recommending Web APIs with no or low interaction in historical data, and improving the recall rate, accuracy rate and other indicators of recommendation.

[0007] The technical means adopted by the present invention are as follows:

[0008] A Web API recommendation method based on a multi-task deep neural network includes the following steps:

[0009] S1. Obtain a Mashup and Web API dataset, wherein the dataset includes Mashup name, Mashup category, Mashup description document, Web API called by Mashup, Web API name, Web API category, and Web API description document; and clean the Mashup description document and Web API description document in the dataset, and delete stop words, emoticons, and special symbols in the document;

[0010] S2, mapping the text information in the cleaned data set into digital codes;

[0011] S3, generating sample data of multiple tasks based on digital coding; the multiple tasks include Mashup and Web API call matching score prediction task, Mashup and Web API category matching score prediction task, Web API category prediction task, Mashup category prediction task;

[0012] S4. Construct a multi-task deep neural network based on multiple tasks; the multi-task deep neural network includes a text semantic feature extraction layer based on BERT, a Mashup and Web API call matching score prediction task layer, a Mashup and WebAPI category matching score prediction task layer, a Mashup and Web API category matching score prediction task layer, a Web A PI category prediction task layer, and a Mashup category prediction task layer; construct a Web API recommendation model based on a multi-task deep neural network;

[0013] S5. Input the sample data into the Web API recommendation model for pre-training to obtain a pre-trained Web API recommendation model;

[0014] S6. Fine-tune the pre-trained Web API recommendation model using Mashup and API matching score prediction tasks;

[0015] S7, the Mashup requirements and the description texts of each Web API are combined into sentence pairs, and the pairs are input into the fine-tuned Mashup and Web API matching score prediction model to obtain a matching score list of the Mashup requirements and each Web API;

[0016] S8. Sort the matching score list between the mashup requirements and each Web API in descending order according to the matching score, and select the top K Web APIs to form the final recommendation list.

[0017] Furthermore, S2 specifically includes the following steps:

[0018] Convert the service category, Web API name, and Mashup name into digital form, map the service category into the service category label, map the Web API name into the Web API code, and map the Mashup name into the Mashup code;

[0019] According to the calling relationship between Mashup and Web API, a Mashup-Web API calling relationship encoding table is generated; the word segmentation tool of the BERT model is used to perform word segmentation processing on the Mashup description document and the Web API description document to obtain the word segmentation encoding of the Mashup description document and the Web API description document.

[0020] Furthermore, S3 specifically includes the following steps:

[0021] Combine the Mashup code, the word segmentation code of the Mashup description document, the Mashup category label and the task category tag to generate training samples for the Mashup category prediction task;

[0022] Combine the Web API code, the word segmentation code of the Web API description document, the Web API category label, and the task category tag to generate training samples for the Web API category prediction task;

[0023] Combine the sentence pairs consisting of Mashup code, Web API code, Mashup description document code and Web API description document code, the corresponding category matching labels and task category tags to obtain the training samples of the Mashup and Web API category matching prediction task, where the category matching label is 1, indicating that the Mashup and Web API categories match, and the category label is 0, indicating that the Mashup and Web API categories do not match.

[0024] The sentence pairs consisting of Mashup encoding, Web API encoding, Mashup description document encoding and Web API description document encoding, the corresponding call relationship labels and task category labels are combined to obtain the training samples of the Mashup and Web API call matching prediction task, where the call relationship label is 1, which means that the Mashup and Web API have a call relationship, and 0, which means that the Mashup and Web API have no call relationship.

[0025] Further, in S4, the Web API recommendation model based on the multi-task deep neural network includes a BERT-based text semantic feature extraction layer and four task layers, the four task layers are a Mashup and Web API call matching score prediction task layer, a Mashup and Web API category matching score prediction task layer, a Mashup and Web API category matching score prediction task layer, a Web A PI category prediction task layer and a Mashup category prediction task layer;

[0026] Among them, the four task layers share the parameters of the BERT-based text semantic feature extraction layer;

[0027] The four task layers are fully connected layers, and the corresponding task prediction results P are calculated according to the following formula t :

[0028]

[0029] Among them, x is the semantic feature output by the text semantic feature extraction layer based on BERT, t is the task type, and W t is the parameter matrix of the fully connected layer corresponding to task t.

[0030] Furthermore, in S4:

[0031] BERT-based text semantic feature extraction layer: The BERT-based text semantic feature extraction layer uses BERT to mine the semantic features of Mashup or Web API to describe text sentences;

[0032] Mashup and Web API call matching score prediction task layer: The Mashup and Web API call matching score prediction task layer uses Mashup and Web API with call relationship as positive samples and Mashup and WebAPI without call relationship as negative samples, and uses the actual call matching relationship between the two to assist the model in learning function-level semantic features;

[0033] For the sentence pairs composed of Mashup and Web API description texts, the BERT semantic feature extraction layer is used to extract their semantic features x, and the probability P of Mashup and Web API having a call matching relationship is calculated according to formula (1): r , and use the cross entropy loss function shown in formula (2) to calculate the task loss and adjust the parameter matrix of the network corresponding to the task layer according to the loss;

[0034]

[0035] Among them, x i represents the i-th sample data of the Mashup and Web API call matching score prediction task, that is, the sentence pair consisting of the Mashup and Web API description text, p is the probability distribution of the real result, and q is the probability distribution of the predicted result;

[0036] Mashup and Web API category matching score prediction task layer: For the sentence pairs consisting of Mashup and Web API, the BERT semantic feature extraction layer is used to extract their semantic features x, and the probability P that Mashup and Web API have a category matching relationship is calculated according to formula (1) c ;

[0037] The task loss is calculated using the cross entropy loss function shown in formula (2);

[0038] Among them, x i represents the i-th sample data of the Mashup and Web API category matching score prediction task, that is, the sentence pair consisting of the Mashup and Web API description text, p is the probability distribution of the true result, and q is the probability distribution of the predicted result;

[0039] Web API category prediction task layer: For the Web API description text, the semantic features x of the Web API description sentence extracted by the BERT semantic feature extraction layer are used to calculate the probability distribution P of the Web API belonging to each category according to formula (1): a =[P1,P2,P i ,...,P n ], where n is the total number of categories, P irepresents the probability that the Web API belongs to the i-th category. According to formula (3), P is selected a The category with the largest probability value As a result of the Web API category prediction.

[0040]

[0041] in, is the category prediction result, Indicates the selection of P a The index with the largest probability value;

[0042] The task loss is calculated using the cross entropy loss function shown in formula (2).

[0043] Among them, x i represents the i-th sample data of the Web API category prediction task, that is, the Web API description text, p is the probability distribution of the true result, and q is the probability distribution of the predicted result;

[0044] Mashup category prediction task layer:

[0045] For the mashup description text, the semantic features x of the mashup description text generated by the BERT semantic feature extraction layer are used to predict the probability distribution P of the mashup belonging to each category according to formula (1). m =[P1,P2,P i ,...,P n ], where n is the total number of categories, P i Indicates the probability that the Mashup belongs to the i-th category;

[0046] The category of Mashup is determined by setting a threshold. The threshold is set to θ. m The probability value P corresponding to the i-th category i , if P i >θ, then the Mashup is judged to belong to the i-th category;

[0047] This task uses the binary cross entropy loss function shown in formula (4) to calculate the task loss:

[0048]

[0049] Among them, y i It represents the true binary label of the i-th sample data of the Mashup category prediction task, which takes the value of 0 or 1. i is the probability of predicting the i-th sample as the positive class.

[0050] Furthermore, S5 specifically includes the following steps:

[0051] Assume that there are n different types of tasks, the current task is denoted as t, and the input data for pre-training is the sample data set T1, T2…, T generated in S3 for each task. n , dataset T for task t t In the example, each sample r i Recorded as a triple (s i ,l i ,c i ), where i represents the i-th sample, s i Indicates the text data of the sample, l i Indicates the true value label of the sample, c i Indicates the task type of this sample;

[0052] Obtain samples of each task for pre-training, divide these samples into small batches, and then merge and shuffle them all;

[0053] Randomly select a small batch of data b t , input the text data set S of the batch samples into the semantic feature extraction layer based on BERT, extract the contextual semantic features of the text data, and obtain the semantic feature set X of the sample. According to the task type C recorded in the sample data in the batch, input X into the corresponding task layer to obtain the predicted value of the task.

[0054] The loss function of the corresponding task type is used to calculate the task loss, and the model parameters are updated using the stochastic gradient descent method until each b t After all epochs are selected, an epoch of pre-training is completed, and the pre-training ends after all epochs are completed.

[0055] Further, S6 includes the following steps:

[0056] S61, obtain Mashup and Web API matching score prediction task fine-tuning samples;

[0057] S62, inputting the text data of the fine-tuning sample of the Mashup and Web API matching score prediction task into the BERT-based semantic feature extraction layer to extract feature vectors of the Mashup text data and the Web API text data;

[0058] S63, inputting the feature vectors of the Mashup text data and the Web API text data into the task layer, and predicting the matching score between the Mashup and the Web API;

[0059] S64. Calculate the task loss using the cross entropy loss function based on the predicted value and the actual value;

[0060] S65, using stochastic gradient descent method to update model parameters according to task loss;

[0061] S66, loop S62 to S65 until each sample has been selected, and then one epoch of fine-tuning training is completed. After all epochs are completed, the fine-tuning is completed and a Mashup and Web API matching score prediction model is obtained.

[0062] The present invention also provides a storage medium, which includes a stored program, wherein when the program is run, any of the above-mentioned Web API recommendation methods based on a multi-task deep neural network is executed.

[0063] The present invention also provides an electronic device, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes any one of the above-mentioned Web API recommendation methods based on a multi-task deep neural network through the computer program.

[0064] Compared with the prior art, the present invention has the following advantages:

[0065] The present invention adopts a network structure of multi-task hard parameter sharing in the pre-training stage, adopts the idea of ​​multi-task learning, and multiple tasks share information to expand the number of samples and implicitly enhance data, effectively alleviate the data sparsity problem, improve the text semantic feature extraction effect of the model, and thus improve the accuracy, recall rate and other indicators of recommendation; the present invention designs the Mashup and Web API call matching score prediction task, the Mashup and Web API category matching score prediction task, the WebAPI category prediction task and the Mashup category prediction task to simultaneously learn the interaction information and category information, and utilizes the characteristics of multi-task learning attention focusing and feature stealing to make the description text feature vectors of the Mashup and Web API of the same category in the vector space closer in representation distance and higher in similarity, so as to prompt the model to pay attention to the category-level semantic association information, and at the same time share these feature knowledge and function-level semantic details, so as to accurately recommend Web APIs with no interaction, low interaction but high matching degree in historical data, and thus improve the accuracy, recall rate and other indicators of recommendation. BRIEF DESCRIPTION OF THE DRAWINGS

[0066] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative labor.

[0067] Figure 1The figure is a flow chart of the method of the present invention.

[0068] Figure 2 This is a comparison chart of the Recall indicators of the two methods in the embodiments of the present invention.

[0069] Figure 3 The figure is a comparison chart of the Precision indicators of the two methods in the embodiments of the present invention.

[0070] Figure 4 This is a comparison chart of the F1 index of the two methods in the embodiments of the present invention.

[0071] Figure 5 This is a comparison chart of MAP indicators of two methods in the embodiments of the present invention. DETAILED DESCRIPTION

[0072] In order to enable those skilled in the art to better understand the scheme of the present invention, the technical scheme in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work should fall within the scope of protection of the present invention.

[0073] It should be noted that the terms "first", "second", etc. in the specification and claims of the present invention and the above-mentioned drawings are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequence. It should be understood that the data used in this way can be interchanged where appropriate, so that the embodiments of the present invention described herein can be implemented in an order other than those illustrated or described herein. In addition, the terms "including" and "having" and any variations thereof are intended to cover non-exclusive inclusions, for example, a process, method, system, product or device that includes a series of steps or units is not necessarily limited to those steps or units that are clearly listed, but may include other steps or units that are not clearly listed or inherent to these processes, methods, products or devices.

[0074] like Figure 1 As shown, the present invention provides a Web API recommendation method based on a multi-task deep neural network, comprising the following steps:

[0075] S1: Clean the Mashup description document and Web API description document.

[0076] Web API description documents are unprocessed texts that contain various symbols and special characters for easy expression. In order to allow the model to focus on learning text features and reduce interference from other information, the text data of the description document needs to be cleaned and stop words, emoticons, and special symbols in the document need to be deleted to facilitate subsequent semantic analysis.

[0077] The Mashup and Web API datasets include: Mashup name, Mashup category, Mashup description document, Web API called by Mashup, Web API name, Web API category, and Web API description document.

[0078] The cleaning method is to delete stop words, emoticons, and special symbols in the document.

[0079] S2: Map text information into digital codes.

[0080] The non-digital information such as service category, Web API name, Mashup name, etc. is converted into digital form and mapped to service category label, Web API code and Mashup code respectively. According to the calling relationship between Mashup and Web API, a Mashup-Web API calling relationship coding table is generated. The word segmentation tool of the BERT model is used to perform word segmentation processing on the Mashup description document and the Web API description document to obtain the word segmentation coding of the Mashup description document and the Web API description document.

[0081] S3: Generate sample data.

[0082] The multi-task deep neural network includes four tasks, namely, Mashup and Web API call matching score prediction task, Mashup and Web API category matching score prediction task, Web API category prediction task, Mashup category prediction task. The input data type and calculation method of each task layer are different, so it is necessary to extract the information required by each task and generate sample data. The detailed process is as follows:

[0083] (1) Combine the mashup code, the word segmentation code of the mashup description document, the mashup category label, and the task category label to generate training samples for the mashup category prediction task;

[0084] (2) Combine the Web API code, the word segmentation code of the Web API description document, the Web API category label, and the task category label to generate training samples for the Web API category prediction task;

[0085] (3) Combine the sentence pairs consisting of Mashup code, Web API code, Mashup description document code and Web API description document code, the corresponding category matching labels and task category tags to obtain the training samples of the Mashup and Web API category matching prediction task, where the category matching label is 1, indicating that the Mashup and Web API categories match, and the category label is 0, indicating that the Mashup and Web API categories do not match;

[0086] (4) Combine the sentence pairs consisting of Mashup code, Web API code, Mashup description document code and Web API description document code, the corresponding call relationship labels and task category labels to obtain training samples for the Mashup and Web API call matching prediction task, where the call relationship label is 1, indicating that there is a call relationship between Mashup and Web API, and 0, indicating that there is no call relationship between Mashup and Web API.

[0087] S4: Building a Web API Recommendation Model Based on Multi-task Deep Neural Network

[0088] Since there may be certain correlations between different tasks, but there may also be conflicts that lead to reduced model performance, it is necessary to design relevant auxiliary tasks and make full use of the characteristics of multi-task learning such as implicit data enhancement, attention focus, feature eavesdropping, inductive bias, and regularization. By sharing information, the extraction effect of text features can be improved, thereby improving the recommendation effect.

[0089] The Web API recommendation model based on multi-task deep neural network constructed in the present invention is divided into five layers, namely, a text semantic feature extraction layer based on BERT, a mashup and Web API call matching score prediction task layer, a mashup and Web API category matching score prediction task layer, a mashup and Web API category matching score prediction task layer, a Web A PI category prediction task layer and a mashup category prediction task layer. The four task layers share the parameters of the text semantic feature extraction layer based on BERT.

[0090] The above four task layers are fully connected layers. The corresponding task prediction results P are calculated according to formula (1). t .

[0091]

[0092] Where x is the semantic feature output by the text semantic feature extraction layer based on BERT, t is the task type, and W t is the parameter matrix of the fully connected layer corresponding to task t.

[0093] (1) Text semantic feature extraction layer based on BERT

[0094] Since the description text of each Mashup and Web API contains a lot of feature information, BERT is used to mine the semantic features of the Mashup or Web API description text sentences.

[0095] (2) Mashup and Web API call matching score prediction task layer

[0096] When a Mashup developer user calls a Web API, he needs to consider the consistency between the functions described in the Web API description text and the development requirements. However, simply using a natural language processing model to calculate the similarity of text only considers the co-occurrence frequency and distribution of words and the similarity of word vectors, without considering the call relationship that reflects the actual application utility, making it difficult to learn high-quality functional-level semantic features. The present invention designs a Mashup and Web API call matching score prediction task, taking Mashups and Web APIs with a call relationship as positive samples and those without a call relationship as negative samples, and using the actual call matching relationship between the two to assist the model in learning functional-level semantic features.

[0097] For the sentence pairs composed of Mashup and Web API description texts, the BERT semantic feature extraction layer is used to extract their semantic features x, and the probability P of Mashup and Web API having a call matching relationship is calculated according to formula (1): r , and use the cross-entropy loss function CE (Cross-Entropy Loss Function) shown in formula (2) to calculate the task loss and adjust the parameter matrix of the network corresponding to the task layer according to the loss.

[0098]

[0099] where x i It represents the i-th sample data of the Mashup and Web API call matching score prediction task, that is, the sentence pair consisting of the Mashup and Web API description text, p is the probability distribution of the real result, and q is the probability distribution of the predicted result.

[0100] (3) Mashup and Web API category matching score prediction task layer

[0101] Since a Mashup only calls 1-3 Web APIs in most cases, and there are many Web APIs that match a Mashup category, using it as an auxiliary task can expand a large amount of training data. The present invention designs a Mashup and Web API category matching score prediction task, and uses Mashups and Web APIs of the same category as positive samples, and Mashups and Web APIs of different categories as negative samples, so that the feature vectors of Mashups and Web APIs of the same category have a closer representation distance in the vector space and a higher similarity, so that the model can deeply mine high-order category-level semantic association information, share these feature knowledge and function-level semantic details, help call matching feature learning, and play the effect of feature eavesdropping, and finally enable the model to learn and extract more comprehensive semantic features of Mashups and Web APIs.

[0102] For the sentence pairs consisting of Mashup and Web API, the BERT semantic feature extraction layer is used to extract its semantic features x, and the probability P that Mashup and Web API have a category matching relationship is calculated according to formula (1): c .

[0103] The cross entropy loss function CE shown in formula (2) is used to calculate the task loss.

[0104] where x i It represents the i-th sample data of the Mashup and Web API category matching score prediction task, that is, the sentence pair consisting of the Mashup and Web API description text, p is the probability distribution of the true result, and q is the probability distribution of the predicted result.

[0105] (4) Web API category prediction task layer

[0106] Since Web API description texts of the same category also have similar language descriptions, predicting the category to which the Web API belongs helps the model learn the semantic features of this part of the category level, discard other unimportant information such as noise in the description language, and achieve the effect of focusing attention. The present invention designs a Web API classification task, classifies Web API description documents to train the model, and improves the ability to capture category-level semantic information.

[0107] For the Web API description text, the semantic features x of the Web API description sentence extracted by the BERT semantic feature extraction layer are used to calculate the probability distribution P of the Web API belonging to each category according to formula (1). a =[P1,P2,P i ,...,P n ], where n is the total number of categories, P irepresents the probability that the Web API belongs to the i-th category. According to formula (3), P is selected a The category with the largest probability value As a result of the Web API category prediction.

[0108]

[0109] in, is the category prediction result, Indicates the selection of P a The index with the largest probability value.

[0110] The cross entropy loss function CE shown in formula (2) is used to calculate the task loss.

[0111] where x i represents the i-th sample data of the Web API category prediction task, that is, the Web API description text, p is the probability distribution of the true result, and q is the probability distribution of the predicted result.

[0112] (5)Mashup Category Prediction Task Layer

[0113] This task is similar to the Web API category prediction task. The design of this task enables the model to focus on category-level semantic information and simultaneously learn the contextual differences that are conducive to aligning Mashup requirements with Web API description text. The difference is that Mashup has diverse functions and may belong to multiple categories, so Mashup category prediction is designed as a multi-label classification task.

[0114] For the mashup description text, the semantic features x of the mashup description text generated by the BERT semantic feature extraction layer are used to predict the probability distribution P of the mashup belonging to each category according to formula (1). m =[P1,P2,P i ,...,P n ], where n is the total number of categories, P i Indicates the probability that the Mashup belongs to the i-th category.

[0115] Since this is a multi-label classification task and the number of categories is uncertain, we use the threshold setting method to determine the category to which the Mashup belongs. Set the threshold to θ, for P m The probability value P corresponding to the i-th category i , if P i >θ, then the Mashup is judged to belong to the i-th category.

[0116] This task uses the binary cross-entropy loss function BCE (Binary Cross-Entropy Loss Function) shown in formula (4) to calculate the task loss.

[0117]

[0118] Among them, y i It represents the true binary label of the i-th sample data of the Mashup category prediction task, which takes the value of 0 or 1. i is the probability of predicting the i-th sample as the positive class.

[0119] S5: Pre-training a Web API recommendation model based on a multi-task deep neural network.

[0120] In order to fully explore the semantic features of Mashup description documents and Web API description documents, obtain high-quality feature vectors and use them to calculate matching scores, it is necessary to pre-train the WebAPI recommendation model based on multi-task deep neural network constructed in step 4.

[0121] Assume that there are n different types of tasks, the current task is denoted as t, and the input data for pre-training is the sample data set T1, T2…, T generated in step 3 for each task. n The dataset T for task t t In the example, each sample r i Recorded as a triple (s i ,l i ,c i ), where i represents the i-th sample, s i Indicates the text data of the sample, l i Indicates the true value label of the sample, c i Indicates the task type of this sample.

[0122] The detailed process of pre-training the Web API recommendation model based on multi-task deep neural network is as follows.

[0123] In the pre-training phase, we first obtain samples of each task for pre-training, divide these samples into small batches, merge them all, and shuffle them. Then, we randomly select a small batch of data b t , input the text data set S of the batch samples into the semantic feature extraction layer based on BERT, extract the contextual semantic features of the text data, and obtain the semantic feature set X of the sample. According to the task type C recorded in the sample data in the batch, input X into the corresponding task layer to obtain the predicted value of the task. Finally, the loss function of the corresponding task type is used to calculate the task loss, and the stochastic gradient descent (SGD) method is used to update the model parameters until each b t After all epochs are selected, an epoch of pre-training is completed, and the pre-training ends after all epochs are completed.

[0124] S6: Fine-tune the Web API recommendation model using Mashup and API matching score prediction tasks.

[0125] The pre-trained Web API recommendation model can learn features that perform well on multiple tasks, but different tasks converge at different speeds. Some simple tasks converge quickly, while others converge slowly, resulting in some tasks being overfitted and some tasks still underfitted after a certain number of model training rounds. In order to maximize the accuracy of matching score predictions and make the model more focused on learning the fine matching features between Mashup requirements and Web API descriptions, thereby improving the accuracy of recommendations, the pre-trained WebAPI recommendation model is fine-tuned using the Mashup and Web API matching score prediction task to obtain the Mashup and Web API matching score prediction model. The detailed process is as follows:

[0126] (1) Obtain fine-tuning samples;

[0127] (2) Input the text data of the fine-tuning sample of the Mashup and Web API matching score prediction task into the BERT semantic feature extraction layer to extract the feature vectors of the Mashup text data and the Web API text data;

[0128] (3) Input the feature vectors of the Mashup text data and the Web API text data into the task layer to predict the matching score between the Mashup and the Web API;

[0129] (4) Calculate the task loss using the cross entropy loss function based on the predicted value and the actual value;

[0130] (5) Use stochastic gradient descent to update model parameters based on task loss;

[0131] (6) The steps described in (2) to (5) above are repeated until each sample has been selected, and one epoch of fine-tuning training is completed. After all epochs are completed, the fine-tuning is completed and the Mashup and Web API matching score prediction model is obtained.

[0132] S7: Predict Mashup requirements and match scores with each Web API.

[0133] The Mashup requirements and the description texts of each Web API are combined into sentence pairs and input into the fine-tuned Mashup and WebAPI matching score prediction model to obtain a matching score list of Mashup requirements and each Web API.

[0134] S8: Sort the matching scores of the Mashup requirements and each Web API to obtain the recommendation results.

[0135] In order to obtain the final recommendation results, it is necessary to sort the matching score list of the Mashup requirements and each Web API in descending order according to the matching score, and select the first K Web APIs to form the final recommendation list, where K is an arbitrarily set number of recommendations.

[0136] Example

[0137] In order to verify the effectiveness of the proposed method, a series of experiments were conducted on a real-world service dataset, and the recommendation results were evaluated using common evaluation metrics for Web API recommendation problems.

[0138] (1) Dataset selection and experimental setup

[0139] The data set used in this invention is real service data crawled from ProgrammableWeb, including the name, description document and category of Mashup and Web API. The Mashup data set includes the name, description document, category and the Web API name called by each Mashup, and the Web API data set includes the name, description document and category of Web API.

[0140] The present invention uses 80% of the data in the data set as a training set for model training, 10% of the data as a validation set for model parameter adjustment, and 10% of the data as a test set to verify the performance of the model.

[0141] (2) Evaluation indicators

[0142] The following are the formal definitions of the four evaluation indicators: accuracy (Precision), recall rate (Recall), F1 score (F1Score), and average accuracy of all categories MAP (Mean Average Precision).

[0143] 1) Precision: It is the ratio of the number of correctly recommended Web APIs to the number of all recommended Web APIs, also known as the precision rate, that is, how many of the recommended Web APIs are actually called by Mashup. The accuracy calculation formula is shown in formula (5), where R(u) is the recommendation result, and T(u) is the set of Web APIs actually called by Mashup on the test set. The larger the precision, the better the recommendation effect.

[0144]

[0145] 2) Recall: It is the ratio of the number of correctly recommended Web APIs to the number of Web APIs actually called by Mashup, also known as the recall rate, that is, how many Web APIs actually called by Mashup are recommended. The recall rate calculation formula is shown in formula (6), where R(u) is the recommendation result, and T(u) is the set of Web APIs actually called by Mashup on the test set. The larger the Recall, the better the recommendation effect.

[0146]

[0147] 3) F1: F1 is the weighted harmonic mean of Precision and Recall, and is calculated as shown in formula (7), where P is the precision and R is the recall. Since there is a negative correlation between precision and recall, and they depend on the length of the recommendation list and are mutually restrained, the F1 indicator is used to comprehensively consider the precision and recall. The larger the F1, the better the recommendation effect.

[0148]

[0149] 4) MAP: It is sensitive to the order of recommendation. The calculation method is shown in formula (8), where K is the length of the recommendation list, m is the number of correctly recommended Web APIs, P(k) refers to the precision from the Web API ranked 1 to the Web API ranked k in the recommendation list, and rel(k) indicates whether the Web API ranked k is correctly recommended. The larger the MAP, the better the recommendation effect.

[0150]

[0151] (3) Experimental comparative analysis

[0152] The present invention conducts comparative experiments from two aspects. The first part is an ablation experiment, which compares the Web API recommendation method based on multi-task deep neural network proposed in the present invention with the single-task recommendation method. The second part is a conventional experimental comparative analysis, which compares the method proposed in the present invention with the current mainstream method.

[0153] 1) Ablation experiment

[0154] The present invention uses the idea of ​​multi-task learning, designs multiple related tasks, constructs a multi-task deep neural network to pre-train the Web API recommendation model, and uses the Mashup and Web API matching scoring tasks as downstream tasks for fine-tuning to improve the accuracy of scoring prediction, thereby improving the recommendation performance. Therefore, in order to verify the effectiveness of the method proposed in the present invention, an ablation experiment is designed to compare the performance of the multi-task method with the single-task method. Usually, Mashup calls about 4 WebAPIs, so the recommendation list length K is most representative when it is 4. The experimental results are shown in Table 1.

[0155] Table 1 Ablation experiment results

[0156]

[0157] It can be seen from Table 1 that when K=4, the method proposed in the present invention has improvements in all indicators, with the recall rate increased by 12.24%, the precision rate increased by 3.72%, the F1 score increased by 5.50%, and the MAP increased by 13.30%, indicating that the method proposed in the present invention has a better recommendation effect. This is because the present invention designs multiple related tasks to construct a multi-task deep neural network. Compared with the model that only relies on a single task, implicit data enhancement can be achieved in the pre-training stage. Without directly adding additional data sets, the training samples of the model are expanded, the data diversity is increased, and richer contextual information is provided for model learning; and the introduction of category-related tasks and interaction-related tasks for simultaneous learning enables the model to not only learn functional-level semantic feature representations, but also deeply explore high-order category-level semantic associations. This refined feature learning mechanism enables the model to obtain higher-quality semantic feature representations; finally, the Web API recommendation model is further fine-tuned through the Mashup and Web API matching scoring task to obtain the Mashup and Web API matching scoring prediction model, which specifically optimizes the scoring accuracy of the model in specific recommendation scenarios to ensure that the model performs optimally in actual scoring predictions, and ultimately achieves improvements in performance indicators such as recommendation recall and accuracy.

[0158] 2) Comparative experiment

[0159] In order to verify the effectiveness of the proposed method, the latest Mashup-oriented Web API recommendation method (MTFM) based on multi-model fusion and multi-task learning is selected as the baseline method. MTFM first uses semantic components and feature interaction components to extract features respectively, then splices the two extracted features, and finally uses the spliced ​​features for the training of two multi-label classification tasks: Web API recommendation and Mashup classification, so that the model has multi-task learning capabilities.

[0160] In the recommendation scenario, the length of the recommendation list K greatly affects the evaluation of the recommendation effect. Smaller K values ​​can evaluate indicators when fewer recommendation targets are provided, which is very important for users who want to choose from a small number of high-confidence recommendations; larger K values ​​can evaluate the performance when providing longer recommendation lists, which is meaningful for users who need more choices or want to explore more possibilities. Therefore, in order to comprehensively evaluate the performance of the two methods at different recommendation list lengths, the performance of the Recall indicator, Precision indicator, F1 indicator and MAP indicator are compared when the recommendation list length K is 1, 2, 3, 4, 5, 10, 20, and 30 respectively. The comparison results are shown below. Figure 2-Figure 5 shown.

[0161] It can be seen that when the length K of the recommendation list is 1, 2, 3, 4, 5, 10, 20, and 30, the recall rate of the present invention is increased by 11.94%, 12.93%, 14.88%, 17.11%, 18.05%, 14.56%, 16.2%, and 16.05%, respectively, with an average increase of 15.22%; the accuracy rate is increased by 7.03%, 2.55%, 1.74%, 2.52%, 1.96%, 0.82%, and 0.91%, respectively. 0.57%, 0.24%, an average increase of 2.18%; F1 score increased by 8.39%, 4.17%, 3.14%, 3.60%, 2.97%, 1.07%, 0.86%, 0.37%, an average increase of 3.07%; MAP increased by 7.03%, 6.44%, 8.20%, 8.10%, 8.75%, 8.52%, 8.03%, 8.72%, an average increase of 7.97%. It can be seen that when K is between 1 and 5, as the length of the recommendation list increases, the increase in recall rate increases continuously. When K is 10, the increase decreases because the recall rate is already high. However, the accuracy and F1 score decrease as the K value increases, so the improvement of the two indicators decreases as the K value increases; the improvement of MAP fluctuates between 7.03% and 8.75%. In general, when the number of recommendations K is different, the proposed method has significant performance advantages in the four evaluation indicators of recall rate, accuracy, F1 score and MAP, which proves the effectiveness of the Web API recommendation method proposed in the present invention. The main reasons are as follows: First, the present invention enables multiple tasks to be learned simultaneously, which can realize implicit data enhancement and expand the number of samples. Compared with MTFM, which only mines interactive features and functional semantic features, the method of the present invention mines more fine-grained category-level semantic association information in semantic information, which can optimize the model learning ability and extract higher quality features; secondly, the multi-task framework designed by the present invention can synchronously integrate the learning process of interactive information and deep semantic information. Compared with MTFM, which only uses Mashup classification as an auxiliary task to utilize the characteristics of multi-task learning inductive bias, the method of the present invention simultaneously learns the category-level semantic association information and sparse function-level semantic association information between Mashup and Web API, and can use the characteristics of multi-task learning attention focus and feature stealing to prompt the model to focus on key category-level semantic associations, while sharing these feature knowledge and function-level semantic details, increasing the score prediction accuracy of the model, thereby improving the recommendation performance of the model.

[0162] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit it. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or replace some or all of the technical features therein with equivalents. However, these modifications or replacements do not cause the essence of the corresponding technical solutions to deviate from the scope of the technical solutions of the embodiments of the present invention.

Claims

1. A Web API recommendation method based on multi-task deep neural network, characterized in that: The steps include: S1. Obtain a Mashup and Web API dataset, wherein the dataset includes Mashup name, Mashup category, Mashup description document, Web API called by Mashup, Web API name, Web API category, and Web API description document; and clean the Mashup description document and Web API description document in the dataset, and delete stop words, emoticons, and special symbols in the document; S2, mapping the text information in the cleaned data set into digital codes; S3, generating sample data of multiple tasks based on digital coding; the multiple tasks include Mashup and Web API call matching score prediction task, Mashup and Web API category matching score prediction task, Web API category prediction task, Mashup category prediction task; S4. Construct a multi-task deep neural network based on multiple tasks; the multi-task deep neural network includes a text semantic feature extraction layer based on BERT, a Mashup and Web API call matching score prediction task layer, a Mashup and Web API category matching score prediction task layer, a Mashup and Web API category matching score prediction task layer, a Web A PI category prediction task layer, and a Mashup category prediction task layer; construct a Web API recommendation model based on a multi-task deep neural network; S5. Input the sample data into the Web API recommendation model for pre-training to obtain a pre-trained Web API recommendation model; S6. Fine-tune the pre-trained Web API recommendation model using Mashup and API matching score prediction tasks; S7, the Mashup requirements and the description texts of each Web API are combined into sentence pairs, and the pairs are input into the fine-tuned Mashup and WebAPI matching score prediction model to obtain a matching score list of the Mashup requirements and each Web API; S8. Sort the matching score list between the mashup requirements and each Web API in descending order according to the matching score, and select the top K Web APIs to form the final recommendation list.

2. The Web API recommendation method based on a multi-task deep neural network according to claim 1, characterized in that: S2 specifically includes the following steps: Convert the service category, Web API name, and Mashup name into digital form, map the service category into the service category label, map the Web API name into the Web API code, and map the Mashup name into the Mashup code; According to the calling relationship between Mashup and Web API, a Mashup-Web API calling relationship encoding table is generated; the word segmentation tool of the BERT model is used to perform word segmentation processing on the Mashup description document and the Web API description document to obtain the word segmentation encoding of the Mashup description document and the Web API description document.

3. The Web API recommendation method based on multi-task deep neural network according to claim 1 is characterized in that: S3 specifically includes the following steps: Combine the Mashup code, the word segmentation code of the Mashup description document, the Mashup category label and the task category tag to generate training samples for the Mashup category prediction task; Combine the Web API code, the word segmentation code of the Web API description document, the Web API category label, and the task category tag to generate training samples for the Web API category prediction task; Combine the sentence pairs consisting of Mashup code, Web API code, Mashup description document code and Web API description document code, the corresponding category matching labels and task category tags to obtain the training samples of the Mashup and Web API category matching prediction task, where the category matching label is 1, indicating that the Mashup and Web API categories match, and the category label is 0, indicating that the Mashup and Web API categories do not match. The sentence pairs consisting of Mashup encoding, Web API encoding, Mashup description document encoding and Web API description document encoding, the corresponding call relationship labels and task category labels are combined to obtain the training samples of the Mashup and Web API call matching prediction task, where the call relationship label is 1, which means that the Mashup and Web API have a call relationship, and 0, which means that the Mashup and Web API have no call relationship.

4. The Web API recommendation method based on multi-task deep neural network according to claim 1 is characterized in that: In S4, the Web API recommendation model based on multi-task deep neural network includes a text semantic feature extraction layer based on BERT and four task layers, the four task layers are a Mashup and Web API call matching score prediction task layer, a Mashup and Web API category matching score prediction task layer, a Mashup and Web API category matching score prediction task layer, a Web A PI category prediction task layer and a Mashup category prediction task layer; Among them, the four task layers share the parameters of the BERT-based text semantic feature extraction layer; The four task layers are fully connected layers, and the corresponding task prediction results P are calculated according to the following formula t : Among them, x is the semantic feature output by the text semantic feature extraction layer based on BERT, t is the task type, and W t is the parameter matrix of the fully connected layer corresponding to task t.

5. The Web API recommendation method based on multi-task deep neural network according to claim 3 is characterized in that: In S4: BERT-based text semantic feature extraction layer: The BERT-based text semantic feature extraction layer uses BERT to mine the semantic features of Mashup or Web API to describe text sentences; Mashup and Web API call matching score prediction task layer: The Mashup and Web API call matching score prediction task layer uses Mashup and Web API with call relationship as positive samples and Mashup and Web API without call relationship as negative samples, and uses the actual call matching relationship between the two to assist the model in learning function-level semantic features; For the sentence pairs composed of Mashup and Web API description texts, the BERT semantic feature extraction layer is used to extract their semantic features x, and the probability P of Mashup and Web API having a call matching relationship is calculated according to formula (1): r , and use the cross entropy loss function shown in formula (2) to calculate the task loss and adjust the parameter matrix of the network corresponding to the task layer according to the loss; Among them, x i represents the i-th sample data of the Mashup and Web API call matching score prediction task, that is, the sentence pair consisting of the Mashup and Web API description text, p is the probability distribution of the real result, and q is the probability distribution of the predicted result; Mashup and Web API category matching score prediction task layer: For the sentence pairs consisting of Mashup and Web API, the BERT semantic feature extraction layer is used to extract their semantic features x, and the probability P that Mashup and Web API have a category matching relationship is calculated according to formula (1) c ; The task loss is calculated using the cross entropy loss function shown in formula (2); Among them, x i represents the i-th sample data of the Mashup and Web API category matching score prediction task, that is, the sentence pair consisting of the Mashup and Web API description text, p is the probability distribution of the true result, and q is the probability distribution of the predicted result; Web API category prediction task layer: For the Web API description text, the semantic features x of the WebAPI description sentence extracted by the BERT semantic feature extraction layer are used to calculate the probability distribution P of the Web API belonging to each category according to formula (1): a =[P1,P2,P i ,...,P n ], where n is the total number of categories, P i represents the probability that the Web API belongs to the i-th category. According to formula (3), P is selected a The category with the largest probability value As a result of the Web API category prediction. in, is the category prediction result, Indicates the selection of P a The index with the largest probability value; The task loss is calculated using the cross entropy loss function shown in formula (2). Among them, x i represents the i-th sample data of the Web API category prediction task, that is, the Web API description text, p is the probability distribution of the true result, and q is the probability distribution of the predicted result; Mashup category prediction task layer: For the mashup description text, the semantic features x of the mashup description text generated by the BERT semantic feature extraction layer are used to predict the probability distribution P of the mashup belonging to each category according to formula (1). m =[P1,P2,P i ,...,P n ], where n is the total number of categories, P i Indicates the probability that the Mashup belongs to the i-th category; The category of Mashup is determined by setting a threshold. The threshold is set to θ. m The probability value P corresponding to the i-th category i , if P i >θ, then the Mashup is judged to belong to the i-th category; This task uses the binary cross entropy loss function shown in formula (4) to calculate the task loss: Among them, y i It represents the true binary label of the i-th sample data of the Mashup category prediction task, which takes the value of 0 or 1. i is the probability of predicting the i-th sample as the positive class.

6. The Web API recommendation method based on multi-task deep neural network according to claim 1 is characterized in that: S5 specifically includes the following steps: Assume that there are n different types of tasks, the current task is denoted as t, and the input data for pre-training is the sample data set T1, T2…, T generated in S3 for each task. n , dataset T for task t t In the example, each sample r i Recorded as a triple (s i ,l i ,c i ), where i represents the i-th sample, s i Indicates the text data of the sample, l i Indicates the true value label of the sample, c i Indicates the task type of this sample; Obtain samples of each task for pre-training, divide these samples into small batches, and then merge and shuffle them all; Randomly select a small batch of data b t , input the text data set S of the batch samples into the semantic feature extraction layer based on BERT, extract the contextual semantic features of the text data, and obtain the semantic feature set X of the sample. According to the task type C recorded in the sample data in the batch, input X into the corresponding task layer to obtain the predicted value of the task. The loss function of the corresponding task type is used to calculate the task loss, and the model parameters are updated using the stochastic gradient descent method until each b t After all epochs are selected, an epoch of pre-training is completed, and the pre-training ends after all epochs are completed.

7. The Web API recommendation method based on multi-task deep neural network according to claim 1 is characterized in that: S6 includes the following steps: S61, obtain Mashup and Web API matching score prediction task fine-tuning samples; S62, inputting the text data of the fine-tuning sample of the Mashup and Web API matching score prediction task into the BERT-based semantic feature extraction layer to extract feature vectors of the Mashup text data and the Web API text data; S63, inputting the feature vectors of the Mashup text data and the Web API text data into the task layer, and predicting the matching score between the Mashup and the Web API; S64. Calculate the task loss using the cross entropy loss function based on the predicted value and the actual value; S65, using stochastic gradient descent method to update model parameters according to task loss; S66, loop S62 to S65 until each sample has been selected, and then one epoch of fine-tuning training is completed. After all epochs are completed, the fine-tuning is completed and a Mashup and Web API matching score prediction model is obtained.

8. A storage medium, characterized in that: The storage medium includes a stored program, wherein when the program is run, the Web API recommendation method based on a multi-task deep neural network described in any one of claims 1 to 7 is executed.

9. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein: The processor executes the Web API recommendation method based on a multi-task deep neural network as described in any one of claims 1 to 7 through the computer program.

Images