A mashup service multi-label classification method based on double manifold regularization width learning

Abstract

The application provides a Mashup service multi-label classification method based on double manifold regularization width learning, mainly comprising: using a hidden Dirichlet distribution topic model to extract features from preprocessed Mashup description documents; linearly mapping a Mashup description document topic feature matrix into n groups of feature nodes respectively; processing the feature nodes through an activation function to generate enhanced nodes; splicing the feature nodes and the enhanced nodes to generate enhanced feature nodes as the input of the model; constructing a target function of a Mashup service multi-label classification model based on double manifold regularization width learning; using a least square method to solve the target function to obtain a weight matrix of a double manifold regularization width learning network; obtaining a description document of a test Mashup service and sending it into a trained model to predict a multi-label classification result. The application improves a width learning model by using double manifold regularization, and realizes a Mashup service multi-label classification function by using an improved BLS model.

Description

TECHNICAL FIELD

[0001] The application relates to the technical field of multi-label classification of Mashup services, in particular to a Mashup service multi-label classification method based on double-manifold regularization width learning. BACKGROUND

[0002] As the research object of service computing, Web services have been expanded in terms of quantity and category, and the number of new services such as API (Application Programming Interface) services and Mashup services has rapidly increased in the information explosion era. As an aggregation model, Mashup services allow software developers to integrate a plurality of publicly published Web APIs to generate reusable, lightweight and user-centered application programs, and build multifunctional service applications to avoid the limitation of single function of API services. However, it is very difficult and time-consuming for Mashup developers to select multiple services meeting the complex function description from a large number of Web services. For example, a Mashup developer wants to create a Mashup service capable of locating a restaurant on a map and seeing the evaluation of the restaurant by other people, and according to the function description, it can be inferred that the sub-services meeting the requirement can be found with a high probability by screening the services in the service categories of "map" and "social". Therefore, if the function description document of the Mashup to be created can be analyzed to find the service categories involved in the function, the search space of Web services can be greatly reduced, and the services in each service category cluster can be more conveniently used, so that the subsequent tasks such as service discovery, service combination and service recommendation become more reasonable and efficient.

[0003] The multi-label classification task of text is a classic problem in the field of natural language processing. A piece of text can correspond to multiple labels. For example, a piece of text describing intelligent medical treatment belongs to both the medical category and the science and technology category. The function description document of Mashup is described by natural language, and the composite function corresponds to multiple different service category labels. For service development registrants, the Mashup service multi-label classification algorithm can quickly label the service categories of Mashup, avoiding the tediousness of manual labeling and the ambiguity problem caused by language gap. In addition, due to the problems of failure and migration of Mashup sub-services, through service category labeling, the developer can replace the services in the determined service category, reduce the service search space and improve the efficiency of service retrieval and recommendation. For service calling personnel, the main function of each Mashup can be intuitively understood through the label, and the service can be filtered to speed up the screening of Mashup services meeting the requirement.

[0004] Currently, multi-label classification methods mainly include two categories: algorithm adaptation and problem transformation.

[0005] Problem transformation converts multi-label classification problem into multiple single-label classification problems, reconstructs multi-label data set, so that data can be directly processed by existing traditional single-label classification algorithm, and data is adapted to algorithm. Common conversion methods include binary relevance (BR) and classifier chain (CC). BR algorithm assigns a binary classifier to each label, thereby decomposing multi-label classification problem into multiple independent binary classification problems, each classifier can be independently trained, and finally the prediction label set of unknown sample is obtained. This method is intuitive and efficient, but ignores the correlation between labels, resulting in reduced classification accuracy. CC algorithm solves the problem of lack of dependence between labels due to the independent assumption of BR method, and links multiple binary classifiers one by one, and the training of each classifier is based on the input data and all classifiers before the chain. However, the classifiers in the chain are randomly arranged, so the relationship between the labels is randomly calculated; in addition, if the prediction performance of the previous classifier is poor, the error will continue to spread as the chain extends.

[0006] The algorithm adaptive method extends the traditional single-label classification algorithm, so that the improved algorithm can directly solve the multi-label classification problem. According to the data characteristics, the algorithm is designed to directly process the multi-label data set. Common algorithms include ML-KNN (Multi-Label K-Nearest Neighbor), Rank-SVM (Ranking Support Vector Machine), and ML-ELM (Multi-Label Extreme Learning Machine). ML-KNN uses the K-nearest neighbor algorithm to obtain the label distribution of the neighbor data, then according to the label of the neighbor sample, the number of neighbor samples belonging to a certain label is counted, and finally the label of the unknown sample is inferred and predicted by maximizing the posterior probability. Rank-SVM adds a ranking loss function with a multi-label model to the original SVM as a constraint condition to handle multi-label classification problems. The above two adaptive algorithms are simple to implement and are not affected by class imbalance. However, due to the characteristics of the algorithm itself, the computational complexity is large. In order to solve the problem of large time consumption, the multi-label classification algorithm ML-ELM based on extreme learning machine is proposed and widely applied. This method designs the network as a single hidden layer feedforward neural network, uses Moore-Penrose generalized inverse to solve the weight coefficients between the hidden layer and the output layer, and performs binary classification on each label in the output layer through the sigmoid function. This model structure is simple in design and has a unique global parameter solution, but only linear or nonlinear single features of the data can be used in the hidden layer, resulting in low generalization performance.

[0007] With the development of deep learning technology, the problem of multi-label classification is solved rapidly by using deep neural network technologies such as deep belief network DBN (Deep Belief Network), recurrent neural network RNN (Recurrent Neural Network), text convolutional neural network TextCNN (Text Convolutional Netural Network) and the like. In order to extract deeper features of data, these neural networks usually extract deep features by deepening the network layers to achieve good approximation results, but the increase of network layers will inevitably increase the parameters of each layer of neurons, which requires a large number of manual parameter adjustment operations. In addition, since these networks use back propagation BP (Back Propagation) to update the weight parameters to minimize the error between the actual output and the target output, the multi-layer parameters are updated by the gradient descent method, which may cause the problems of gradient explosion or disappearance and the model being easily trapped in a local optimal solution. Broad learning system (BLS) aims to achieve the fitting of results by expanding the network width instead of deepening the network layers, and the overall structure of the network is similar to ELM, which has an input layer, a hidden layer (mapping layer) and an output layer. The neurons between each layer are fully connected, the output weight coefficient is solved by solving the pseudo-inverse matrix to avoid the problem of being trapped in a local optimal solution by solving the parameters layer by layer, and the problems of gradient explosion or disappearance will not occur. Therefore, BLS has been widely used in image classification, pattern recognition, anomaly detection and other fields due to its fast and convenient parameter solving method and simple network structure. Broad learning system generates feature nodes by randomly linearly mapping the original input data, and then generates enhanced nodes by non-linearly mapping the feature nodes. The two parts of nodes are collectively used as the input of the final network and are directly fully connected with the output layer. Finally, the parameters of the model are learned by solving the pseudo-inverse matrix between the input nodes and the output labels. However, this method has the following shortcomings: the random mapping often makes the data in the new space present a non-linear distribution that is difficult to predict; and it only solves the target parameters by minimizing the training error, and cannot mine the non-linear geometric structure of the mapped data, which will affect the accuracy of the final classification. SUMMARY

[0008] In view of the deficiencies of the prior art, the present application provides a Mashup service multi-label classification method based on double manifold regularization broad learning, which improves the broad learning model by using double manifold regularization, and realizes the Mashup service multi-label classification function of topic-service category label matching by using the improved BLS model.

[0009] The technical means adopted by the present application are as follows:

[0010] The application discloses a Mashup service multi-label classification method based on double-manifold regularized width learning, and comprises the following steps:

[0011] A Mashup service dataset is acquired, wherein the Mashup service dataset comprises the name, description document, API service called respectively and the name and category of the API service of the Mashup service, and the description document and binary output label of the Mashup service are preprocessed;

[0012] An implicit Dirichlet distribution topic model is used to extract features from the preprocessed Mashup description document, so as to generate a Mashup description document topic distribution vector;

[0013] The Mashup description document topic distribution vector is taken as original input data of width learning, and a linear transformation function and n sets of parameter matrices of random mapping are used to linearly map a Mashup description document topic feature matrix into n sets of feature nodes respectively;

[0014] The feature nodes are processed through a nonlinear activation function, so as to generate enhanced nodes;

[0015] The feature nodes and the enhanced nodes are spliced to generate enhanced feature nodes, and the enhanced feature nodes are taken as input of a Mashup service multi-label classification model based on double-manifold regularized width learning;

[0016] A target function of the Mashup service multi-label classification model based on double-manifold regularized width learning is constructed by using a manifold regularization idea;

[0017] The target function is solved by using a least square method, a weight matrix of a double-manifold regularized width learning network is acquired, and training of the Mashup service multi-label classification model based on double-manifold regularized width learning is completed;

[0018] A description document of a test Mashup service is acquired, text preprocessing and topic feature extraction are performed, a topic distribution vector matrix of the test Mashup service is obtained, the topic distribution vector matrix is mapped to generate enhanced feature nodes, and finally the input of a test sample is obtained, and then the input is sent into the Mashup service multi-label classification model based on double-manifold regularized width learning which has been trained, so that the classification result of the Mashup service is predicted.

[0019] Further, the preprocessing of the description document and the binary output label of the Mashup service comprises the following steps:

[0020] All letters in the description document are converted into lowercase letters;

[0021] Remove punctuation marks and meaningless special symbols in the description document;

[0022] Remove stop words in the description document using the corpus in the nltk package;

[0023] Convert the complex form of the word to the basic form based on the dictionary using the WordNetLemmatizer tool in the nltk package;

[0024] Convert the crawled Mashup call record to 0-1 digital, generate Mashup-service category call matrix Y=(y ij ) MN×CN , where MN is the number of Mashup services, CN is the number of service categories, y ij =1 indicates that Mashup service M i calls a service in category j, y ij =0 indicates that no service in category j is called.

[0025] Further, the preprocessed Mashup description document is extracted using the latent Dirichlet distribution topic model, and then the Mashup description document topic feature matrix is generated, including:

[0026] The function description document set of Mashup is used as the corpus of the latent Dirichlet distribution topic model;

[0027] After training using the latent Dirichlet distribution topic model, the topic set {K1,K2,…,K T} is obtained, where T represents the number of topics set in the latent Dirichlet distribution topic model, and each Mashup description document corresponds to a probability distribution on the topic set {K1,K2,…,K T}, that is, the topic feature of the i-th Mashup description document is represented as a 1×T vector Further, the topic distribution vector of the entire corpus is obtained

[0028] Further, the feature node is obtained according to the following formula:

[0029] Z n ≡[Z1,…,Z n ](1)

[0030] Where Z i represents the i-th group of feature nodes, containing k feature nodes, n×k is the total number of feature nodes,

[0031]

[0032] where φ(·) is a linear mapping function, and The weight coefficient matrix and bias matrix of the i-th group of feature nodes are generated by random mapping and conform to normal distribution.

[0033] Further, the enhanced nodes are obtained according to the following formula:

[0034] H m = ξ(Z n · W h + β h ) (3)

[0035] where m is the number of enhanced nodes, generated by the feature nodes once, and respectively as the weight coefficient matrix and bias matrix of the enhanced nodes, generated by random mapping and conform to normal distribution, and then orthogonal normalized, and ξ(·) is a nonlinear activation function.

[0036] Further, the enhanced feature nodes are obtained according to the following formula:

[0037] X = [Z n | H m ] (4)

[0038] where is the enhanced feature node matrix, the rows of which represent the number of samples, and the columns represent the total number of network nodes;

[0039] Therefore, the input and output of the Mashup service multi-label classification model based on the dual manifold regularization width learning can be associated by the following formula:

[0040] Y = Xβ (5)

[0041] where Y is the service category binary multi-label vector output by the Mashup service multi-label classification model based on the dual manifold regularization width learning, and β is the weight matrix that the width learning network needs to learn.

[0042] Further, the objective function of the Mashup service multi-label classification model based on the dual manifold regularization width learning is constructed by fusing the dual manifold regularization, including:

[0043] The data manifold regularization term of the enhanced feature node X is constructed, specifically including constructing the data manifold regularization constraint of the enhanced feature node X as:

[0044]

[0045] where x i,： and x j,：is the i-th row and the j-th row of the enhanced feature node X, representing the training sample, y i and y j is the label of the sample x i,： and x j,： , corresponding to the embedding of the sample in the high-dimensional space in the low-dimensional label space, W ij represents the similarity between the samples x i,： and x j,： , the p nearest neighbor samples to the sample are determined by the K nearest neighbor algorithm, and the Gaussian kernel function is used to calculate:

[0046]

[0047] W ij is calculated according to the following formula:

[0048]

[0049] where N p (xi,:) represents the p nearest neighbors of the sample xi,:, and t is the bandwidth parameter of the Gaussian kernel function, which is used to control the local action range of the function;

[0050] The feature manifold regularization term of the enhanced feature node X is constructed, which specifically includes constructing the similarity matrix S between the features:

[0051]

[0052] where x i,： and x j,： are the i-th column and the j-th column of the enhanced feature node X, representing the sample features;

[0053] The feature manifold regularization constraint of the enhanced feature node X is to minimize the distance between the weights β :,i and β :,j corresponding to the approximate data features x i and x j , which can be expressed as:

[0054]

[0055] The objective function of the Mashup service multi-label classification model based on the double manifold regularization width learning is constructed as:

[0056]

[0057] where, is the quadratic norm, λ is the L2 regularization factor of the width learning, which is used to balance the empirical risk and the structural risk, C1 is the penalty factor of the data manifold regularization, and C2 is the penalty factor of the feature manifold regularization;

[0058] The target function of the Mashup service multi-label classification model based on double manifold regularized width learning is simplified, and the simplified target function is:

[0059]

[0060] Wherein, trace(·) represents the trace operation of a matrix, L1 and L2 are Laplacian matrices of data and features, which can be obtained from data similarity matrix W and feature similarity matrix S respectively.

[0061] Further, the least square method is used to solve the target function, and the weight matrix of the double manifold regularized width learning network is obtained, and the training of the Mashup service multi-label classification model based on double manifold regularized width learning is completed, including:

[0062] The derivative of the target function with respect to β is obtained, and the derivative is 0, and the following is obtained:

[0063] β=(X T X+λI+C1X T L1X+C2L2) -1 X T Y (13)

[0064] Wherein, β is the weight matrix of the double manifold regularized width learning network.

[0065] Further, the classification result of the Mashup service is obtained according to the following formula:

[0066] Pred=X * β (14)

[0067] Wherein, Pred is the classification result of the Mashup service, β is the weight matrix of the double manifold regularized width learning network, X * is the final input of the test sample.

[0068] Compared with the prior art, the present application has the following advantages:

[0069] The present application improves the original width learning in the presence of problems in Mashup service multi-label classification. In the original width learning model, the constraints of data manifold and feature manifold are added, the row and column information of the input sample matrix is fully utilized, the local geometric structure of data and features is maintained, the width learning model parameters solved are more accurate, and the model classification effect is improved. The experiment proves that the improved double manifold regularized width learning Mashup service multi-label classification method of the present application is not only superior to the original width learning, but also has obvious advantages compared with other commonly used multi-label classification algorithms. BRIEF DESCRIPTION OF DRAWINGS

[0070] In order to more clearly illustrate the technical solutions in the embodiments of the present application or the prior art, the following will briefly introduce the drawings needed to be used in the embodiments or prior art description. Obviously, the drawings in the following description are some embodiments of the present application, and other drawings can be obtained by those skilled in the art without any creative effort based on these drawings.

[0071] Figure 1 Flow chart of the Mashup service multi-label classification method based on double manifold regularization width learning of the present application.

[0072] Figure 2 MLMS-DMBLS network model diagram of the present application. DETAILED DESCRIPTION

[0073] In order to make the person skilled in the art better understand the present application scheme, the following will combine the drawings in the embodiments of the present application to clearly and completely describe the technical solutions in the embodiments of the present application. Obviously, the described embodiments are only some of the embodiments of the present application, not all. Based on the embodiments in the present application, all other embodiments obtained by those skilled in the art without any creative effort shall belong to the scope of protection of the present application.

[0074] In view of the problems that the data existing in the width learning presents unpredictable nonlinear distribution in the new space, and the final classification accuracy is not high due to solving the target parameters by minimizing the training error, the width learning model is improved by using the idea of manifold learning, and the improved width learning model is used for Mashup service multi-label classification. The manifold learning assumes that some data is embedded in a low-dimensional manifold in a high-dimensional space, and its goal is to mine the nonlinear geometric structure of the data when the high-dimensional data is mapped to the low-dimensional space, and to preserve the original local relationship of the adjacent points. The manifold regularization uses the manifold learning to enable the data to maintain the original geometric distribution in the new feature space, that is, the sample points that are very close in the original data space can also be very close in the projected label space, so as to solve the problem of loss of classification accuracy caused by the random mapping in BLS which cannot restore the geometric distribution of the feature space data. The Mashup service description data belongs to the text data which has been proved to meet the above manifold assumption. The feature attribute of the Mashup service description data is used, the local manifold structure constraint of the feature is introduced, it is considered that the feature vectors with similar relationship should also have similar weight parameters, and the double manifold regularization constraints of the data and the feature are integrated into the target function of the width learning, so that the structural features of the Mashup service description can be maximized, the local geometric structure of the sample and the feature can be mined, and the accuracy of the Mashup service multi-label classification using the width learning is improved.

[0075] The application provides a Mashup service multi-label classification method MLMS-DMBLS (Multi-label Classification of Mashup Services Based on Double Manifold Broad Learning System) based on double manifold regularization width learning, and a process of the method is as shown in the figure. Figure 1 The model input is a topic vector of a Mashup description document, and the output is a matching probability of each service category label, the width learning model is improved by using double manifold regularization, and the improved BLS model is used to realize the Mashup service multi-label classification function of topic-service category label matching. Specifically, the following steps are included.

[0076] (1) Training phase

[0077] Step one: Mashup description document and label preprocessing.

[0078] Firstly, we crawled the real Mashup service dataset from ProgrammableWeb platform, which contains the name of Mashup service, description document, API services called by each Mashup service and the name of API service, description document and the category of API service. Assume the number of Mashup service is MN, and the number of service category is CN.

[0079] In order to obtain the relevant topic features of Mashup service, we need to preprocess the description document of Mashup service, the detailed process is as follows:

[0080] 1) Convert all letters to lowercase;

[0081] 2) Remove punctuation marks and meaningless special symbols, such as ', ','; ','$ ','【 ','& ', etc.

[0082] 3) Use the corpus corpus in nltk package to remove stop words in English that often appear but have no actual meaning, such as'of ','by ','about ','under ', etc.

[0083] 4) Use the WordNetLemmatizer tool in nltk package to convert the complex form of the word into the basic form based on the dictionary, such as'does ','did ','done'unified to'do ','men'to'man ', etc., which is convenient for subsequent semantic analysis of computer.

[0084] In order to obtain the binary output label of MLMS-DMBLS, we need to convert the crawled Mashup calling record to 0-1 number, and generate Mashup-service category calling matrix Y = (y ij ) MN×CN , where MN is the number of Mashup service, CN is the number of service category, y ij = 1 indicates that Mashup service M i called a service in category j, y ij = 0 indicates that no service in category j is called.

[0085] Step two: Mashup description document topic feature extraction based on LDA model.

[0086] In order to fully understand the semantic characteristics of Mashup service function description documents and map them to different service category labels, this paper uses the Latent Dirichlet Allocation (LDA) topic model to mine the potential topic vectors of the pre-processed Mashup service description documents. A Mashup service usually corresponds to multiple abstract topics. If the weight of a service topic vector on a certain topic exceeds a threshold, it can be inferred that part of the function of the service will correspond to a service category similar to this topic. The set of Mashup function description documents is used as the corpus of the topic model, and after training the LDA model, a set of topics {K1, K2, …, K T} is obtained, where T represents the number of topics set in the LDA model. Each Mashup description document corresponds to a probability distribution over the topic set {K1, K2, …, K T}, i.e. the topic characteristics of the i-th Mashup description document can be represented as a 1 × T vector The topic distribution vector of the entire corpus can be represented as a two-dimensional matrix

[0087] Step three: generate feature nodes.

[0088] The original data input by the width learning is a two-dimensional matrix θ composed of Mashup description document topic vectors. The linear transformation function and the parameter matrix of n groups of random mapping are used to linearly map θ into n groups of feature nodes Z n , i.e.

[0089] Z n ≡ [Z1, …, Z n ] (1)

[0090] where Z i represents the i-th group of feature nodes, containing k feature nodes. n × k is the total number of feature nodes, which is denoted as nk in the following.

[0091]

[0092] φ(·) is a linear mapping function, and as the weight coefficient matrix and bias matrix of the i-th group of feature nodes, are generated by random mapping and conform to the normal distribution.

[0093] Step four: generate enhanced nodes.

[0094] The nonlinear features of the data are obtained by using the feature nodes in equation (1) through the activation function, thereby generating enhanced nodes H m , i.e.

[0095] H m = ξ(Z n · W h + β h ) (3)

[0096] where m is the number of augmented nodes, which are generated by the feature nodes once, and are the weight coefficient matrix and bias matrix of the augmented nodes, respectively, which are generated by random mapping and conform to normal distribution, and then are orthonormalized, and ξ(·) is a nonlinear activation function.

[0097] Step five: concatenating the feature nodes and the augmented nodes.

[0098] The feature nodes Z n and the augmented nodes H m are merged to obtain the augmented feature nodes X, see equation (4), and are input into the MLMS-DMBLS.

[0099] X = [Z n | H m ] (4)

[0100] The rows of the matrix represent the number of samples, and the columns represent the total number of network nodes. The network output of the MLMS-DMBLS is a binary multi-label vector Y of service categories, which can be represented as:

[0101] Y = Xβ (5)

[0102] β is the weight matrix that the wide learning network needs to learn. The network model of the MLMS-DMBLS is shown in Figure 2 .

[0103] Step six: fusion of double manifold regularization to construct the objective function of the MLMS-DMBLS.

[0104] The augmented feature nodes generated by the Mashup description document topic vector matrix through steps three, four, and five are Each row represents a training sample, and there are MN rows in total; each column represents a node. In order to facilitate description, the present application sets d = nk + m, and uses d to represent the total number of nodes, which represents the d-dimensional features of the augmented feature nodes X. Let x i,： (1≤i≤MN) represent the ith row of the augmented feature nodes X, and x :,j (1≤j≤d) represent the jth column of the augmented feature nodes X.

[0105] 1) Construct the data manifold regularization term of the augmented feature nodes X

[0106] By adding a data manifold regularization constraint to the objective function of a Mashup service multi-label classification model based on original width learning, the local geometric structure of samples is preserved when they are embedded from a high-dimensional space to a low-dimensional space. That is, sample points that were initially very close should also be very close in the new low-dimensional projection space after feature mapping. Based on this assumption, the data manifold regularization constraint for enhancing feature node X is to minimize the number of similar data samples x. i,： and x j,： The corresponding label y i and y j The distance between them can be represented in Table 5:

[0107]

[0108] Among them, y i and y j It is sample x i,： and x j,： The label corresponds to the embedding of a sample in the high-dimensional space in the low-dimensional label space, W. ij Indicates sample x i,： and x j,： The similarity between samples is determined by using the K-nearest neighbor algorithm to identify the p nearest neighbor samples, and then using the Gaussian kernel function to calculate the similarity between the neighbor samples and the sample, thus obtaining the similarity matrix W between the samples.

[0109]

[0110] W ij The specific calculation method is shown in equation (8).

[0111]

[0112] Where N p (x i,： ) represents sample x i,： The nearest p neighbors, where t is the bandwidth parameter of the Gaussian kernel function, used to control the local scope of the function.

[0113] 2) Construct the feature manifold regularization term to enhance feature node X.

[0114] To further enhance the generalization performance of width learning, following the duality principle, sample features should also adhere to the manifold assumption, meaning that feature vectors with similar relationships should also have similar weight parameters, which aligns with general rules. Similar to the idea of ​​data manifold regularization, feature manifold regularization also requires constructing a similarity matrix S between features, i.e.:

[0115]

[0116] Meanwhile, the feature manifold regularization constraint of feature node X is to minimize the distance between the approximate data features x :,i and x :,j corresponding to the weight β i and β j , which can be expressed as:

[0117]

[0118] 3) Constructing the objective function of MLMS-DMBLS

[0119] After considering the data manifold regularization and feature manifold regularization, formula (6) and (10) are integrated into the objective function of the Mashup service multi-label classification model based on the original width learning. By minimizing the error between the predicted results and the actual output, the weight coefficient β is solved under the constraint of the double manifold regularization term. In addition, in order to prevent model overfitting, the empirical risk and structural risk of the model are both minimized by reducing the quadratic norm of the output coefficient. Therefore, the objective function of MLMS-DMBLS can be described as:

[0120]

[0121] wherein, is the quadratic norm, λ is the L2 regularization factor of width learning, which is used to balance the empirical risk and structural risk, C1 is the penalty factor of data manifold regularization, and C2 is the penalty factor of feature manifold regularization.

[0122] 4) Simplifying the objective function of MLMS-DMBLS

[0123] In order to facilitate the solution of the MLMS-DMBLS model parameters, the double manifold regularization term of formula (11) needs to be matrix transformed and deduced. The non-constant term of data manifold regularization is deduced and simplified as follows:

[0124]

[0125] wherein, trace(·) represents the trace operation of the matrix, Y represents the output label matrix of the sample, L1 is the data Laplacian matrix, which can be calculated according to formula (13) according to the similarity matrix W.

[0126] L1=D-W (13)

[0127] D is the degree matrix, which can be calculated according to the corresponding similarity matrix, and the calculation method is shown in formula (14).

[0128]

[0129] Similarly, the regularized term of the feature manifold is transformed into the transformed objective function as equation (15) by the matrix transformation of equation (12).

[0130] trace(β T L2β)(15)

[0131] where L2 is the feature Laplacian matrix, which can be obtained from the feature similarity matrix S as equation (16).

[0132] L2 = D - S (16)

[0133]

[0134] Finally, the objective function of MLMS-DMBLS is:

[0135]

[0136] Step seven: calculate the weight coefficients of MLMS-DMBLS using the least squares method.

[0137] To solve the minimum value of β in equation (18), the derivative of the whole equation with respect to β is taken and set to zero, so the four parts of the equation are taken with respect to β in turn, and the specific derivation process is as follows:

[0138]

[0139]

[0140]

[0141]

[0142] Integrate the derivatives of parts ①, ②, ③, and ④ with respect to β and set them equal to zero, that is:

[0143] X T Xβ-X T Y+λβ+C1X T L1Xβ+C2L2β=0 (19)

[0144] We can get:

[0145] β=(X T X+λI+C1X T L1X+C2L2) -1 X T Y (20)

[0146] (2) Test phase

[0147] The application takes the description documents of part of Mashup services in the data set as a test sample set for experiment M is the number of test samples, and the matrix is mapped to generate enhanced feature nodes to obtain the final input X of the test sample * Then, it is sent to the trained double manifold regularization width learning multi-label classification model, and the classification result of the Mashup service is predicted according to formula (21).

[0148] Pred=X * β (21)

[0149] In order to generate a binary multi-label result vector with only 0 and 1 from the predicted category probability vector, a threshold setting method is used. For the output probability in each label binary classifier, if the predicted value is greater than the threshold, it is marked as 1, indicating that the Mashup adds the service category label, otherwise it is marked as 0, indicating that the Mashup does not add the service category label. Each threshold in the set {0.1, 0.2…0.8, 0.9} is taken for multiple experiments, and it is finally determined that the threshold is 0.5 when the classification effect is best. The solving process of the Mashup service multi-label classification model based on double manifold regularization width learning is shown in algorithm 1.

[0150]

[0151]

[0152] In order to verify the effectiveness of the method proposed in the application, a series of experiments are carried out on the real-world service data set, and five evaluation standards suitable for multi-label classification are selected to measure the performance of the experiment. The evaluation indexes and specific experiments are described in detail as follows.

[0153] (1) Evaluation index

[0154] For multi-label classification problems, traditional classification evaluation indexes such as accuracy and recall rate cannot directly evaluate the results. In order to verify the beneficial effects of the method proposed in the application, five evaluation indexes commonly used in multi-label classification are used for performance evaluation, including Hamming Loss (HL), One-error (OE), Coverage (CV), Ranking Loss (RL) and Average Precision (AP). The formal definitions of the five evaluation indexes are given as follows.

[0155]

[0156] Hamming loss is used to evaluate the number of times the sample label is misclassified, where represents the XOR operation, which is used to calculate the number of different real labels and predicted labels. The smaller the HL, the better the model performance.

[0157]

[0158] 1 - Error rate reflects the number of times the highest probability label in the predicted label is not correctly classified, mainly focusing on whether the most relevant label is predicted correctly. Where f(·,·) represents the prediction function, corresponding to the output probability vector p * above. The smaller the OE, the better the model performance.

[0159]

[0160] Coverage rate reflects how many steps are needed on average to cover all relevant labels after sorting the predicted labels in descending order. The smaller the CV, the better the model performance.

[0161]

[0162] Ranking loss is used to evaluate the average score of the reverse sorted label pair. Where Y i and respectively represent the number of labels marked 0 and marked 1 for sample x i . The smaller the RL, the better the model performance.

[0163]

[0164] Average precision is used to evaluate the average score of the correct label ranked higher than a specific label y∈Y i . Where rank f (x i ,y) represents the ranking position of label y in all labels in sample x i , and rank f (x,y')≤rank f (x,y) represents that label y' is ranked before label y. The larger the AP, the better the model performance.

[0165] (2) Data set selection and experimental setting

[0166] The data set used in the application is from the data published by the Key Laboratory of Knowledge Processing and Networked Manufacturing of Hunan University of Science and Technology, which is real service data crawled from ProgrammableWeb.com website, and specifically includes 6206 Mashup services and 12919 Web APIs, as well as the description documents of the services and the categories to which the Web APIs belong and the like information. In order to evaluate the effectiveness of the method proposed in the application, the performance of the algorithm is comprehensively measured by the above-mentioned five commonly used evaluation indexes of multi-label classification, which are abbreviated as HL↓, OE↓, CV↓, RL↓ and AP↑ respectively, ↓ represents that the lower the numerical value of the evaluation index is, the better, and ↑ represents that the higher the numerical value of the evaluation index is, the better. The application performs comparative experiments from two parts, the first part is ablation experiment comparative analysis, the results of Mashup multi-label classification by the MLMS-DMBLS proposed in the application and the manifold regularization width learning M-BLS (Manifold-Broad Learning System) and the width learning BLS are compared. The second part is regular experiment comparative analysis, the MLMS-DMBLS and the multi-label classification algorithms of the classical ML-KNN, the RELM based on the original extreme learning machine ELM, the RMLDM introducing the L2,1 regularization term and the extreme learning machine introducing data and feature manifold regularization term are compared.

[0167] For the MLMS-DMBLS method proposed in the application, the number of width learning feature nodes and enhanced nodes is determined by grid search method with fixed step cycle search; the value range of the regularization factor λ, C1 and C2 is {2 -20 , 2 -19 , … 2 19 , 2 20}; the value range of the theme number T is {10, 20, … 90, 100}, and after multiple experimental parameter adjustment, the detailed parameter settings involved in the application are shown in Table 1.

[0168] Table 1 Model parameter setting

[0169]

[0170] In order to evaluate the model, the application uses 80% of the data set as a training set for model training, and 20% of the data as a test set to verify the performance of the model. In order to reduce the generation of experimental random errors, each test algorithm is tested for 10 times under the same parameters, the mean value of each evaluation index is taken as the final experimental result, and the best result is marked in bold font.

[0171] (3) Comparative experiment

[0172] The application utilizes the idea of manifold learning to constrain the target function of the wide learning model, thereby improving the performance of the model.

[0173] Table 2: Test results of ablation experiment

[0174]

[0175] As can be seen from Table 2, in multiple experiments, the method proposed in the application achieves the optimal level in various indicators, indicating that the MLMS-DMBLS method has better classification performance and stability. This is because the method introduces manifold regularization, utilizes the double constraints of data manifold and feature manifold, so that the input text features can not only maintain the geometric distribution characteristics of the original samples after feature mapping, but also ensure that the feature vectors with similar relationships have similar weight parameters, thereby improving the classification performance of the model.

[0176] At the same time, in order to verify the effectiveness of the method proposed in the application, the test results of other multi-label classification algorithms for Mashup service multi-label classification are compared, and the results are shown in Table 3.

[0177] Table 3: Test results of various classification algorithms

[0178]

[0179] It can be seen that the present application has different percentage performance improvement in various evaluation indicators. For HL↓, the present application has little improvement compared with the second-ranked ML-KNN method, only reducing the loss by 0.03%, but has obvious improvement in OE↓ and AP↑, reducing the 1-error rate by 19.56% and increasing the average precision by 18.69%. It can be seen that ML-KNN can approach the method of the present application in Hamming loss and coverage, but in other indicators, the present application is still in a dominant position. In the OE↓ indicator, MLMS-DMBLS has no obvious difference with RMLDM, but compared with the other two extreme learning machine-based methods ELM and RELM, it reduces by 5.71 and 2.25 percent respectively. In the AP↑ indicator, compared with the second-ranked method RMLDM, it increases the precision by 2.49%. At the same time, compared with the three extreme learning machine-based methods, the loss in CV↓ and RL↓ is particularly significant, which further proves that the feature extraction capability of width learning is significantly better than that of extreme learning machine due to the joint action of feature nodes and enhanced nodes, especially in the double manifold regularization constrained width learning model, the overall performance is better than some other multi-label classification models. In addition, according to the evaluation indicator results of ELM, RELM and RMLDM, adding appropriate constraints to the objective function of the model can learn better parameters, to a certain extent, improve the classification performance of the model, which is consistent with the conclusion drawn by the present application in the ablation experiment results in Table 2.

[0180] Finally, it should be noted that: the above embodiments are only used to illustrate the technical solutions of the present application, and not to limit them; although the present application has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that they can still modify the technical solutions recorded in the foregoing embodiments, or make equivalent replacement for part or all of the technical features; and these modifications or replacements do not make the essence of the corresponding technical solutions deviate from the scope of the technical solutions of the embodiments of the present application.

Claims

1. A multi-label classification method for Mashup services based on bimanifold regularized width learning, characterized in that, Includes the following steps: Obtain the Mashup service dataset, which includes the name of the Mashup service, the description document, the API service called by each service, the name of the API service and its category, and preprocess the description document and the binary output label of the Mashup service. The hidden Dirichlet distribution topic model is used to extract features from the preprocessed Mashup description documents, thereby generating the topic distribution vector of the Mashup description documents; The Mashup description document topic distribution vector is used as the original input data for width learning. The Mashup description document topic feature matrix is ​​linearly mapped into n sets of feature nodes using a linear transformation function and a parameter matrix of n sets of random mappings. Enhanced nodes are generated by processing feature nodes using a non-linear activation function. The feature nodes and enhancement nodes are concatenated to generate enhanced feature nodes, which are then used as input to a Mashup service multi-label classification model based on dual-manifold regularized width learning. We construct the objective function for a Mashup service multi-label classification model based on bimanifold regularization width learning using the manifold regularization concept. The objective function is solved using the least squares method to obtain the weight matrix of the dual-manifold regularized width learning network, and the training of the multi-label classification model based on the dual-manifold regularized width learning Mashup service is completed. Obtain the description document of the test Mashup service, perform text preprocessing and topic feature extraction to obtain the topic distribution vector matrix of the test Mashup service, map the topic distribution vector matrix to generate enhanced feature nodes, and then obtain the final input of the test sample. Then, feed it into the pre-trained Mashup service multi-label classification model based on dual-manifold regularized width learning to predict the classification result of the Mashup service.

2. The Mashup service multi-label classification method based on dual-manifold regularized width learning according to claim 1, characterized in that, The steps for preprocessing the description document and the binary output tags of the Mashup service include: Convert all letters in the description document to lowercase. Remove punctuation marks and meaningless special characters from the description document; Use the corpus corpus in the nltk package to remove stop words from description documents; Using the WordNetLemmatizer tool in the nltk package, complex word forms are transformed into basic forms based on a dictionary; The crawled Mashup call records are converted from 0 to 1 numbers to generate a Mashup-service type call matrix. ,in The number of services provided to Mashups For the number of service types, Indicates Mashup service A service from type j was invoked. This indicates that no service in type j was invoked.

3. The Mashup service multi-label classification method based on dual-manifold regularized width learning according to claim 1, characterized in that, A hidden Dirichlet distribution topic model is used to extract features from the preprocessed Mashup description documents, thereby generating a topic feature matrix for the Mashup description documents, including: The collection of functional description documents of Mashup is used as the corpus for the Hidden Dirichlet Distribution topic model; The topic set was obtained by training it with a Hidden Dirichlet Distribution (HDD) topic model. Where T represents the number of topics set in the Hidden Dirichlet Distribution topic model, and each Mashup description document corresponds to a set of topics. A probability distribution on the , that is, the topic features of the i-th Mashup description document are represented as a vector This allows us to obtain the topic distribution vector of the entire corpus. .

4. The Mashup service multi-label classification method based on dual-manifold regularized width learning according to claim 1, characterized in that, The feature nodes are obtained according to the following formula: in, Let i represent the i-th group of feature nodes, which contains k feature nodes. , The total number of feature nodes. in, It is a linear mapping function. and The weight coefficient matrix and bias matrix of the i-th feature node are generated by random mapping and conform to a normal distribution.

5. The multi-label classification method for Mashup services based on dual-manifold regularized width learning according to claim 1, characterized in that, The enhancement node is obtained according to the following formula: Where m is the number of enhancement nodes, generated all at once from the feature nodes. and The weight coefficient matrix and bias matrix, respectively, for the augmentation nodes, are generated through random mapping and conform to a normal distribution, and then orthogonally normalized. It is a non-linear activation function.

6. The Mashup service multi-label classification method based on dual-manifold regularized width learning according to claim 5, characterized in that, The enhanced feature nodes are obtained according to the following formula: in, To enhance the feature node matrix, the rows of the matrix represent the number of samples, and the columns represent the total number of network nodes; Therefore, the input and output of the Mashup service multi-label classification model based on dual-manifold regularized width learning can be related by the following formula: in, This is the binary multi-label vector of service categories output by a Mashup service multi-label classification model based on dual-manifold regularized width learning. The weight matrix that a width-learning network needs to learn.

7. The Mashup service multi-label classification method based on dual-manifold regularized width learning according to claim 6, characterized in that, The objective function of a Mashup service multi-label classification model based on bimanifold regularization width learning is constructed by fusing bimanifold regularization, including: Constructing the data manifold regularization term for the enhanced feature node X specifically includes constructing the data manifold regularization constraint for the enhanced feature node X as follows: in, and It is an enhanced feature node X The i row and number j The row represents a training sample. and It is a sample and The labels correspond to the embedding of samples in the high-dimensional space in the low-dimensional label space. Indicates sample and The similarity between samples is determined by the K-nearest neighbor algorithm to identify the sample closest to the nearest neighbor. The neighbor samples are used to calculate and obtain: Calculate using the following formula: in Indicates sample Recent A neighbor, It is the bandwidth parameter of the Gaussian kernel function, used to control the local scope of the function; Constructing the feature manifold regularization term to enhance feature node X specifically includes constructing the similarity matrix between features. : in, and It is an enhanced feature node X The i Column and number j Columns represent sample features; Simultaneously, the feature manifold regularization constraint of feature node X is enhanced to minimize the features of mutually approximate data features. and The corresponding weight and The distance can be expressed as: The objective function for constructing a multi-label classification model for Mashup services based on bimanifold regularized width learning is: （11） in, It is a quadratic norm. The L2 regularization factor for breadth learning is used to balance empirical risk and structural risk. The penalty factor for data manifold regularization. The penalty factor for regularization of the characteristic manifold; The objective function of the Mashup service multi-label classification model based on dual-manifold regularized width learning is simplified. The simplified objective function is as follows: in, Represents the trace operation of a matrix. and It is the Laplace matrix of data and features, which can be derived from the data similarity matrix. and feature similarity matrix The conclusion is as follows.

8. The Mashup service multi-label classification method based on dual-manifold regularized width learning according to claim 7, characterized in that, The objective function is solved using the least squares method to obtain the weight matrix of the dual-manifold regularized width learning network, and the training of the multi-label classification model based on the dual-manifold regularized width learning Mashup service is completed, including: For the objective function as a whole Taking the derivative and setting it to 0, we get: in Let be the weight matrix of the dual-manifold regularized width learning network.

9. A multi-label classification method for Mashup services based on dual-manifold regularized width learning according to claim 8, characterized in that, The classification result for the Mashup service is predicted using the following formula: in, To predict the classification results of the Mashup service, Here is the weight matrix of a two-manifold regularized width learning network. This is the final input for the test sample.

Images