Web service class label recommendation method fusing multi-channel semantic information and label collaboration

Abstract

The application discloses a Web service category label recommendation method fusing multi-channel semantic information and label cooperation, through crawling Web services, performing data cleaning on Web service description, adopting RoBERTa and CNN to obtain multi-channel feature word semantic information, establishing a global semantic extraction model with a fast rule approximate attention mechanism, adopting a multi-level semantic feature fusion mechanism to obtain a Web service function vector; in addition, a service label cooperation correlation graph is constructed, a biased random walk strategy fusing label combination popularity is designed, a graph attention network LCGAT fusing edge weight calculation attention is established, and a final label cooperation vector is output; the service function vector and the label cooperation vector are fused, the label cooperation vector judgment score is integrated into the output score of a sigmoid classifier, the Web service function similarity and the cooperation compatibility between label combinations are taken into account, the category recommendation quality is improved, and the recommendation complexity is reduced.

Description

Technical Field

[0001] The present invention belongs to the field of cloud computing, and in particular relates to a Web service category tag recommendation method that integrates multi-channel semantic information and tag collaboration. Background Art

[0002] With the maturity of next-generation information technologies such as cloud computing, the Internet of Things, and mobile internet, software developed and deployed using service-oriented architectures has seen widespread adoption and application. An increasing number of enterprises are encapsulating computing resources or business functions as web services and publishing them on various online platforms, leading to a rapid increase in the number of web services on the internet. Reusing and integrating existing web services can effectively shorten business system development cycles, reduce development costs, and facilitate the expansion of enterprise business scope.

[0003] Developers often use functional keywords to search for web services on the internet that meet their needs. Currently, there are over 25,000 registered web services on ProgrammableWeb, a well-known web service registration platform, covering more than 500 application types. Faced with the large number and diverse types of web services, category tags are crucial for the storage, organization, and discovery of services. Web services often use short natural language text to describe service functions and provide one or more category tags to indicate the functional scope of the service. Reasonable category tags can help improve the efficiency and accuracy of web service discovery. In addition, category tags are also the main basis for the service registry to organize and store web services. Reasonable category tags facilitate the classified management of web services. Therefore, assigning reasonable category tags to newly released web services is an effective way to improve service discovery efficiency and the rationality of storage organization.

[0004] When recommending category labels for Web services, one or more functional categories need to be given based on its functional description. Web service category label recommendation is largely a multi-label classification problem for text. Currently, mainstream multi-label recommendation methods usually use topic models or neural network models to generate service function vectors for service function descriptions, and then calculate the similarity between the service function vector and the label vector to achieve multi-label category recommendation. In addition, the association information between category labels has gradually attracted the attention of researchers. In existing technical solutions, the structural dependency information between labels is studied by serializing service labels and integrating it into the category recommendation task. Related research has shown that fully exploring the association patterns between labels can improve the accuracy of category label recommendations for Web services. Since service category labels are unordered, this method of using label sequences to assist category recommendation is prone to introduce semantic association noise between labels, thereby affecting the quality of Web service category label recommendations.

[0005] From the above analysis, we can see that there are several technical difficulties that affect the quality of service label recommendation when recommending web service category labels:

[0006] 1. The semantic feature density of service description text is sparse, making it difficult for existing methods and models to generate high-quality service capability vectors. Web service descriptions are typically brief, with low and uneven feature density. The granularity of feature words in the description text varies. Consequently, traditional topic models and neural network models have limited ability to extract low-density, sparse semantic features and struggle to achieve a reasonable semantic fusion of feature words of varying granularity.

[0007] 2. Existing category recommendation methods that consider tag collaboration typically recommend tags based on tag sequence groups. This approach simplifies category recommendation complexity but contradicts the fact that web service tags are unordered. When recommending tag sequence groups, they artificially introduce sequential dependencies between tags, introducing unnecessary semantic noise. This makes it difficult for the model to reasonably and accurately extract tag collaboration relationships, thus affecting the accuracy of web service category tag recommendations.

[0008] 3. Existing methods fail to consider the intensity and popularity of tag collaboration. They treat all tag collaborations equally, without distinguishing between the number of collaborations and the popularity of tag combinations. This approach reduces the recommendation chances of some high-frequency tag combinations, thereby reducing the quality of category tag recommendations. Summary of the Invention

[0009] In response to the problems of sparse semantic features and poor completeness in the existing technology when generating service function vectors, the present invention proposes a Web service category label recommendation method that integrates multi-channel semantic information and label collaboration to recommend reasonable category labels for Web services, thereby improving service discovery efficiency and storage organization rationality.

[0010] The present invention is implemented by adopting the following technical solution: a method for recommending web service category tags by integrating multi-channel semantic information and tag collaboration, comprising the following steps:

[0011] Step A: Crawl the web service.

[0012] Step B: Clean the Web service description and construct the service function vector:

[0013] Step B1: Establish a multi-channel word vector semantic feature extraction model to generate an aggregated word vector c containing local context feature information of different granularity window sizes. t ;

[0014] Step B2: Establish a multi-channel sentence vector semantic feature extraction model MCFRASRU to generate a service function vector sfv(s) for the service description text that takes into account both local and global context features;

[0015] Step C: Determine the web service tag set and construct the tag collaboration vector:

[0016] Step C1: construct a tag collaboration association graph of Web services based on the collaboration association between service tags;

[0017] Step C2: Design a wandering strategy with tag combination popularity correction. Based on the biased wandering probability, design a node wandering probability calculation method that incorporates tag combination popularity.

[0018] Step C3: Combine the node wandering probability strategy that incorporates the popularity of the tag combination with the Node2vec model to generate an initial collaboration vector for the tag corresponding to each node in the tag collaboration association graph;

[0019] Step C4: Establish a graph attention network LCGAT that integrates edge weight calculation attention and outputs the final label collaboration vector;

[0020] Step D: Combine the service function vector and the label collaboration vector to output the Web service category label.

[0021] Furthermore, the step B1 is specifically implemented in the following manner:

[0022] The CNN convolutional layer network is constructed by using a variety of convolution kernels to construct semantic information extraction channels for feature words of different lengths. Each channel inputs the word vector v pre-trained by RoBERTa. t After processing through different channels, the final generated aggregate word vector c t Contains local context feature information of different granularity window sizes:

[0023] c t =ReLU(Con·V t:t+h-1 +b) (1)

[0024] Among them, V t:t+h-1 ∈R d*h is the word vector matrix with the tth word as the center word and length h, and d represents the word vector v t The dimension of , b is the bias vector.

[0025] Furthermore, the step B2 specifically includes the following steps:

[0026] Step B21: Build an SRU++ model with a fast attention mechanism: FRASRU, which is the aggregate word vector c tFurther aggregate contextual features to generate a global word vector hw with global context features t :

[0027] Step B22, hw t with c t The concatenation is used as the final word vector of the t-th word, and the text vector in the current channel is generated for the service description text information based on the attention mechanism;

[0028] Step B23: Calculate the average of the text vectors generated by the service description text information in different channels to generate a service function vector for the Web service that integrates multi-channel semantic information.

[0029] Furthermore, step B21 is implemented in the following manner:

[0030] The self-attention mechanism in SRU++ is replaced by the fast rule approximate attention mechanism, and the functional semantics of each word is modeled from both the positive and negative directions. The positive semantic vector is h t1 , the reverse semantic vector is h t2 , concatenate the two to get word w t The global word vector hw of the global context feature t :

[0031] hw t =[h t1 ||h t2 ] (12)

[0032] Here, “||” represents vector concatenation.

[0033] Furthermore, in step B22, considering that each word in the service description has different importance in generating the service function vector, the contribution of the word to the construction of the service function vector is calculated through the attention mechanism, and the weight vector u of each word is t for:

[0034] u t =tanh(M g [hw t ||c t ]+b g ) (13)

[0035] Among them, M g is the learnable parameter matrix, tanh() is the activation function, b g is the bias vector, and the weight vector is normalized to obtain the attention weight of each word:

[0036]

[0037] Combine the word vector at each position with the corresponding attention weight α tMultiply and sum to get the text vector sv(sd) of the service description sd in the current channel:

[0038] sv(sd)=∑ t α t [hw t ||c t ] (15)

[0039] Among them, hw t is the global word vector of the global context feature.

[0040] Furthermore, the final service function vector in step B23 is sfv(s):

[0041]

[0042] Where s is the Web service, m is the number of semantic channels, j = 1, 2, 3…m, sd is the service description text information, sv j (sd) is the text vector of s in the jth channel.

[0043] Furthermore, in step C2, the following method is specifically adopted:

[0044] (1) In the original Node2vec, for node v i , which is for the next hop node v j The probability reference formula is:

[0045]

[0046] Among them, π vivj is node v i and v j The unnormalized transition probability between, z is the normalization constant, π vivj =a pq (i,j)·w i,j , where w i,j is node v i and v j The edge weight between pq (i,j) is calculated by formula (18):

[0047]

[0048] Among them, d i,j Represents node v j and v i The shortest path distance between nodes, q and p are two hyperparameters used to define the transition probability between nodes;

[0049] (2) Calculate the popularity of tag combinations:

[0050]

[0051] where Ns(l) is the number of Web services containing label l, the numerator in equation (19) is the sum of Web services belonging to label l i and l j , respectively, represent the Web services containing label l i and l j , respectively, L is the set of service category labels. i and l j , respectively, L is the set of service category labels.

[0052] (3) The label combination popularity is integrated into the biased random walk probability of Node2vec, and the modified transition probability is:

[0053]

[0054] Node2vec modifies the walk probability using the label combination popularity when performing random walk.

[0055] Further, in step C, the following method is used to achieve it:

[0056] (1) In the GAT model, the similarity coefficient of the node v i in the label collaboration graph and its first-order neighbor nodes is calculated:

[0057] e i,j =a(W1u i ,W2u j ),j∈N i (21)

[0058] where u i , u j are the collaborative association vectors of v i and neighbor nodes v j , W1, W2∈R F×F ′ are trainable matrices, F and F′ are the current layer initialization vector dimension and output dimension, respectively, and a is a shared attention mechanism coefficient.

[0059] (2) The current layer bias vector is constructed as:

[0060]

[0061] where b l w is the current layer bias vector, b l’ w is the bias vector of the previous layer, W l w is a trainable weight matrix, and ui is the current layer node v i Vector representation of ;

[0062] (3) Combine the bias vector and edge weight w i,j Combined with the attention calculation process, a new attention coefficient is obtained:

[0063]

[0064] Therefore, according to the multi-head attention mechanism, v is obtained in the last layer i Corresponding label l i The final collaboration vector lcv(l i ):

[0065]

[0066] Among them, σ is the sigmoid activation function, a k i,j is the kth attention mechanism (a k ) calculated the normalized attention coefficient, W k is the weight matrix of the corresponding linear transformation of the input.

[0067] Furthermore, the step D comprises the following steps:

[0068] (1) The service function vector sfv(s) of service s is combined with the collaboration vector lcv(l j ) Through the MLP layer, we get the label l j The predicted probability of a label appearing in the label set of service s:

[0069] p(s,l j )=sigmoid(W1sfv(s)+W2lcv(l j )+b) (25)

[0070] Among them, W1 and W2 are trainable weight matrices, and b is the bias term;

[0071] (2) Given a label set L = {l1,l2,…,l j ,…,l n}, the set of label prediction probabilities that the labels in L belong to the label set of service s is {p(s,l1),p(s,l2),…,p(s,l j ),…,p(s,l n )}, and train the model using the binary cross entropy loss function:

[0072]

[0073] Where, Ys={y s,l1,y s,l2 ,…,y s,lj ,...,y s,ln} represents the true label of service s, n is the total number of labels, if l j ∈L exists in the label list of s, corresponding to y s,lj Set the value to 1, otherwise set the value to 0, p(s,l j ) is the label prediction probability in formula (25);

[0074] (3) When making tag recommendations, let PYs = {py s,l1 ,py s,l2 ,…,py s,lj ,...,py s,ln} is the label prediction set of service s, if p(s,l j )≥0.5, depending on the label l j is a positive example, which exists in the label list of s, corresponding to py s,lj Set the value to 1; if p(s,l j )<0.5, visual label l j is a negative example, which does not exist in the label list of s, corresponding to py s,lj Set the value to 0.

[0075] Compared with the prior art, the advantages and positive effects of the present invention are:

[0076] 1. Service function vector generation:

[0077] (1) Using RoBERTa and CNN to obtain the semantics of multi-channel feature words solves the problem of incomplete feature extraction caused by existing research ignoring the aggregation of local context information of feature words of different granularity;

[0078] (2) Establish a global semantic extraction model with a fast rule-based approximate attention mechanism. By focusing on the association between feature words through built-in attention, the association between feature words of different granularity and the global context is extracted, solving the problem of insufficient modeling of feature word context relationships in existing studies.

[0079] (3) In view of the fact that local feature words in Web services contribute differently to the text compared with other words in multi-label classification, a multi-level semantic feature fusion mechanism is used to obtain the Web service function vector, which fully integrates local context information and global context information to achieve complete and accurate extraction of the functional features corresponding to the Web service description.

[0080] 2. Label collaboration vector generation:

[0081] (1) Constructing a service tag collaboration association graph, the historical collaboration information of service tags is intuitively represented in the form of weighted edges. Compared with existing tag sequence research methods, this method can reasonably model tag associations based on the logical facts of tag collaboration, and improve the quality of association modeling without introducing irrelevant semantic association noise.

[0082] (2) A biased random walk strategy that integrates the popularity of tag combinations is designed. Under this strategy, the nodes with the largest association strength and combination popularity in the collaborative association graph are preferentially sampled into the path sequence. This solves the problem that the existing technology only samples based on the number of co-occurrences, which makes it difficult to ensure that the nodes with the highest possibility of co-calling are sampled into the path sequence.

[0083] (3) We propose a GAT network model that integrates edge weights to calculate attention. This model incorporates edge weights into attention calculations, enabling it to learn intuitive collaborative relationships between labels and neighbors. Compared to traditional models, this model can more quickly and effectively process dense, complex, and high-order edge collaborative graphs, improving the quality of graph node embedding modeling and the extraction of local collaborative information from labels.

[0084] 3. Finally, the service function vector and the label collaboration vector are integrated, and the label collaboration vector judgment score is incorporated into the output score of the sigmoid classifier. This takes into account the similarity of Web service functions and the collaborative compatibility between label combinations, improving the quality of category recommendations while reducing the complexity of recommendations. BRIEF DESCRIPTION OF THE DRAWINGS

[0085] Figure 1 This is a flowchart of a method for recommending Web service category labels according to an embodiment of the present invention;

[0086] Figure 2 This is a sample diagram of the Web service structure;

[0087] Figure 3 This is a schematic diagram of a tag collaboration association diagram according to an embodiment of the present invention;

[0088] Figure 4 Schematic diagram of the framework of the Web service category label recommendation method according to an embodiment of the present invention. DETAILED DESCRIPTION

[0089] In order to more clearly understand the above-mentioned objects, features and advantages of the present invention, the present invention is further described below with reference to the accompanying drawings and embodiments. In the following description, many specific details are set forth to facilitate a full understanding of the present invention. However, the present invention can also be implemented in other ways than those described herein. Therefore, the present invention is not limited to the specific embodiments disclosed below.

[0090] This embodiment proposes a web service category tag recommendation method that integrates multi-channel semantic information and tag collaboration, such as Figure 1 and Figure 4 As shown, the following steps are included:

[0091] Step A: Crawl the web service.

[0092] Step B: Clean the Web service description and construct the service function vector:

[0093] Step B1: Establish a multi-channel word vector semantic feature extraction model to generate an aggregated word vector c containing local context feature information of different granularity window sizes. t ;

[0094] Step B2: Establish a multi-channel sentence vector semantic feature extraction model MCFRASRU to generate a service function vector sfv(s) for the service description text that takes into account both local and global context features;

[0095] Step C: Determine the web service tag set and construct the tag collaboration vector:

[0096] Step C1: construct a tag collaboration association graph of Web services based on the collaboration association between service tags;

[0097] Step C2: Design a wandering strategy with tag combination popularity correction. Based on the biased wandering probability, design a node wandering probability calculation method that incorporates tag combination popularity.

[0098] Step C3: Combine the node wandering probability strategy that incorporates the popularity of the tag combination with the Node2vec model to generate an initial collaboration vector for the tag corresponding to each node in the tag collaboration association graph;

[0099] Step C4: Establish a graph attention network LCGAT that integrates edge weight calculation attention and outputs the final label collaboration vector;

[0100] Step D: Combine the service function vector and the label collaboration vector to output the Web service category label.

[0101] Specifically, the following is a detailed introduction to this embodiment:

[0102] 1. Crawling Web Services:

[0103] Definition 1. Web service: A Web service is defined as a four-tuple, s = (Id, n, L, sd), where Id is the service identification number, n is the service name, L is the service category label set, and sd is the service description text information.

[0104] Crawling Web services from the service registration platform, cleaning the Web service description, removing useless special characters, and segmenting the text. Figure 2 As shown in the figure, Social and Webhooks are category labels, and the text below the category labels is the service description text information.

[0105] 2. Building a Service Function Vector

[0106] 2.1. Build a multi-channel word vector semantic feature extraction model to generate aggregated word vectors that can aggregate local context feature information of different granularity window sizes;

[0107] A multi-channel CNN architecture is used, with three layers of convolutions having the same number of kernels but different window sizes. Each channel is fed with word vectors pre-trained with RoBERTa. After processing, the resulting aggregated word vectors contain local contextual features of varying window sizes.

[0108] For the tth word w in the service description text sd t , generate word vector v through RoBERTa t . Let v t The dimension is d, through the convolution kernel Con∈R h*d Perform a sliding window operation on the word vector matrix input by RoBERTa, processing h adjacent word vectors each time. t The word vector v t The vector obtained after the convolution layer is denoted as c t :

[0109] c t =ReLU(Con·V t:t+h-1 +b) (1)

[0110] Among them, V t:t+h-1 ∈R d*h is the word vector matrix with the tth word as the center word and length h, and b is the bias vector. After the word vector is processed by three convolution kernels with different window sizes, each channel is used to generate a word vector that aggregates the contextual feature information of the corresponding granularity window size.

[0111] This embodiment uses RoBERTa and CNN to acquire the semantics of multi-channel feature words, aggregates local context feature information of different granularity window sizes for words, solves the problem of incomplete feature extraction caused by ignoring the distinction between feature word granularity and failing to integrate the semantic information of feature words of different granularity in existing technical solutions, and improves the semantic completeness of the service function vector by aggregating context information of window sizes of multiple granularity.

[0112] 2.2. Establish a multi-channel sentence vector semantic feature extraction model MCFRASRU to generate service function vectors for service description texts that take into account both local and global context features:

[0113] 2.21 Build the SRU++ model with fast attention mechanism: FRASRU. For the aggregate word vector c t Further aggregate contextual features to generate a global word vector hw with global context features t :

[0114] In SRU, using the cell state x t Pass information on all word positions and finally output the hidden state h t , thereby removing the dependence of the t-1th word vector on the t-1th word vector when calculating the t-th word vector in RNN and LSTM, and processing most operations in parallel, and only a small amount of operations in serial. The parallel processing process can be seen in formulas (2)-(5):

[0115] f t =σ(M t c t +v f ex t-1 +l f ) (2)

[0116]

[0117] r t =σ(M r c t +v r ex t-1 +l r ) (4)

[0118] h t =r t ⊙x t +(1-r t )⊙c, (5)

[0119] Among them, e represents the element-by-element product, and σ represents the sigmoid function. t 、M r and M is the weight matrix to be learned, l f 、l r 、v f and v r is the parameter vector that needs to be learned during training, c t is the tth word w t Word vectors processed by RoBERTa and CNN. t c t 、Mc t 、M r c tis the matrix multiplication for batch processing at all positions. Assuming that each c dimension in the input sequence C is D, the length is L, and the hidden layer dimension is d, the batch multiplication is shown in formula (6), where M t 、M r , M∈R d×D , U∈R L×3d .

[0120]

[0121] SRU++ combines the SRU recurrent temporal network and the attention mechanism to further improve the training speed based on SRU, and can achieve the same test results in a shorter training time. The attention matrix operation is shown in Equations (7)-(9):

[0122] Q=M q C T (7)

[0123] K=M k Q (8)

[0124] V=M v Q (9)

[0125] Among them, Q is the attention component query matrix, K is the keyword matrix, and V is the value vector matrix. q ∈R d′×d , M k ∈R d′×d′ , M v ∈R d′×d′ is a trainable projection matrix, L is the sentence length, and d′ is the attention dimension less than d. To save parameters, K and V T All are derived from Q. Attention output A T ∈R d′×L , the calculation method is shown in formula (10):

[0126]

[0127] This paper replaces the self-attention mechanism in SRU++ with a fast rule-based approximation attention mechanism, further improving the training speed and achieving better training results. By decomposing the attention matrix with positive random features, the matrix multiplication in formula (6) is reordered based on matrix decomposition.

[0128] Decompose the matrix A using the random eigenfunction (u) to obtain matrices Q′ and K′. Then adjust the matrix multiplication order of Q′, K′, and V to Q((K) T V), and thus the matrix approximation form of formula (6) after recalculation is obtained, as shown in formula (11):

[0129] U=M o (Q+a·A) (11)

[0130] Among them, M o ∈R 3d×d′ and α are trainable parameters. (Q+α·A) represents the residual connection, which improves gradient propagation and stabilizes training.

[0131] The functional semantic modeling of each word is performed according to formulas (2)-(5) from the positive and negative directions. The positive semantic vector is h t1 , the reverse is h t2 , concatenate the two to get word w t The global context feature word vector hw t , “||” indicates vector concatenation:

[0132] hw t =[ht1||ht2] (12)

[0133] 2.22 will hw t with c t The concatenation is used as the final word vector of the t-th word, and the text vector sv(sd) in the current channel is generated for the service description sd based on the attention mechanism.

[0134] hw t with c t The concatenation is performed so that the resulting word vector contains both its own functional semantics and the local and global contextual features of the word in the service description text. Given that each word in the service description has different importance in generating the service function vector, the contribution of the word to the construction of the service function vector is calculated through the attention mechanism. The weight vector of each word is:

[0135] u t =tanh(M g [hw t ||c t ]+b g ) (13)

[0136] Among them, M g is the learnable parameter matrix, tanh() is the activation function, b g is the bias vector. Normalize the weight vector to get the attention weight of each word:

[0137]

[0138] Finally, the word vector at each position is combined with the corresponding attention weight α t Multiply and sum to get the text vector sv(sd) of sd in the current channel:

[0139] sv(sd)=∑t α t [hw t ||c t ] (15)

[0140] 2.23 The text vectors generated by the service description in different channels are averaged to generate a service function vector for the Web service that integrates multi-channel semantic information:

[0141] The present invention takes word vectors that aggregate local context feature information of different granularity window sizes as input through different channels. When passing through FRASRU, global context features (context-related features) are aggregated for each word vector, so that the word vector takes into account both local and global context features, thereby making the semantic feature density and completeness of the sentence vector high, effectively alleviating the semantic sparsity problem.

[0142] In this embodiment, three semantic channels (j=1, 2, 3) are constructed. For Web service s, its final service function vector is sfv(s):

[0143]

[0144] This embodiment constructs a FRASRU model to further aggregate context-related features for the aggregated word vector, so as to extract the association dependency between feature words of different granularities and the global context, realize high-speed parallel computing, and improve the density of semantic features of the service function vector.

[0145] 3. Constructing Tag Collaboration Association Vector

[0146] 3.1. Based on the collaborative association between service tags, construct a tag collaborative association graph for Web services:

[0147] Definition 2. Collaborative association.

[0148] There exists a Web service s such that l i ∈sL∧l j .∈sL, then the label l i With label l j There is a collaborative relationship, recorded as:

[0149] Definition 3. Label collaboration graph.

[0150] The tag collaboration graph is an undirected weighted graph LCG = (V, E, W), where:

[0151] (1) V = {v1, v2, ..., v n} is a set of label nodes, node v i Indicates service tag l i ;

[0152] (2) E={e1,e2,…,e m} is the set of collaborative associated edges: e=(v i ,v j )∈E i With v j The corresponding label l i and l j satisfy

[0153] (3)W={w ij} is the edge weight set.

[0154] The edges in the label collaboration association graph are collaboration association edges, such as Figure 3 As shown. If two labels l i With l j The tag set belonging to the same Web service, node v in LCG i and v j A collaborative association edge is established between them, and the weight of the collaborative association edge is label l i With l j The number of co-participating web services.

[0155] 3.2. Design a wandering strategy with tag combination popularity correction. Based on the biased wandering probability, design a node wandering probability calculation method that incorporates tag combination popularity:

[0156] (1) In the original Node2vec, for node v i , which is for the next hop node v j The probability reference formula is:

[0157]

[0158] Among them, π vivj is node v i and v j The unnormalized transition probability between , z is the normalization constant. Considering that the jth step is at v i The random walk on v i Connected edges (v i , v j ) on the transition probability π vivj Decision. vivj =a pq (i,j)·w i,j , where w i,j is node v i and v j The edge weight between them. pq (i, j) is calculated by formula (18):

[0159]

[0160] Among them, d i,j Represents node v j and v i The shortest path distance between nodes. q and p are two hyperparameters that define the transition probability between nodes. q controls the degree of "depth-first" or "breadth-first" approach in the walk. Larger q values ​​tend to favor exploring nodes farther from the initial node, i.e., depth-first walk, which yields more global structural information. Smaller q values ​​tend to favor exploring neighboring nodes, i.e., breadth-first walk, which preserves more local neighborhood information.

[0161] The parameter p controls the "return probability" of the walk. Larger p values ​​make the walk more likely to repeatedly wander between adjacent nodes, preserving more local neighborhood information. Smaller p values ​​tend to explore new nodes more, generating more global structural information.

[0162] (2) Calculate the popularity of tag combinations and design the following calculation formula:

[0163]

[0164] Where Ns(l) is the number of web services containing label l. i With l j The total number of Web services to which it belongs. i ) and Is(l j ) respectively indicate the inclusion of label l i With l j Let l be a web service containing label l i With l j All the tags in the tag set of the Web service l are taken as the denominator, and the ratio of the numerator to the denominator is the tag combination popularity.

[0165] (3) Incorporating the label combination popularity into the biased random walk probability of Node2vec, the modified transition probability is:

[0166]

[0167] Thanks to this, Node2vec can use the popularity of label combinations to correct the walk probability when performing random walks, ensuring that the collaboration strength and co-occurrence call probability of nodes in the node sequence are optimized at the same time.

[0168] 3.3. The node wandering probability strategy that incorporates the popularity of tag combinations is combined with the Node2vec model to generate an initial collaboration vector for the tag corresponding to each node in the tag collaboration association graph.

[0169] The walking strategy with label combination popularity correction designed in 3.2 is used to perform sequence sampling in the label collaboration association graph, and the generated sampling sequence is processed using Node2Vec to implement node embedding for the label collaboration association graph. i Generate initial collaboration vector u i .

[0170] This embodiment addresses the problem that existing graph embedding node vector generation technology cannot solve the node vector generation problem in weighted label collaboration association graphs with unbalanced edge density. A graph node embedding vector generation method based on an improved Node2vec+GAT model is proposed. The label collaboration vector is pre-trained using Node2vec with fused label combination popularity, and the fused edge weight is input into the attention GAT model. This method can improve the computational rationality of label collaboration strength in complex and dense label graphs, and is more conducive to accurately recommending service category labels.

[0171] 3.4. Establish a graph attention network LCGAT that integrates edge weight calculation attention and outputs the final label collaboration vector. In the GAT model, for the node v in the label collaboration graph i , calculate the similarity coefficient with its first-order neighbor nodes:

[0172] e i,j =a(W1u i ,W2u j ),j∈N i (twenty one)

[0173] Among them, u i 、u j v i With neighbor node v j Collaborative association vector, W1, W2∈R F×F ′ is a trainable matrix, F and F′ are the initialization vector dimension and output dimension of the current layer respectively, and a is the coefficient of the shared attention mechanism. Then, the current layer bias vector is constructed, see formula (21):

[0174]

[0175] Among them, b l w is the current layer bias vector, b l’ w is the bias vector of the previous layer. l w is the trainable weight matrix, u i is the current layer node v i Next, the bias vector is combined with the edge weight w i,jCombined with the attention calculation process, a new attention coefficient is obtained, see formula (22):

[0176]

[0177] Therefore, according to the multi-head attention mechanism, v is obtained in the last layer i Corresponding label l i The final label collaboration vector lcv(l i ):

[0178]

[0179] Among them, σ is the sigmoid activation function, a k i,j is the kth attention mechanism (a k ) calculated the normalized attention coefficient, W k is the weight matrix of the corresponding linear transformation of the input.

[0180] Fourth, integrate the service function vector and label collaboration vector to achieve web service category label recommendation through multi-label classifier;

[0181] 4.1 Construct a Web service category recommendation method based on multi-channel semantic information and label collaboration to achieve category output.

[0182] The present invention regards the multi-label classification of Web services as a binary classification problem. The service function vector sfv(s) of service s is combined with the j-th label collaboration vector lcv(l j ) Through the MLP layer, we get the label l j The predicted probability of a label appearing in the label set of service s:

[0183] p(s,l j )=sigmoid(W1sfv(s)+W2lcv(l j )+b) (25)

[0184] Where W1 and W2 are trainable weight matrices and b is the bias term. Given a label set L = {l1,l2,…,l j ,…,l n}, the set of label prediction probabilities that the labels in L belong to the label set of service s is {p(s,l1),p(s,l2),…,p(s,l j ),…,p(s,l n )}. The model is trained using the binary cross entropy loss function:

[0185]

[0186] Where, Ys={ys,l1 ,y s,l2 ,…,y s,lj ,...,y s,ln} represents the true label of service s, and n is the total number of labels. j ∈L exists in the label list of s, corresponding to y s,lj Set to 1, otherwise set to 0. j ) is the label prediction probability in formula (25).

[0187] The goal of loss is to s,lj is 1, minimizing the predicted l j is the negative logarithm of the probability of a positive example; when y s,lj is 0, minimizing the predicted l j is the negative logarithm of the probability of negative examples. Therefore, the label prediction probability p(s,l j ) and the true label y s,lj The closer the distance, the smaller the loss; conversely, the larger the loss. When the loss converges, the model achieves the optimal effect.

[0188] Simulation experiment results:

[0189] In the experimental test phase of tag recommendation, let PYs = {py s,l1 ,py s,l2 ,…,py s,lj ,...,py s,ln} is the label prediction set of service s. If p(s,l j )≥0.5, depending on the label l j is a positive example, which exists in the label list of s, corresponding to py s,lj Set the value to 1; if p(s,l j )<0.5, visual label l j is a negative example, which does not exist in the label list of s, corresponding to py s,lj Set the value to 0. Use the evaluation metric to evaluate the model performance based on the label prediction set PYs and the true label set Ys.

[0190] The present invention combines the semantic information of Web services with the collaborative association information between tags to assist Web service vectors in classification and recommendation. It has good adaptability to Web service data with different numbers of tags and uneven distribution, and can significantly improve the quality of category recommendation.

[0191] 5. Experimental Results and Analysis

[0192] 5.1 Dataset and Evaluation Criteria

[0193] We crawled real web services from the Programmable Web Service Registry Platform. After removing services with short descriptions or very few categories, we generated a dataset containing 21,328 web services. We used a ten-fold cross-validation method, with the average of the ten rounds of results serving as the final experimental result. We used the following commonly used classification evaluation metrics as evaluation indicators for our proposed method.

[0194] (1) Precision refers to the proportion of samples correctly predicted as positive examples to all predicted samples. This indicator reflects the precision of the model.

[0195]

[0196] (2) Recall rate refers to the ratio of samples correctly predicted as positive examples to all samples that are actually positive examples. This indicator can reflect the recall rate of the model.

[0197]

[0198] (3) The micro-F1 value is calculated by the weighted harmonic mean of precision and recall, which is a comprehensive measure of precision and recall.

[0199]

[0200] In the above formula, TP represents the number of positive samples predicted as positive examples, FP represents the number of negative samples predicted as positive examples, FN represents the number of positive samples predicted as negative examples, and FP represents the number of negative samples predicted as positive examples.

[0201] 5.2 Model Comparison

[0202] 5.2.1 Verification of the rationality of word embedding layer

[0203] Four different word embedding generation models, RoBERTa, Glove, ELMO, and Word2vec, are used in combination with the proposed MCFRASRU to generate service function vectors. The rationality of RoBERTa as a word embedding layer is verified by the recommendation accuracy of category labels.

[0204] Table 1. Verification of rationality of word embedding layer

[0205]

[0206]

[0207] As shown in Table 1, compared with Glove, ELMO, and Word2vec, using RoBERTa as the word embedding model improves the label category precision by 2.6%, 8.6%, and 4%, respectively; the recall rate increases by 2%, 1.1%, and 4.2%, respectively; and the F1 value increases by 2.2%, 3.2%, and 5.9%, respectively. This achieves higher-precision multi-label classification quality for web services, indicating that using RoBERTa as a word embedding model can generate higher-quality web service function vectors.

[0208] 5.2.2 Evaluation of Multi-channel and Fast Rule Approximate Attention Performance and Speed ​​Improvement

[0209] This section uniformly uses RoBERTa as the word embedding model. In the full-text semantic extraction module, the SRU model is compared with the FRASRU and MCFRASRU models to verify:

[0210] (1) The superiority of the multi-channel mechanism in improving the generation quality of Web service function vectors;

[0211] (2) The introduction of the fast rule approximate attention mechanism has the advantage of accelerating training speed;

[0212] This article uses min_time / time (a unit speed metric) to verify the training speed improvements for SRU, FRASRU, and MCFRASRU. The numerator, min_time, represents the average training time of the model with the shortest training time after 10 training rounds for each of the three models, while the denominator, time, represents the average training time for each model. Therefore, in the same set of comparative experiments, min_time is a fixed value. Models with a smaller average training time, time, have a larger min_time / time ratio, indicating faster training.

[0213] Table 2 Evaluation of multi-channel and fast rule approximate attention performance and speed improvement

[0214]

[0215] As shown in Table 2, the F1 index of FRASRU is improved by 1.08% compared with SRU, the recall rate and precision rate are both improved by about 1.1%, and the unit speed is increased by 7.3%. This shows that after the introduction of the fast rule approximate attention mechanism, the speed of SRU is significantly improved, and the accuracy of its multi-label classification of Web services is also slightly improved. That is, the fast rule approximate attention can improve the training speed of the model, improve the robustness of the model, and make it easier to train better results, making the model classification

[0216] Improved accuracy;

[0217] The F1 index of MCFRASRU is improved by 4.7% compared to FRASRU, and the recall rate and precision are improved by 5.9% and 5.9% respectively

[0218] The unit speed is improved by 0.5%, which shows that after introducing the multi-channel mechanism, the model can analyze the Web service description representation according to the local feature words of different granularities and their context association, achieving the effect of slightly improving the training speed while effectively improving the generation quality of Web service function vectors.

[0219] 5.2.3 Performance improvement comparison of MCFRASRU-LCGAT method (inventive method)

[0220] The method of adding label collaboration association verification in this patent is named MCFRASRU-LCGAT, and the comparative experiment with RoBERTa-MCFRASRU is carried out to verify the improvement of the recommendation quality after the integration of label collaboration association.

[0221] Table 3 Performance improvement comparison of MCFRASRU-Node method

[0222]

[0223]

[0224] As shown in Table 3, compared with RoBERTa-MCFRASRU, after integrating service labels for compatibility determination, MCFRASRU-LCGAT improves F1, precision and recall by 4.6%, 5% and 4.1% respectively, which shows that service label association can well balance the functional similarity of service labels and the compatibility of label combination, thereby improving the accuracy of compatibility determination. Specifically, the introduction of service label association can make the model better learn the similarity between labels and consider more relevant information, so that the compatibility determination is more accurate.

[0225] 5.2.4 Comparison of MCFRASRU-LCGAT method (inventive method) and other methods

[0226] The leading model in the field of multi-label text classification is selected together with the inventive method for Web service multi-label classification task experiment to verify the advancement of the method, which mainly includes the following methods:

[0227] (1) SGM: The multi-label classification task is regarded as a sequence generation problem. The encoder part of the sequence generation model is improved to integrate the correlation between labels, and the key information of the input text is automatically obtained through the attention mechanism to generate a text representation vector. The label sequence is serialized and the classification result is output.

[0228] (2) AttentionXML: A deep learning model based on a label tree is proposed. By using BiLSTM to capture the long-distance dependencies between words, labels can be mapped to different corresponding fragments in a given text. It is trained using a shallow and wide PLT and a top-down hierarchical model.

[0229] (3) DSRM-DNN: This paper proposes a multi-label text classification method that combines a dynamic semantic representation model and a deep neural network. It uses a word embedding model and a clustering algorithm to extract semantic words from the text. The selected words are used as the output of the DSRM-DNN model and are quantified by a weighted combination of word attributes in the text. Finally, a multi-label text classifier is constructed by combining a deep belief network and a backpropagation neural network.

[0230] (4) LightXML: This model is trained in an end-to-end manner and negative labels are sampled dynamically. The model generates sampled negative and positive labels in the label recall part, distinguishes positive labels from other labels in the label ranking part, and dynamically samples negative labels by using the same text representation.

[0231] (5) BERT multi-scale CNN: This method uses BERT to extract text sentence vectors and designs a multi-scale convolution module to extract text segments of different lengths and enrich the representation vectors to improve multi-label classification performance.

[0232] Table 4 Comparison of RoBERTa-MCFRASRU model with other methods

[0233]

[0234] As shown in Table 4, on a real-world Web service dataset, our proposed model achieves the highest scores across all metrics compared to methods without label collaboration and association integration. The quality of the generated Web service feature vectors is superior to that of other existing multi-label classification methods. Compared to the other five methods, the F1 index improves by 17.7%, 11.3%, 6.3%, 4%, and 3.1%, respectively; the precision improves by 17.3%, 10.6%, 6.4%, 2.1%, and 3%, respectively; and the recall improves by 18.1%, 12.1%, 6.1%, 6%, and 0.18%, respectively. This demonstrates that the results of multi-label recommendation for Web service categories using our proposed model are more consistent with real-world scenarios, and the Web service feature vectors it generates are of higher quality.

[0235] The above description is merely a preferred embodiment of the present invention and does not constitute any other form of limitation to the present invention. Any person skilled in the art may utilize the technical contents disclosed above to change or modify them into equivalent embodiments with equivalent changes for application in other fields. However, any simple modification, equivalent change, and modification of the above embodiments made in accordance with the technical essence of the present invention without departing from the technical solution of the present invention shall still fall within the scope of protection of the technical solution of the present invention.

Claims

1. A web service category tag recommendation method that integrates multi-channel semantic information and tag collaboration is characterized by: The following steps are involved: Step A: Crawl the web service. Step B: Clean the Web service description and construct the service function vector: Step B1: Establish a multi-channel word vector semantic feature extraction model to generate an aggregated word vector c containing local context feature information of different granularity window sizes. t ; The CNN convolutional layer network is constructed by using a variety of convolution kernels to construct semantic information extraction channels for feature words of different lengths. Each channel inputs the word vector v pre-trained by RoBERTa. t ; After processing through different channels, the final generated aggregate word vector c t Contains local context feature information of different granularity window sizes: c t =ReLU(With·V t:t+h-1 +b) (1) Among them, V t:t+h-1 ∈R d*h is the word vector matrix with the tth word as the center word and length h, and d represents the word vector v t The dimension of , b is the bias vector; Step B2: Establish a multi-channel sentence vector semantic feature extraction model MCFRASRU to generate a service function vector sfv(s) for the service description text that takes into account both local and global context features; Step B21: Build an SRU++ model with a fast attention mechanism: FRASRU, which is the aggregate word vector c t Further aggregate contextual features to generate a global word vector hw with global context features t : Step B22, hw t with c t The concatenation is used as the final word vector of the t-th word, and the text vector in the current channel is generated for the service description text information based on the attention mechanism; Step B23: averaging the text vectors generated by the service description text information in different channels to generate a service function vector for the Web service that integrates multi-channel semantic information; Step C: Determine the web service tag set and construct the tag collaboration vector: Step C1: construct a tag collaboration association graph of Web services based on the collaboration association between service tags; Step C2: Design a wandering strategy with tag combination popularity correction. Based on the biased wandering probability, design a node wandering probability calculation method that incorporates tag combination popularity. Step C3: Combine the node wandering probability strategy that incorporates the popularity of the tag combination with the Node2vec model to generate an initial collaboration vector for the tag corresponding to each node in the tag collaboration association graph; Step C4: Establish a graph attention network LCGAT that integrates edge weight calculation attention and outputs the final label collaboration vector; Step D: Combine the service function vector and the label collaboration vector to output the Web service category label.

2. The method for recommending web service category tags by integrating multi-channel semantic information and tag collaboration according to claim 1 is characterized by: Step B21 is implemented in the following manner: The self-attention mechanism in SRU++ is replaced by the fast rule approximate attention mechanism, and the functional semantics of each word is modeled from both the positive and negative directions. The positive semantic vector is h t1 , the reverse semantic vector is h t2 , concatenate the two to get word w t The global word vector hw of the global context feature t : hw t =[h t1 ||h t2 ] (12) Here, "||" indicates vector concatenation.

3. The method for recommending web service category tags by integrating multi-channel semantic information and tag collaboration according to claim 1 is characterized by: In step B22, considering that each word in the service description has different importance in generating the service function vector, the contribution of the word to the construction of the service function vector is calculated through the attention mechanism, and the weight vector u of each word is t for: u t =tanh(M g [hw t ||c t ]+b g ) (13) Among them, M g is the learnable parameter matrix, tanh() is the activation function, b g is the bias vector, and the weight vector is normalized to obtain the attention weight of each word: Combine the word vector at each position with the corresponding attention weight α t Multiply and sum to get the text vector sv(sd) of the service description sd in the current channel: sv(sd)=∑ t a t [hw t ||c t ] (15) Among them, hw t is the global word vector of the global context feature.

4. The method for recommending web service category tags by integrating multi-channel semantic information and tag collaboration according to claim 1 is characterized by: The final service function vector in step B23 is sfv(s): Where s is the Web service, m is the number of semantic channels, j = 1, 2, 3…m, sd is the service description text information, sv j (sd) is the text vector of s in the jth channel.

5. The method for recommending web service category tags by integrating multi-channel semantic information and tag collaboration according to claim 1 is characterized by: In the step C2, the following method is specifically adopted: (1) In the original Node2vec, for node v i , which is for the next hop node v j The probability reference formula is: Among them, π vivj is node v i and v j The unnormalized transition probability between, z is the normalization constant, π vivj =a pq (i,j)·w i,j , where w i,j is node v i and v j The edge weight between pq (i,j) is calculated by formula (18): Among them, d i,j Represents node v j and v i The shortest path distance between nodes, q and p are two hyperparameters used to define the transition probability between nodes; (2) Calculate the popularity of tag combinations: Where Ns(l) is the number of web services containing label l, and the numerator in formula (19) is label l i With l j The total number of Web services to which it belongs, Is(l i ) and Is(l j ) respectively indicate the inclusion of label l i With l j Web services, L is the service category label set; (3) Incorporating the label combination popularity into the biased random walk probability of Node2vec, the modified transition probability is: When Node2vec performs random walks, it uses the popularity of label combinations to correct the walk probability.

6. The method for recommending web service category tags by integrating multi-channel semantic information and tag collaboration according to claim 1 is characterized by: In the step C, the following method is used: (1) In the GAT model, for the node v in the label collaboration graph i Calculate the similarity coefficient with its first-order neighbor nodes: e i,j =a(W1u i ,W2u j ),j∈N i (21) Among them, u i 、u j v i With neighbor node v j Collaborative association vector, W1, W2∈R F×F ′ is a trainable matrix, F and F′ are the initialization vector dimension and output dimension of the current layer respectively, and a is the coefficient of the shared attention mechanism; (2) Construct the current layer bias vector: Among them, b l w is the current layer bias vector, b l’ w is the bias vector of the previous layer, W l w is the trainable weight matrix, u i is the current layer node v i Vector representation of ; (3) Combine the bias vector and edge weight w i,j Combined with the attention calculation process, a new attention coefficient is obtained: Therefore, according to the multi-head attention mechanism, v is obtained in the last layer i Corresponding label l i The final collaboration vector lcv(l i ): Among them, σ is the sigmoid activation function, a k i,j is the kth attention mechanism (a k ) calculated the normalized attention coefficient, W k is the weight matrix of the corresponding linear transformation of the input.

7. The method for recommending web service category tags by integrating multi-channel semantic information and tag collaboration according to claim 1 is characterized by: Described step D comprises the following steps: (1) The service function vector sfv(s) of service s is combined with the collaboration vector lcv(l j ) Through the MLP layer, we get the label l j The predicted probability of a label appearing in the label set of service s: p(s,l j )=sigmoid(W1sfv(s)+W2lcv(l j )+b) (25) Among them, W1 and W2 are trainable weight matrices, and b is the bias term; (2) Given a label set L = {l1,l2,…,l j ,…,l n }, the set of predicted label probabilities that the labels in L belong to the label set of service s is {p(s,l1),p(s,l2),…,p(s,l j ),…,p(s,l n )}, and train the model using the binary cross entropy loss function: Where, Ys={y s,l1 ,y s,l2 ,…,y s,lj ,...,y s,ln } represents the true label of service s, n is the total number of labels, if l j ∈L exists in the label list of s, corresponding to y s,lj Set the value to 1, otherwise set the value to 0, p(s,l j ) is the label prediction probability in formula (25); (3) When making tag recommendations, let PYs = {py s,l1 ,py s,l2 ,…,py s,lj ,...,py s,ln } is the label prediction set of service s, if p(s,l j )≥0.5, depending on the label l j is a positive example, which exists in the label list of s, corresponding to py s,lj Set the value to 1; if p(s,l j )<0.5, visual label l j is a negative example, which does not exist in the label list of s, corresponding to py s,lj Set the value to 0.

Images