Sequence perception service recommendation method based on double-tower model

Abstract

A sequence sensing service recommendation method based on a double-tower model comprises the following steps: step 1, creating a service data set, obtaining service interaction data, and constructing a service network diagram by using the double-tower model; 2, aiming at a graph structure in the service network graph, designing a query perception self-attention mechanism aggregation service embedding vector, and carrying out service feature extraction by utilizing a graph convolutional network; 3, further extracting important service information by using graph pooling taking graph coarsening as a core; and step 4, capturing the dynamically changing focus service and performing graph-level representation reading, and obtaining final output embedding according to the focus service representation and the graph-level representation reading so as to obtain the probability of interaction between the target service and the user. According to the invention, the construction of the service network diagram is realized, the service embedding is effectively aggregated, and the service recommendation accuracy is improved.

Description

Technical Field

[0001] The invention belongs to the field of service computing, and in particular relates to a sequence-aware service recommendation method based on a double-tower model. Background Art

[0002] Service-oriented computing (SOC) is the mainstream paradigm of Web applications. It uses the Internet to develop agile applications in the form of service calls, aiming to build flexible applications that can quickly respond to business needs. Web API is an application programming interface that can be reused and programmed. As a basic module for developers to quickly build applications, it has gradually become the backbone of the development of service-oriented computing.

[0003] Service recommendation technology can effectively help developers find suitable API services, attracting a large number of researchers in the field of service computing to study it. For example, Ren et al. proposed a collaborative filtering service recommendation method based on SVM, obtained a separating hyperplane from historical rating data, and then used the distance between the point representing the service and the separating hyperplane to measure the user's preference for a certain service, so as to recommend services. Wang et al. designed a structural interaction component to encode the potential structural relationship between Mashups and APIs in heterogeneous networks and capture the topological structure signals between different Mashups and APIs. Then, text embedding vectors were generated for Mashups and APIs to meet multi-level semantic requirements, and finally the output vectors were fused to obtain a service recommendation list.

[0004] The twin-tower model is a machine learning framework that constructs two independent neural network branches to process different data features respectively, and uses the interoperability layer to achieve effective fusion or comparison between features. It has received widespread attention in many fields such as search engines, recommendation systems, and social networks. For example, Su et al. introduced an all-to-all interaction module to enhance the interaction between queries and items, and designed a discrete code and sparse inverted index, which takes into account both the efficiency of retrieval and the accuracy of candidate item matching, providing a new idea for modeling complex data relationships. Lee et al. proposed a flexible twin-tower recommendation model to improve the accuracy of cold start recommendations. Although the twin-tower model has many applications in recommendation systems, no researchers in the field of service computing have applied it to service recommendations.

[0005] Sequential recommendation technology predicts the user's next behavior preference based on the user's historical behavior sequence modeling. Compared with traditional recommendation methods, sequential recommendation focuses on the user's behavior pattern and contextual information, thereby better capturing the user's preference changes and behavior patterns. For example, Du et al. applied sliding window technology to long user sequences in bidirectional Transformers to capture the user's behavior pattern in a more fine-grained manner, and combined cloze mask and discard mask to generate high-quality positive samples. Li et al. considered sequence patterns and high-order heterogeneous collaborative signals to improve recommendation performance. Zhang et al. proposed a framework that integrates user sequences and dynamic collaborative signals, connects different user sequences to construct a dynamic graph, and designs a dynamic graph recommendation network to capture user behavior preferences. However, in service recommendation, no researchers have paid attention to the dynamic changes in developer behavior, and lack consideration of the needs of personalized service recommendations.

[0006] Graph convolutional neural network was proposed by Thomas Kipf and Max Welling in 2016. As an extension of convolutional neural network, graph convolutional neural network can process non-Euclidean data. Its core idea is to iteratively aggregate the feature information of nodes and their neighbors in the graph to effectively capture the complex relationships in graph data. It is widely used in recommendation systems, transportation networks and knowledge graphs. Zhang et al. proposed a semantic variational graph autoencoder based on graph convolutional neural network to form a recommendation model. Wang et al. reused the initial feature information of the project using the knowledge graph-aware deep graph convolutional network with initial residual connections, and reused the feature information of the project at each layer using the knowledge graph-aware deep graph convolutional network with dense connections to improve the recommendation ability of high-order projects. Although the above methods have improved the accuracy of recommendations to a certain extent, they ignore the differences in complex signals in the interaction between users and API services. Treating all signals equally cannot accurately capture the role of different signals in API service mining. Summary of the invention

[0007] In order to overcome the shortcomings of the existing technology, the present invention proposes a sequence-aware service recommendation method based on the dual-tower model. The method first uses the dual-tower model to mine the similarity of service features and construct a service network graph, from which the focus services are differentiated; secondly, in the process of graph convolution, a query-aware self-attention mechanism is proposed to aggregate service information; then a graph pooling method with a coarsening strategy as the core is adopted to further extract service information, and dynamically capture the focus service and read out the graph-level representation; finally, the probability of the target service interacting with the user at the next moment is predicted.

[0008] The technical solution adopted by the present invention to solve its technical problem is:

[0009] A sequence-aware service recommendation method based on a dual-tower model, the method comprising the following steps:

[0010] Step 1: Create a service dataset, obtain service interaction data, and build a service network diagram using the dual-tower model;

[0011] Step 2: Design a query-aware self-attention mechanism to aggregate service embedding vectors based on the graph structure in the service network graph, and use a graph convolutional network to extract service features. The process is as follows:

[0012] Step (2.1) designs a query-aware self-attention mechanism in the graph convolutional network;

[0013] Step (2.2) adjusts the weight of the service embedding vector in information aggregation according to the user query, and updates the embedding representation through graph convolution operation;

[0014] Step 3: Use graph pooling with graph coarsening as the core to further extract important service information;

[0015] Step 4: Capture the dynamically changing focus service and read out the graph-level representation. The final output embedding is obtained based on the focus service representation and the graph-level representation readout, and then the probability of the target service interacting with the user is obtained. The process is as follows:

[0016] Step (4.1) The focal service is a densely connected service in the service network diagram, representing the main needs of users;

[0017] Step (4.2) captures the user's dynamically changing focus service under the multi-head self-attention mechanism;

[0018] Step (4.3) uses the average function to calculate the average node embedding of all service nodes after the forward calculation of the propagation layer, generates the graph-level representation of the service network graph G before coarsening, and assigns the result to The process is expressed in the formula It indicates that the symbol Mean represents the average function;

[0019] Step (4.4) predicts the probability of the user interacting with the target service and makes recommendations.

[0020] Further, the process of step 1 is as follows:

[0021] Step (1.1) The dual-tower model is a deep learning framework whose basic idea is to learn two different data features from two towers respectively, and then measure the similarity between the two input data;

[0022] Step (1.2) creates a service data set, crawls user information and API services from relevant websites, analyzes and organizes the collected results, and obtains a service data set;

[0023] Step (1.3) sorts the interaction sequences in chronological order to form an interaction sequence set;

[0024] Step (1.4) constructs a service network diagram and converts the interaction sequence set into a service network diagram through the dual tower model.

[0025] Preferably, in step (1.2), the service data set includes the following information:

[0026] Step (1.2.1) User: The developer who calls the API service, represented by the symbol u i express;

[0027] Step (1.2.2) API service: An interface that allows communication and data exchange between different software applications. It is the recommended object in service recommendation and is represented by symbol a i express;

[0028] Step (1.2.3) Interaction sequence: The API service called by the user at a timestamp constitutes an interaction sequence, represented by the symbol seq.

[0029] More preferably, the process of step (1.3) is:

[0030] Step (1.3.1) constructs the set of interaction sequences, represented by symbol S;

[0031] Step (1.3.2) All interaction sequences are arranged in chronological order to obtain an interaction sequence set. This process is done by Indicates, symbol m * Represents the total number of interaction sequences.

[0032] More preferably, in step (1.4), the process of constructing the service network graph is:

[0033] Step (1.4.1) defines the service network graph, represented by symbol G, the node set of G is represented by symbol V, the edge set of G is represented by E, and the adjacency matrix is ​​represented by A;

[0034] Step (1.4.2) replaces a in step (1.2.2) i Convert to node v i ;

[0035] Step (1.4.3) inputs the API service node into the dual-tower model to calculate the similarity between services and construct a service network graph.

[0036] The process of step (1.4.3) is:

[0037] Step (1.4.3.1) uses one-hot encoding to convert any two API service nodes into an embedding vector h i and h j , the one-hot encoding is an encoding method that converts categorical variables into numerical format;

[0038] Step (1.4.3.2) encodes the description document and label information of the API service in the interaction sequence;

[0039] Step (1.4.3.3) combines the API service embedding vector and the semantic vector to obtain a service vector containing semantic information. The process is expressed in the formula Indicates, symbol Represents the fusion function, which combines two features into a single representation using a weighted sum method, symbol θ i and the symbol ω i All are fusion function parameters;

[0040] Step (1.4.3.4) uses weighted cosine similarity to measure the similarity between two embedding vectors and assigns the result to sim i,j , the process is expressed as Indicates that the symbol cos(·) represents the cosine similarity measurement method, the symbol w represents a learnable weight vector, and the symbol ⊙ represents the Hadamard product. Weighted cosine similarity is a method for measuring the similarity between two vectors. It adds the concept of weight on the basis of cosine similarity to better reflect the importance of different features.

[0041] Step (1.4.3.5) takes any two API service nodes v with similarity i and v j Connect an edge between them, using the symbol (v i ,e i,j ,v j ) indicates that a service network diagram is constructed based on this;

[0042] The process described in (1.4.3.2) is:

[0043] Step (1.4.3.2.1) The description document of the API service is a document used to record the functions, features and interface information of the service;

[0044] Step (1.4.3.2.2) The service label is a short text used to identify the service function type;

[0045] Step (1.4.3.2.3) performs word segmentation and removes stop words on the description documents and tag information of any two API services;

[0046] Step (1.4.3.2.4) uses the BERT (Bidirectional Encoder Representations from Transformers) model to encode the description documents and label information of the two processed API services to obtain the semantic vector s iand j ,The BERT model is a natural language processing model;

[0047] Furthermore, the process of the query-aware self-attention mechanism in step (2.1) is:

[0048] Step (2.1.1) The embedding vector of the API service queried by the user i'th time is represented by Indicates that the superscript Indicates that the current service vector is the user's query vector;

[0049] Step (2.1.2) User query service embedding according to time order is represented by symbol Indicates that t * Represents the length of time;

[0050] Step (2.1.3) calculates the attention score of Q at different times and assigns the result to θ j , the process is expressed as Indicates that the symbol W q and W k is a trainable time parameter, Representatives will Transpose, the symbol d' represents the query representation dimension, For square root operation, Attention is a two-layer feedforward neural network with LeakyReLU (Leaky Rectified Linear Unit) as the activation function;

[0051] Step (2.1.4) calculates the attention weights in the API service node aggregation in a non-destructive way and assigns the result to β j , the process is expressed as Indicates, symbol W k ' and W v ' represents the learnable weight matrix, is the scaling factor to avoid the inner product being too large. represents the source node, represents the target node, softmax is an activation function, and the non-destructive method is to use only and The influence coefficient will not affect More efficient modeling is achieved by controlling the information source of the self-attention component.

[0052] The process of step (2.2) is:

[0053] Step (2.2.1) Embed the API service node into the matrix Input the l+1th layer, the symbol n represents the number of node embeddings, ∈ is a mathematical symbol that represents the relationship of belonging, represents a set of d-dimensional real vectors, where d is the embedding size of the node;

[0054] Step (2.2.2) calculates the service embedding representation of the l+1th layer and assigns the result to The process is expressed in the formula Indicates, symbol Represents node v i The set of neighbor nodes, ∑ is a summation symbol, v k Represents v i A neighbor node of .

[0055] Furthermore, the process of step 3 is as follows:

[0056] Step (3.1) uses the global graph content attribute extraction technology 2GCont (Global Graph Content) and the cross-attention mechanism CAMe (Cross-Attention Mechanism) to coarsen the graph. Graph coarsening is a technology used to reduce the size of the graph while retaining its overall structure.

[0057] Step (3.2) uses the service node embedding matrix before coarsening Generate the service cluster embedding of the coarsened graph and assign the result to This process is described by the formula Indicates that the symbol L represents the number of layers of the final graph convolution, the symbol m represents the number of service cluster embeddings, and the symbol M T Represents the transpose of the cross-attention mechanism M in step (3.1.2);

[0058] Step (3.3) uses the service node scores {γ1,γ2,...,γ n Generate the service clustering score of the coarsened graph and assign the result to The process is expressed in the formula Indicates, symbol M T Represents the transpose of the cross-attention mechanism M in step (3.1.2), symbol γ i is the value of θ in step (2.1.3) i The service node v obtained by applying softmax i Importance score.

[0059] Preferably, in the step (3.1), the graph coarsening process is as follows:

[0060] Step (3.1.1) calculates the global graph content matrix and assigns the result to C. This process is expressed as X represents,the symbol X represents the parameterized weight matrix;

[0061] Step (3.1.2) calculates the cross attention mechanism and assigns the result to M. This process is expressed in formula Indicates, symbol represents a trainable shared attention parameter, symbol represents the service node embedding of dimension N before coarsening, represents the service cluster embedding with dimension N' after coarsening, [·||·] represents the concatenation operation;

[0062] Step (3.1.3) for M ij Normalize and assign the result to M. The process is expressed as Indicates that the symbol exp is the natural exponential operation, and the symbol M ig represents all elements in the i-th row, and the symbol N' represents the number of columns in the i-th row of M;

[0063] The process of step (4.2) is:

[0064] Step (4.2.1) The multi-head self-attention mechanism is a variant of the self-attention mechanism, which can extract information from different subspaces at different positions;

[0065] Step (4.2.2) represents the cluster embedding matrix obtained in step (3.2) with the symbol H, that is, symbol Represents the i * Service cluster embedding;

[0066] Step (4.2.3) captures the dynamically changing focus service and assigns the result to The process is expressed in the formula Indicates that the symbol h * represents the total number of heads, the symbol || represents the splicing operation, and the symbol Represents from the i ** The embedding representation learned in each head is expressed by the formula Indicates that the symbol W O , and All of them are learnable parameters, and the attention mechanism ATTENTION(·) is implemented by scaling the dot product and the softmax function.

[0067] The process of step (4.4) is as follows:

[0068] Step (4.4.1) will and Concatenate and learn the embedding combination through the fully connected layer to get the final output embedding, and assign the result to The process is expressed in the formula It indicates that the symbol Readout represents the readout function;

[0069] Step (4.4.2) will Perform a dot product operation with the target service embedding to obtain the probability of the user interacting with the API service, and assign the result to The process is expressed in the formula express,

[0070] Step (4.4.3) constructs a cross entropy loss function training model and assigns the loss function result to The process is expressed in the formula The symbol |V| represents the service dataset, and the symbol r n',t Represents the true label, when r n',t =1 means the user has interacted with the service, r n',t =0 means that the user has not interacted with the service. The symbol log represents the logarithmic operation. The cross entropy loss function is a commonly used loss function in classification problems, which is used to measure the difference between the model output and the true label.

[0071] The beneficial effects of the present invention are mainly manifested in:

[0072] 1. For massive API service data, the dual-tower model is used to learn service embedding expressions, fully explore the correlation between services, and realize the construction of service network graphs;

[0073] 2. Design a query-aware self-attention mechanism in the graph convolutional network to avoid the interference of useless information and effectively aggregate service embedding;

[0074] 3. Utilize graph pooling operations with graph coarsening as the core to further capture high-order associations between services. The dynamic changes of the focus service are captured through the multi-head self-attention mechanism, and the service network graph before coarsening is read out in a weighted manner to effectively constrain the importance of each service node and improve the accuracy of service recommendations. BRIEF DESCRIPTION OF THE DRAWINGS

[0075] Figure 1 A diagram illustrating the process of building a service network diagram based on the two-tower model.

[0076] Figure 2 This is a diagram illustrating the hyper-parameter experimental results obtained by the present invention on the experimental dataset. DETAILED DESCRIPTION

[0077] The present invention will be further described below in conjunction with the accompanying drawings.

[0078] Reference Figure 1 and Figure 2 , a sequence-aware service recommendation method based on a dual-tower model, comprising the following steps:

[0079] Step 1: Create a service dataset, obtain service interaction data, and use the dual-tower model to build a service network diagram. Figure 1 The preprocessing in refers to the removal of segmentation and stop words; the process is as follows:

[0080] Step (1.1) The dual-tower model is a deep learning framework whose basic idea is to learn two different data features from two towers respectively, and then measure the similarity between the two input data;

[0081] Step (1.2) creates a service data set by crawling user information and API services from relevant online websites such as ProgrammableWeb and FastAPi, and analyzing and collating the collected results to obtain a service data set;

[0082] In the step (1.2), the service data set includes the following information:

[0083] Step (1.2.1) User: The developer who calls the API service, represented by the symbol u i express;

[0084] Step (1.2.2) API service: An interface that allows communication and data exchange between different software applications. It is the recommended object in service recommendation and is represented by symbol a i express.

[0085] Step (1.2.3) Interaction sequence: The API service called by the user at a timestamp constitutes an interaction sequence, represented by the symbol seq;

[0086] Step (1.3) sorts the interaction sequences in chronological order to form an interaction sequence set. The process is as follows:

[0087] Step (1.3.1) constructs the set of interaction sequences, represented by symbol S;

[0088] Step (1.3.2) All interaction sequences are arranged in chronological order to obtain an interaction sequence set. This process is done by Indicates, symbol m * represents the total number of interaction sequences;

[0089] Step (1.4) constructs a service network diagram, and converts the interaction sequence set into a service network diagram through the two-tower model;

[0090] In the step (1.4), the construction process of the service network diagram is as follows:

[0091] Step (1.4.1) defines the service network graph, represented by symbol G, the node set of G is represented by symbol V, the edge set of G is represented by E, and the adjacency matrix is ​​represented by A;

[0092] Step (1.4.2) replaces a in step (1.2.2) i Convert to node v i ;

[0093] Step (1.4.3) inputs the API service node into the dual-tower model to calculate the similarity between services and construct a service network diagram. The process is:

[0094] Step (1.4.3.1) uses one-hot encoding to convert any two API service nodes into an embedding vector h i and h j , the one-hot encoding is an encoding method that converts categorical variables into numerical format;

[0095] Step (1.4.3.2) encodes the description document and label information of the API service in the interaction sequence. The process is as follows:

[0096] Step (1.4.3.2.1) The description document of the API service is a document used to record the functions, features and interface information of the service;

[0097] Step (1.4.3.2.2) The service label is a short text used to identify the service function type;

[0098] Step (1.4.3.2.3) performs word segmentation and removes stop words on the description documents and tag information of any two API services;

[0099] Step (1.4.3.2.4) uses the BERT (Bidirectional Encoder Representations from Transformers) model to encode the description documents and label information of the two processed API services to obtain the semantic vector s i and j ,The BERT model is a natural language processing model;

[0100] Step (1.4.3.3) combines the API service embedding vector and the semantic vector to obtain a service vector containing semantic information. The process is expressed in the formula Indicates, symbol Represents the fusion function, which combines two features into a single representation using a weighted sum method, symbol θ i and the symbol ω i All are fusion function parameters;

[0101] Step (1.4.3.4) uses weighted cosine similarity to measure the similarity between two embedding vectors and assigns the result to sim i,j, the process is expressed as Indicates that the symbol cos(·) represents the cosine similarity measurement method, the symbol w represents a learnable weight vector, and the symbol ⊙ represents the Hadamard product. Weighted cosine similarity is a method for measuring the similarity between two vectors. It adds the concept of weight on the basis of cosine similarity to better reflect the importance of different features.

[0102] Step (1.4.3.5) takes any two API service nodes v with similarity i and v j Connect an edge between them, using the symbol (v i ,e i,j ,v j ) indicates that a service network diagram is constructed based on this;

[0103] Step 2: Based on the graph structure in the service network graph, a query-aware self-attention mechanism is designed to aggregate service embedding vectors and a graph convolutional network is used to extract service features. The process is as follows:

[0104] Step (2.1) designs a query-aware self-attention mechanism in the graph convolutional network. The process is:

[0105] Step (2.1.1) The embedding vector of the API service queried by the user i'th time is represented by Indicates that the superscript q′ i′ Indicates that the current service vector is the user's query vector;

[0106] Step (2.1.2) User query service embedding according to time order is represented by symbol Indicates that t * Represents the length of time;

[0107] Step (2.1.3) calculates the attention score of Q at different times and assigns the result to θ j , the process is expressed as Indicates that the symbol W q and W k is a trainable time parameter, Representatives will Transpose, the symbol d' represents the query representation dimension, For square root operation, Attention is a two-layer feedforward neural network with LeakyReLU (Leaky Rectified Linear Unit) as the activation function;

[0108] Step (2.1.4) calculates the attention weights in the API service node aggregation in a non-destructive way and assigns the result to β j , the process is expressed as Indicates, symbol W k ' and W v ' represents the learnable weight matrix, is the scaling factor to avoid the inner product being too large. represents the source node, represents the target node, softmax is an activation function, and the non-destructive method is to use only and The influence coefficient will not affect More efficient modeling by controlling the information source of the self-attention component;

[0109] Step (2.2) adjusts the weight of the service embedding vector in information aggregation according to the user query, and updates the embedding representation through graph convolution operation. The process is:

[0110] Step (2.2.1) Embed the API service node into the matrix Input the l+1th layer, the symbol n represents the number of node embeddings, ∈ is a mathematical symbol that represents the relationship of belonging, represents a set of d-dimensional real vectors, where d is the embedding size of the node;

[0111] Step (2.2.2) calculates the service embedding representation of the l+1th layer and assigns the result to The process is expressed in the formula Indicates, symbol

[0112] Represents node v i The set of neighbor nodes, ∑ is a summation symbol, v k Represents v i A neighbor node of

[0113] Step 3: Use graph pooling with graph coarsening as the core to further extract important service information. The process is as follows:

[0114] Step (3.1) uses the global graph content attribute extraction technology 2GCont (Global Graph Content) and the cross-attention mechanism CAMe (Cross-Attention Mechanism) to coarsen the graph. Graph coarsening is a technology used to reduce the size of the graph while retaining its overall structure. The process is:

[0115] Step (3.1.1) calculates the global graph content matrix and assigns the result to C. This process is expressed as Indicates that the symbol X represents the parameterized weight matrix;

[0116] Step (3.1.2) calculates the cross attention mechanism and assigns the result to M. This process is expressed in formula Indicates that the symbol a T represents a trainable shared attention parameter, symbol represents the service node embedding of dimension N before coarsening, represents the service cluster embedding with dimension N' after coarsening, [·||·] represents the concatenation operation;

[0117] Step (3.1.3) for M ij Normalize and assign the result to M. The process is expressed as Indicates that the symbol exp is the natural exponential operation, and the symbol M ig represents all elements in the i-th row, and the symbol N' represents the number of columns in the i-th row of M;

[0118] Step (3.2) uses the service node embedding matrix before coarsening Generate the service cluster embedding of the coarsened graph and assign the result to This process is described by the formula Indicates that the symbol L represents the number of layers of the final graph convolution, the symbol m represents the number of service cluster embeddings, and the symbol M T Represents the transpose of the cross-attention mechanism M in step (3.1.2);

[0119] Step (3.3) uses the service node scores {γ1,γ2,...,γ n Generate the service clustering score of the coarsened graph and assign the result to The process is expressed in the formula Indicates, symbol M T Represents the transpose of the cross-attention mechanism M in step (3.1.2), symbol γ i is the value of θ in step (2.1.3) i The service node v obtained by applying softmax i Importance score;

[0120] Step 4: Capture the dynamically changing focus service and read out the graph-level representation. The final output embedding is obtained based on the focus service representation and the graph-level representation readout, and then the probability of the target service interacting with the user is obtained. The process is as follows:

[0121] Step (4.1) The focal service is a densely connected service in the service network diagram, representing the main needs of users;

[0122] Step (4.2) captures the user's dynamically changing focus service under the multi-head self-attention mechanism. The process is:

[0123] Step (4.2.1) The multi-head self-attention mechanism is a variant of the self-attention mechanism, which can extract information from different subspaces at different positions;

[0124] Step (4.2.2) represents the cluster embedding matrix obtained in step (3.2) with the symbol H, that is, symbol Represents the i * Service cluster embedding;

[0125] Step (4.2.3) captures the dynamically changing focus service and assigns the result to The process is expressed in the formula Indicates that the symbol h * represents the total number of heads, the symbol || represents the splicing operation, and the symbol Represents from the i ** The embedding representation learned in each head is expressed by the formula Indicates that the symbol W O , and All of them are learnable parameters, and the attention mechanism ATTENTION(·) is implemented by scaling the dot product and the softmax function.

[0126] Step (4.3) uses the average function to calculate the average node embedding of all service nodes after the forward calculation of the propagation layer, generates the graph-level representation of the service network graph G before coarsening, and assigns the result to The process is expressed in the formula It indicates that the symbol Mean represents the average function;

[0127] Step (4.4) predicts the probability of the user interacting with the target service and makes recommendations. The process is:

[0128] Step (4.4.1) will and Concatenate and learn the embedding combination through the fully connected layer to get the final output embedding, and assign the result to The process is expressed in the formula It indicates that the symbol Readout represents the readout function;

[0129] Step (4.4.2) will Perform a dot product operation with the target service embedding to obtain the probability of the user interacting with the API service, and assign the result to The process is expressed in the formula express,

[0130] Step (4.4.3) constructs a cross entropy loss function training model and assigns the loss function result to The process is expressed in the formula The symbol |V| represents the service dataset, and the symbol r n',t Represents the true label, when r n',t =1 means the user has interacted with the service, rn',t =0 means that the user has not interacted with the service. The symbol log represents the logarithmic operation. The cross entropy loss function is a commonly used loss function in classification problems, which is used to measure the difference between the model output and the true label.

[0131] This embodiment analyzes the actual effect of the invention with specific service data, and the implementation plan is as follows:

[0132] Step 1: The data of Programmable Web from 2019 to 2020 are selected as the experimental dataset. The specific information is shown in Table 1. The dataset contains 3102 users, 11930 API service data, and 6206 interaction sequences.

[0133]

[0134] Table 1

[0135] Step 2: Test the service recommendation performance of the API service node under different embedding dimensions.

[0136] Step 3: Use HR indicators to evaluate the accuracy of service recommendation results. The calculation method is as follows:

[0137]

[0138] Among them, HR represents the accuracy of the recommendation result, hit(v i ) represents the service recommendation result, and N represents the actual interaction data between the user and the API service. The larger the value of the HR indicator, the higher the accuracy of the recommendation. When the length of the service recommendation list is 10, the HR indicator can be recorded as HR@10.

[0139] Step 4: Use the NDCG indicator to evaluate the relevance of the service recommendation results. The calculation method is as follows:

[0140]

[0141] Among them, NDCG represents the relevance of the recommendation results, DCG represents the weighted sum index of the relevance scores in the recommendation results, B represents the length of the recommendation list, r(b) represents the relevance score of the b-th API service, and IDCG represents the DCG in an ideal situation. The larger the value of the NDCG indicator, the higher the quality of the sorting result and the more relevant it is to the user's development needs. When the length of the service recommendation list is 10, the NDCG indicator can be recorded as NDCG@10.

[0142] Step 5: Use 70% of the experimental data in the dataset as the training set and the remaining 30% as the test set. Figure 2 shown.

[0143] according to Figure 2 The results show that the recommendation performance of the present invention improves with the increase of the embedding dimension. When the embedding dimension is 64, the performance reaches the best, and the HR@10 index and NDCG@10 index reach 0.5027 and 0.4821 respectively. After that, the recommendation performance gradually decreases as the embedding dimension increases. Analyzing the reasons, a lower embedding dimension may cause information loss, a higher embedding dimension can capture more information and make the model have better expression ability, but too high an embedding dimension may cause the model to overfit. In summary, it can be considered that the present invention has better recommendation accuracy and relevance when the embedding dimension is 64.

[0144] The contents described in the embodiments of this specification are merely enumerations of implementation forms of the inventive concept and are for illustrative purposes only. The protection scope of the present invention should not be considered to be limited to the specific forms described in this embodiment, and the protection scope of the present invention also extends to equivalent technical means that can be thought of by ordinary technicians in this field based on the inventive concept.

Claims

1. A sequence-aware service recommendation method based on a dual-tower model, characterized in that: The method comprises the following steps: Step 1: Create a service dataset, obtain service interaction data, and build a service network diagram using the dual-tower model; Step 2: Design a query-aware self-attention mechanism to aggregate service embedding vectors based on the graph structure in the service network graph, and use a graph convolutional network to extract service features. The process is as follows: Step (2.1) designs a query-aware self-attention mechanism in the graph convolutional network; Step (2.2) adjusts the weight of the service embedding vector in information aggregation according to the user query, and updates the embedding representation through graph convolution operation; Step 3: Use graph pooling with graph coarsening as the core to further extract important service information; Step 4: Capture the dynamically changing focus service and read out the graph-level representation. The final output embedding is obtained based on the focus service representation and graph-level representation readout, and then the probability of the target service interacting with the user is obtained. The process is as follows: Step (4.1) The focal service is a densely connected service in the service network diagram, representing the main needs of users; Step (4.2) captures the user's dynamically changing focus service under the multi-head self-attention mechanism; Step (4.3) uses the average function to calculate the average node embedding of all service nodes after the forward calculation of the propagation layer, generates the graph-level representation of the service network graph G before coarsening, and assigns the result to Step (4.4) predicts the probability of the user interacting with the target service and makes recommendations.

2. The sequence-aware service recommendation method based on the dual-tower model according to claim 1, characterized in that: The process of step 1 is as follows: Step (1.1) The dual-tower model is a deep learning framework whose basic idea is to learn two different data features from two towers respectively, and then measure the similarity between the two input data; Step (1.2) creates a service data set, crawls user information and API services from relevant websites, analyzes and organizes the collected results, and obtains a service data set; Step (1.3) sorts the interaction sequences in chronological order to form an interaction sequence set; Step (1.4) constructs a service network diagram and converts the interaction sequence set into a service network diagram through the dual tower model.

3. The sequence-aware service recommendation method based on the dual-tower model as claimed in claim 2, characterized in that: In the step (1.2), the service data set includes the following information: Step (1.2.1) User: The developer who calls the API service, represented by the symbol u i express; Step (1.2.2) API service: An interface that allows communication and data exchange between different software applications. It is a recommended object in service recommendation and is represented by the symbol a i express; Step (1.2.3) Interaction sequence: The API service called by the user at a timestamp constitutes an interaction sequence, represented by the symbol seq.

4. The sequence-aware service recommendation method based on the dual-tower model as claimed in claim 2, characterized in that: The process of step (1.3) is: Step (1.3.1) constructs the set of interaction sequences, represented by symbol S; Step (1.3.2) All interaction sequences are arranged in chronological order to obtain an interaction sequence set. This process is done by Indicates, symbol m * represents the total number of interaction sequences; In the step (1.4), the construction process of the service network diagram is as follows: Step (1.4.1) defines the service network graph, represented by symbol G, the node set of G is represented by symbol V, the edge set of G is represented by E, and the adjacency matrix is ​​represented by A; Step (1.4.2) replaces a in step (1.2.2) i Convert to node v i ; Step (1.4.3) inputs the API service node into the dual-tower model to calculate the similarity between services and construct a service network graph.

5. The sequence-aware service recommendation method based on the dual-tower model according to claim 4, characterized in that: The process of step (1.4.3) is: Step (1.4.3.1) uses one-hot encoding to convert any two API service nodes into an embedding vector h i and h j , the one-hot encoding is an encoding method that converts categorical variables into numerical format; Step (1.4.3.2) encodes the description document and label information of the API service in the interaction sequence; Step (1.4.3.3) combines the API service embedding vector and the semantic vector to obtain a service vector containing semantic information. The process is expressed in the formula Indicates, symbol Represents the fusion function, which combines two features into a single representation using a weighted sum method, symbol θ i and the symbol ω i All are fusion function parameters; Step (1.4.3.4) uses weighted cosine similarity to measure the similarity between two embedding vectors and assigns the result to sim i,j , the process is expressed as Indicates that the symbol cos(·) represents the cosine similarity measurement method, the symbol w represents a learnable weight vector, and the symbol ⊙ represents the Hadamard product. Weighted cosine similarity is a method for measuring the similarity between two vectors. It adds the concept of weight on the basis of cosine similarity to better reflect the importance of different features. Step (1.4.3.5) takes any two API service nodes v with similarity i and v j Connect an edge between them, using the symbol (v i ,e i,j ,v j ) indicates that a service network diagram is constructed based on this.

6. The sequence-aware service recommendation method based on the dual-tower model according to claim 5, characterized in that: The process described in (1.4.3.2) is: Step (1.4.3.2.1) The description document of the API service is a document used to record the functions, features and interface information of the service; Step (1.4.3.2.2) The service label is a short text used to identify the service function type; Step (1.4.3.2.3) performs word segmentation and removes stop words on the description documents and tag information of any two API services; Step (1.4.3.2.4) uses the BERT model to encode the description documents and label information of the two processed API services to obtain the semantic vector s i and j , the BERT model is a natural language processing model.

7. The sequence-aware service recommendation method based on the dual-tower model according to any one of claims 1 to 6, characterized in that: The process of the query-aware self-attention mechanism in step (2.1) is: Step (2.1.1) The API service embedding vector of the user's i'th query is represented by Indicates that the superscript q′ i′ Indicates that the current service vector is the user's query vector; Step (2.1.2) User query service embedding according to time order is represented by symbol Indicates that t * Represents the length of time; Step (2.1.3) calculates the attention scores of Q at different times and assigns the results to The process is expressed in the formula Indicates that the symbol W q and W k is a trainable time parameter, Representatives will Transpose, the symbol d' represents the query representation dimension, is a square root operation. Attention is a two-layer feedforward neural network with LeakyReLU as the activation function. Step (2.1.4) calculates the attention weights in the API service node aggregation in a non-destructive way and assigns the result to β j , the process is expressed as Indicates, symbol W k ' and W v ' represents the learnable weight matrix, is the scaling factor to avoid the inner product being too large. represents the source node, represents the target node, softmax is an activation function, and the non-destructive method is to use only and The influence coefficient will not affect More efficient modeling by controlling the information source of the self-attention component; The process of step (2.2) is: Step (2.2.1) Embed the API service node into the matrix Input the l+1th layer, the symbol n represents the number of node embeddings, ∈ is a mathematical symbol that indicates the relationship of belonging, represents a set of d-dimensional real vectors, where d is the embedding size of the node; Step (2.2.2) calculates the service embedding representation of the l+1th layer and assigns the result to The process is expressed in the formula Indicates, symbol Represents node v i The set of neighbor nodes, ∑ is a summation symbol, v k Represents v i A neighbor node of .

8. The sequence-aware service recommendation method based on the dual-tower model according to any one of claims 1 to 6, characterized in that: The process of step 3 is as follows: Step (3.1) uses the global graph content attribute extraction technology 2GCont and the cross attention mechanism CAMe to perform graph coarsening. Graph coarsening is a technology used to reduce the size of a graph while retaining its overall structure. Step (3.2) uses the service node embedding matrix before coarsening Generate the service cluster embedding of the coarsened graph and assign the result to This process is described by the formula Indicates that the symbol L represents the number of layers of the final graph convolution, the symbol m represents the number of service cluster embeddings, and the symbol M T Represents the transpose of the cross-attention mechanism M in step (3.1.2); Step (3.3) uses the service node scores {γ1,γ2,...,γ n Generate the service clustering score of the coarsened graph and assign the result to The process uses the formula Indicates, symbol M T Represents the transpose of the cross-attention mechanism M in step (3.1.2), symbol γ i is the value in step (2.1.3) The service node v obtained by applying softmax i Importance score.

9. The sequence-aware service recommendation method based on the dual-tower model according to claim 8, characterized in that: In the step (3.1), the graph coarsening process is as follows: Step (3.1.1) calculates the global graph content matrix and assigns the result to C. This process is expressed as Indicates that the symbol X represents the parameterized weight matrix; Step (3.1.2) calculates the cross attention mechanism and assigns the result to M. This process is expressed as Indicates that the symbol a T represents a trainable shared attention parameter, symbol represents the service node embedding of dimension N before coarsening, represents the service cluster embedding with dimension N' after coarsening, [·||·] represents the concatenation operation; Step (3.1.3) for M ij Normalize and assign the result to M. The process is expressed as Indicates that the symbol exp is the natural exponential operation, and the symbol M ig Represents all elements in the i-th row, and the symbol N' represents the number of columns in the i-th row of M.

10. The method for recommending sequence-aware services based on a dual-tower model according to any one of claims 1 to 6, characterized in that: The process of step (4.2) is: Step (4.2.1) The multi-head self-attention mechanism is a variant of the self-attention mechanism, which can extract information from different subspaces at different positions; Step (4.2.2) represents the cluster embedding matrix obtained in step (3.2) with the symbol H, that is, symbol Represents the i * Service cluster embedding; Step (4.2.3) captures the dynamically changing focus service and assigns the result to The process uses the formula Indicates that the symbol h * represents the total number of heads, the symbol || represents the splicing operation, and the symbol Represents from the i ** The embedding representation learned in each head is expressed by the formula Indicates that the symbol W O , and All are learnable parameters. The attention mechanism ATTENTION(·) is implemented by scaling dot products and softmax function. The process of step (4.4) is as follows: Step (4.4.1) will and Concatenate and learn the embedding combination through the fully connected layer to get the final output embedding, and assign the result to The process uses the formula It indicates that the symbol Readout represents the readout function; Step (4.4.2) will Perform a dot product operation with the target service embedding to obtain the probability of the user interacting with the API service, and assign the result to The process uses the formula express, Step (4.4.3) constructs a cross entropy loss function training model and assigns the loss function result to The process uses the formula The symbol |V| represents the service dataset, and the symbol r n',t Represents the true label, when r n',t =1 means the user has interacted with the service, r n',t =0 means that the user has not interacted with the service, and the symbol log represents logarithmic operation.

Images