Method, device and computer equipment for determining the applicability of graph neural network models

Abstract

The present application relates to the field of artificial intelligence technology and provides a method, apparatus, and computer device for determining the applicability of a graph neural network model. The method comprises: obtaining a graph data sample, the graph data sample including a training sample and a test sample, training a selected graph neural network model based on the training sample to obtain a target graph neural network model, inputting the test sample into the target neural network model, obtaining a link prediction result corresponding to the test sample based on the association relationship prediction result between the test sample nodes output by the target neural network model, and determining the applicability of the graph neural network model and the graph data sample based on the link prediction result. This method can determine whether the selected graph neural network model is suitable for analyzing graph data, and can quickly select a model suitable for graph data during the model selection process.

Description

Technical Field

[0001] The present application relates to the field of artificial intelligence technology, and in particular to a method, apparatus, and computer device for determining the applicability of a graph neural network model. Background Art

[0002] With the development of artificial intelligence technology, neural network models have been widely used in various fields, and deep learning has achieved great success.

[0003] Traditional machine learning uses data in Euclidean space. The most notable characteristic of data in Euclidean space is its regular spatial structure. For example, images are regular square grids, and speech data is a one-dimensional sequence. These data can be represented using one- or two-dimensional matrices, making convolution operations more efficient. At the same time, a core assumption is that samples are independent of each other. However, in real life, much data lacks a regular spatial structure. In other words, it exists in non-Euclidean space, such as abstract graphs from electronic transactions and recommendation systems. The connections between each node in the graph are not fixed. Graph neural networks can model data in non-Euclidean space and capture the data's internal dependencies. Graph neural networks are irregular and unordered. Graph neural networks include graph convolutional networks and graph attention networks.

[0004] Experiments have shown that not all tasks are suitable for graph neural networks, and graph neural networks also have limitations. For some tasks, the results achieved by applying graph neural networks are worse than those achieved by simple models. However, there is currently a lack of a method to determine the applicability of graph neural network models. Summary of the Invention

[0005] Based on this, it is necessary to provide a graph neural network model applicability determination method, device, computer equipment and storage medium that can determine the applicability of the graph neural network model in response to the above technical problems.

[0006] A method for determining the applicability of a graph neural network model, the method comprising:

[0007] Obtain graph data samples, which include training samples and test samples;

[0008] Train the selected graph neural network model according to the training samples to obtain the target graph neural network model;

[0009] The test sample is input into the target neural network model, and the link prediction result corresponding to the test sample is obtained based on the association relationship prediction result between the test sample nodes output by the target neural network model;

[0010] Based on the link prediction results, the applicability of the graph neural network model and the graph data sample is determined.

[0011] A device for determining the applicability of a graph neural network model, comprising:

[0012] A sample acquisition module is used to obtain graph data samples, which include training samples and test samples;

[0013] The model training module is used to train the selected graph neural network model based on the training samples to obtain the target graph neural network model;

[0014] The model testing module is used to input the test sample into the target neural network model, and obtain the link prediction result corresponding to the test sample based on the association relationship prediction result between the test sample nodes output by the target neural network model;

[0015] The applicability determination module is used to determine the applicability of the graph neural network model and graph data samples based on the link prediction results.

[0016] A computer device includes a memory and a processor, wherein the memory stores a computer program, and when the processor executes the computer program, the following steps are performed:

[0017] Obtain graph data samples, which include training samples and test samples;

[0018] Train the selected graph neural network model according to the training samples to obtain the target graph neural network model;

[0019] The test sample is input into the target neural network model, and the link prediction result corresponding to the test sample is obtained based on the association relationship prediction result between the test sample nodes output by the target neural network model;

[0020] Based on the link prediction results, the applicability of the graph neural network model and the graph data sample is determined.

[0021] A computer-readable storage medium stores a computer program, which, when executed by a processor, implements the following steps:

[0022] Obtain graph data samples, which include training samples and test samples;

[0023] Train the selected graph neural network model according to the training samples to obtain the target graph neural network model;

[0024] The test sample is input into the target neural network model, and the link prediction result corresponding to the test sample is obtained based on the association relationship prediction result between the test sample nodes output by the target neural network model;

[0025] Based on the link prediction results, the applicability of the graph neural network model and the graph data sample is determined.

[0026] A computer program product or computer program includes computer instructions stored in a computer-readable storage medium. A processor of a computer device reads the computer instructions from the computer-readable storage medium and executes the computer instructions, causing the computer device to perform the following steps:

[0027] Obtain graph data samples, which include training samples and test samples;

[0028] Train the selected graph neural network model according to the training samples to obtain the target graph neural network model;

[0029] The test sample is input into the target neural network model, and the link prediction result corresponding to the test sample is obtained based on the association relationship prediction result between the test sample nodes output by the target neural network model;

[0030] Based on the link prediction results, the applicability of the graph neural network model and the graph data sample is determined.

[0031] The above-mentioned graph neural network model applicability determination method, device, computer equipment and storage medium obtain graph data samples, train the selected graph neural network model according to the training samples in the graph data samples, obtain the target graph neural network model, implement model training, and use the test samples in the graph data samples to test the trained model. By inputting the test samples into the target neural network model, the link prediction results corresponding to the test samples are obtained according to the association relationship prediction results between the test sample nodes output by the target neural network model to detect whether the trained model has learned the structure of the graph data. According to the link prediction results, the applicability of the graph neural network model and the graph data samples is further determined to determine whether the selected graph neural network model is suitable for the analysis of graph data. In the process of model selection, a model suitable for graph data can be quickly selected. BRIEF DESCRIPTION OF THE DRAWINGS

[0032] Figure 1 Schematic diagram of an application environment of a method for determining the applicability of a neural network model in one embodiment;

[0033] Figure 2 Schematic diagram of a flow chart of a method for determining the applicability of a neural network model in one embodiment;

[0034] Figure 3 Schematic diagram of a flow chart of a method for determining the applicability of a neural network model in another embodiment;

[0035] Figure 4 Schematic diagram of a flow chart of a method for determining the applicability of a neural network model in another embodiment;

[0036] Figure 5 A schematic diagram of the relationship between graph data nodes in a method for determining the applicability of a graph neural network model in one embodiment;

[0037] Figure 6 A schematic diagram of the relationship between the semantic relevance accuracy and graph signal-to-noise ratio of a graph neural network model applicability determination method in another embodiment;

[0038] Figure 7 Schematic diagram of a flow chart of a method for determining the applicability of a neural network model in another embodiment;

[0039] Figure 8 Schematic diagram of a flow chart of a method for determining the applicability of a neural network model in another embodiment;

[0040] Figure 9 Schematic diagram of a flow chart of a method for determining the applicability of a neural network model in another embodiment;

[0041] Figure 10 Schematic diagram of a flow chart of a method for determining the applicability of a neural network model in another embodiment;

[0042] Figure 11 FIG is a structural block diagram of a device for determining the applicability of a neural network model in an embodiment;

[0043] Figure 12 FIG. 1 is a diagram showing the internal structure of a computer device in one embodiment. DETAILED DESCRIPTION

[0044] In order to make the purpose, technical solutions and advantages of this application more clear, the following further describes this application in detail with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain this application and are not intended to limit this application.

[0045] Artificial Intelligence (AI) refers to the theories, methods, techniques, and application systems that use digital computers or machines controlled by digital computers to simulate, extend, and expand human intelligence, to perceive the environment, acquire knowledge, and use that knowledge to achieve optimal results. In other words, AI is a comprehensive technology within computer science that seeks to understand the essence of intelligence and produce new intelligent machines that can respond in a manner similar to human intelligence. AI also involves studying the design principles and implementation methods of various intelligent machines, enabling them to possess the capabilities of perception, reasoning, and decision-making.

[0046] Artificial intelligence (AI) technology is a comprehensive discipline encompassing a wide range of fields, encompassing both hardware and software technologies. Foundational AI technologies generally include sensors, specialized AI chips, cloud computing, distributed storage, big data processing, operating / interaction systems, and mechatronics. AI software technologies primarily encompass computer vision, speech processing, natural language processing, and machine learning / deep learning.

[0047] With the research and advancement of artificial intelligence technology, artificial intelligence technology has been studied and applied in many fields, such as common smart homes, smart wearable devices, virtual assistants, smart speakers, smart marketing, unmanned driving, autonomous driving, drones, robots, smart medical care, smart customer service, etc. It is believed that with the development of technology, artificial intelligence technology will be applied in more fields and play an increasingly important role.

[0048] The solution provided in the embodiment of the present application relates to a method for determining the applicability of an artificial intelligence graph neural network model, which can be applied to Figure 1 In the application environment shown. The terminal 102 communicates with the server 104 through the network. The server 104 obtains the graph data samples uploaded by the user through the terminal 102, and the graph data samples include training samples and test samples; the selected graph neural network model is trained according to the training samples to obtain the target graph neural network model; the test sample is input into the target neural network model, and the link prediction result corresponding to the test sample is obtained according to the association relationship prediction result between the test sample nodes output by the target neural network model; according to the link prediction result, the applicability of the graph neural network model and the graph data sample is determined. The terminal 102 can be, but is not limited to, various personal computers, laptops, smart phones, tablet computers and portable wearable devices, and the server 104 can be implemented with an independent server or a server cluster composed of multiple servers. In other embodiments, the above-mentioned graph neural network model applicability determination method can also be implemented on a separate terminal or server with data processing capabilities.

[0049] In one embodiment, Figure 2 As shown in the figure, a method for determining the applicability of a graph neural network model is provided. Figure 1 The server in is used as an example to illustrate, including the following steps 202 to 208.

[0050] Step 202: Obtain graph data samples, which include training samples and test samples.

[0051] Graph data samples refer to graph data used for model training and testing. Graph data is a data representation of a node graph of related data. Related data can be a user's historical behavior data. Historical behavior data can be quite large, and the data associated with a particular historical behavior can be one or more items.

[0052] For example, if a user wants to search for "Double Color Ball," they enter "Double Color Ball" in the terminal's search box and send a search request to the server. In response to the user's search request sent through the terminal, the server returns relevant documents corresponding to "Double Color Ball" to the terminal. At this point, if the user clicks on one of the documents, A, the data associated with "Double Color Ball" will be the document A that the user clicked. If the user feels that none of the documents displayed on the current interface are the results they want, and re-enters "Welfare Lottery" in the search box, the data associated with "Double Color Ball" will be the "Welfare Lottery" that the user searched for again. It can be understood that if both of the above situations occur, the data associated with "Double Color Ball" will include both Document A and "Welfare Lottery." Graph data allows data with associations to be displayed through connected nodes.

[0053] Training samples refer to graph data samples used to train the model. Test samples refer to graph data samples used to test the model. Compared to graph data used in applications, the relationships between nodes in graph data samples can be accurately determined in advance. In applications, however, models are required to predict whether relationships exist between nodes.

[0054] In one embodiment, training samples and test samples can be obtained by allocating graph data samples in a proportional manner. For example, the ratio of training samples to test samples can be set to 9:1, and the graph data samples can be allocated at a 9:1 ratio to obtain training samples and test samples. In other embodiments, training samples and test samples can also be allocated according to other configuration rules. By allocating graph data samples to obtain training samples and test samples, the test samples can maintain a relatively similar level to the training samples, thereby improving the accuracy of the model test results.

[0055] Step 204: Train the selected graph neural network model based on the training samples to obtain a target graph neural network model.

[0056] In an embodiment, model training can be achieved through supervised learning. Supervised learning is a machine learning task that infers a function from labeled training data. The model is trained by predetermining the desired outcome of the training samples and using that outcome as the training objective. In an embodiment, the graph data samples carry annotation information, which represents the desired output, i.e., the relationships between nodes that the model is expected to predict.

[0057] The selected graph neural network model can be one of a model set consisting of multiple graph neural network models, and the graph neural network includes: graph convolutional neural network, graph attention network, etc. Different graph neural network models have certain differences in their processing capabilities and principles for graph data, so the processing results of graph data will also be different. In an embodiment, the selected graph neural network model can be randomly selected or selected according to the configured rules. For example, the configured rules can be selected based on the matching relationship between the application scenario of the pre-configured graph data and the graph neural network model. For example, if the application scenario of text search matches the graph convolutional neural network, the graph convolutional neural network can be preferentially selected for training. Among them, the matching relationship between the application scenario and the graph neural network model can be based on the applicability judgment results of the historical graph neural network model and the graph data sample or based on the actual application of the model.

[0058] The selected graph neural network model is trained based on the training samples to obtain the target graph neural network model. The training process of the graph neural network model can determine the termination condition of the training by limiting the number of iterations or by limiting the model accuracy.

[0059] In step 206, the test sample is input into the target neural network model, and the link prediction result corresponding to the test sample is obtained based on the association relationship prediction result between the test sample nodes output by the target neural network model.

[0060] During the training process, the target neural network model is essentially learning the graph structure. The server inputs the test samples into the target neural network model and processes the test samples according to the learning results of the target neural network model to predict the association relationship between the test sample nodes. The data output by the target neural network model is the predicted result of the association relationship between the test sample nodes.

[0061] The server calculates link prediction metrics based on the actual relationships between test sample nodes and the predicted relationships between test sample nodes to determine the difference between the predicted and actual results. The goal of link prediction is to derive new links (weights / features) based on known nodes and links, and to determine the accuracy of these new links.

[0062] In an embodiment, the link prediction result may be probability data with a value range of [0, 1]. In an embodiment, the link prediction result may be at least one of AUC (Area Under Curve, the area under the ROC curve and the coordinate axis, a performance indicator for measuring the quality of a learner) or Precision.

[0063] The AUC measures the overall accuracy of the algorithm. After training, the link prediction algorithm can obtain a similarity value, Sim (edge ​​similarity value), for each pair of nodes in the network. The AUC metric is based on a comparison of the similarity values ​​of edges in the test set with the similarity values ​​of non-existent edges (i.e., using the non-existent edges as the benchmark).

[0064] If Sim 测试 >Sim 不存在 , then the numerator of the value is increased by 1 (this proves that the prediction effect is good);

[0065] If Sim 测试 =Sim 不存在 , then the numerator of the value is increased by 0.5 (this is equivalent to random selection)

[0066] If Sim 测试 <Sim 不存在 , the numerator of the value is added with 0 (the prediction effect is very poor in this case);

[0067] The denominator of the value is the number of times the similarity value of the edge in the test set is compared with the similarity value of the non-existent edge. The AUC indicator is the ratio of the numerator to the denominator.

[0068] Precision only considers the accuracy of the predictions for the top L edges. After training, the link prediction algorithm obtains similarity values ​​between node pairs. Edges in the training set are removed, and only the similarity values ​​of the edges in the test set and the set of non-existent edges are sorted. The top L edges are selected after sorting. Assuming that N of the L edges belong to the test set, the Precision value is N / L.

[0069] Step 208: Determine the applicability of the graph neural network model and the graph data sample based on the link prediction results.

[0070] The link prediction result is a probability metric used to describe the accuracy of the prediction, with a value range of [0, 1]. Based on the link prediction result, we can distinguish between model problems and graph data problems. Specifically, when the link prediction result approaches 1, it indicates that the model has learned the graph structure, and the problem may lie in the graph data. When the link prediction result approaches 0, it indicates that the model has not learned the graph structure, and the problem lies in the selected model.

[0071] The above-mentioned method for determining the applicability of a graph neural network model obtains graph data samples, trains the selected graph neural network model according to the training samples in the graph data samples, obtains a target graph neural network model, implements model training, and uses test samples in the graph data samples to test the trained model. By inputting the test samples into the target neural network model, the link prediction results corresponding to the test samples are obtained according to the association relationship prediction results between the test sample nodes output by the target neural network model to detect whether the trained model has learned the structure of the graph data. According to the link prediction results, the applicability of the graph neural network model and the graph data samples is further determined to determine whether the selected graph neural network model is suitable for the analysis of graph data. In the process of model selection, a model suitable for graph data can be quickly selected.

[0072] In one embodiment, Figure 2 As shown, based on the link prediction results, the applicability of the graph neural network model and the graph data sample is determined, that is, step 208, including steps 302 to 304.

[0073] Step 302: When the link prediction result is less than the preset index threshold, it is determined that the graph neural network model is not suitable for the graph data sample.

[0074] Step 304: When the link prediction result is not less than the preset indicator threshold, the applicability of the graph neural network model and the graph data sample is determined based on the graph signal-to-noise ratio of the test sample.

[0075] The preset indicator threshold is a limit value set based on the specific application scenario requirements and used to evaluate link prediction results. Taking the preset indicator threshold of 0.5 as an example, when the link prediction result is less than 0.5, such as the link prediction result of 0.2, the representation model has not learned the structure of the graph, and the problem lies in the selected model. In this case, it is determined that the graph neural network model is not suitable for the graph data sample. Furthermore, if there is a problem with the selected model, the model can be retrained by replacing the graph neural network model. When the link prediction result is not less than 0.5, such as the link prediction result of 0.8, the representation model has learned the structure of the graph, and the problem may lie in the graph data. In this case, the graph data can be further judged. Specifically, the applicability of the graph neural network model to the graph data sample can be determined based on the graph signal-to-noise ratio of the test sample.

[0076] In the above embodiment, on the one hand, the judgment process of the situation where the graph neural network model is not suitable for the graph data sample is simplified. Through the judgment condition that the link prediction result is less than the preset index threshold, the inappropriate models can be quickly screened. On the other hand, based on the combination of link prediction results and graph signal-to-noise ratio, the applicability of the graph neural network model and the graph data sample is judged through secondary detection, which can achieve effective selection of the graph neural network model suitable for graph data analysis.

[0077] In one embodiment, when the link prediction result is not less than a preset indicator threshold, the applicability of the graph neural network model and the graph data sample is determined based on the graph signal-to-noise ratio of the test sample, i.e., step 304, including steps 402 to 408.

[0078] Step 402 : When the link prediction result is not less than a preset index threshold, the average semantic similarity between the node and the first-order neighbor nodes is determined based on the node corresponding data in the test sample and the first-order neighbor node corresponding data associated with the node.

[0079] Step 404: Obtain the image signal-to-noise ratio of the test sample based on the average semantic similarity.

[0080] Step 406: When the graph signal-to-noise ratio is less than the preset graph signal-to-noise ratio threshold, it is determined that the graph data sample is not suitable for the graph neural network model.

[0081] Step 408: When the graph signal-to-noise ratio is not less than the preset graph signal-to-noise ratio threshold, it is determined that the graph data sample is suitable for the graph neural network model.

[0082] The nodes in the test sample refer to the nodes that constitute the graph data, and the nodes are associated with each other through edges. For any node, the node that can be directly associated with it by one edge is the first-order neighbor node associated with it. Similarly, the node that needs to be indirectly associated with it by at least two edges is the second-order neighbor node associated with it, and so on. In an embodiment, the graph data can be a directed graph, that is, the edge from the first node to the second node has an indicated direction, wherein the indicated direction can be represented by an arrow. In an embodiment, the first-order neighbor node of a directed graph is a node that is directly adjacent to a node along the direction indicated by the arrow. In other embodiments, the graph data can also be an undirected graph, that is, the direct association relationship between the nodes does not have a clear directional relationship, and as long as the nodes can be connected by an edge, they are first-order neighbor nodes to each other.

[0083] The data corresponding to a node refers to the content used to characterize the node. For example, the data corresponding to a node in a data search scenario is the content searched by the user and the content of the text clicked. In an embodiment, the data search scenario involves a user historical behavior session, and a session refers to the user's search behavior over a period of time, including switching queries (questions) and clicking on docs (documents). Among them, query and doc are heterogeneous. Specifically, two texts are heterogeneous in that the semantic spaces of text A and text B to be matched are different. For example, text A corresponds to the query input by the user, and text B corresponds to the retrieved document. Heterogeneity is reflected in the following aspects: the query length is generally shorter, while the doc length is longer. The words that make up the query are more colloquial, and the words that make up the doc contain more proper nouns. In an embodiment, the user's search, click and other behaviors in the service search will leave records in the background, and these data can be obtained offline to obtain the user's historical behavior session.

[0084] In an embodiment, the average semantic similarity between a node and its first-order neighbor nodes can be calculated based on the text corresponding to the node in the test sample and the text corresponding to the first-order neighbor nodes associated with the node. The average semantic similarity can be achieved using any method such as cosine similarity, Euclidean distance, Minkowski distance, Jaccard similarity coefficient, Pearson correlation coefficient, or Manhattan distance.

[0085] It should be noted that if a node is a query, its associated first-order neighbor node can be a query or a doc. Figure 5 As shown in the figure, after searching for "Double Color Ball" (q1), the user finds that the options on the corresponding page don't have the data they want. They then search for "Welfare Lottery" (q2) again, click "Welfare Lottery Results" in the options for "Welfare Lottery" to jump to a new page (d1). They then find that this new page doesn't have the data they need. They then return to the previous page and search for "Lottery Results" (q3) again. Click "Lottery Results" in the options for "Lottery Results" to jump to a new page (d2). In the graph data, q1, q2, q3, d1, and d2 are nodes. q1's first-order neighbor is q2, q2's first-order neighbors are q3 and d1, and q3's first-order neighbor is d2.

[0086] Figure 1. Signal-to-Noise Ratio (SNR) i Used to characterize the situation of noise data in graph data, where the noise data can be used to determine whether the relationship between two nodes is noise data by the average semantic similarity between the node and the first-order neighbor node.

[0087]

[0088] Among them, match-score (text i ,text j ) represents the average semantic similarity score between node i and node j, N i It refers to the number of first-order neighbor nodes of the i-th node.

[0089] When the graph signal-to-noise ratio is less than the preset graph signal-to-noise ratio threshold, the graph data sample is determined to be unsuitable for the graph neural network model. When the graph signal-to-noise ratio is not less than the preset graph signal-to-noise ratio threshold, the graph data sample is determined to be suitable for the graph neural network model.

[0090] like Figure 6 As shown in the figure, by calculating the semantic relevance accuracy of neighboring nodes in the graph data, it is found that the semantic relevance accuracy is strongly correlated with the graph signal-to-noise ratio. This further verifies the reliability of using the graph signal-to-noise ratio to determine the suitability of graph data samples for the graph neural network model. Specifically, the semantic relevance accuracy can be obtained by calculating the proportion of semantically relevant nodes in the manually annotated embedding neighboring nodes. The calculation formula is as follows:

[0091]

[0092] Among them, label ij Represents the manually labeled labels between neighboring nodes i and j, test_set i Refers to the test set corresponding to the i-th node.

[0093] If the graph signal-to-noise ratio (SNR) is lower than the preset SNR threshold, it indicates that the data distribution of the current graph data is not suitable for the graph neural network model. In this case, you should clean the data to improve the SNR, or switch to another model type, such as a deep neural network model.

[0094] In one embodiment, Figure 7 As shown, when the image signal-to-noise ratio is less than the preset image signal-to-noise ratio threshold, step 702 is further included.

[0095] Step 702 : performing data cleaning on the graph data sample until the graph signal-to-noise ratio of the graph data sample is not less than a preset graph signal-to-noise ratio threshold.

[0096] It is understandable that step 702 may be performed simultaneously with the determination step in step 406 , or may be performed before or after the determination step.

[0097] Data cleaning refers to the process of processing noise data in graph data samples. Specifically, the semantic similarity between nodes can be used to determine whether the relationship between nodes is noise data. When the semantic similarity between nodes is less than a preset similarity threshold, the relationship between the nodes can be determined to be noise data. When the semantic similarity between the nodes is not less than the preset similarity threshold, the relationship between the nodes is determined to be normal data.

[0098] Data cleaning can remove noise from graph data samples to adjust their distribution and make them suitable for the selected graph neural network model. If, after data cleaning, the graph data sample's signal-to-noise ratio (SNR) still cannot be maintained at or above the preset SNR threshold, the selected graph neural network model should be replaced.

[0099] In one embodiment, Figure 8 As shown, when the link prediction result is less than the preset index threshold, step 802 is further included. It is understood that step 802 can be performed simultaneously with the determination step in step 406, or can be performed before or after the determination step.

[0100] Step 802: Based on a preset graph neural network model set, the selected graph neural network model is replaced until the replaced graph neural network model meets the preset test conditions after model training based on the training samples.

[0101] The test conditions include that the link prediction result corresponding to the test sample is not less than a preset indicator threshold, and the graph signal-to-noise ratio corresponding to the test sample is not less than a preset graph signal-to-noise ratio threshold.

[0102] Model replacement can be implemented based on multiple graph neural network models in the model collection. The replacement object can be any unselected one in the model collection, or the one with the highest matching degree can be selected from the model collection based on the configured selection rules.

[0103] For the replaced graph neural network model, it is necessary to retrain and test the model based on the graph data samples. The specific training and testing process is the same as the training and testing process of the graph neural network model before the replacement. The link prediction results and graph signal-to-noise ratio corresponding to the test samples can also be obtained in the same way, so they will not be repeated here.

[0104] If the replaced graph neural network model is trained based on the training samples and then tested using the test samples, and the link prediction results corresponding to the test samples are not less than the preset indicator threshold, and the graph signal-to-noise ratio corresponding to the test samples is not less than the preset graph signal-to-noise ratio threshold, then it can be determined that the replaced graph neural network model has good applicability with the graph data samples.

[0105] By replacing the graph neural network model and training and testing the replaced graph neural network model, it is possible to find a graph neural network model suitable for analyzing graph data samples.

[0106] In one embodiment, Figure 9 As shown, obtaining graph data samples includes steps 902 to 904.

[0107] Step 902: Obtain user historical behavior data.

[0108] Step 904: Obtain a graph data sample based on the recorded data corresponding to user operations in the user's historical behavior data. User operations can be reflected through user historical behavior data sessions. A session includes the user's search behavior over a period of time, including at least one of a search switch operation (switching queries) and a view operation (clicking on a document). A search switch operation refers to an operation in which a user successively enters at least two search keywords to conduct a search. The recorded data corresponding to the search switch operation includes the at least two search keywords entered successively. The recorded data corresponding to the search switch operation is used to characterize the similarity between the at least two search keywords entered successively. For example, a search switch operation may be an operation in which a user enters a first search keyword to conduct a search, then enters a second search keyword to conduct another search. In this case, the recorded data corresponding to the search switch operation can be used to characterize the similarity between the first and second search keywords. A view operation refers to an operation in which a user enters a search keyword, triggering a search result and entering the search result data details viewing interface. The recorded data corresponding to the view operation includes the entered search keyword and the text corresponding to the search result. The recorded data corresponding to the view operation can be used to characterize the correlation between the search keyword and the search result. Corresponding user historical behavior data is recorded for each user. By performing graph modeling on the session, graph data corresponding to the user's user historical behavior data is constructed.

[0109] In one embodiment, obtaining a graph data sample based on recorded data corresponding to user operations in historical user behavior data includes: establishing connections between the nodes using the recorded data corresponding to the user operations as nodes and the user operations as edges between the nodes; determining weights corresponding to the edges based on the number of operations corresponding to the user operations; and obtaining a graph data sample based on the nodes, edges, and weights corresponding to the edges.

[0110] It can be understood that the user operation can be only a search switching operation or only a viewing operation, or it can include both a search switching operation and a viewing operation. At the same time, the search switching operation and the viewing operation are used as data processing objects, and the words associated with the input search terms and the text associated with the input search terms can be obtained, so as to facilitate the subsequent accurate search results.

[0111] In one embodiment, taking a scenario involving both search switching and viewing operations as an example, the query and document are considered nodes, and switching the query or clicking the document is considered an edge. The edge weight is determined by the proportion of the query volume (qv) of the switching query or clicking the document in the user's historical behavior data. Specifically, for a node A, it has four edges, where edge 1 has 1 repetition, edge 2 has 2 repetitions, edge 3 has 3 repetitions, and edge 4 has 4 repetitions. The sum of all operations corresponding to this node is 1+2+3+4=10. Therefore, the weight of edge 1 is 1 / 10=0.1, the weight of edge 2 is 2 / 10=0.2, the weight of edge 3 is 3 / 10=0.3, and the weight of edge 4 is 4 / 10=0.4. A graph data sample is constructed based on the nodes, the edges between nodes, and the corresponding edge weights. By calculating edge weights, the strength of the associations between data can be more accurately and effectively described, improving the accuracy of the graph data.

[0112] In one embodiment, Figure 10 As shown, a method for determining the applicability of a graph neural network model is provided, which specifically includes the following steps 1002 to 1026.

[0113] Step 1002: Obtain user historical behavior data.

[0114] Step 1004 : Using the record data corresponding to the user operation as a node and the user operation as an edge between the nodes, a connection relationship between the nodes is established. The user operation includes a search switching operation and a viewing operation.

[0115] Step 1006: Determine the weight corresponding to the edge according to the number of operations corresponding to the user operation.

[0116] Step 1008: Obtain graph data samples based on the nodes, edges, and weights corresponding to the edges, and divide the graph data samples into training samples and test samples.

[0117] Step 1010: Train the selected graph neural network model based on the training samples to obtain a target graph neural network model.

[0118] Step 1012: Input the test sample into the target neural network model, and obtain the link prediction result corresponding to the test sample based on the association relationship prediction result between the test sample nodes output by the target neural network model.

[0119] Step 1014: When the link prediction result is less than the preset index threshold, it is determined that the graph neural network model is not suitable for the graph data sample, and the process goes to step 1026.

[0120] Step 1016 , when the link prediction result is not less than the preset index threshold, the average semantic similarity between the node and the first-order neighbor nodes is determined based on the node corresponding data in the test sample and the first-order neighbor node corresponding data associated with the node.

[0121] Step 1018: Obtain the image signal-to-noise ratio of the test sample based on the average semantic similarity.

[0122] Step 1020: When the graph signal-to-noise ratio is less than the preset graph signal-to-noise ratio threshold, it is determined that the graph data sample is not suitable for the graph neural network model, and the process goes to step 1024 or step 1026.

[0123] Step 1022: When the graph signal-to-noise ratio is not less than the preset graph signal-to-noise ratio threshold, it is determined that the graph data sample is suitable for the graph neural network model.

[0124] Step 1024 : performing data cleaning on the graph data sample until the graph signal-to-noise ratio of the graph data sample is not less than a preset graph signal-to-noise ratio threshold.

[0125] Step 1026: Based on the preset graph neural network model set, the selected graph neural network model is replaced until the replaced graph neural network model meets the preset test conditions after model training based on the training samples.

[0126] This application also provides an application scenario, which applies the above-mentioned graph neural network model applicability determination method. Specifically, the application of the graph neural network model applicability determination method in WeChat search is used as an example for explanation.

[0127] SouYisou is a function in WeChat that can search Moments, articles, public accounts, novels, music, emoticons, etc. based on keywords. In SouYisou, it specifically refers to a certain type of result search, such as public account search, mini-program search, etc., which belong to vertical search. Among them, vertical search is a professional search engine for a certain industry. It is a subdivision and extension of the search engine. It integrates a certain type of specialized information in the library, extracts the required data by targeted fields, processes it, and then returns it to the user in some form. Vertical search also includes service search. Service search can directly display services that meet the user's query to the user. For example, when searching for a nanny, service search can directly provide a menu for finding nanny services.

[0128] The online processing of service search involves query intent identification, recall ranking, refined ranking, and finally the posterior module. This solution, applied to data mining in the posterior module, aims to offline discover highly correlated query-doc sets based on user behavior data. For example, if user logs reveal that most users click on a certain doc for a certain query, we can conclude that the two query-doc sets are highly correlated. These highly correlated query-doc sets serve as the foundational data for service search. By ensuring the reproducibility of these query-doc sets, we ensure a stable service search experience.

[0129] Using graph neural networks as a model can solve the query-doc correlation detection problem, but it is found that there is still room for improvement. Based on this, a method to measure the feasibility of graph neural network models is proposed, which can provide guidance for future solution selection.

[0130] First, based on the user's historical behavior session, we use the query and document as nodes, switching the query or clicking the document as the edge, and the proportion of the number of times the query is switched or the document is clicked in the user's historical behavior data as the weight of the edge to obtain the graph data.

[0131] Step 1: Train the selected graph neural network model (graph convolutional network GCN or graph attention network GAT) based on the training data in the graph data.

[0132] Step 2 calculates the link prediction metric based on the test data in the graph data to distinguish between model problems and graph data problems. If the link prediction task performs poorly, it indicates that the selected model has not learned the graph structure and the problem lies with the selected model. A new model needs to be selected and Step 1 repeated.

[0133] If the Link prediction task performs well, the selected representation model has learned the graph structure. The problem lies not in the model but in the data, so we proceed to Step 3.

[0134] Step 3: Calculate the signal-to-noise ratio (SNR) of the test data. i (signal noise ratio), which is the average semantic similarity between a node and its first-order neighbor nodes.

[0135]

[0136] By calculating the semantic relevance accuracy (the proportion of semantically relevant nodes in manually labeled embedding neighbor nodes), it can be found that the semantic relevance accuracy is strongly correlated with the graph signal-to-noise ratio.

[0137]

[0138] If the graph signal-to-noise ratio (SNRi) is very low, it indicates that the current data distribution is not suitable for a graph neural network model. In this case, data cleaning should be performed to improve the graph signal-to-noise ratio, or a different model type, such as a deep neural network, should be used. This process can quickly identify any issues with the matching of the data to the selected graph neural network solution. This allows accurate identification of the root cause of any problems during the model selection process, minimizing unnecessary steps and wasted effort, and enabling rapid selection of a model suitable for the graph data.

[0139] It should be understood that, although the various steps in the flowcharts involved in the above-mentioned embodiments are displayed in sequence according to the instructions of the arrows, these steps are not necessarily executed in sequence in the order indicated by the arrows. Unless otherwise specified herein, there is no strict order restriction on the execution of these steps, and these steps can be executed in other orders. Moreover, at least a portion of the steps in the flowcharts involved in the above-mentioned embodiments may include multiple steps or multiple stages, and these steps or stages are not necessarily executed at the same time, but can be executed at different times, and the execution order of these steps or stages is not necessarily to be carried out in sequence, but can be executed in turn or alternately with other steps or at least a portion of steps or stages in other steps.

[0140] In one embodiment, Figure 11 As shown, a graph neural network model applicability determination device 1100 is provided. The device can adopt a software module or a hardware module, or a combination of the two to form a part of a computer device. The device specifically includes: a sample acquisition module 1102, a model training module 1104, a model testing module 1106 and a applicability determination module 1108, wherein:

[0141] A sample acquisition module 1102 is used to acquire graph data samples, where the graph data samples include training samples and test samples;

[0142] A model training module 1104 is used to train the selected graph neural network model based on the training samples to obtain a target graph neural network model;

[0143] The model testing module 1106 is used to input the test sample into the target neural network model, and obtain the link prediction result corresponding to the test sample based on the association relationship prediction result between the test sample nodes output by the target neural network model;

[0144] The applicability determination module 1108 is used to determine the applicability of the graph neural network model and the graph data sample based on the link prediction results.

[0145] In one embodiment, the applicability determination module is also used to determine that the graph neural network model is not applicable to the graph data sample when the link prediction result is less than a preset indicator threshold; when the link prediction result is not less than the preset indicator threshold, the applicability of the graph neural network model to the graph data sample is determined based on the graph signal-to-noise ratio of the test sample.

[0146] In one embodiment, the applicability determination module is also used to determine the average semantic similarity between the node and the first-order neighbor nodes based on the corresponding data of the node in the test sample and the corresponding data of the first-order neighbor nodes associated with the node when the link prediction result is not less than a preset index threshold; obtain the graph signal-to-noise ratio of the test sample based on the average semantic similarity; when the graph signal-to-noise ratio is less than the preset graph signal-to-noise ratio threshold, determine that the graph data sample is not suitable for the graph neural network model; when the graph signal-to-noise ratio is not less than the preset graph signal-to-noise ratio threshold, determine that the graph data sample is suitable for the graph neural network model.

[0147] In one embodiment, the graph neural network model applicability determination device further includes a data cleaning module, which is used to perform data cleaning on the graph data sample until the graph signal-to-noise ratio of the graph data sample is not less than a preset graph signal-to-noise ratio threshold.

[0148] In one embodiment, the graph neural network model applicability determination device also includes a model replacement module, which is used to replace the selected graph neural network model based on a preset graph neural network model set until the replaced graph neural network model meets the preset test conditions after model training based on the training sample, wherein the test conditions include that the link prediction result corresponding to the test sample is not less than a preset indicator threshold, and the graph signal-to-noise ratio corresponding to the test sample is not less than a preset graph signal-to-noise ratio threshold.

[0149] In one embodiment, the sample acquisition module is also used to obtain user historical behavior data; based on the record data corresponding to the user operations in the user historical behavior data, a graph data sample is obtained, the user operations include at least one of a search switching operation and a viewing operation, the record data corresponding to the search switching operation is used to characterize the similarity between at least two search keywords successively obtained based on the search switching operation, and the record data corresponding to the viewing operation is used to characterize the correlation between the search keywords and the search results.

[0150] In one embodiment, the sample acquisition module is also used to use the recorded data corresponding to the user operation as the node and the user operation as the edge between the nodes to establish a connection relationship between the nodes; determine the weight corresponding to the edge based on the number of operations corresponding to the user operation; and obtain a graph data sample based on the nodes, edges and the weights corresponding to the edges.

[0151] The above-mentioned graph neural network model applicability judgment device obtains graph data samples, trains the selected graph neural network model according to the training samples in the graph data samples, obtains the target graph neural network model, realizes model training, and uses the test samples in the graph data samples to test the trained model. By inputting the test samples into the target neural network model, the link prediction results corresponding to the test samples are obtained according to the association relationship prediction results between the test sample nodes output by the target neural network model to detect whether the trained model has learned the structure of the graph data. According to the link prediction results, the applicability of the graph neural network model and the graph data samples is further judged to determine whether the selected graph neural network model is suitable for the analysis of graph data. In the process of model selection, the model suitable for graph data can be quickly selected.

[0152] For the specific limitations of the graph neural network model applicability determination device, please refer to the limitations of the graph neural network model applicability determination method above, which will not be repeated here. The various modules in the above-mentioned graph neural network model applicability determination device can be implemented in whole or in part by software, hardware, and a combination thereof. The above-mentioned modules can be embedded in or independent of the processor in the computer device in the form of hardware, or can be stored in the memory of the computer device in the form of software, so that the processor can call and execute the operations corresponding to the above modules.

[0153] In one embodiment, a computer device is provided. The computer device may be a server, and its internal structure diagram may be as follows: Figure 12 As shown. The computer device includes a processor 1210, a memory 1220 and a network interface 1250 connected via a system bus. Among them, the processor of the computer device is used to provide computing and control capabilities. The memory 1220 of the computer device includes a non-volatile storage medium 1230 and an internal memory 1240. The non-volatile storage medium 1230 stores an operating system 1232, a computer program 1234 and a database 1236. The internal memory provides an environment for the operation of the operating system and computer program in the non-volatile storage medium. The database of the computer device is used to record data of user operations and constructed graph data. The network interface 1250 of the computer device is used to communicate with an external terminal through a network connection. When the computer program is executed by the processor, it implements a method for determining the applicability of a graph neural network model.

[0154] Those skilled in the art will understand that Figure 12 The structure shown in the figure is only a block diagram of a part of the structure related to the solution of the present application, and does not constitute a limitation on the computer device to which the solution of the present application is applied. The specific computer device may include more or fewer components than shown in the figure, or combine certain components, or have a different component arrangement.

[0155] In one embodiment, a computer device is further provided, including a memory and a processor. The memory stores a computer program, and the processor implements the steps in the above method embodiments when executing the computer program.

[0156] In one embodiment, a computer-readable storage medium is provided, storing a computer program, which implements the steps in the above-mentioned method embodiments when executed by a processor.

[0157] In one embodiment, a computer program product or computer program is provided, the computer program product or computer program including computer instructions stored in a computer-readable storage medium. A processor of a computer device reads the computer instructions from the computer-readable storage medium and executes the computer instructions, causing the computer device to perform the steps of each of the above-described method embodiments.

[0158] Those skilled in the art will appreciate that all or part of the processes in the above-described method embodiments can be implemented by instructing the relevant hardware through a computer program. The computer program can be stored in a non-volatile computer-readable storage medium. When executed, the computer program can include the processes of the above-described method embodiments. Any reference to memory, storage, database, or other media used in the embodiments provided herein may include at least one of non-volatile and volatile memory. Non-volatile memory may include read-only memory (ROM), magnetic tape, floppy disk, flash memory, or optical storage. Volatile memory may include random access memory (RAM) or external cache memory. By way of illustration and not limitation, RAM can be in various forms, such as static random access memory (SRAM) or dynamic random access memory (DRAM).

[0159] The data collected by this application are used within a reasonable and legal scope.

[0160] The technical features of the above embodiments can be combined arbitrarily. To make the description concise, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0161] The above embodiments merely represent several implementation methods of the present application. While the descriptions are relatively specific and detailed, they should not be construed as limiting the scope of the present invention. It should be noted that a person skilled in the art could make various modifications and improvements without departing from the spirit of the present application, all of which fall within the scope of protection of the present application. Therefore, the scope of protection of the present patent application shall be determined by the appended claims.

Claims

1. A method for determining the applicability of a graph neural network model, characterized in that: The method comprises: Acquire graph data samples, where the graph data samples include training samples and test samples; Training the selected graph neural network model according to the training samples to obtain a target graph neural network model; Inputting the test sample into the target graph neural network model, and obtaining a link prediction result corresponding to the test sample based on the association relationship prediction result between the test sample nodes output by the target graph neural network model; When the link prediction result is not less than a preset indicator threshold, the applicability of the graph neural network model to the graph data sample is determined based on the graph signal-to-noise ratio of the test sample; the graph signal-to-noise ratio represents the average semantic similarity between a node and its first-order neighbor nodes.

2. The method according to claim 1, characterized in that The method further comprises: When the link prediction result is less than a preset index threshold, it is determined that the graph neural network model is not suitable for the graph data sample.

3. The method according to claim 2, characterized in that When the link prediction result is not less than a preset indicator threshold, determining the applicability of the graph neural network model to the graph data sample according to the graph signal-to-noise ratio of the test sample includes: When the link prediction result is not less than a preset index threshold, determining the average semantic similarity between the node and the first-order neighbor nodes based on the node corresponding data in the test sample and the first-order neighbor node corresponding data associated with the node; Obtaining a signal-to-noise ratio of the test sample according to the average semantic similarity; When the graph signal-to-noise ratio is less than a preset graph signal-to-noise ratio threshold, determining that the graph data sample is not suitable for the graph neural network model; When the graph signal-to-noise ratio is not less than a preset graph signal-to-noise ratio threshold, it is determined that the graph data sample is suitable for the graph neural network model.

4. The method according to claim 3, characterized in that When the image signal-to-noise ratio is less than a preset image signal-to-noise ratio threshold, the method further includes: The graph data samples are cleaned until a graph signal-to-noise ratio of the graph data samples is not less than a preset graph signal-to-noise ratio threshold.

5. The method according to claim 3, characterized in that When the link prediction result is less than a preset indicator threshold, the method further includes: Based on a preset graph neural network model set, the selected graph neural network model is replaced until the replaced graph neural network model meets the preset test conditions after model training based on the training sample, wherein the test conditions include that the link prediction result corresponding to the test sample is not less than a preset indicator threshold, and the graph signal-to-noise ratio corresponding to the test sample is not less than a preset graph signal-to-noise ratio threshold.

6. The method according to claim 1, characterized in that The acquiring of the graph data sample comprises: Obtain user historical behavior data; A graph data sample is obtained based on the recorded data corresponding to the user operations in the user historical behavior data, where the user operations include at least one of a search switching operation and a viewing operation. The recorded data corresponding to the search switching operation is used to characterize the similarity between at least two search keywords successively obtained based on the search switching operation, and the recorded data corresponding to the viewing operation is used to characterize the correlation between the search keywords and the search results.

7. The method according to claim 6, characterized in that The obtaining of the graph data sample according to the record data corresponding to the user operation in the user historical behavior data includes: Using the recorded data corresponding to the user operation as a node and the user operation as an edge between the nodes to establish a connection relationship between the nodes; Determining a weight corresponding to the edge according to the number of operations corresponding to the user operation; A graph data sample is obtained according to the nodes, the edges, and the weights corresponding to the edges.

8. A device for determining the applicability of a graph neural network model, characterized in that: The device comprises: A sample acquisition module is used to acquire graph data samples, wherein the graph data samples include training samples and test samples; A model training module, configured to train the selected graph neural network model according to the training samples to obtain a target graph neural network model; A model testing module is used to input the test sample into the target graph neural network model, and obtain the link prediction result corresponding to the test sample based on the association relationship prediction result between the test sample nodes output by the target graph neural network model; The applicability determination module is used to determine the applicability of the graph neural network model and the graph data sample based on the graph signal-to-noise ratio of the test sample when the link prediction result is not less than a preset indicator threshold; the graph signal-to-noise ratio represents the average semantic similarity between the node and the first-order neighbor node.

9. The graph neural network model applicability determination device according to claim 8, characterized in that: The applicability determination module is also used to determine that the graph neural network model is not applicable to the graph data sample when the link prediction result is less than a preset indicator threshold.

10. The graph neural network model applicability determination device according to claim 9, characterized in that: The applicability determination module is also used to determine the average semantic similarity between the node and the first-order neighbor node based on the corresponding data of the node in the test sample and the corresponding data of the first-order neighbor node associated with the node when the link prediction result is not less than a preset index threshold; obtain the graph signal-to-noise ratio of the test sample based on the average semantic similarity; when the graph signal-to-noise ratio is less than the preset graph signal-to-noise ratio threshold, determine that the graph data sample is not suitable for the graph neural network model; when the graph signal-to-noise ratio is not less than the preset graph signal-to-noise ratio threshold, determine that the graph data sample is suitable for the graph neural network model.

11. The graph neural network model applicability determination device according to claim 10, characterized in that: The device also includes a data cleaning module, which is used to perform data cleaning on the graph data sample when the graph signal-to-noise ratio is less than a preset graph signal-to-noise ratio threshold, until the graph signal-to-noise ratio of the graph data sample is not less than the preset graph signal-to-noise ratio threshold.

12. The graph neural network model applicability determination device according to claim 10, characterized in that: The device also includes a model replacement module, which is used to replace the selected graph neural network model based on a preset graph neural network model set when the link prediction result is less than a preset indicator threshold, until the replaced graph neural network model meets the preset test conditions after model training based on the training sample, wherein the test conditions include that the link prediction result corresponding to the test sample is not less than the preset indicator threshold, and the graph signal-to-noise ratio corresponding to the test sample is not less than the preset graph signal-to-noise ratio threshold.

13. The graph neural network model applicability determination device according to claim 8, characterized in that: The sample acquisition module is also used to obtain user historical behavior data; based on the record data corresponding to the user operations in the user historical behavior data, a graph data sample is obtained, the user operation includes at least one of a search switching operation and a viewing operation, the record data corresponding to the search switching operation is used to characterize the similarity between at least two search keywords successively obtained based on the search switching operation, and the record data corresponding to the viewing operation is used to characterize the correlation between the search keywords and the search results.

14. The graph neural network model applicability determination device according to claim 13, characterized in that: The sample acquisition module is also used to use the recorded data corresponding to the user operation as a node and the user operation as an edge between the nodes to establish a connection relationship between the nodes; determine the weight corresponding to the edge according to the number of operations corresponding to the user operation; and obtain a graph data sample based on the nodes, the edges and the weights corresponding to the edges.

15. A computer device comprising a memory and a processor, wherein the memory stores a computer program, wherein: When the processor executes the computer program, the method according to any one of claims 1 to 7 is implemented.

16. A computer-readable storage medium storing a computer program, characterized in that: When the computer program is executed by a processor, the method according to any one of claims 1 to 7 is implemented.

17. A computer program product comprising computer instructions, characterized in that When the computer instructions are executed by a processor, the method according to any one of claims 1 to 7 is implemented.

Images