Scalable graph learning based on matrix decomposition and locality hashing techniques

Abstract

A method for scalable graph learning is disclosed. The method includes receiving a graph dataset and determining a node feature matrix and a convolution matrix associated with the graph dataset. A first sketch matrix for the node feature matrix and a second sketch matrix for the convolution matrix are determined. Forward propagation rules associated with a GNN model are determined, based on the first sketch matrix and the second sketch matrix to select active nodes associated with the set of layers of the GNN model. Node representations are determined. A training loss of the GNN model is determined, based on the node representations, the training loss being associated with a prediction error associated with a downstream prediction task of the GNN model, and the GNN model being configured to be trained to generate a prediction result for the downstream prediction task, based on the training loss.

Description

SCALABLE GRAPH LEARNING BASED ON MATRIX DECOMPOSITION AND LOCALITY HASHING TECHNIQUES FIELD

[0001] The embodiments discussed in the present disclosure are related to scalable graph learning based on matrix decomposition and locality hashing techniques. BACKGROUND

[0002] Data of many application domain may be represented as graph data. Examples of such application domains include, financial, transportation, supply chain, medical, pharmaceutical, social network, speech recognition, computer vision and the like. Graph Neural Network (GNN) models may be applied on the graph data to perform tasks such as, but not limited to, node classification, edge prediction, sub- graph classification, and the like. GNN models may capture complex relationships and dependencies in graph-structured data. However, the training the GNN models may often require substantial computational resources and memory. Traditional approaches for training the GNN models may involve iterative computations over the entire graph, which can be computationally expensive, especially for large-scale graphs. This drawback becomes more pronounced when dealing with inherently large graphs or when training GNNs on limited computational resources. Also, these inherently large graphs may often require storing and processing of an entire graph structure during training. This may lead to high memory requirements, making it challenging to train GNN models on devices with limited memory capacity. Hence, there is a need to develop an improved technique to train GNN models that canleverage the available graph data to proactively manage training time and memory usage.

[0003] The subject matter claimed in the present disclosure is not limited to embodiments that solve any disadvantages or that operate only in environments such as those described above. Rather, this background is only provided to illustrate one example technology area where some embodiments described in the present disclosure may be practiced.SUMMARY

[0004] According to an aspect of an embodiment, a method may include a set of operations which may include receiving a graph dataset associated with an application domain. The set of operations may further include determining a node feature matrix and a convolution matrix associated with the graph dataset. The set of operations may further include determining a first sketch matrix for the node feature matrix and a second sketch matrix for the convolution matrix, based on the matrix decomposition technique. The set of operations may further include determining forward propagation rules associated with a Graph Neural Network (GNN) model, based on the first sketch matrix and the second sketch matrix to select active nodes associated with the set of layers of the GNN model, based on a locality hashing technique. Node representations may be determined for a final layer of the GNN model, based on the active nodes, the forward propagation rules and an un-sketching technique. The set of operations may further include determining a training loss of the GNN model, based on the node representations for the final layer, the training loss being associated with a prediction error associated with a downstream prediction task of the GNN model, and the GNN model being configured to be trained to generate a prediction result for the downstream prediction task, based on the training loss to control rendering of the first information including the prediction result associated with the downstream prediction task, based on the GNN model.

[0005] The objects and advantages of the embodiments will be realized and achieved at least by the elements, features, and combinations particularly pointed out in the claims.

[0006] Both the foregoing general description and the following detailed description are given as examples and are explanatory and are not restrictive of the invention, as claimed. BRIEF DESCRIPTION OF THE DRAWINGS

[0007] Example embodiments will be described and explained with additional specificity and detail through the use of the accompanying drawings in which:

[0008] FIG.1 is a diagram representing an example network environment related to scalable graph learning based on matrix decomposition and locality hashing technique;

[0009] FIG.2 is a block diagram that illustrates an exemplary electronic device for scalable graph learning based on matrix decomposition and locality hashing technique;

[0010] FIG. 3 is a diagram that illustrates an exemplary execution pipeline for scalable graph learning based on matrix decomposition and locality hashing technique;

[0011] FIG.4 is a diagram that illustrates an exemplary scenario of a sketch unit for performing a matrix decomposition based on a matrix decomposition technique;

[0012] FIG.5 is a diagram that illustrates an exemplary scenario of determination of a sketch matrix of node representations at a next layer of a graph neural network (GNN) model;

[0013] FIG.6 is a diagram that illustrates an exemplary scenario of un-sketching sketches of node feature matrix at a layer of a graph neural network (GNN) model;

[0014] FIG.7 is a diagram that illustrates an exemplary scenario of locality hashing technique for selection of active nodes associated with a set of layers of a graph neural network (GNN) model;

[0015] FIG.8 is a diagram that illustrates an exemplary scenario of scalable graph learning based on matrix decomposition and locality hashing technique;

[0016] FIG.9 is a diagram that illustrates an exemplary scenario of scalable graph learning based on matrix decomposition and locality hashing technique; and

[0017] FIG.10 is a diagram that illustrates a flowchart of an exemplary method for scalable graph learning based on matrix decomposition and locality hashing technique,

[0018] all according to at least one embodiment described in the present disclosure. DESCRIPTION OF EMBODIMENTS

[0019] Some embodiments described in the present disclosure may relate to methods and electronic devices for scalable graph learning based on matrix decomposition and locality hashing techniques. In the present disclosure, graph dataset (for example, Cora Dataset, PubMed Dataset, MUTAG Dataset, Reddit Dataset, and the like) associated with an application domain (for example, but not limited to, financial domain, social network domain, and recommendation system domain) may be received. A node feature matrix and a convolution matrix associated with the graph dataset may be determined. A first sketch matrix for the node feature matrix and second sketch matrix for the convolution matrix may be determined based on a matrix decomposition technique. Forward propagation rules associated with a graph neural network (GNN) model may be determined, based on the first sketch matrixand the second sketch matrix to select active nodes associated with a set of layers of the GNN model based on a locality hashing technique. Node representations may be determined for a final layer of the GNN model based on the active nodes, the forward propagation rules and un-sketching technique. A training loss of the GNN model may be determined based on the node representations for the final layer. The training loss may be associated with a prediction error associated with a downstream prediction task of the GNN model and the GNN model may be configured to be trained to generate a prediction result for the downstream prediction task, based on the training loss. Rendering of first information, including the prediction result associated with the downstream prediction task, based on the GNN model, may be controlled.

[0020] The technological field of graph learning may be improved by configuring an electronic device to train a GNN model on a graph dataset using scalable graph learning based on matrix decomposition and locality hashing techniques. The electronic device may receive a graph dataset associated with the application domain to determine the node feature matrix and convolution matrix associated with the graph dataset. The electronic device may determine the first sketch matrix for the node feature matrix and the second sketch matrix for the convolution matrix, based on the matrix decomposition technique. Forward propagation rules associated with the graph neural network (GNN) model may be determined, based on the first sketch matrix and the second sketch matrix. Active nodes associated with the set of layers of the GNN model may be selected, based on the locality hashing technique and node representations for the final layer of the GNN model may be determined based on the active nodes, the forward propagation rules and an un-sketching technique. Thetraining loss being associated with a prediction error associated with a downstream prediction task of the GNN model and the GNN model being configured to be trained to generate a prediction result for the downstream prediction task, based on the training loss. The electronic device may control rendering of first information including the prediction result associated with the downstream prediction task, based on the GNN model.

[0021] The disclosed approach may offer several advantages. Enhanced memory efficiency and reduced training time may be achieved by utilizing techniques such as, a locality hashing technique (such as, a Locality-Sensitive hashing (LSH)) without additional computation associated with LSH, like updating a hash table associated with the LSH during the training. As the projection of the update rule in the lower dimension may be used, memory requirement and time complexity may be reduced. Based on the application of update rules in a reduced dimension, the computational burden of performing matrix-matrix multiplications within the graph space may be eliminated. Further, the proposed technique uses a fixed Proper Orthogonal Decomposition (POD) projection matrix throughout the training, which may be leveraged to use techniques like LSH to further reduce training time. Further, the determination of the POD projection matrix may be agnostic to the underlying architecture of the GNN model. Thus, the present disclosure may provide scalable graph learning based on matrix decomposition and locality hashing technique. This approach may be optimized for minimal memory usage and fast execution.

[0022] Embodiments of the present disclosure are explained with reference to the accompanying drawings.

[0023] FIG.1 is a diagram representing an example network environment related to scalable graph learning based on matrix decomposition and locality hashing technique, arranged in accordance with at least one embodiment described in the present disclosure. With reference to FIG.1, there is shown an environment 100. The environment 100 may include an electronic device 102, a graph neural network (GNN) model 104, a server 106, a database 108, a communication network 112, and a test graph 114. The server 106 may host the database 108. Further, the electronic device 102 may be communicatively coupled to the server 106, via the communication network 112.

[0024] The electronic device 102 may include suitable logic, circuitry, interfaces and / or code that may be configured to receive a graph dataset 110 associated with an application domain. The electronic device 102 may determine a node feature matrix and convolution matrix associated with the graph dataset 110. The electronic device 102 may further determine a first sketch matrix for the node feature matrix and a second sketch matrix for the convolution matrix, based on a matrix decomposition technique. Also, the electronic device 102 may determine forward propagation rules associated with the GNN model 104, based on the first sketch matrix and the second sketch matrix. The electronic device 102 may select active nodes associated with a set of layers of the GNN model 104, based on a locality hashing technique to determine node representations for a final layer of the GNN model 104. The node representations for the final layer may be based on the active nodes, the forward propagation rules and an un-sketching technique. The electronic device 102 may further determine a training loss of the GNN model 104, based on the node representations for the final layer. The training loss may be associated with aprediction error associated with a downstream prediction task of the GNN model 104, and the GNN model 104 may be configured to be trained to generate a prediction result for the downstream prediction task, based on the training loss. The electronic device 102 may control rendering of first information including the prediction result associated with the downstream prediction task, based on the GNN model 104.

[0025] The electronic device 102 may further receive the test graph 114 associated with the application domain and apply the GNN model 104 on the test graph 114 based on model parameters associated with the GNN model 104. The model parameters may be determined based on the training of the GNN model 104 to generate the prediction result. The prediction result associated with the test graph 114 may be generated based on the final layer of the GNN model 104.

[0026] In an embodiment, the electronic device 102 may control a display device (e.g., a display device 206A of FIG. 2). The display device 206A may be communicatively coupled to the electronic device 102 or may be a standalone device configured to render the first information including the prediction result associated with the downstream prediction task, based on the GNN model 104. This allows the prediction result associated with the downstream prediction task to be displayed. The generation of the prediction result may be associated with the test graph 114 based on the final layer of the GNN model 104 and the forward propagation rules. Examples of the electronic device 102 may include, but may not be limited to, a computing device, a smartphone, a mainframe machine, a server, a consumer electronic (CE) device, a computer workstation, and / or a device with a graph- processing capability (such as, a device with a set of graphic processor units (GPU)).

[0027] The GNN model 104 may include suitable logic, circuitry, interfaces, and / or code that may configured to classify or analyze input graph data to generate an output result for a particular real-time application. For example, a trained GNN model 104 may recognize different nodes in the input graph data, and edges between each node in the input graph data. The edges may correspond to different connections or relationship between each node in the input graph data. Based on the recognized nodes and edges, the trained GNN model may classify different nodes within the input graph data, into different labels or classes. In an example, a particular node of the input graph data may include a set of features (e.g., the node feature matrix) associated therewith. The set of features may include, but are not limited to, a matrix indicative of the incoming and outgoing edges and respective edge weights for a particular node . Further, each edge may connect with different nodes having similar set of features. The electronic device 102 may be configured to encode the set of features to generate a feature vector using the GNN model 104. After the encoding, information may be passed between the particular node and the neighboring nodes connected through the edges. Based on the information passed to the neighboring nodes, a final vector may be generated for each node. Such final vector may include information associated with the set of features for the particular node as well as the neighboring nodes, thereby providing reliable and accurate information associated with the particular node. As a result, the GNN model 104 may analyze the information represented as the input graph data. The GNN model 104 may be implemented using hardware including a processor, a microprocessor (e.g., to perform or control performance of one or more operations), a field-programmable gate array (FPGA), or an application-specific integrated circuit (ASIC). In some other instances, the GNNmodel 104 may be a code, a program, or set of software instruction. The GNN model 104 may be implemented using a combination of hardware and software.

[0028] In some embodiments, the GNN model 104 may correspond to multiple classification layers for classification of different nodes in input graph data, where each successive layer may use an output of a previous layer as input. During training, the GNN model 104 may be configured to filter the nodes based on the node representations at various layers of the GNN model 104, based on the locality hashing technique and further provide an output result (i.e., a graph representation) of the GNN model 104. Examples of the GNN model 104 may include, but are not limited to, a graph convolution network (GCN), a Graph Spatial-Temporal Networks with GCN, Graph attention networks (GAT), a Graph SAGE, a deep Bayesian graph neural network, and / or a combination of such networks.

[0029] The electronic device 102 and server 106 may be communicatively coupled with each other via the communication network 112. In some embodiment, the electronic device 102 may include the GNN model 104 and a display device (e.g., a display device 206A of FIG.2). In some other embodiments, the display device 206A may be separately placed out of the electronic device 102. The server 106 may be communicatively coupled with the database 108.

[0030] The server 106 may include logic, circuitry, interfaces, and / or code configured to store a graph dataset (e.g., the graph dataset 110) associated with a certain application domain. In an example, the server 106 may store the graph dataset 110 on the database 108. In some embodiments, the server 106 may also store the GNN model 104 and / or the test graph 114 on the database 108. The server 106 may be configured to retrieve data (for example, the graph dataset 110, the GNNmodel 104, and / or the test graph 114) from the database 108 and transmit the retrieved data to the electronic device 102.

[0031] The server 106 may be implemented as a cloud server and may execute operations through web applications, cloud applications, hypertext transport protocol (HTTP) requests, repository operations, file transfer, and the like. Other example implementations of the server 106 may include, but are not limited to, a database server, a file server, a web server, a media server, an application server, a mainframe server, a cloud computing server, and / or any device with a graph- processing capability (such as, a device with a set of graphic processor units (GPU)).

[0032] In at least one embodiment, the server 106 may be implemented as a plurality of distributed cloud-based resources by use of several technologies that are well known to those ordinarily skilled in the art. In certain embodiments, the functionalities of the server 106 may be incorporated in its entirety or at least partially in the electronic device 102, without a departure from the scope of the disclosure. In an embodiment, the server 106 may be configured to train the GNN model 104 and the electronic device 102 may be configured to perform inference on downstream prediction tasks, based on the trained GNN model 104.

[0033] The database 108 may include suitable logic, circuitry, interfaces, and / or code that may be configured to store graph data. For example, the graph data stored on the database 108 may include the graph dataset 110 and / or the test graph 114. The database 108 may further store the GNN model 104. The database 108 may be derived from data off a relational or non-relational database, or a set of comma- separated values (csv) files in a conventional storage or a big-data storage. The database 108 may be stored or cached on a device, such as, the server 106 or theelectronic device 102. The device storing the database 108 may be configured to receive a query for the graph data or the GNN model 104. In response, the device storing the database 108 may be configured to retrieve and transmit the graph data or the GNN model 104 to the electronic device 102.

[0034] In accordance with an embodiment, the database 108 may be hosted on a plurality of servers stored at same or different locations. The operations of the database 108 may be executed using hardware including a processor, a microprocessor (for example, to perform or control performance of one or more operations), a field-programmable gate array (FPGA), or an application-specific integrated circuit (ASIC). In some other instances, the database 108 may be implemented using software.

[0035] A person with ordinary skill in the art will understand that the scope of the disclosure may not be limited to the implementation of the database 108 and the server 106 (or the electronic device 102) as two separate entities. In certain embodiments, the functionalities of the database 108 can be incorporated in its entirety or at least partially in the server 106 (or the electronic device 102), without a departure from the scope of the disclosure.

[0036] The communication network 112 may include various communication media through which the electronic device 102 may communicate with server 106 or devices storing the graph dataset 110. Examples of the communication network 112 may include, but are not limited to, the Internet, a cloud network, a Wireless Fidelity (Wi-Fi) network, a Personal Area Network (PAN), a Local Area Network (LAN), a cellular network (such as, a Long-term evolution (or 4G) cellular network or a 5G cellular network), a satellite network (such as, a network of low earth orbitsatellites), and / or a Metropolitan Area Network (MAN)). Various devices in the environment 100 may connect to the communication network 112 using various wired and wireless communication protocols, including TCP / IP, UDP, HTTP, FTP, ZigBee, EDGE, IEEE 802.11, Li-Fi, IEEE 802.16, multi-hop communication, wireless access point (AP), device-to-device communication, cellular communication protocols, and Bluetooth.

[0037] In operation, the electronic device 102 may receive graph dataset 110 that may represent a graph associated with a specific application domain. The graph may include nodes and their relationships. To process the graph dataset 110, the electronic device 102 may perform various operations such as, but not limited to, determination of a node feature matrix. The node feature matrix may capture features or attributes of each node in the graph dataset 110. For instance, in a social network graph, the node feature matrix may include information such as age, gender, and interests of each user. Additionally, the electronic device 102 may determine a convolution matrix, which represents relationships between each node in the graph. The convolution matrix may be indicative of information flow and propagation through the graph of the graph dataset 110. The reception of the graph and the determination of the node feature matrix and the convolution matrix are described further, for example, with reference to FIG.3.

[0038] The electronic device 102 may determine a first sketch matrix for the node feature matrix and a second sketch matrix for the convolution matrix, based on a matrix decomposition technique. The electronic device 102 may determine forward propagation rules for the GNN model 104, based on the first sketch matrix and the second sketch matrix. The determination of the first sketch matrix, the second sketchmatrix, and the forward propagation rules are described further, for example, with reference to FIG.3 and FIG.5.

[0039] The electronic device 102 may use first sketch matrix and the second sketch matrix in conjunction with a locality hashing technique to select active nodes for each layer of the GNN model 104. This helps in focusing on the most relevant nodes for a task. Active nodes refer to the nodes or elements in a convolution matrix that have non-zero values or contribute significantly to the node representations for the final layer of the GNN model 104. Selecting only the active nodes may optimize memory usage by storing and processing only the relevant information. The selection of the active nodes is described further, for example, in FIG.3 and FIG.6.

[0040] The electronic device 102 may determine node representations for the final layer of the GNN model 104, based on the active nodes, the forward propagation rules and un-sketching technique. These node representations can be used for various downstream tasks, such as node classification or link prediction. To train the GNN model 104, the electronic device 102 may determine a training loss based on the node representations of the final layer. The training loss measures a prediction error associated with a downstream prediction task. For example, in a recommendation system, the training loss could quantify how well the GNN model 104 predicts user preferences based on the node representations. The GNN model 104 is then trained using the training loss to improve its prediction capabilities. The GNN model 104 learns to generate more accurate prediction results for the downstream prediction task based on the training loss. The determination of the node representations and the training of the GNN model are described further, for example, in FIG.3, FIG.8, and FIG.9.

[0041] The electronic device 102 may control rendering or display of first information, which includes the prediction result associated with the downstream prediction task. The rendering of the first information is controlled based on the GNN model 104 and its training loss. This ensures that the prediction result is reliable and useful for the intended application. Thus, the electronic device 102 may process graph datasets, extract meaningful information, and make accurate predictions by optimizing training time and memory of the electronic device 102 based on the learned node representations. The control of the rendering of the first information is described further, for example, in FIG.3.

[0042] In certain scenarios, the matrix decomposition technique may correspond to a Proper Orthogonal Decomposition (POD)-technique. The POD technique may include generation of a covariance matrix. The generation of the covariance matrix may be based on the node feature matrix. The node feature matrix represents the data or variables associated with each graph node in the electronic device 102 or the database 108. A predetermined number of eigen vectors may be determined from the covariance matrix. The eigen vectors may indicate the directions along which the data varies the most. The eigen vector may capture dominant modes or patterns present in the data. Based on the determined number of eigen vectors, a POD matrix may be generated. The POD matrix may be constructed by combining the eigen vectors corresponding to the dominant modes of variation in the data. The determination of the first sketch matrix and the second sketch matrix may be further based on the generation of the POD matrix. The first sketch matrix and the second sketch matrix may be typically used for dimensionality reduction or approximation purposes.

[0043] In some aspects, determination of a singular vector may be performed by generating a first random vector and a second random vector of a predetermined size. Each first element of the first random vector and each second element of the second random vector may be sampled from a set of numbers corresponding to a dimensionality of the node feature matrix. An estimated feature matrix associated with the node feature matrix may be determined based on the first random vector and the second random vector. A singular vector may be determined based on the estimated feature matrix. The generation of the POD matrix may be further based on the singular vector.

[0044] Modifications, additions, or omissions may be made to FIG. 1 without departing from the scope of the present disclosure. For example, the environment 100 may include more or fewer elements than those illustrated and described in the present disclosure. For instance, in some embodiments, the environment 100 may include the electronic device 102 but not the database 108. In addition, in some embodiments, the functionality of each of the database 108 may be incorporated into the electronic device 102, without a deviation from the scope of the disclosure.

[0045] FIG.2 is a block diagram that illustrates an exemplary electronic device 102 for scalable graph learning based on matrix decomposition and locality hashing technique, arranged in accordance with at least one embodiment described in the present disclosure. FIG.2 is explained in conjunction with elements from FIG.1. With reference to FIG.2, there is shown a block diagram 200 of the electronic device 102. The electronic device 102 may include a processor 202, a memory 204, the GNN model 104, an input / output (I / O) device 206, a network interface 208. The I / Odevice 206 may include the display device 206A. The memory 204 may include the graph dataset 110.

[0046] The processor 202 may include suitable logic, circuitry, interfaces, and / or code that may be configured to execute program instructions associated with different operations to be executed by the electronic device 102. The operations may include, but are not limited to, graph dataset reception, node feature matrix and convolution matrix determination, first sketch matrix and second sketch matrix determination, forward propagation rule determination, active node selection, node representation determination, training loss determination, rendering control, The processor 202 may include any suitable special-purpose or general-purpose computer, computing entity, or processing device, including various computer hardware or software modules, and may be configured to execute instructions stored on any applicable computer-readable storage media. For example, the processor 202 may include a microprocessor, a microcontroller, a digital signal processor (DSP), an application-specific integrated circuit (ASIC), a Field-Programmable Gate Array (FPGA), or any other digital or analog circuitry configured to interpret and / or to execute program instructions and / or to process data.

[0047] Although illustrated as a single processor in FIG.2, the processor 202 may include any number of processors configured to, individually or collectively, perform or direct performance of any number of operations of the electronic device 102, as described in the present disclosure. Additionally, one or more of the processors may be present on one or more different electronic devices 102, such as different servers.

[0048] In some embodiments, the processor 202 may be configured to interpret and / or execute program instructions and / or process data stored in the memory 204.In some embodiments, the processor 202 may fetch program instructions from the GNN model 104 and load the program instructions in the memory 204. After the program instructions are loaded into memory 204, the processor 202 may execute the program instructions. Some of the examples of the processor 202 may be a Graphical Processing Unit (GPU), a Central Processing Unit (CPU), a Reduced Instruction Set Computer (RISC) processor, an Application-Specific Integrated Circuit (ASIC) processor, a Complex Instruction Set Computer (CISC) processor, a co- processor, and / or a combination thereof.

[0049] The memory 204 may include suitable logic, circuitry, interfaces, and / or code that may be configured to store program instructions executable by the processor 202. In certain embodiments, the memory 204 may be configured to store information, such as, but not limited to, the graph dataset 110, the node feature matrix, the convolution matrix, the forward propagation rules, the test graph 114, and the GNN model 104. The memory 204 may further store a set of elements of the POD matrix. The set of elements of the POD matrix may be selected based on the locality hashing technique. The POD matrix may be converted to a sparse matrix representation, based on a count-sketch technique. The sparse matrix may be stored in the memory 204.

[0050] The memory 204 may include computer-readable storage media for carrying or having computer-executable instructions or data structures stored thereon. Such computer-readable storage media may include any available media that may be accessed by a general-purpose or special-purpose computer, such as the processor 202. By way of example, and not limitation, such computer-readable storage media may include tangible or non-transitory computer-readable storagemedia, including but not limited to, a CPU cache, a Hard Disk Drive (HDD), a Solid- State Drive (SSD), Random Access Memory (RAM), Read-Only Memory (ROM), Electrically Erasable Programmable Read-Only Memory (EEPROM), Compact Disc Read-Only Memory (CD-ROM), a Secure Digital (SD) card, or other optical disk storage, magnetic disk storage or other magnetic storage devices, or flash memory devices (e.g., solid state memory devices). The computer-readable storage may also include any other storage medium which may be used to carry or store particular program code in the form of computer-executable instructions or data structures and which may be accessed by a general-purpose or special-purpose computer. Combinations of the above may also be included within the scope of computer- readable storage media. Computer-executable instructions may include, for example, instructions and data configured to cause the processor 202 to perform a certain operation or group of operations associated with the electronic device 102.

[0051] The I / O device 206 may include suitable logic, circuitry, interfaces, and / or code that may be configured to receive a user input. For example, the user input may indicate a selection of the graph dataset 110 or the test graph 114. The I / O device 206 may be further configured to provide an output in response to the user input. For example, the output may correspond to the first information. The I / O device 206 may include various input and output devices, which may be configured to communicate with the processor 202 and other components, such as the network interface 210. Examples of the input devices may include, but are not limited to, a touch screen, a keyboard, a mouse, a joystick, and / or a microphone. Examples of the output devices may include, but are not limited to, the display device 206A and aspeaker. The I / O device 206 may be configured within the electronic device 102 or outside of the electronic device 102.

[0052] The display device 206A may include logic, circuitry, and interfaces configured to display the first information including the prediction result associated with the downstream prediction task. The display device 206A may be a touch screen which may enable a user to provide user-inputs via the display device 206A. The touch screen may be at least one of a resistive touch screen, a capacitive touch screen, or a thermal touch screen. The display device 206A may be realized through several known technologies such as, but not limited to, a Liquid Crystal Display (LCD) display, a Light Emitting Diode (LED) display, a plasma display, or an Organic LED (OLED) display technology, or other display devices. In accordance with an embodiment, the display device 206A may refer to a display screen of a head mounted device (HMD), a smart-glass device, a see-through display, a projection-based display, an electro- chromic display, or a transparent display.

[0053] The network interface 208 may include suitable logic, circuitry, and interfaces that may be configured to facilitate communication between the processor 202 (i.e., the electronic device 102) and the server 106, via the communication network 112. The network interface 208 may be implemented by use of various known technologies to support wired or wireless communication of the electronic device 102 with the communication network 112. The network interface 208 may include, but is not limited to, antenna, a radio frequency (RF) transceiver, one or more amplifiers, a tuner, one or more oscillators, a digital signal processor, a coder- decoder (CODEC) chipset, a subscriber identity module (SIM) card, or a local buffer circuitry.

[0054] The network interface 208 may be configured to communicate via wireless communication with networks, such as the Internet, an Intranet, or a wireless network, such as a cellular telephone network, a wireless local area network (LAN), and a metropolitan area network (MAN). The wireless communication may be configured to use one or more of a plurality of communication standards, protocols and technologies, such as Enhanced Data GSM Environment (EDGE), wideband code division multiple access (W-CDMA), Long Term Evolution (LTE), 5thGeneration (5G) New Radio (NR), Global System for Mobile Communications (GSM), code division multiple access (CDMA), time division multiple access (TDMA), Bluetooth, Wireless Fidelity (Wi-Fi) (such as IEEE 802.11a, IEEE 802.11b, IEEE 802.11g or IEEE 802.11n), voice over Internet Protocol (VoIP), light fidelity (Li-Fi), Worldwide Interoperability for Microwave Access (Wi-MAX), a protocol for email, instant messaging, and a Short Message Service (SMS).

[0055] Modifications, additions, or omissions may be made to the example electronic device 102 without departing from the scope of the present disclosure. For example, in some embodiments, the example electronic device 102 may include any number of other components that may not be explicitly illustrated or described for the sake of brevity.

[0056] FIG. 3 is a diagram that illustrates an exemplary execution pipeline for scalable graph learning based on matrix decomposition and locality hashing technique, in accordance with an embodiment of the disclosure. FIG.3 is described in conjunction with elements from FIG.1 and FIG.2. With reference to FIG.3, there is shown an execution pipeline 300. The exemplary execution pipeline 300 may include a sequence of operations that may be executed by the processor 202 of the electronicdevice 102 of FIG.1 for scalable graph learning based on matrix decomposition and locality hashing technique.

[0057] The execution pipeline 300 includes an input graph dataset 302A, an operation for determination of sketch matrices 302, operations for approximate forward propagations 304A and 304B, an operation for determination of un- sketched matrix 306 (of node representations at a layer “l”), a SoftMax layer (for classification) 308, an operation for training of the GNN model 310, a test graph 312, a layer-1 of the GNN model 314A, a layer-2 of the GNN model 314B, and a SoftMax layer (for classification) 314C.

[0058] Though only two approximate forward propagations and two layers of the GNN model 104 are shown in FIG.3, the scope of the disclosure may not be so limited. The GNN models may include more than two approximate forward propagations and more than two layers of the GNN model 104, without departure from the scope of the disclosure.

[0059] In the training phase of the GNN model 104, the operations 302, 304A, 304B, 306, and 310 may be executed. At 302, an operation for determination of sketch matrices may be executed. The processor 202 of the electronic device 102 may be configured to determine the sketch matrices. In an embodiment, the processor 202 may be configured to receive the input graph dataset 302A, which may be associated with a certain application domain. The application domain may correspond to one of, but not limited to, a financial domain (e.g., for fraud / tax evasion detection), a social network domain (e.g., for community detection, profile prediction, and trend prediction), or a recommendation system domain (e.g., for product / content recommendation). The application domain may further include abiological and a bio-informatics domain (e.g., for protein function prediction and drug-target interaction prediction), a chemistry, a bio-chemistry and a material science domain (e.g., for compound property prediction), or a citation network domain (e.g., for research paper classification).

[0060] The processor 202 may be further configured to determine the node feature matrix and the convolution matrix associated with input graph dataset 302A. In an example, the node feature matrix may correspond to an adjacency matrix associated with the input graph dataset 302A. For example, each row of the node feature matrix may represent a certain node and each column may represent a particular feature of the node. Examples of node features may include information on adjoining nodes, incoming edges, outgoing edges, edge weights, or a weight associated with the node. In an embodiment, the node feature matrix may correspond to node representations at a certain layer “l” of the GNN model 104. In an example, a spatial convolution operation may be applied on the input graph dataset 302A to determine the convolution matrix. For example, the spatial convolution operation may include application of a weighted kernel to the node feature matrix to update the features of the nodes of the input graph dataset 302A. Based on the convolution operation, the features of the nodes may be updated based on a message passing function, via the edges associated with the respective nodes. Based on the update of the features of a node, information may be aggregated for the node from its neighborhood.

[0061] The processor 202 may be configured to determine the first sketch matrix for the node feature matrix and a second sketch matrix for the convolution matrix, based on a matrix decomposition technique. The matrix decomposition techniquemay correspond to a Proper Orthogonal Decomposition (POD)-technique. The POD technique corresponds to a dimensionality reduction technique that may capture the dominant spatial patterns or modes inherent in high-dimensional data through a Singular Value Decomposition (SVD)-technique. A low rank SVD of a matrix (such as, the node feature matrix or the convolution matrix) may be determined based on randomization techniques. The determination of the first sketch matrix and the second sketch matrix are described further, for example, in FIG.4 and FIG.8.

[0062] At 304A and 304B, the operations for approximate forward propagations may be executed. The processor 202 may be configured to determine the forward propagation rules associated with a graph neural network (GNN) model (e.g., the GNN model 104), based on the first sketch matrix and the second sketch matrix. In an embodiment, the determination of the forward propagation rules (e.g., the operations for approximate forward propagations 304A and 304B) may be further based on at least one of a POD matrix, a mean associated with the node feature matrix, an activation function, the convolution matrix, a weight matrix associated with a layer of the set of layers, the first sketch matrix, the second sketch matrix, or a bias matrix associated with the layer of the set of layers. Based on the determination of the forward propagation rules, a sketch matrix of the node feature matrix (i.e., the node representations) may be determined for a next layer (k+1) of the GNN model 104. The sketch matrix of the next layer (k+1) may be used to determine a sketch matrix of another next layer (k+2) based on the forward propagation rules. Similarly, the sketch matrix may be determined for each layer till the final layer of the GNN model 104, based on at least one of a connectivity (i.e., edges) between the nodes of the GNN model 104, weights associated with the nodes, or weights associated withthe edges. The determination of the forward propagation rules is described further, for example, in FIG.5.

[0063] At 306, an operation for determination of un-sketched matrix (of node representations at a layer “l”) may be executed. The processor 202 may be configured to determine the un-sketched matrix of node representations at a layerIn an embodiment, the processor 202 may be configured to select active nodes associated with a set of layers of the GNN model 104, based on a locality hashing technique. The processor 202 may be further configured to determine node representations (and weights) for a final layer (e.g., a SoftMax layer) of the GNN model 104, based on the active nodes, the forward propagation rules (i.e., the approximate forward propagation 304A and 304B), and an un-sketching technique. The un-sketched matrices may indicate the weights and the node representations for the final layer of the GNN model 104. In an example, the processor 202 may be configured to apply the forward propagation rules (i.e., the approximate forward propagation 304A and 304B) to the active nodes of the set of layers of the GNN model 104. Herein, the node representations for the final layer of the GNN model 104 may be further based on the application of the forward propagation rules. Herein, the node representations may correspond to the matrices un-sketched to an original size of the node feature matrix or the convolution matrix. The selection of active nodes and the determination of the node representations are described further, for example, in FIG.6 and FIG.7.

[0064] At 310, an operation for training of the GNN model may be executed. The processor 202 may be configured to train the GNN model 104. To train the GNN model 104, the processor 202 may pass the weights and the node representations associated with the final layer of the GNN model 104 to the final layer, such as, theSoftMax layer (for classification) 308. Thereafter, the processor 202 may determine a training loss of the GNN model 104, based on the output of the final layer, such as, the SoftMax layer (for classification) 308. The training loss may be associated with a prediction error associated with a downstream prediction task of the GNN model 104. Further, the GNN model 104 may be configured to be trained to generate a prediction result for the downstream prediction task, based on the training loss. In an example, in case the GNN model 104 corresponds to a Simplified Graph Convolution (SGC) update rule, a memory complexity in the training of the GNN model 104 for “d1” active neurons, sketch dimension of size “c”, “L” layers of the GNNmodel 104, and an independent graph dimension “n” may be “^^(^^^^1^^) + ^^(^^2)”. Further, the forward propagation rules may be in a reduced dimension (i.e., a sketch dimension), thereby reducing an overhead of performing matrix-matrix multiplications in a graph space.

[0065] The processor 202 may be configured to determine model parameters associated with the GNN model 104, based on the GNN model 104 being configured to be trained to generate the prediction result. The determination of the model parameters may be based on the training loss. For example, the determined model parameters may include information on at least one of, but not limited to, an indication of active nodes and edges of the GNN model 104, weights and biases associated with the GNN model 104, the training loss, and tunable model hyperparameters associated with the GNN model 104.

[0066] In the inference phase of the GNN model 104, the model parameters associated with the GNN model 104 may be transferred from a trained GNN model (i.e., a GNN model trained in the training phase) to a live GNN model (e.g., the GNNmodel 104). The model parameters may be used to configure the GNN model 104 for an inference task (e.g., a downstream prediction task). For example, the processor 202 may receive the test graph 312 associated with a downstream prediction task. In an embodiment, the test graph 312 may be of the same application domain or same as the input graph dataset 302A. The processor 202 may configure the weights, the edges, and node representations of the GNN model 104, based on the model parameters determined for the GNN model 104 and the training loss. The information about the configuration of the weights, the edges, and the node representations of the GNN model 104 may be passed to the layer-1 of the GNN model 314A, the layer-2 of the GNN model 314B, and subsequently to the SoftMax layer (for classification) 314C. Based on the configuration of the GNN model 104 for the inference task, the processor 202 may determine a prediction result (e.g., an edge classification or a node classification) associated with the downstream prediction task associated with the test graph 312.

[0067] The disclosed approach may offer several advantages. Enhanced memory efficiency and reduced training time may be achieved by utilizing techniques like a locality hashing technique (such as, a Locality-Sensitive hashing (LSH)) for un- sketching without additional computation of updating the hash table associated with LSH. As the projection of the update rule in the lower dimension may be used, memory requirement and time complexity may be reduced. Based on the application of update rules in a reduced dimension, the computational burden of performing matrix-matrix multiplications within the graph space may be eliminated. Further, the proposed technique uses a fixed Proper Orthogonal Decomposition (POD) projection matrix throughout the training, which may be leveraged to usetechniques like LSH to further reduce training time. Further, the determination of the POD projection matrix may be agnostic to the underlying architecture of the GNN model 104. Thus, the present disclosure may provide scalable graph learning based on matrix decomposition and locality hashing technique. This approach may be optimized for minimal memory usage and fast execution.

[0068] FIG.4 is a diagram that illustrates an exemplary scenario of a sketch unit for performing a matrix decomposition based on a matrix decomposition technique, in accordance with an embodiment of the disclosure. FIG. 4 is described in conjunction with elements from FIG. 1, FIG. 2, and FIG. 3. With reference to FIG. 4, there is shown an exemplary scenario 400 of a POD sketch unit for performing a matrix decomposition based on a matrix decomposition technique. The scenario 400 may include a node feature matrix 402A, a convolution matrix 402B, a mean 404 (associated with the node feature matrix 402A), a matrix sketching model 406, an operation for covariance matrix generation 406A, an operation for Eigen vector determination 406B, an operation for Proper-Orthogonal Decomposition (POD)- matrix generation 406C, and an output 408.

[0069] The POD sketch unit (not shown in FIG. 4) may include suitable logic, circuitry, interfaces and / or code that may be configured to determine the first sketch matrix of node feature matrix 402A and the second sketch matrix of the convolution matrix 402B associated with the GNN model 104. In an example, the processor 202 may use the POD sketch unit to determine the first sketch matrix and the second sketch matrix. The processor 202 may determine the node feature matrix 402A and the convolution matrix 402B associated with the GNN model 104. The determination of the node feature matrix and the convolution matrix are described further, forexample, in FIG. 3. The processor 202 may be further configured to determine the mean 404 (associated with the node feature matrix 402A), based on elements of the node feature matrix 402A. The node feature matrix 402A may be represented based on an expression (1), as follows: ^^(0) ∈ ^^^^×^^where “n” may represent a number of nodes of the GNN model 104; “d” may represent a feature dimension of the GNN model 104; and “^^(0)” may represent the node feature matrix 402A.

[0070] The convolution matrix 402B may be represented based on an expression (2), as follows: ^^ ∈ ^^^^×^^ (2)where “C” may represent the convolution matrix 402B.

[0071] The mean 404 (associated with the node feature matrix 402A) may be represented based on an expression (3), as follows: ^̅^ (3) where “^̅^” may represent a mean value of an input node feature matrix (e.g., the node feature matrix 402A)

[0072] The processor 202 may feed the node feature matrix 402A, the convolution matrix 402B, and the mean 404 to the matrix sketching model 406. The matrix sketching model 406 may correspond to a set of mathematical operations that may be used to obtain the output 408 (i.e., the first sketch matrix and the second sketch matrix). For example, the matrix sketching model 406 may encompass operations such as, the operation for covariance matrix generation 406A, the operation for Eigen vector determination 406B, and the operation for POD-matrix generation 406C.

[0073] At 406A, the operation for covariance matrix generation may be executed. The processor 202 may generate a covariance matrix associated with the node feature matrix 402A. For example, the processor 202 may use an expression (4), as follows, to generate the covariance matrix associated with the node feature matrix 402A:where “^^(0) ^^” may represent an ithdata point of the node feature matrix 402A; and “Q” may represent the covariance matrix associated with the node feature matrix 402A.

[0074] At 406B, the operation for Eigen vector determination may be executed. The processor 202 may determine a predetermined number of Eigen vectors associated with the covariance matrix (i.e., “Q”). For example, the processor 202 may determine “c” leading Eigen vectorsi = 1, 2, …c) of the covariance matrix “Q”. In an embodiment, the Eigen vector of the covariance matrix (i.e., “Q”) may correspond to a vector which on multiplying with the covariance matrix “Q” results in a scaler multiple of the covariance matrix “Q”.

[0075] At 406C, the operation for POD-matrix generation may be executed. The processor 202 may generate a POD matrix based on the predetermined number (e.g., “c”) of Eigen vectorsi = 1, 2, …c). Herein, the determination of each of the first sketch matrix and the second sketch matrix may be further based on the generation of the POD matrix. For example, the processor 202 may generate the POD matrix, based on an expression (5), as follows:where “^^^^, i = 1, 2, …c“ may represent “c” leading Eigen vectors of the covariance matrix (i.e., “Q”); and “^^” may represent the generated POD matrix of the dimensions “c x n”.

[0076] At 408, an output may be determined. The processor 202 may determine an output based on the generation of the POD matrix “^^”. The output may include a sketch (i.e., the first sketch matrix) of input node feature matrix (“^^^^(0)”) and a sketch (i.e., the second sketch matrix) of convolution matrix (“^^^^”). The output may be determined based on expressions (6) and (7), as follows:where “^^” may represent the POD matrix; “^^(0)” may represent the input node feature matrix 402A; “C” may represent the convolution matrix; and “^̅^” may represent a mean value of an input node feature matrix (e.g., the node feature matrix 402A).

[0077] In an embodiment, for determination of the POD matrix from large graphs, the processor 202 may generate a first random vector and a second random vector of a predetermined size. Each first element of the first random vector and each second element of the second random vector being sampled from a set of numbers corresponding to a dimensionality of the node feature matrix. For example, the sketch dimension “c” may be given by an expression (8), as follows:^^ = ⌈^^^^⌉ (8)where “r” may represent a random number.

[0078] In an example, in case “^^ > ^^”, random vectors “r1” (i.e., the first randomvector) and “r2” (i.e., the second random vector), each of size “^^ − ^^”, may begenerated. Each element of “r1(j)” (i.e., the first random vector) and “r2(j)” (i.e., the second random vector) may be sampled from the set “[1, 2, …d]”, which may correspond to a dimensionality (i.e., “d”) of the node feature matrix 402A.

[0079] The processor 202 may determine an estimated feature matrix associated with the node feature matrix 402A, based on the first random vector (i.e., “r1”) and the second random vector (i.e., “r2”). For example, the processor 202 may determine the estimated feature matrix associated with the node feature matrix 402A, based on an expression (9), as follows:where ”^̃^(0)” may represent the estimated feature matrix associated with the node feature matrix 402A; and “⊙” may represent a Hadamard product operation, which may correspond to an element wise multiplication of matrices of same size.

[0080] The processor 202 may determine a singular vector based on the estimated feature matrix. Herein, the generation of the POD matrix may be further based on the singular vector. For example, the processor 202 may determine a mean of the estimated feature matrix “^̃^(0)”, i.e., “[^̅^]”, and may determine a transpose of the POD matrix, i.e., “^^^^” from left singular vectors of a matrix determined based on an expression (10), as follows:

[0081] The scenario 400 of FIG. 4 is for exemplary purposes and should not be construed to limit the scope of the disclosure.

[0082] FIG.5 is a diagram that illustrates an exemplary scenario of determination of a sketch matrix of node representations at a next layer of a graph neural network (GNN) model, in accordance with an embodiment of the disclosure. FIG. 5 is described in conjunction with elements from FIG. 1, FIG. 2, FIG. 3, and FIG. 4. With reference to FIG.5, there is shown an exemplary scenario 500 of determination of a sketch matrix of a feature matrix for a next layer of the graph neural network (GNN) model 104. The scenario 500 may include sketches 502 of node representations at a current layer “l” of the GNN model 104, a bias matrix 504 (associated with a current layer “l” of the GNN model 104), forward propagation rules 506, node representations 508 (for a current layer “l” of the GNN model 104), and sketches 510 of the node representations 508 for the next layer “l+1” of the GNN model 104. The forward propagation rules 506 may include an operation 506A for determination of node representations for a specific layer, an operation 506B for bias matrix determination, and an operation 506C for approximation of the forward propagation rules 506.

[0083] The processor 202 may feed the sketches 502 (e.g., a first sketch matrix 502A and a second sketch matrix 502B) of the node representations from a current layer “l” (denoted as 502A) of the GNN model 104 and the bias matrix 504 (associated with a current layer “l” of the GNN model 104) to the forward propagation rules 506.The processor 202 may select active nodes (e.g., “d1” neurons) associated with a set of layers of the GNN model 104, based on a locality hashing technique, as described further, for example, in FIG.7.

[0084] At 506A, a feature matrix may be determined for the specific layer “l” of the GNN model 104. The processor 202 may determine the feature matrix for the specific layer “l” based on an expression (11), as follows: ^^ = ^^(^^) ^^ ^^^^ + ^^1 (11)where “Z” may represent the feature matrix for the layer “l”; “SC” may represent a sketch (i.e., the second sketch matrix 502B) of the convolutionmatrix, given by “S^^ ∈ ^^^^×^^”; “SX(l)” may represent a sketch (i.e., the first sketch matrix 502A) of the node feature matrix for the layer “l”, given by “^^(^^)∈ ^^^^×^^1^^ ” (wherein “d1” is the total neurons atlayer “l”); and “M1” may represent an intermediate matrix associated with “d1” neurons selected from the GNN model 104.may be determined based on an expression (12), as follows: ^^1 = [^^ ^^ ^̅^]^^×^^1 (12)where “^̅^” may represent a mean vector of the node feature matrix 402A (i.e., “^^(0)”)

[0085] At 506B, a bias matrix for a layer “l” of the GNN model 104 may be determined. The processor 202 may determine the bias matrix 504 (associated with the specific layer “l” of the GNN model 104) based on the mean associated with the node feature matrix and a weight matrix associated with the layer “l” of the set of layers of the GNN model 104. The processor 202 may determine the bias matrix 504 based on an expression (13), as follows:^^ = ^^ Θ(^^) (13)where “M” may represent a transformed feature matrix at layer “l”( of the GNN model 104); and “Θ(^^)” may represent a weight matrix associated with the layer “l” of the GNN model 104.

[0086] At 506C, the forward propagation rules may be approximated. The processor 202 may determine approximation of the forward propagation rules 506 based on at least one of, but not limited to, the POD matrix, the mean associated with the node feature matrix, an activation function, the convolution matrix, the weight matrix associated with a layer (e.g., “l”) of the set of layers, the first sketch matrix, the second sketch matrix, or the bias matrix associated with a layer (e.g., “l”) of the set of layers. The processor 202 may approximate the forward propagation rules 506 based on expressions (14), (15), and (16), as follows:^^ = ^^(^^) (15)^^2 = ^^(: , ^^) ^^ − [^̅^]^^×^^2 (16)where “Q” may represent an un-sketched node representation corresponding to “M”; “A” may represent a set of active neurons; may represent a number of neurons of the layer “l+1”, based on a selection (using a locality hashing technique) of “A” active neurons from the layer “l”; “^^^^” may represent a transpose of the POD matrix; “V” may represent an output from thenodes of the layer “l+1”;“^^(. )” may represent an activation function, such as, a Sigmoidal function; and“V2” may represent an approximation of the forward propagation rules 506 for the layer “l+1”.

[0087] Based on the approximation of the forward propagation rules 506, the processor 202 may determine the node representations 508 for the next layer of the GNN model 104, and also determine the sketches 510 of the node representations 508 for the next layer. For example, based on the expression (16), the processor 202 may determine a sketch matrix 510 (i.e., “SX(l+1)”) of the node representations 508 for a next layer (i.e., layer “l+1”) of the GNN model 104. The processor 202 may similarly continue the approximation of the forward propagation rules 506 based on the sketch matrix the node representations for the next layer, till the final layer of the GNN model 104 is reached. The sketch matrix for the next layer may be used for the approximation of the forward propagation rules 506 for another next layer of the GNN model 104, and so on for the remaining layers of the GNN model 104. Once the forward propagation rules 506 are approximated for the final layer, the processor 202 may determine the node representations for the final layer of the GNN model 104, and also determine sketches of the node representations for the final layer. In an embodiment, the approximation of the forward propagation rules 506 may be represented by an expression (17), as follows:where “^^ ∈ ^^^^×^^” may represent the POD matrix (“c” being a sketch dimension and“n” being a number of nodes in the GNN model 104); “^̅^” may represent a mean of the input node feature matrix 402A; may represent an activation function (e.g., a Sigmoidal function);“^^(^^)” may denote a node representation at a layer “^^”; “^^(^^)” may represent a “qth” convolution matrix;“^^^^(^^,^^) = ^^^^(^^)^^^^” may represent a sketch of the “qth” convolution matrix at the layer “^^”; “^^(^^)” may represent sketches of node representations at the layer “^^ ∈ ^^^^×^^1^^ ”;“Θ(^^,^^)” may represent a weight matrix at the layer “^^” and filter “^^ ∈ ^^^^1×^^2” assuming“^^1” neurons at the layer “^^” and “^^2” neurons at layer “(l+1)”; and“^^ = [^^^^(^^,^^) ^̅^]^^×^^ ” may represent the bias matrix 504 (associated with the layer “l” of the GNN model 104).

[0088] In an embodiment, to optimize the approximation of the forward propagation rules 506, the processor 202 may select a set of elements of the POD matrix based on the locality hashing technique (such as, a locality-sensitive hashing (LSH) technique). Further, the processor 202 may store, in the memory 204, the setof elements of the POD matrix. For example, the complete POD matrix “^^ ∈ ^^^^×^^” maynot be stored in the memory 204, but only selected rows of the matrix, given by the locality hashing technique, may be transferred to the memory 204 for storage.

[0089] In another embodiment, to further optimize the approximation of the forward propagation rules 506, the processor 202 may convert the POD matrix into a sparse matrix representation, based on a count-sketch technique. The processor 202 may store, in the memory 204, the sparse matrix representation. For example, to convert the POD matrix into the sparse matrix, the processor 202 may determine an expression (18), as follows: ^^ ^^(^^^^^^) (18)where “^^ ∈ ^^^^×^^^^”;“^^^^” may represent a number of neurons at a layer “^^”; and “^^” may represent a non-linear activation function.

[0090] The processor 202 may determine an expression (19) based on an expression (19), as follows:where R(k)may represent a “kth” count-sketch of “R”;

[0091] In an example, the count-sketch technique may correspond to a dimensionality reduction technique that may project an n-dimensional vector to a smaller c-dimensional space based on a random hash table h : [n] → [c] and a binary Rademacher variable s : [n] → {±1}, where [n] = {1, . . . , n}. Count sketch may be defined based on an expression (20), as follows:where “CS(u)” may represent a linear transformation of u, i.e., “CS(u) = Ru” and Here, “R ∈ Rc×n” may represent a count sketch matrix, which may have exactly one non-zero element per column.

[0092] Based on the expression (19) and using the count-sketch technique, the processor 202 may determine the sparse matrix representation. For example, the sparse matrix representation may be given by an expression (21), as follows: ^^(^^)^^^^ ∈ ^^^^×^^ (21)where ^^(^^) ∈ ^^^^×^^ may be a sparse matrix. Instead of storage of the POD matrix (i.e.,“^^”), the processor 202 may store the sparse matrix given by the expression (21).

[0093] The scenario 500 of FIG. 5 is for exemplary purposes and should not be construed to limit the scope of the disclosure.

[0094] FIG.6 is a diagram that illustrates an exemplary scenario of un-sketching sketches of node feature matrix at a layer of a graph neural network (GNN) model, in accordance with an embodiment of the disclosure. FIG.6 is described in conjunction with elements from FIG.1, FIG.2, FIG.3, FIG.4, and FIG.5. With reference to FIG.6, there is shown an exemplary scenario 600 of un-sketching sketches of node feature matrix at a layer of the graph neural network (GNN) model 104. The scenario 600 may include sketches 602 of node representations 508 at a next layer “l+1” of the GNN model 104, an un-sketch unit 604, and an un-sketch node representations 606 at the layer “l+1”.

[0095] The processor 202 may feed the sketches 602 of the node representations 508 at a layer “l+1” of the GNN model 104 to the un-sketch unit 604. The un-sketch unit 604 may include suitable logic, circuitry, interfaces and / or code that may be configured to receive the sketches 602 of the node representations 508 at a layer “l+1” of the GNN model 104. The processor 202 may use the un-sketch unit 604 to determine the un-sketch node representations 606 at the layer “l+1”. The un-sketch node representations may have the same size as the original node feature matrix 402A. The processor 202 may determine the un-sketched node representations 606 based on an expression (22), as follows:wheremay represent the un-sketch node representations 606 at the layer“^^ + 1”;may represent a sketched representation of the noderepresentations (i.e., the sketches 602 of the feature matrix) at the layer “^^ + 1”;“^^” may represent the POD matrix; and“^^1” may represent a number of active neurons selected using the locality hashing technique;

[0096] The scenario 600 of FIG. 6 is for exemplary purposes and should not be construed to limit the scope of the disclosure.

[0097] FIG.7 is a diagram that illustrates an exemplary scenario of locality hashing technique for selection of active nodes associated with a set of layers of a graph neural network (GNN) model, in accordance with an embodiment of the disclosure. FIG.7 is described in conjunction with elements from FIG.1, FIG.2, FIG.3, FIG.4, FIG. 5, and FIG.6. With reference to FIG.7, there is shown an exemplary scenario 700 of locality hashing technique for selection of active nodes associated with a set of layers of the graph neural network (GNN) model 104. The scenario 700 may include an element of sketch of node representations for a particular layer (say, a layer “l”) (S) 702, an un-sketch matrix for the particular layer (say, a layer “l”) (U) 704, a first un- sketch vector 706A (i.e., an element of the un-sketch matrix (U) 702 (for example, second column of the un-sketch matrix (U) 702)), a second un-sketch vector 706B (i.e., an element of the un-sketch matrix (U) 702 (for example, second column of the un-sketch matrix U 702), a hash table 708, and a SoftMax output 710.

[0098] The locality hashing technique (for example, a locality-sensitive hashing (LSH) technique) may correspond to a technique for efficient approximation and execution of similarity searches on high-dimensional data. In an embodiment, the locality hashing technique may include hashing of input data in such a way that similar items of the data may map to same "buckets" with high probability, while dissimilar items may likely be mapped to different buckets. The technique may be used to get an approximate SoftMax at the output layer based on a nearestneighborhood search. An advantage of the locality hashing technique in the context of the disclosure may lie in the fact that weights of the hash table 708 may not be required to be updated while the GNN model 104 is being trained, thereby improving the training efficiency of the GNN model 104.

[0099] For example, as shown in FIG.7, based on data in the first un-sketch vector 706A (such as, column “2” of the un-sketch matrix (U) 704) and the second un-sketch vector 706B (such as, column “5” of the un-sketch matrix (U) 704). Thus, the processor 202 may hash the columns of the un-sketch matrix U 704, such as, the first un-sketch vector 706A (i.e., the column “2”) and the second un-sketch vector 706B (i.e., the column “5”), to a bucket associated with the hash values “00”;”01” in the hash table 708. Accordingly, the processor 202 may select active nodes as nodes that correspond to the first un-sketch vector 706A and the second un-sketch vector 706B at the particular layerof the GNN model 104.

[0100] The processor 202 may receive an input matrix, such as, the element of the sketch of the node representations (S) 702 and may query the hash table 708, based on a hash value of the input matrix (such as, the sketch of the node representations (S) 702). In an example, the input matrix (such as, the element of the sketch of the node representations (S) 702) may be similar to the elements in the un-sketch matrix given by the bucket associated with hash values “00”;”01”. In an embodiment, the processor 202 may multiply the sketch of the node representations (S) 702 with a matrix including columns “2” and “5” of the un-sketch matrix (U) 704 to determine the SoftMax output 710 associated with the sketch of the node representations (S) 702. For example, the processor 202 may determine the SoftMax output 710 based on an expression (23), as follows:^^^^^^^^^^^^^^ ^^^^^^^^^^^^ = ^^ ^^^^ . ^^ (23)where “^^^^^^” may represent the matrix including selected columns “A” of the weight matrix (W) 704.

[0101] Thus, the element of the sketch of the node representations (S) 702 need only be multiplied with selected columns of the un-sketch matrix (U) 704, which may lead to a saving of time and computational cost required to determine the SoftMax output 710.

[0102] The scenario 700 of FIG. 7 is for exemplary purposes and should not be construed to limit the scope of the disclosure.

[0103] FIG.8 is a diagram that illustrates an exemplary scenario of scalable graph learning based on matrix decomposition and locality hashing technique, in accordance with an embodiment of the disclosure. FIG.8 is described in conjunction with elements from FIG. 1, FIG. 2, FIG. 3, FIG. 4, FIG. 5, FIG. 6, and FIG. 7. With reference to FIG. 8, there is shown an exemplary scenario 800 of scalable graph learning based on matrix decomposition and locality hashing technique. The scenario 800 may include a POD sketch unit 802 including a node feature matrix 804A, a convolution matrix 804B, a mean 804C associated with the node feature matrix 804A, a POD sketch matrix 806A, a covariance matrix 806B, a matrix multiplication 806C of the covariance matrix 806B, a first sketch matrix 808A, a second sketch matrix 808B, and a bias matrix (M) 808C. The scenario 800 may further include approximate forward propagation (AFP) 810, an un-sketch unit 812, un-sketch node representations 814 at layer “l+1” of the GNN model 104, and sketches 816 of node representations at a layer "l+1" of the GNN model 104.

[0104] The POD sketch unit 802 may include suitable logic, circuitry, interfaces and / or code that may be configured to determine the first sketch matrix 808A, the second sketch matrix 808B, and the bias matrix (M) 808C. The processor 202 may use the POD sketch unit 802 to determine the first sketch matrix 808A, the second sketch matrix 808B, and the bias matrix (M) 808C. For example, as shown in FIG.8, the processor 202 may determine the POD sketch matrix 806A, based on the node feature matrix 804A. Further, the processor 202 may determine the covariance matrix 806B, based on the convolution matrix 804B. The processor 202 may determine the matrix multiplication 806C of the covariance matrix 806B, based on the mean 804C associated with the node feature matrix 804A and the covariance matrix 806B. For example, the processor 202 may multiply elements of the covariance matrix 806B with corresponding elements of the mean 804C to determine the matrix multiplication 806C. The processor 202 may determine the first sketch matrix 808A and the second sketch matrix 808B based on the POD sketch matrix 806A and the covariance matrix 806B, respectively. Further, the processor 202 may determine the bias matrix (M) 808C based on the matrix multiplication 806C of the covariance matrix 806B. The determination of the first sketch matrix 808A and the second sketch matrix 808B are described further, for example, in FIG.4.

[0105] The processor 202 may determine the approximate forward propagation (AFP) 810 to train the GNN model 104, based on the first sketch matrix 808A, the second sketch matrix 808B, and the bias matrix (M) 808C. The determination of the approximate forward propagation (AFP) 810 is described further, for example, in FIG.5. The processor 202 may select the active nodes associated with the set of layers of the GNN model 104, based on a locality hashing technique, as described further,for example, in FIG. 3 and FIG. 7. Further, the processor 202 may determine node representations for the final layer of the GNN model 104, based on the active nodes, the forward propagation rules, and an un-sketching technique (as described further).

[0106] The un-sketch unit 812 may include suitable logic, circuitry, interfaces and / or code that may be configured to receive the sketches (i.e., the first sketch matrix 808A) of a feature matrix (e.g., the node feature matrix 402A) at a layer “l” of the GNN model 104. The processor 202 may use the un-sketch unit 812 to determine an un-sketched node feature matrix at a next layer, such as, the layer “l+1”. The un- sketched node feature matrix may be un-sketch node representations 606 at the layer “l+1”. The un-sketch node feature matrix may have the same size as the original node feature matrix 804A.

[0107] With reference to FIG. 5, the processor 202 may determine the node representations of the next layer “l+1” of the GNN model 104. Further, based on the expression (16) and (17), the processor 202 may determine a sketch matrix (i.e., “SX(l+1)”) of the node representations for the next layer (i.e., the layer “l+1”) of the GNN model 104. Thus, the processor 202 may determine the sketched node representations for the next layer “l+1”. The processor 202 may similarly continue the determination of the approximate forward propagation (AFP) 810 till the final layer of the GNN model 104 is reached. The sketch matrix for the next layer and un- sketched node representations for the next layer may be used for the determination of the approximate forward propagation (AFP) 810 for a layer succeeding the next layer of the GNN model 104, and so on for the remaining layers of the GNN model 104. Once the determination of the approximate forward propagation (AFP) 810 iscomplete for the final layer, the processor 202 may determine the node representations for the final layer of the GNN model 104.

[0108] The scenario 800 of FIG. 8 is for exemplary purposes and should not be construed to limit the scope of the disclosure.

[0109] FIG.9 is a diagram that illustrates an exemplary scenario of scalable graph learning based on matrix decomposition and locality hashing technique, in accordance with an embodiment of the disclosure. FIG.9 is described in conjunction with elements from FIG.1, FIG.2, FIG.3, FIG.4, FIG.5, FIG.6, FIG.7, and FIG.8. With reference to FIG. 9, there is shown an exemplary scenario 900 of scalable graph learning based on matrix decomposition and locality hashing technique. The scenario 900 may include an input graph 902A, an operation for POD sketching 904, and a training block 906 including an approximate update rule layer-1 906A and an approximate update rule layer-2906B, an operation for POD un-sketching 908, and a SoftMax layer 910A. The scenario 900 may further include an input graph 902B and an inference block 912 including a simplified graph convolution network (SGCN) layer-1912A, an SGCN layer-2912B, and a SoftMax layer 910B.

[0110] Though two approximate update rule layers and two SGCN layers are shown in FIG. 9, the disclosure may not be so limited. The GNN model 104 may include more than two approximate update rule layers and / or more than two SGCN layers, without departure from the scope of the disclosure.

[0111] The processor 202 may receive the input graph 902A to train the GNN model 104. In an example, the input graph 902A may correspond to a graph with “2708” nodes, wherein each node may have “1433” features. A total number of classes in which the nodes of the input graph 902A may be classified may be “7”. Forexample, the input graph 902A may correspond to a “Cora” dataset associated with “7” classes. Examples of the “7” classes may be a case-based class (category “0”), a genetic algorithm class (category “1”), a neural network class (category “2”), a probabilistic method class (category “3”), a reinforcement learning class (category “4”), a rule-learning class (category “5”), and a theoretical machine learning class (category “6”).

[0112] At 904, an operation for POD sketching may be executed. To execute the POD sketching, the processor 202 may determine the first sketch matrix and the second sketch matrix based on the node feature matrix and the convolution matrix associated with the GNN model 104, respectively. For example, based on the node feature matrix, the processor 202 may determine a sketch matrix (i.e., the first sketch matrix) of a size “54 x 1433”. Further, based on the convolution matrix, the processor 202 may determine a sketch matrix (i.e., the second sketch matrix) of a size “54 x 54”. In an example, the determination of the first sketch matrix and the second sketch matrix may be based on a sketch ratio of “0.02”. An advantage of the disclose may lie in the fact that POD sketching can be performed at a low sketch ratio without compromising on the accuracy of the training of the GNN model 104.

[0113] The processor 202 may feed the first sketch matrix and the second sketch matrix to the training block 906. The processor 202 may determine approximate forward propagations across the approximate update rule layer-1 906A and the approximate update rule layer-2 906B based on the first sketch matrix and the second sketch matrix. The determination of the approximate forward propagations may correspond to a sketch space associated with the GNN model 104. In an example, a size of a weight matrix associated with the approximate update rule layer 1 may be“1433 x 16”. Further, a size of a weight matrix associated with the approximate update rule layer 2 may be “16 x 7”. The determination of the approximate forward propagations is described further, for example, in FIG.5. Based on the determination of the approximate forward propagations, the processor 202 may determine the node feature matrix for the final layer of the GNN model 104 and the sketches of the node feature matrix for a next layer (e.g., a layer “3”, not shown in FIG.9) of the GNN model 104.

[0114] The processor 202 may select the active nodes associated with the set of layers of the GNN model 104, based on a locality hashing technique, as described further, for example, in FIG.3 and FIG.7. Further, the processor 202 may determine node representations for the final layer of the GNN model 104, based on the active nodes, the forward propagation rules, and an un-sketching technique (as described further, for example, at 908).

[0115] At 908, an operation for POD un-sketching may be executed. To execute the POD un-sketching, the processor 202 may convert the sketches of the node feature matrix (i.e., the node representations) from a sketch space to an un-sketched node representations that may have the same size as that of the original node feature matrix. The POD un-sketching is described further, for example, in FIG.3 and FIG.6.

[0116] To train the GNN model 104, the processor 202 may pass the weights and the node representations (e.g., the un-sketch node feature matrix) associated with the final layer of the GNN model 104 to the final layer, such as, the SoftMax layer 910A. Thereafter, the processor 202 may determine a training loss of the GNN model 104, based on the weights and the node representations for the final layer, such as, the SoftMax layer 910A. The training loss may be associated with a prediction errorassociated with a downstream prediction task of the GNN model 104. Further, the GNN model 104 may be configured to be trained to generate a prediction result for the downstream prediction task, based on the training loss.

[0117] The processor 202 may be configured to determine model parameters associated with the GNN model 104, based on the GNN model 104 being configured to be trained to generate the prediction result. The determination of the model parameters may be based on the training loss. For example, the determined model parameters may include information on at least one of, but not limited to, an indication of active nodes and edges of the GNN model 104, weights and biases associated with the GNN model 104, the training loss, and tunable model hyperparameters associated with the GNN model 104.

[0118] In the inference block 912 of the GNN model 104, the model parameters associated with the GNN model 104 may be transferred from a trained GNN model (i.e., a GNN model trained in the training phase) to a live GNN model (e.g., the GNN model 104). The model parameters may be used to configure the GNN model 104 for an inference task (e.g., a downstream prediction task). For example, the processor 202 may receive a test graph (e.g., the input graph 902B) associated with a downstream prediction task. In an embodiment, the test graph (e.g., the input graph 902B) may be of the same application domain or same as the input graph 902A. The processor 202 may configure the weights, the edges, and node representations of the GNN model 104, based on the model parameters determined for the GNN model 104 and the training loss. The information about the configuration of the weights, the edges, and the node representations of the GNN model 104 may be passed to the SGCN layer-1912A, the SGCN layer-2912B of the GNN model 104, and subsequentlyto the SoftMax layer 910B. In an example, a size of a weight matrix at the SGCN layer- 1912A may be “1433 x 16”. Further, a size of a weight matrix at the SCGN layer-2 may be “16 x 7”. Based on the configuration of the GNN model 104 for the inference task, the processor 202 may determine a prediction result (e.g., an edge classification or a node classification) associated with the downstream prediction task associated with the test graph. For example, the processor 202 may determine a prediction result corresponding to one of the “7” classes associated with the input graph 902B.

[0119] The scenario 900 of FIG. 9 is for exemplary purposes and should not be construed to limit the scope of the disclosure.

[0120] FIG.10 is a diagram that illustrates a flowchart of an exemplary method for scalable graph learning based on matrix decomposition and locality hashing technique, in accordance with an embodiment of the disclosure. FIG.10 is described in conjunction with elements from FIG.1, FIG.2, FIG. 3, FIG.4, FIG.5, FIG.6, FIG.7, FIG.8, and FIG. 9. With reference to FIG. 9, there is shown an exemplary flowchart 1000 of a method for scalable graph learning based on matrix decomposition and locality hashing technique. The flowchart 1000 may include operations 1002 to 1018, which may be executed by the processor 202 (of FIG.2) of the electronic device 102 (of FIG.1). The flowchart 1000 may start at 1002 and proceed to 1004.

[0121] At 1004, a graph dataset associated with an application domain may be received. The processor 202 may be configured to receive the graph dataset 110 associated with an application domain. The reception of the graph dataset is described further, for example, in FIG.3.

[0122] At 1006, a node feature matrix and a convolution matrix associated with the graph dataset may be determined. The processor 202 may be configured todetermine the node feature matrix 402A and the convolution matrix 402B associated with the graph dataset 110. The determination of the node feature matrix and the convolution matrix is described further, for example, in FIG.3.

[0123] At 1008, a first sketch matrix for the node feature matrix and a second sketch matrix for the convolution matrix may be determined, based on a matrix decomposition technique. The processor 202 may be configured to determine the first sketch matrix 502A for the node feature matrix 402A and the second sketch matrix 502B for the convolution matrix 402B, based on a matrix decomposition technique. For example, matrix decomposition technique may correspond to a Proper Orthogonal Decomposition (POD)-technique. The determination of the first sketch matrix and the second sketch matrix is described further, for example, in FIG. 4.

[0124] At 1010, forward propagation rules associated with a GNN model may be determined, based on the first sketch matrix and the second sketch matrix. The processor 202 may be configured to determine the forward propagation rules 506 associated with the GNN model 104, based on the first sketch matrix 502A and the second sketch matrix 502B. The determination of the forward propagation rules is described further, for example, in FIG.5.

[0125] At 1012, active nodes associated with a set of layers of the GNN model may be selected, based on a locality hashing technique. The processor 202 may be configured to select the active nodes associated with the set of layers of the GNN model 104, based on the locality hashing technique. The selection of active nodes and the locality hashing technique are described further, for example, in FIG.5 and FIG. 7, respectively.

[0126] At 1014, node representations for a final layer of the GNN model may be determined, based on the active nodes, the forward propagation rules, and an un- sketching technique. The processor 202 may be configured to determine the node representations for the final layer of the GNN model 104, based on the active nodes, the forward propagation rules, and the un-sketching technique. The determination of the node representations is described further, for example, in FIG.6.

[0127] At 1016, a training loss of the GNN model may be determined, based on the node representations for the final layer, wherein the training loss may be associated with a prediction error associated with a downstream prediction task of the GNN model, and wherein the GNN model may be configured to be trained to generate the prediction result for the downstream prediction task, based on the training loss. The processor 202 may be configured to determine the training loss of the GNN model 104, based on the node representations for the final layer. The training loss may be associated with a prediction error associated with a downstream prediction task of the GNN model 104. Further, the GNN model 104 may be configured to be trained to generate the prediction result for the downstream prediction task, based on the training loss. The training of the GNN model is described further, for example, in FIG.3 and FIG.9.

[0128] At 1018, a rendering of first information including the prediction result associated with the downstream prediction task may be controlled, based on the GNN model. The processor 202 may be configured to control the rendering of the first information. The first information may include the prediction result associated with the downstream prediction task, based on the (trained) GNN model 104. For example, the processor 202 may feed an input graph (e.g., the test graph 312)associated with a downstream prediction task (e.g., a node classification task) to the (trained) GNN model 104. The (trained) GNN model 104 may be applied on the input graph to determine a prediction result (e.g., node labels). The control of the rendering of the first information is described further, for example, in FIG.3. Control may pass to end.

[0129] Although the flowchart 1000 is illustrated as discrete operations, such as 1004, 1006, 1008, 1010, 1012, 1014, 1016, and 1018, the disclosure is not so limited. However, in certain embodiments, such discrete operations may be further divided into additional operations, combined into fewer operations, or eliminated, depending on the particular implementation without detracting from the essence of the disclosed embodiments.

[0130] Various embodiments of the disclosure may provide one or more non- transitory computer-readable storage media configured to store instructions that, in response to being executed, cause a system (such as the example electronic device 102) to perform a set of operations. The set of operations may include receiving a graph dataset (e.g., the graph dataset 110) associated with an application domain. The set of operations may further include determining a node feature matrix and a convolution matrix associated with the graph dataset 110. The set of operations may further include determining a first sketch matrix for the node feature matrix and a second sketch matrix for the convolution matrix, based on the matrix decomposition technique. The set of operations may further include determining forward propagation rules associated with a Graph Neural Network (GNN) model (e.g., the GNN model 104), based on the first sketch matrix and the second sketch matrix to select active nodes associated with the set of layers of the GNN model 104, based ona locality hashing technique. Node representations may be determined for a final layer of the GNN model 104, based on the active nodes, the forward propagation rules and an un-sketching technique. The set of operations may further include determining a training loss of the GNN model 104, based on the node representations for the final layer, the training loss being associated with a prediction error associated with a downstream prediction task of the GNN model 104, and the GNN model 104 being configured to be trained to generate a prediction result for the downstream prediction task, based on the training loss to control rendering of the first information including the prediction result associated with the downstream prediction task, based on the GNN model 104.

[0131] As used in the present disclosure, the terms “module” or “component” may refer to specific hardware implementations configured to perform the actions of the module or component and / or software objects or software routines that may be stored on and / or executed by general purpose hardware (e.g., computer-readable media, processing devices, etc.) of the computing system. In some embodiments, the different components, modules, engines, and services described in the present disclosure may be implemented as objects or processes that execute on the computing system (e.g., as separate threads). While some of the system and methods described in the present disclosure are generally described as being implemented in software (stored on and / or executed by general purpose hardware), specific hardware implementations or a combination of software and specific hardware implementations are also possible and contemplated. In this description, a “computing entity” may be any computing system as previously defined in thepresent disclosure, or any module or combination of modulates running on a computing system.

[0132] Terms used in the present disclosure and especially in the appended claims (e.g., bodies of the appended claims) are generally intended as “open” terms (e.g., the term “including” should be interpreted as “including, but not limited to,” the term “having” should be interpreted as “having at least,” the term “includes” should be interpreted as “includes, but is not limited to,” etc.).

[0133] Additionally, if a specific number of an introduced claim recitation is intended, such an intent will be explicitly recited in the claim, and in the absence of such recitation no such intent is present. For example, as an aid to understanding, the following appended claims may contain usage of the introductory phrases "at least one" and "one or more" to introduce claim recitations. However, the use of such phrases should not be construed to imply that the introduction of a claim recitation by the indefinite articles "a" or "an" limits any particular claim containing such introduced claim recitation to embodiments containing only one such recitation, even when the same claim includes the introductory phrases "one or more" or "at least one" and indefinite articles such as "a" or "an" (e.g., “a” and / or “an” should be interpreted to mean “at least one” or “one or more”); the same holds true for the use of definite articles used to introduce claim recitations.

[0134] In addition, even if a specific number of an introduced claim recitation is explicitly recited, one of ordinary skill in the art will recognize that such recitations should be interpreted to mean at least the recited number (e.g., the bare recitation of "two recitations," without other modifiers, means at least two recitations, or two or more recitations). Furthermore, in those instances where a convention analogous to“at least one of A, B, and C, etc.” or “one or more of A, B, and C, etc.” is used, in general such a construction is intended to include A alone, B alone, C alone, A and B together, A and C together, B and C together, or A, B, and C together, etc.

[0135] Further, any disjunctive word or phrase presenting two or more alternative terms, whether in the description, claims, or drawings, should be understood to contemplate the possibilities of including one of the terms, either of the terms, or both terms. For example, the phrase “A or B” should be understood to include the possibilities of “A” or “B” or “A and B.”

[0136] All examples and conditional language recited in the present disclosure are intended for pedagogical objects to aid the reader in understanding the present disclosure and the concepts contributed by the inventor to furthering the art and are to be construed as being without limitation to such specifically recited examples and conditions. Although embodiments of the present disclosure have been described in detail, various changes, substitutions, and alterations could be made hereto without departing from the spirit and scope of the present disclosure.

Claims

What is claimed is:

1. A method, executed by a processor, comprising: receiving a graph dataset associated with an application domain; determining a node feature matrix and a convolution matrix associated with the graph dataset; determining a first sketch matrix for the node feature matrix and a second sketch matrix for the convolution matrix, based on a matrix decomposition technique; determining forward propagation rules associated with a graph neural network (GNN) model, based on the first sketch matrix and the second sketch matrix; selecting active nodes associated with a set of layers of the GNN model, based on a locality hashing technique; determining node representations for a final layer of the GNN model, based on the active nodes, the forward propagation rules and an un-sketching technique; determining a training loss of the GNN model, based on the node representations for the final layer, the training loss being associated with a prediction error associated with a downstream prediction task of the GNN model, and the GNN model being configured to be trained to generate a prediction result for the downstream prediction task, based on the training loss; and controlling rendering of first information including the prediction result associated with the downstream prediction task, based on the GNN model.

2. The method according to claim 1, wherein the application domain corresponds to one of a financial domain, a social network domain, a recommendation system domain, a biological domain, a bio-informatics domain, a chemistry domain, a bio- chemistry domain, a material science domain, or a citation network domain.

3. The method according to claim 1, wherein the matrix decomposition technique corresponds to a Proper Orthogonal Decomposition (POD)-technique.

4. The method according to claim 1, further comprising: generating a covariance matrix associated with the node feature matrix; determining a predetermined number of eigen vectors associated with the covariance matrix; and generating a POD matrix based on the predetermined number of eigen vectors, the determination of each of the first sketch matrix and the second sketch matrix being further based on the generation of the POD matrix.

5. The method according to claim 4, further comprising: generating a first random vector and a second random vector of a predetermined size, each first element of the first random vector and each second element of the second random vector being sampled from a set of numbers corresponding to a dimensionality of the node feature matrix; determining an estimated feature matrix associated with the node feature matrix, based on the first random vector and the second random vector; anddetermining a singular vector based on the estimated feature matrix, the generation of the POD matrix being further based on the singular vector.

6. The method according to claim 4, wherein the determination of the forward propagation rules is further based on at least one of: the POD matrix, a mean associated with the node feature matrix, an activation function, the convolution matrix, a weight matrix associated with a layer of the set of layers, the first sketch matrix, the second sketch matrix, or a bias matrix associated with the layer of the set of layers.

7. The method according to claim 4, further comprising: selecting a set of elements of the POD matrix based on the locality hashing technique; and storing, in a memory, the set of elements of the POD matrix.

8. The method according to claim 4, further comprising: converting the POD matrix into a sparse matrix representation, based on a count-sketch technique; and storing, in a memory, the sparse matrix representation.

9. The method according to claim 1, further comprising applying the forward propagation rules to the active nodes of the set of layers of the GNN model, the node representations for the final layer of the GNN model being further based on the application of the forward propagation rules.

10. The method according to claim 1, further comprising determining model parameters associated with the GNN model, based on the GNN model being configured to be trained to generate the prediction result.

11. The method according to claim 10, further comprising: receiving a test graph associated with the application domain; applying the GNN model on the test graph based on the model parameters; and generating the prediction result associated with the test graph based on the final layer of the GNN model and the forward propagation rules.

12. One or more non-transitory computer-readable storage media configured to store instructions that, in response to being executed, cause an electronic device to perform operations, the operations comprising: receiving a graph dataset associated with an application domain; determining a node feature matrix and a convolution matrix associated with the graph dataset;determining a first sketch matrix for the node feature matrix and a second sketch matrix for the convolution matrix, based on a matrix decomposition technique; determining forward propagation rules associated with a graph neural network (GNN) model, based on the first sketch matrix and the second sketch matrix; selecting active nodes associated with a set of layers of the GNN model, based on a locality hashing technique; determining node representations for a final layer of the GNN model, based on the active nodes, the forward propagation rules and an un-sketching technique; determining a training loss of the GNN model, based on the node representations for the final layer, the training loss being associated with a prediction error associated with a downstream prediction task of the GNN model, and the GNN model being configured to be trained to generate a prediction result for the downstream prediction task, based on the training loss; and controlling rendering of first information including the prediction result associated with the downstream prediction task, based on the GNN model.

13. The one or more non-transitory computer-readable storage media according to claim 12, wherein the matrix decomposition technique corresponds to a Proper Orthogonal Decomposition (POD)-technique.

14. The one or more non-transitory computer-readable storage media according to claim 12, the operations further comprising: generating a covariance matrix associated with the node feature matrix; determining a predetermined number of eigen vectors associated with the covariance matrix; and generating a POD matrix based on the predetermined number of eigen vectors, the determination of each of the first sketch matrix and the second sketch matrix being further based on the generation of the POD matrix.

15. The one or more non-transitory computer-readable storage media according to claim 14, the operations further comprising: generating a first random vector and a second random vector of a predetermined size, each first element of the first random vector and each second element of the second random vector being sampled from a set of numbers corresponding to a dimensionality of the node feature matrix; determining an estimated feature matrix associated with the node feature matrix, based on the first random vector and the second random vector; and determining a singular vector based on the estimated feature matrix, the generation of the POD matrix being further based on the singular vector.

16. The one or more non-transitory computer-readable storage media according to claim 14, wherein the determination of the forward propagation rules is further based on at least one of: the POD matrix,a mean associated with the node feature matrix, an activation function, the convolution matrix, a weight matrix associated with a layer of the set of layers, the first sketch matrix, the second sketch matrix, or a bias matrix associated with the layer of the set of layers.

17. The one or more non-transitory computer-readable storage media according to claim 12, the operations further comprising applying the forward propagation rules to the active nodes of the set of layers of the GNN model, the node representations for the final layer of the GNN model being further based on the application of the forward propagation rules.

18. The one or more non-transitory computer-readable storage media according to claim 12, the operations further comprising determining model parameters associated with the GNN model, based on the GNN model being configured to be trained to generate the prediction result.

19. The one or more non-transitory computer-readable storage media according to claim 18, further comprising: receiving a test graph associated with the application domain; applying the GNN model on the test graph based on the model parameters; andgenerating the prediction result associated with the test graph based on the final layer of the GNN model and the forward propagation rules.

20. An electronic device, comprising: a memory configured to store instructions; and a processor, coupled to the memory, configured to execute the instructions to perform a process comprising: receiving a graph dataset associated with an application domain; determining a node feature matrix and a convolution matrix associated with the graph dataset; determining a first sketch matrix for the node feature matrix and a second sketch matrix for the convolution matrix, based on a matrix decomposition technique; determining forward propagation rules associated with a graph neural network (GNN) model, based on the first sketch matrix and the second sketch matrix; selecting active nodes associated with a set of layers of the GNN model, based on a locality hashing technique; determining node representations for a final layer of the GNN model, based on the active nodes, the forward propagation rules and an un-sketching technique; determining a training loss of the GNN model, based on the node representations for the final layer, the training loss being associated with a prediction error associated with a downstream prediction task of the GNNmodel, and the GNN model being configured to be trained to generate a prediction result for the downstream prediction task, based on the training loss; and controlling rendering of first information including the prediction result associated with the downstream prediction task, based on the GNN model.

Images