Data processing method and device, electronic equipment and storage medium

Abstract

The invention discloses a data processing method and device, electronic equipment and a storage medium, and the method comprises the steps: obtaining to-be-processed graph data, determining a target cache priority of node data of each node in the graph data according to the cache attribute information of a preset accelerator set, the accelerator set comprises a first accelerator and a second accelerator, the first bandwidth of the first accelerator is larger than the second bandwidth of the second accelerator, and according to the target cache priority, the feature data and the structure data are cached into the first accelerator in sequence so as to perform acceleration training of the graph neural network model in the first accelerator. By adopting the embodiment of the invention, the feature data and the structure data can be cached in the same accelerator at the same time, so that the training process of the graph neural network model can be accelerated in the same accelerator.

Description

Technical Field

[0001] The present invention relates to the field of data processing technology, and in particular to a data processing method, device, electronic equipment and storage medium. Background Art

[0002] Graph neural networks in machine learning are a type of deep learning model that has the ability to extract features from graph data (graph data includes multiple nodes and edges representing connection relationships between nodes) and learn the implicit information of nodes in graph data. Graph neural network models are trained by sampling from graph data. Under the existing technology, the sampling method in the training process of graph neural network models is: select nodes in the graph data according to a pre-acquired probability distribution and save them as multiple sub-graph data. Among them, each sub-graph data is a part of the graph data, and also contains a certain number of nodes and the connection relationships between these nodes. These sub-graphs are used as input data for training graph neural network models to learn the implicit information of nodes in the graph data.

[0003] In the training process of graph neural network models, accelerators are usually used to speed up the training process of the model. In related technologies, the two commonly used accelerators are CPU and GPU. However, the CPU cannot meet the requirements in terms of computing performance based on its serial execution structure. Although the GPU has obvious advantages in computing performance, it cannot break through the power consumption barrier like the CPU. It can usually only completely cache the structural data of the graph but cannot completely cache the feature data of the graph. As a result, when the graph scale is large and the GPU memory is limited, the training process of the graph neural network model cannot be effectively accelerated. Summary of the invention

[0004] The purpose of the embodiments of the present invention is to provide a data processing method, device, electronic device and storage medium to solve the technical problem that the training process of the graph neural network model cannot be effectively accelerated.

[0005] In a first aspect, an embodiment of the present invention provides a data processing method, including:

[0006] Get the graph data to be processed;

[0007] Determining, according to cache attribute information of a preset accelerator set, a target cache priority of node data of each node in the graph data, wherein the node data includes feature data and structure data, the accelerator set includes a first accelerator and a second accelerator, and a first bandwidth of the first accelerator is greater than a second bandwidth of the second accelerator;

[0008] According to the target cache priority, the feature data and the structure data are cached in the first accelerator in sequence to perform accelerated training of the graph neural network model in the first accelerator.

[0009] In a second aspect, an embodiment of the present invention provides a data processing device, including:

[0010] A first acquisition module, used to acquire graph data to be processed;

[0011] a determination module, configured to determine a target cache priority of node data of each node in the graph data according to cache attribute information of a preset accelerator set, wherein the node data includes feature data and structure data, the accelerator set includes a first accelerator and a second accelerator, and a first bandwidth of the first accelerator is greater than a second bandwidth of the second accelerator;

[0012] A cache module is used to cache the feature data and the structure data in the first accelerator in sequence according to the target cache priority, so as to perform accelerated training of the graph neural network model in the first accelerator.

[0013] In a third aspect, an embodiment of the present invention provides an electronic device, comprising a processor, a memory, and a computer program stored in the memory and executable on the processor, wherein the processor implements the steps of any of the above-mentioned data processing methods when executing the computer program.

[0014] In a fourth aspect, an embodiment of the present invention provides a computer-readable storage medium, wherein the computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the steps in any of the above-mentioned data processing methods are implemented.

[0015] An embodiment of the present invention provides a data processing method, device, electronic device and storage medium. The method determines the target cache priority of the feature data and structure data of each node in the graph data, so that the feature data and structure data can be stored in an accelerator in sequence according to the target cache priority, thereby facilitating accelerated training of the graph neural network model in the accelerator. BRIEF DESCRIPTION OF THE DRAWINGS

[0016] Figure 1 It is a flow chart of a data processing method provided by an embodiment of the present invention;

[0017] Figure 2 It is a flow chart of a method for calculating coldness and heatness provided by an embodiment of the present invention;

[0018] Figure 3 is a schematic diagram of a method for calculating coldness and heatness information provided by an embodiment of the present invention;

[0019] Figure 4 is a structural schematic diagram of a data processing device provided by an embodiment of the present invention;

[0020] Figure 5 is another structural schematic diagram of a data processing device provided by an embodiment of the present invention;

[0021] Figure 6 is a structural schematic diagram of an electronic device provided by an embodiment of the present invention;

[0022] Figure 7 This is another structural schematic diagram of an electronic device provided by an embodiment of the present invention. DETAILED DESCRIPTION

[0023] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0024] It should be understood that the various steps described in the method embodiments of the present disclosure may be performed in different orders and / or in parallel. In addition, the method embodiments may include additional steps and / or omit the steps shown. The scope of the present disclosure is not limited in this respect.

[0025] The term "including" and its variations used herein are open inclusions, i.e., "including but not limited to". The term "based on" means "based at least in part on". The term "one embodiment" means "at least one embodiment"; the term "another embodiment" means "at least one additional embodiment"; the term "some embodiments" means "at least some embodiments". The relevant definitions of other terms will be given in the following description.

[0026] In the training process of graph neural network models, accelerators are usually used to speed up the training process of the model. Specifically, a graph caching strategy can be used to accelerate the training process of graph neural network models: the graph data is divided, and each accelerator caches its own graph partition, and caches the feature data of the nodes in its own memory from large to small according to the node out-degree until all available memory is used up. Because the feature data of nodes with high out-degree will be accessed more frequently, caching them in the accelerator can speed up the feature extraction process during training and improve training performance.

[0027] In related technologies, the two most commonly used accelerators are CPU and GPU. However, the CPU cannot meet the requirements in terms of computing performance ideally due to its serial execution structural characteristics. Although the GPU has obvious advantages in computing performance, it cannot break through the power consumption barrier like the CPU. It can usually only completely cache the structural data and some feature data of the graph. When the graph is large and the GPU memory is limited, it cannot effectively accelerate the training process of the graph neural network model.

[0028] In order to solve the technical problems existing in the related art, the embodiment of the present invention provides a data processing method, see Figure 1 , Figure 1 is a flow chart of a data processing method provided by an embodiment of the present invention, the method comprising steps 101 to 103;

[0029] Step 101, obtaining graph data to be processed.

[0030] In this embodiment, the graph data to be processed provided by this embodiment is the graph data needed for training the graph neural network model. The graph data provided by this embodiment is a graph structure composed of a number of nodes and the lengths of the edges connecting the nodes. The graph data is usually used as the input for training the graph neural network model, so that the graph neural network model can learn the implicit vector representation of each node in the graph data, and help the graph neural network model classify each node.

[0031] Step 102: Determine the target cache priority of the node data of each node in the graph data according to the cache attribute information of the preset accelerator set.

[0032] The node data includes feature data and structure data, the accelerator set includes a first accelerator and a second accelerator, and a first bandwidth of the first accelerator is greater than a second bandwidth of the second accelerator.

[0033] In this embodiment, the cache attribute information provided by this embodiment includes bandwidth information and storage capacity information. The first accelerator provided by this embodiment can be a graphics processing unit (Graphics Processing Unit, GPU), and the second accelerator can be a central processing unit (Central Processing Unit, CPU). In the related art, since the computing performance of the GPU is more advantageous, the GPU is usually used to cache the graph data to accelerate the feature extraction process in the training process of the graph neural network model and improve the training performance. Among them, the GPU is usually classified into a sampler and a trainer. The structural data of the graph is completely cached in the sampler video memory, and the feature data of the graph is partially cached on the trainer. The sampling and feature extraction are pipelined to accelerate the training process through the above means. However, the sampler only supports the complete caching of the structural data of the graph. When the graph scale is large and the GPU memory is limited, the training process of the graph neural network model cannot be effectively accelerated.

[0034] Therefore, this embodiment needs to cache both the structural data and the feature data of the graph data to effectively accelerate the training process of the graph neural network model. Specifically, before determining the target cache priority of the node data of each node in the graph data according to the cache attribute information of the preset accelerator set, the data processing method provided by this embodiment may also include: obtaining the target hotness information of the node data of each node in the graph data; obtaining the first bandwidth and the first storage capacity of the first accelerator in the preset accelerator set, and the second bandwidth and the second storage capacity of the second accelerator in the accelerator set.

[0035] The heat and coldness information provided in this embodiment is used to characterize the expectation of sampling the characteristic data and structural data of the nodes in the graph data when sampling under the same metric standard. After obtaining the target heat and coldness information of the node data of each node in the graph data, and the cache attribute information of each accelerator, the target cache priority of the node data of each node in the graph data can be determined. Specifically, the step of determining the target cache priority of the node data of each node in the graph data based on the cache attribute information of a preset accelerator set provided in this embodiment may include: determining the target cache priority of the node data of each node in the graph data based on the first bandwidth, the first storage capacity, the second bandwidth, the second storage capacity, and the target heat and coldness information and the amount of node data of each node.

[0036] As an optional embodiment, the target cache priority of each node data can be calculated by a preset priority calculation formula. Specifically, the priority calculation formula provided in this embodiment is:

[0037]

[0038] Among them, V is the target cache priority corresponding to the node data, H is the hotness information corresponding to the node data, and R h is the size of the node data cache in the second accelerator, B h is the second bandwidth of the second accelerator, R g is the size of the node data cache in the first accelerator, B g is the first bandwidth of the first accelerator, S is the data size of the node data, The reduction in the read time of the node data cached in the first accelerator relative to the data cached in the second accelerator.

[0039] In some embodiments, the target hotness and coldness information of the node data provided in this embodiment is mainly calculated based on the expected value of each node being sampled. For details, please refer to Figure 2 , Figure 2 : is a flow chart of a method for calculating coldness and heatness provided by an embodiment of the present invention. Figure 2 As shown, the step of obtaining target hotness and coldness information of node data of each node in the graph data provided in this embodiment may include steps 201 to 203;

[0040] Step 201, determine the training nodes in the graph data.

[0041] The training node is preset with the first hotness and coldness information, and the remaining nodes are preset with the second hotness and coldness information. The remaining nodes are nodes other than the training node in the graph data.

[0042] In this embodiment, the hotness and coldness information provided by this embodiment is mainly determined during the sampling process. Therefore, this embodiment is mainly explained by taking the calculation of the hotness and coldness information of the node data during the sampling process as an example.

[0043] As an optional embodiment, in the sampling process, the training nodes provided in this embodiment are determined in advance or randomly. After the training nodes are determined, the heat and coldness information of the training nodes in the graph data can be set to the first heat and coldness information, for example, it can be set to 1, and then the heat and coldness information of the remaining nodes in the graph data can be set to the second heat and coldness information, for example, it can be set to 0. In this way, the target heat and coldness information of the node data of each node in the sampling process can be calculated based on the heat and coldness information pre-set for each node.

[0044] Step 202, according to the number of the training node, the neighboring nodes adjacent to the training node, and the preset number of sampling times, determine the expected value of each neighboring node to be sampled.

[0045] In this embodiment, the preset number of sampling times in the sampling process provided in this embodiment may be pre-set, such as 2 times or 3 times. Thus, according to the preset number of sampling times and the data of the neighboring nodes corresponding to the training node, the expected value of each neighboring node of the training node being sampled can be determined.

[0046] For example, when a training node has five neighboring nodes and the preset sampling times is three, it can be determined that the expected value of the neighboring nodes of the training node being sampled is 3 / 5, that is, 0.6.

[0047] Step 203, determining target coldness and heat information of node data of each node in the graph data according to the expected value, the number of sampling layers of the graph data, and the first coldness and heat information and the second coldness and heat information.

[0048] In this embodiment, the target heat information provided by this embodiment includes the third heat information of the structural data and the fourth heat information of the feature data. Specifically, the step of determining the target heat information of the node data of each node in the graph data according to the expected value, the sampling layer number of the graph data, and the first heat information and the second heat information provided by this embodiment may include: when the sampling layer number of the graph data is the first sampling layer, the first heat information of each training node and the second heat information of each neighbor node are determined as the third heat information of the structural data of each node in the graph data; when the sampling layer number of the graph data is the second sampling layer, the heat information corresponding to each node in the graph data and the sampled expected value are added to obtain the fourth heat information of the feature data of each node in the graph data, and the second sampling layer is the layer below the first sampling layer.

[0049] As an optional embodiment, in the sampling calculation process of different layers, the structural data and characteristic data of the node can be obtained. For details, see Figure 3 , Figure 3 is a schematic diagram of a method for calculating hot and cold information provided by an embodiment of the present invention. Figure 3 As shown, Figure 3 A sampling calculation process with a sampling layer number l=1 is shown. The sampling calculation process is as follows: First, in order to facilitate the use of graph information flow to assist the sampling calculation process, this embodiment needs to reversely process all edges in the graph data, and set the first hotness information H=1 of all training nodes, the second hotness information H=0 of the remaining nodes, and set the number of iterations i=0. Figure 3 As shown, the nodes with a value of 1 in the circle are training nodes, and the nodes with a value of 0 in the circle are other nodes. Afterwards, the sampling number #picks in the sampling process is determined. The sampling number #picks provided in this embodiment may be 3.

[0050] Among them, the sampling process provided by this embodiment can be probability-free sampling or probability sampling. In some embodiments, if the sampling process provided by this embodiment is probability-free sampling, the expected value of the neighbor nodes of the training node to be sampled is calculated based on each sampling edge in the graph data, msg = min (1, H * #picks / out_degree), where out_degree is the out-degree of the training node; if the sampling process provided by this embodiment is probability sampling (the sampling probability carried is pre-set), the sampling probability probs of the sampling edge of each node is normalized, and the expected value of the neighbor nodes of the training node to be sampled is calculated based on each sampling edge guard unit msg = min (1, H * #picks * probs).

[0051] Please continue to see Figure 3 , after determining the number of sampling #picks and the expected value msg, the expected value msg on all incident edges of each node can be collected and aggregated, and added to the node's own hotness information H, so as to obtain the iteration result of this layer, that is, the target hotness information of the node data, and make the number of iterations i = i + 1. Among them, if the number of iterations i is less than the number of sampling layers l, the expected value msg of each node sampled in the next sampling layer will continue to be calculated, otherwise, the calculation ends.

[0052] In order to determine the hotness and coldness information corresponding to the structural data and characteristic data of the node, as shown in FIG. 3, this embodiment determines the iterative result of the l-1th layer as the first hotness and coldness information H of the structural data. s The iterative result of the first layer is determined as the second hotness information H of the feature data f .

[0053] In this way, by adopting the method for calculating the hotness and coldness information provided by the embodiment of the present invention, the third hotness and coldness information of the structural data of each node in the graph data and the fourth hotness and coldness information of the feature data of each node in the graph data can be calculated, so that the cache priority of the structural data and the feature data of each node can be calculated according to the hotness and coldness information of the structural data and the feature data of each node, so as to facilitate the subsequent caching of the structural data and the feature data into the accelerator at the same time.

[0054] In some embodiments, the target cache priority provided by this embodiment includes a first cache priority for structural data and a second cache priority for feature data. Specifically, the step of determining the target cache priority of node data of each node in the graph data according to the first bandwidth, the first storage capacity, the second bandwidth, the second storage capacity, and the target heat information and the node data volume of the node data of each node provided by this embodiment may include: determining the first cache priority of the structural data of each node according to the first bandwidth, the first storage capacity, the second bandwidth, the second storage capacity, and the third heat information and the first data volume of the structural data of each node; determining the second cache priority of the feature data of each node according to the first bandwidth, the first storage capacity, the second bandwidth, the second storage capacity, and the fourth heat information and the second data volume of the feature data of each node.

[0055] The priority calculation formula provided in the above embodiment may be directly used to calculate the cache priority corresponding to the structural data and feature data of each node.

[0056] Step 103, according to the target cache priority, cache the feature data and the structure data in the first accelerator in sequence to perform accelerated training of the graph neural network model in the first accelerator.

[0057] In this embodiment, after obtaining the cache priority corresponding to the structural data and feature data of each node, this embodiment sequentially caches the feature data and structural data into the first accelerator according to the cache priority corresponding to the structural data and feature data of each node, so as to perform accelerated training of the graph neural network model in the first accelerator. Specifically, the step of sequentially caching the feature data and structural data into the first accelerator according to the target cache priority provided in this embodiment to perform accelerated training of the graph neural network model in the first accelerator may include: sorting all structural data and feature data in order of priority according to the first cache priority of the structural data of each node in the graph data and the second cache priority of the feature data; starting from the node data with the highest priority, sequentially caching each structural data and feature data into the first accelerator to perform accelerated training of the graph neural network model in the first accelerator.

[0058] Among them, this embodiment can cache all structural data and feature data into the GPU in sequence according to the priority sorting order, or can cache the first N data in the sorting order into the GPU in sequence. The specific amount of cached data can be set according to the cache attribute information of the GPU or according to the actual application requirements, and is not specifically limited here.

[0059] In some embodiments, when caching structural data and feature data into a GPU, there may be multiple GPUs, so there are two situations: GPUs with or without NVLink. Specifically, when multiple GPUs are without NVLink, each GPU is independent of each other and performs static cache processing. At this time, the training nodes of the graph data can be evenly and randomly assigned to each GPU, and the cache priority calculation of the structural data and feature data is performed on each GPU. Then, according to the cache priority and combined with the storage capacity of each GPU, the structural data and feature data that need to be cached are selected, and then the feature extraction process in the training process of the graph neural network model is accelerated on each GPU, thereby improving the training performance of the model.

[0060] When multiple PGUs have NVLink, cache data can be shared between GPUs through NVLink. At this time, as with no NVLink, the training nodes of the graph data can be evenly and randomly assigned to each GPU, and the cache priority calculation of the structural data and feature data is performed on each GPU. Then the cache priorities calculated by all GPUs are collected, and according to the cache priority sorting, multiple GPUs are treated as a unified GPU, and the structural data and feature data that need to be cached are selected in combination with the storage space of all GPUs. After selecting the data that needs to be cached, the data that needs to be cached is randomly and evenly assigned to each GPU for caching, and the feature extraction process during the training of the graph neural network model is accelerated on each GPU, thereby improving the training performance of the model.

[0061] At this point, the simultaneous caching process of the structural data and feature data of each node in the graph data is completed. Compared with the solution of only completely caching the structural data through the GPU in the related art, this embodiment can cache the feature data and structural data of the graph in the GPU at the same time when the graph scale is large and the GPU memory is limited, thereby accelerating the feature extraction process in the training process of the graph neural network model and improving the training performance of the model.

[0062] In order to illustrate that the data processing method provided in this embodiment can effectively accelerate the feature extraction process in the training process of the graph neural network model and improve the training performance of the model, this embodiment provides a specific comparative experiment:

[0063] Experiment 1

[0064] Experiment configuration information:

[0065] Hardware device: 4*Tesla V100 16GB;

[0066] Dataset: ogbn-papers100M (18.8GB structure [with probs], 52.8GB feature);

[0067] Model Model: GraphSAGE with hidden_dim=256;

[0068] Task distribution Fan_out: [10,10,10];

[0069] Batch size: 5000

[0070] Training nodes: Based on the original setting of ogbn-papers100M, 10% of the nodes in the entire graph are randomly selected.

[0071] Experimental comparison objects:

[0072] SOTA: caches according to the situation without NVLink provided in the above embodiment, but only caches feature data;

[0073] Embodiment of the present invention: The feature data and structure data are cached simultaneously in the case of no NVLink provided in the above embodiment.

[0074] The results are shown in the following table:

[0075]

[0076] According to the results shown in the above table, compared with SOTA, the embodiment of the present invention slightly sacrifices a certain amount of feature data loading acceleration, but achieves a significant sampling acceleration, thereby improving the overall performance of the graph neural network model training process.

[0077] Experiment 2

[0078] Experiment configuration information: Cache setting of each trainer

[0079] Hardware device: 4*Tesla V100 16GB with NVLink connection (150GB / sec bandwidth);

[0080] Dataset: ogbn-papers100M (18.8GB structure [with probs], 52.8GB feature);

[0081] Model Model: GraphSAGE with hidden_dim=256;

[0082] Task distribution Fan_out: [10,10,10];

[0083] Batch size: 5000

[0084] Training nodes: Based on the original setting of ogbn-papers100M, 10% of the nodes in the entire graph are randomly selected.

[0085] Experimental comparison objects:

[0086] SOTA: caches according to the situation with NVLink provided in the above embodiment, but only caches feature data;

[0087] Embodiment of the present invention: According to the above embodiment, when there is NVLink, feature data and structure data are cached simultaneously.

[0088] The results are shown in the following table:

[0089]

[0090] According to the results shown in the above table, compared with SOTA, the embodiments of the present invention can effectively improve the overall performance of the graph neural network model training process.

[0091] According to the two comparative experiments provided in the above embodiment, it can be determined that the solution of caching feature data and structure data simultaneously provided in this embodiment can achieve a substantial performance improvement compared with the SOTA solution of only caching feature data.

[0092] In summary, an embodiment of the present invention provides a data processing method, which includes obtaining graph data to be processed, determining the target cache priority of node data of each node in the graph data according to cache attribute information of a preset accelerator set, the node data including feature data and structure data, the accelerator set including a first accelerator and a second accelerator, the first bandwidth of the first accelerator being greater than the second bandwidth of the second accelerator, and caching the feature data and the structure data in the first accelerator in sequence according to the target cache priority, so as to perform accelerated training of the graph neural network model in the first accelerator. By adopting an embodiment of the present invention, the feature data and the structure data can be cached in the same accelerator at the same time, so that the training process of the graph neural network model can be accelerated in the same accelerator.

[0093] According to the method described in the above embodiments, this embodiment will be further described from the perspective of a data processing device. The data processing device can be implemented as an independent entity or integrated into an electronic device, such as a terminal. The terminal can include a mobile phone, a tablet computer, etc.

[0094] See also Figure 4 , Figure 4 is a structural diagram of a data processing device provided by an embodiment of the present invention, such as Figure 4 As shown, the data processing device 300 provided by the embodiment of the present invention includes: a first acquisition module 301, a determination module 302 and a cache module 303;

[0095] The first acquisition module 301 is used to acquire graph data to be processed.

[0096] The determination module 302 is used to determine the target cache priority of the node data of each node in the graph data according to the cache attribute information of the preset accelerator set, the node data includes feature data and structure data, the accelerator set includes a first accelerator and a second accelerator, and the first bandwidth of the first accelerator is greater than the second bandwidth of the second accelerator.

[0097] The cache module 303 is used to cache the feature data and the structure data in the first accelerator in sequence according to the target cache priority, so as to perform accelerated training of the graph neural network model in the first accelerator.

[0098] In some embodiments, see Figure 5 , Figure 5 is another structural diagram of a data processing device provided by an embodiment of the present invention, such as Figure 5 As shown, the data processing device 300 provided by the embodiment of the present invention further includes: a second acquisition module 304 and a third acquisition module 305;

[0099] The second acquisition module 304 is used to acquire target hotness and coldness information of node data of each node in the graph data.

[0100] The third acquisition module 305 is used to acquire a first bandwidth and a first storage capacity of a first accelerator in a preset accelerator set, and a second bandwidth and a second storage capacity of a second accelerator in the accelerator set.

[0101] In this embodiment, the determination module 302 provided in this embodiment is specifically used to determine the target cache priority of the node data of each node in the graph data based on the first bandwidth, the first storage capacity, the second bandwidth, the second storage capacity, and the target hotness information and node data volume of the node data of each node.

[0102] In some embodiments, the second acquisition module 304 provided in this embodiment is specifically used to: determine the training nodes in the graph data, the training nodes are preset with the first hot and cold information, and the remaining nodes are preset with the second hot and cold information, and the remaining nodes are nodes in the graph data other than the training nodes; determine the expected value of each neighbor node being sampled according to the number of training nodes, neighbor nodes adjacent to the training nodes, and the preset number of sampling times; determine the target hot and cold information of the node data of each node in the graph data according to the expected value, the number of sampling layers of the graph data, and the first hot and cold information and the second hot and cold information.

[0103] In some embodiments, the target heat information includes the third heat information of the structural data and the fourth heat information of the feature data. The second acquisition module 304 provided in this embodiment is also specifically used for: when the sampling layer number of the graph data is the first sampling layer, the first heat information of each training node and the second heat information of each neighbor node are determined as the third heat information of the structural data of each node in the graph data; when the sampling layer number of the graph data is the second sampling layer, the heat information corresponding to each node in the graph data and the sampled expected value are added to obtain the fourth heat information of the feature data of each node in the graph data, and the second sampling layer is the layer below the first sampling layer.

[0104] In some embodiments, the target cache priority includes a first cache priority for the structural data and a second cache priority for the characteristic data. The determination module 302 provided in this embodiment is further specifically used to: determine the first cache priority for the structural data of each node based on the first bandwidth, the first storage capacity, the second bandwidth, the second storage capacity, and the third hotness information and the first data volume of the structural data of each node; determine the second cache priority for the characteristic data of each node based on the first bandwidth, the first storage capacity, the second bandwidth, the second storage capacity, and the fourth hotness information and the second data volume of the characteristic data of each node.

[0105] In some embodiments, the cache module 303 provided in this embodiment is specifically used to: sort all the structural data and feature data in order of priority according to the first cache priority of the structural data of each node in the graph data, and the second cache priority of the feature data; start from the node data with the highest priority, cache each structural data and feature data in the first accelerator in turn, so as to perform accelerated training of the graph neural network model in the first accelerator.

[0106] During specific implementation, the above modules and / or units can be implemented as independent entities, or can be arbitrarily combined to be implemented as the same or several entities. The specific implementation of the above modules and / or units can refer to the previous method embodiments. The specific beneficial effects that can be achieved can also refer to the beneficial effects in the previous method embodiments, which will not be repeated here.

[0107] Also, see Figure 6 , Figure 6 1 is a schematic diagram of a structure of an electronic device provided by an embodiment of the present invention, and the electronic device may be a mobile terminal such as a smart phone, a tablet computer, or the like. Figure 6 As shown, the electronic device 400 includes a processor 401 and a memory 402. The processor 401 is electrically connected to the memory 402.

[0108] The processor 401 is the control center of the electronic device 400. It uses various interfaces and lines to connect various parts of the entire electronic device. It executes various functions of the electronic device 400 and processes data by running or loading applications stored in the memory 402 and calling data stored in the memory 402, thereby monitoring the electronic device 400 as a whole.

[0109] In this embodiment, the processor 401 in the electronic device 400 will load the instructions corresponding to the processes of one or more applications into the memory 402 according to the following steps, and the processor 401 will run the application stored in the memory 402, thereby implementing any step in the data processing method provided in the above embodiment.

[0110] The electronic device 400 can implement the steps in any embodiment of the data processing method provided in the embodiments of the present invention, and therefore can achieve the beneficial effects that can be achieved by any data processing method provided in the embodiments of the present invention. Please refer to the previous embodiments for details and will not be repeated here.

[0111] See also Figure 7 , Figure 7 is another structural schematic diagram of an electronic device provided by an embodiment of the present invention, such as Figure 7 As shown, Figure 7 The specific structural block diagram of the electronic device provided by the embodiment of the present invention is shown, and the electronic device can be used to implement the data processing method provided in the above embodiment. The electronic device 500 can be a mobile terminal such as a smart phone or a laptop computer.

[0112] The RF circuit 510 is used to receive and send electromagnetic waves, realize the mutual conversion between electromagnetic waves and electrical signals, and thus communicate with a communication network or other devices. The RF circuit 510 may include various existing circuit elements for performing these functions, such as antennas, radio frequency transceivers, digital signal processors, encryption / decryption chips, user identity module (SIM) cards, memories, etc. The RF circuit 510 can communicate with various networks such as the Internet, corporate intranets, wireless networks, or communicate with other devices through wireless networks. The above-mentioned wireless networks may include cellular telephone networks, wireless local area networks, or metropolitan area networks. The above-mentioned wireless networks can use various communication standards, protocols and technologies, including but not limited to Global System for Mobile Communication (GSM), Enhanced Data GSM Environment (EDGE), Wideband Code Division Multiple Access (WCDMA), Code Division Multiple Access (CDMA), Time Division Multiple Access (TDMA), Wireless Fidelity (Wi-Fi) (such as Institute of Electrical and Electronics Engineers standards IEEE 802.11a, IEEE 802.11b, IEEE802.11g and / or IEEE802.11n), Voice over Internet Protocol (VoIP), Worldwide Interoperability for Microwave Access (Wi-Max), other protocols for email, instant messaging and short messages, and any other suitable communication protocols, even those that have not yet been developed.

[0113] The memory 520 may be used to store software programs and modules, such as program instructions / modules corresponding to the data processing method in the above-mentioned embodiment. The processor 580 executes various functional applications and processes data by running the software programs and modules stored in the memory 520.

[0114] The memory 520 may include a high-speed random access memory, and may also include a non-volatile memory, such as one or more magnetic storage devices, flash memory, or other non-volatile solid-state memory. In some examples, the memory 520 may further include a memory remotely arranged relative to the processor 580, and these remote memories may be connected to the electronic device 500 via a network. Examples of the above-mentioned network include, but are not limited to, the Internet, an intranet, a local area network, a mobile communication network, and combinations thereof.

[0115] The input unit 530 can be used to receive input digital or character information, and generate keyboard, mouse, joystick, optical or trackball signal input related to user settings and function control. Specifically, the input unit 530 may include a touch-sensitive surface 531 and other input devices 532. The touch-sensitive surface 531, also known as a touch display screen or a touch pad, can collect user touch operations on or near it (such as operations performed by users using fingers, styluses, or any other suitable objects or accessories on or near the touch-sensitive surface 531), and drive corresponding connection devices according to a pre-set program. Optionally, the touch-sensitive surface 531 may include a touch detection device and a touch controller. Among them, the touch detection device detects the user's touch orientation, detects the signal brought by the touch operation, and transmits the signal to the touch controller; the touch controller receives the touch information from the touch detection device, converts it into touch point coordinates, and then sends it to the processor 580, and can receive and execute commands sent by the processor 580. In addition, the touch-sensitive surface 531 can be implemented in various types such as resistive, capacitive, infrared, and surface acoustic wave. In addition to the touch-sensitive surface 531, the input unit 530 may further include other input devices 532. Specifically, the other input devices 532 may include, but are not limited to, one or more of a physical keyboard, function keys (such as volume control keys, switch keys, etc.), a trackball, a mouse, a joystick, and the like.

[0116] The display unit 540 can be used to display information input by the user or information provided to the user and various graphical user interfaces of the electronic device 500, which can be composed of graphics, text, icons, videos and any combination thereof. The display unit 540 may include a display panel 541, and optionally, the display panel 541 may be configured in the form of LCD (Liquid Crystal Display), OLED (Organic Light-Emitting Diode), etc. Further, the touch-sensitive surface 531 may cover the display panel 541, and when the touch-sensitive surface 531 detects a touch operation on or near it, it is transmitted to the processor 580 to determine the type of the touch event, and then the processor 580 provides a corresponding visual output on the display panel 541 according to the type of the touch event. Although in the figure, the touch-sensitive surface 531 and the display panel 541 are implemented as two independent components to implement input and output functions, in some embodiments, the touch-sensitive surface 531 and the display panel 541 can be integrated to implement input and output functions.

[0117] The electronic device 500 may also include at least one sensor 550, such as a light sensor, a motion sensor, and other sensors. Specifically, the light sensor may include an ambient light sensor and a proximity sensor, wherein the ambient light sensor may adjust the brightness of the display panel 541 according to the brightness of the ambient light, and the proximity sensor may generate an interrupt when the flip cover is closed or closed. As a type of motion sensor, the gravity acceleration sensor can detect the magnitude of acceleration in each direction (generally three axes), and can detect the magnitude and direction of gravity when stationary, which can be used for applications that identify the posture of the mobile phone (such as horizontal and vertical screen switching, related games, magnetometer posture calibration), vibration recognition related functions (such as pedometer, tapping), etc.; as for other sensors such as gyroscopes, barometers, hygrometers, thermometers, infrared sensors, etc. that can also be configured in the electronic device 500, they will not be repeated here.

[0118] The audio circuit 560, the speaker 561, and the microphone 562 can provide an audio interface between the user and the electronic device 500. The audio circuit 560 can transmit the electrical signal converted from the received audio data to the speaker 561, which is converted into a sound signal for output; on the other hand, the microphone 562 converts the collected sound signal into an electrical signal, which is received by the audio circuit 560 and converted into audio data, and then the audio data is output to the processor 580 for processing, and then sent to another terminal through the RF circuit 510, or the audio data is output to the memory 520 for further processing. The audio circuit 560 may also include an earplug jack to provide communication between an external headset and the electronic device 500.

[0119] The electronic device 500 can help the user receive requests, send information, etc. through the transmission module 570 (such as a Wi-Fi module), which provides the user with wireless broadband Internet access. Although the transmission module 570 is shown in the figure, it is understandable that it is not a necessary component of the electronic device 500 and can be omitted as needed without changing the essence of the invention.

[0120] The processor 580 is the control center of the electronic device 500. It uses various interfaces and lines to connect various parts of the entire mobile phone. By running or executing software programs and / or modules stored in the memory 520, and calling data stored in the memory 520, it executes various functions of the electronic device 500 and processes data, thereby monitoring the electronic device as a whole. Optionally, the processor 580 may include one or more processing cores; in some embodiments, the processor 580 may integrate an application processor and a modem processor, wherein the application processor mainly processes the operating system, user interface, and application programs, etc., and the modem processor mainly processes wireless communications. It can be understood that the above-mentioned modem processor may not be integrated into the processor 580.

[0121] The electronic device 500 also includes a power supply 590 (such as a battery) for supplying power to various components. In some embodiments, the power supply can be logically connected to the processor 580 through a power management system, so that the power management system can manage charging, discharging, and power consumption management. The power supply 590 can also include any components such as one or more DC or AC power supplies, recharging systems, power failure detection circuits, power converters or inverters, and power status indicators.

[0122] Although not shown, the electronic device 500 also includes a camera (such as a front camera, a rear camera), a Bluetooth module, etc., which will not be described in detail here. Specifically in this embodiment, the display unit of the electronic device is a touch screen display, and the mobile terminal also includes a memory, and one or more programs, wherein one or more programs are stored in the memory, and are configured to be executed by one or more processors to implement any step of the data processing method provided in the above embodiment.

[0123] In specific implementation, the above modules can be implemented as independent entities, or can be arbitrarily combined and implemented as the same or several entities. The specific implementation of the above modules can be found in the previous method embodiments, which will not be repeated here.

[0124] Those skilled in the art can understand that all or part of the steps in the various methods of the above embodiments can be completed by instructions, or by controlling related hardware through instructions, and the instructions can be stored in a computer-readable storage medium and loaded and executed by a processor. To this end, an embodiment of the present invention provides a storage medium, which stores multiple instructions, and the instructions can be executed by the processor to implement any step of the data processing method provided in the above embodiments.

[0125] The storage medium may include: a read-only memory (ROM), a random access memory (RAM), a magnetic disk or an optical disk, etc.

[0126] Since the instructions stored in the storage medium can execute the steps in any embodiment of the data processing method provided in the embodiments of the present invention, the beneficial effects that can be achieved by any data processing method provided in the embodiments of the present invention can be achieved. Please refer to the previous embodiments for details and will not be repeated here.

[0127] The above is a detailed introduction to a data processing method, device, electronic device and storage medium provided in the embodiments of the present application. Specific examples are used in this article to illustrate the principles and implementation methods of the present application. The description of the above embodiments is only used to help understand the method and its core idea of ​​the present application. At the same time, for those skilled in the art, according to the idea of ​​the present application, there will be changes in the specific implementation methods and application scope. In summary, the content of this specification should not be understood as a limitation on the present application. Moreover, for those of ordinary skill in the art, without departing from the principles of the present invention, several improvements and modifications can be made, and these improvements and modifications are also regarded as the protection scope of the present invention.

Claims

1. A data processing method, characterized in that: include: Get the graph data to be processed; Determining, according to cache attribute information of a preset accelerator set, a target cache priority of node data of each node in the graph data, wherein the node data includes feature data and structure data, the accelerator set includes a first accelerator and a second accelerator, and a first bandwidth of the first accelerator is greater than a second bandwidth of the second accelerator; According to the target cache priority, the feature data and the structure data are cached in the first accelerator in sequence to perform accelerated training of the graph neural network model in the first accelerator.

2. The method according to claim 1, characterized in that Before the step of determining the target cache priority of the node data of each node in the graph data according to the cache attribute information of the preset accelerator set, the method further includes: Obtain target hotness and coldness information of node data of each node in the graph data; Acquire a first bandwidth and a first storage capacity of a first accelerator in a preset accelerator set, and a second bandwidth and a second storage capacity of a second accelerator in the accelerator set; The step of determining the target cache priority of the node data of each node in the graph data according to the cache attribute information of the preset accelerator set includes: The target cache priority of the node data of each node in the graph data is determined according to the first bandwidth, the first storage capacity, the second bandwidth, the second storage capacity, and the target hotness information and node data volume of the node data of each node.

3. The method according to claim 2, characterized in that The step of obtaining target hotness and coldness information of node data of each node in the graph data includes: Determine a training node in the graph data, wherein the training node is preset with first hotness and coldness information, and the remaining nodes are preset with second hotness and coldness information, and the remaining nodes are nodes in the graph data other than the training node; Determine the expected value of each neighbor node being sampled according to the training node, the number of neighbor nodes adjacent to the training node, and a preset number of sampling times; Target heat information of node data of each node in the graph data is determined according to the expected value, the number of sampling layers of the graph data, and the first heat information and the second heat information.

4. The method according to claim 3, characterized in that The target temperature information includes the third temperature information of the structure data and the fourth temperature information of the feature data; The step of determining target heat and coldness information of node data of each node in the graph data according to the expected value, the number of sampling layers of the graph data, and the first heat and coldness information and the second heat and coldness information includes: When the sampling layer number of the graph data is the first sampling layer, the first hotness information of each training node and the second hotness information of each neighboring node are determined as the third hotness information of the structural data of each node in the graph data; When the sampling layer number of the graph data is the second sampling layer, the hotness and coldness information corresponding to each node in the graph data and the sampled expected value are added to obtain the fourth hotness and coldness information of the feature data of each node in the graph data, and the second sampling layer is the next layer of the first sampling layer.

5. The method according to claim 4, characterized in that The target cache priority includes a first cache priority for structure data and a second cache priority for feature data; Determining the target cache priority of the node data of each node in the graph data according to the first bandwidth, the first storage capacity, the second bandwidth, the second storage capacity, and the target hotness information and the node data volume of each node includes: Determine a first cache priority of the structure data of each of the nodes according to the first bandwidth, the first storage capacity, the second bandwidth, the second storage capacity, and the third hotness information and the first data volume of the structure data of each of the nodes; A second cache priority of the feature data of each of the nodes is determined according to the first bandwidth, the first storage capacity, the second bandwidth, the second storage capacity, the fourth hotness information and the second data volume of the feature data of each of the nodes.

6. The method according to claim 5, characterized in that The step of sequentially caching the feature data and the structure data in the first accelerator according to the target cache priority, so as to perform accelerated training of the graph neural network model in the first accelerator, includes: According to the first cache priority of the structural data of each node in the graph data and the second cache priority of the characteristic data, all the structural data and characteristic data are sorted in order of priority; Starting from the node data with the highest priority, each of the structural data and the feature data is cached in the first accelerator in turn to perform accelerated training of the graph neural network model in the first accelerator.

7. A data processing device, characterized in that: include: A first acquisition module, used to acquire graph data to be processed; a determination module, configured to determine a target cache priority of node data of each node in the graph data according to cache attribute information of a preset accelerator set, wherein the node data includes feature data and structure data, the accelerator set includes a first accelerator and a second accelerator, and a first bandwidth of the first accelerator is greater than a second bandwidth of the second accelerator; A cache module is used to cache the feature data and the structure data in the first accelerator in sequence according to the target cache priority, so as to perform accelerated training of the graph neural network model in the first accelerator.

8. The device according to claim 7, characterized in that The device also includes: A second acquisition module is used to acquire target hotness and coldness information of node data of each node in the graph data; A third acquisition module, configured to acquire a first bandwidth and a first storage capacity of a first accelerator in a preset accelerator set, and a second bandwidth and a second storage capacity of a second accelerator in the accelerator set; The determination module is specifically used to determine the target cache priority of the node data of each node in the graph data according to the first bandwidth, the first storage capacity, the second bandwidth, the second storage capacity, and the target hotness information and node data volume of the node data of each node.

9. An electronic device, characterized in that: The electronic device comprises a processor, a memory, and a computer program stored in the memory and executable on the processor, and the processor implements the steps in the method according to any one of claims 1 to 6 when executing the computer program.

10. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the steps in the method according to any one of claims 1 to 6 are implemented.

Images