Graph data processing method and apparatus, and related device

Abstract

A graph data processing method, apparatus, and related equipment relate to the field of graph computing technology. The graph data processing apparatus acquires graph data and, based on the characteristic values ​​of each node in the graph data, determines multiple partitions of the space to which the multiple nodes belong, corresponding to the space-filling curve. The apparatus then determines the arrangement order of the multiple partitions according to their positions on the space-filling curve, and sorts the multiple nodes according to this order. Based on this arrangement order, the multiple nodes are stored sequentially. Since the space-filling curve exhibits spatial clustering, determining the arrangement order of the multiple nodes based on the arrangement order of the partitions determined by the space-filling curve, and storing the nodes in this order, allows spatially close nodes to be stored in adjacent locations, thereby improving the cache hit rate when subsequently reading the graph data.

Description

Technical Field

[0001] This application relates to the field of graph computing technology, and in particular to a graph data processing method, apparatus and related equipment. Background Technology

[0002] Graph data is an abstract data structure used to represent the relationships between objects. It typically consists of multiple vertices connected by edges. Nodes represent objects, such as users or products; edges represent relationships between different objects, such as the similarity between different products. In scenarios such as search engine optimization and scientific computing, graph data can be used for computation to achieve information mining and data analysis.

[0003] In graph computation, it is often necessary to read data from multiple nodes connected by edges in the graph, such as reading information about multiple consecutive products with similar attributes. However, in real-world applications, the data of multiple nodes that need to be accessed simultaneously and are connected by edges may be stored in locations that are far apart. This can lead to low efficiency in reading the data from these multiple nodes, resulting in low overall efficiency of graph computation. Summary of the Invention

[0004] This application provides a graph data processing method to improve the efficiency of accessing data from multiple nodes connected by edges, thereby improving the overall efficiency of graph computation. Furthermore, this application also provides a corresponding graph data processing apparatus, a computer-readable storage medium, and a computer program product.

[0005] Firstly, this application provides a graph data processing method, which can be executed by a corresponding graph data processing device. Specifically, the graph data processing device acquires graph data, which includes multiple nodes. In practical applications, these multiple nodes can be connected by edges. Then, based on the feature values ​​of each node (e.g., coordinate values ​​of the node in multiple dimensions), the graph data processing device determines multiple partitions of the space to which the multiple nodes belong, where each partition can be a gridded unit in a multi-dimensional space, and each of the multiple nodes belongs to one of these partitions. Next, the graph data processing device determines the arrangement order of the multiple partitions according to their positions in the space filling curve, and sorts the multiple nodes according to the arrangement order of the multiple partitions to obtain the arrangement order of the multiple nodes. For example, if the sorting position of partition A is before the sorting position of partition B, then the sorting position of at least one node belonging to partition A will be at the sorting position of each node belonging to partition B. Finally, the graph data processing device sequentially stores the multiple nodes according to their arrangement order.

[0006] Since the storage of graph data, which includes multiple nodes, involves sorting the partitions to which the nodes belong using space-fill curves and storing the nodes sequentially according to their arrangement, and given that space-fill curves exhibit spatial clustering, determining the node arrangement order based on the partition order defined by the space-fill curves and storing the nodes according to this order ensures that spatially close nodes are stored in adjacent locations. This allows for the subsequent prefetching of data from multiple nodes connected by edges in the graph data, improving cache hit rate and thus increasing the efficiency of reading the data from these nodes, thereby enhancing the overall efficiency of graph computation.

[0007] In one possible implementation, when the graph data processing device determines multiple partitions of the space to which the space-filling curve belongs based on the feature values ​​of each of the multiple nodes, it may first determine multiple sets of candidate parameters based on the feature values ​​of each node. Each set of candidate parameters is used to determine a partitioning result, and each partitioning result can be used to indicate multiple partitions of the space to which the space-filling curve belongs. Each set of candidate parameters includes a partition size or a zero-point coordinate, wherein the zero-point coordinate is used to calculate the distance between the coordinates of each node in each dimension and the zero-point coordinate. Then, the graph data processing device optimizes the multiple sets of candidate parameters based on the feature values ​​of each node to obtain a target candidate parameter among the multiple sets of candidate parameters, and determines the partitioning result corresponding to the target candidate parameter. Thus, by optimizing multiple sets of candidate parameters, the best candidate parameters that can improve the effect of storing spatially close nodes in mutually adjacent positions can be determined from multiple sets of candidate parameters. Accordingly, in the subsequent process of reading the data of multiple nodes connected by edges in the graph data, the data of multiple spatially close nodes are more likely to be prefetched into the cache together, which can further improve the cache hit rate, that is, further improve the efficiency of reading the data of these multiple nodes, thereby further improving the overall efficiency of graph computation.

[0008] In one possible implementation, each set of candidate parameters includes the partition size and the zero-point coordinates. When the graph data processing device determines multiple sets of candidate parameters based on the feature values ​​of each node, it can specifically determine the range of values ​​for the partition size based on the distances between multiple pairs of nodes with connected edges. The distances between nodes can be calculated based on the feature values ​​of the nodes. Then, the graph data processing device determines multiple possible values ​​for the partition size within this range. Thus, the graph data processing device can calculate multiple sets of zero-point coordinate values ​​based on the multiple partition size values ​​and the feature values ​​of each node. In this way, for each piece of graph data, the graph data processing device can generate multiple sets of candidate parameters that are suitable for the graph data, so that the optimal target candidate parameters can be determined from these multiple sets of candidate parameters, thereby improving the effect of storing spatially close nodes in mutually adjacent locations.

[0009] In one possible implementation, before determining the range of partition sizes, the graph data processing device can further determine multiple pairs of nodes with connected edges based on the edge information included in the graph data. The edge information indicates the connection relationships between the multiple nodes. Alternatively, the graph data processing device can determine multiple pairs of nodes with a distance less than the truncation radius that are connected by edges. Thus, the graph data processing device can determine the connection relationships between different nodes based on the edge information carried in the graph data or based on the truncation radius, so that spatially proximate nodes can be stored in adjacent locations as much as possible based on the connection relationships between different nodes.

[0010] In one possible implementation, when the graph data processing device optimizes multiple sets of candidate parameters based on the feature values ​​of each node, it can specifically calculate the node distance corresponding to each set of candidate parameters based on the feature values ​​of each node. The linear distance includes the distance between multiple pairs of nodes with connected edges after sorting. Therefore, the graph data processing device can determine the target candidate parameter from the multiple sets of candidate parameters based on the linear distance corresponding to each set of candidate parameters, where the linear distance corresponding to the target candidate parameter is the smallest. Since the distance between multiple pairs of nodes with connected edges is the smallest after sorting, the effect of storing spatially close nodes in adjacent positions is better. Therefore, determining the order of storing multiple nodes based on the target candidate parameter based on the linear distance corresponding to each set of candidate parameters can improve the effect of storing spatially close nodes in adjacent positions, thereby helping to improve the cache hit rate during subsequent graph data reading.

[0011] In one possible implementation, the graph data processing device can also determine the order of the space-filling curve based on the partition size in the target candidate parameters. Thus, the graph data processing device determines the arrangement order of multiple partitions based on the space-filling curve adapted to the target candidate parameters, which can help improve the effect of storing spatially close nodes in adjacent locations as much as possible.

[0012] In one possible implementation, when the graph data processing device determines multiple partitions of the space to which the space-filling curves of multiple nodes belong based on the feature values ​​of each node, it can specifically reduce the dimensionality of the initial features of each node to obtain the target features of each node, where the dimensionality of the initial features is higher than that of the target features. Then, the graph data processing device determines multiple partitions of the space to which the space-filling curves of multiple nodes belong based on the values ​​of the target features of each node. In this way, by reducing the dimensionality of node features and determining the arrangement order of multiple nodes based on the reduced node feature values, the graph data processing device can help reduce the computational load of determining the arrangement order of multiple nodes, thereby helping to reduce the consumption of computing resources.

[0013] Secondly, this application provides a graph data processing apparatus, comprising: an acquisition module for acquiring graph data, the graph data including multiple nodes; a determination module for determining multiple partitions of the space to which the multiple nodes belong, based on the feature values ​​of each node among the multiple nodes, each node among the multiple nodes belonging to one of the multiple partitions; a sorting module for determining the arrangement order of the multiple partitions according to their positions in the space filling curve; and a storage module for sorting the multiple nodes according to the arrangement order of the multiple partitions to obtain the arrangement order of the multiple nodes; and a storage module for sequentially storing the multiple nodes according to the arrangement order of the multiple nodes.

[0014] In one possible implementation, when determining multiple partitions of the space to which the space-filling curve belongs, based on the feature values ​​of each of the multiple nodes, the determining module is configured to: determine multiple sets of candidate parameters based on the feature values ​​of each node, each set of candidate parameters being used to determine a partitioning result, the partitioning result being used to indicate multiple partitions of the space to which the space-filling curve belongs, each set of candidate parameters including partition size or zero-point coordinates, the zero-point coordinates being used to calculate the distance between the coordinates of each node in each dimension and the zero-point coordinates; optimize the multiple sets of candidate parameters based on the feature values ​​of each node to obtain target candidate parameters among the multiple sets of candidate parameters; and determine the partitioning result corresponding to the target candidate parameters.

[0015] In one possible implementation, each set of candidate parameters includes the partition size and the zero point coordinates; when determining multiple sets of candidate parameters based on the feature values ​​of each node, the determining module is used to: determine the range of values ​​for the partition size based on the distance between multiple pairs of nodes with connected edges; determine multiple values ​​for the partition size within the range of values ​​for the partition size; and calculate multiple sets of values ​​for the zero point coordinates based on the multiple values ​​for the partition size and the feature values ​​of each node.

[0016] In one possible implementation, the determining module is further configured to: before determining the range of values ​​for the partition size, determine multiple pairs of nodes with connected edges among the multiple nodes based on the edge information included in the graph data, wherein the edge information is used to indicate the connection relationship between the multiple nodes; or, before determining the range of values ​​for the partition size, determine multiple pairs of nodes with a distance less than the cutoff radius among the multiple nodes with connected edges.

[0017] In one possible implementation, when the determining module optimizes multiple sets of candidate parameters based on the feature values ​​of each node, it is used to: calculate the node distance corresponding to each set of candidate parameters based on the feature values ​​of each node, wherein the linear distance includes the distance between multiple pairs of nodes with connected edges in the sorted sequence; and determine the target candidate parameter from the multiple sets of candidate parameters based on the linear distance corresponding to each set of candidate parameters, wherein the linear distance corresponding to the target candidate parameter is the smallest.

[0018] In one possible implementation, the determining module is further configured to: determine the order of the space-filling curve based on the partition size in the target candidate parameters.

[0019] In one possible implementation, when determining multiple partitions of the space to which the space-filling curve belongs based on the feature values ​​of each of the multiple nodes, the determining module is configured to: reduce the dimensionality of the initial features of each of the multiple nodes to obtain the target features of each node, wherein the number of dimensions of the initial features is higher than the number of dimensions of the target features; and determine multiple partitions of the space to which the space-filling curve belongs based on the value of the target features of each node.

[0020] Thirdly, this application provides a computing device, which includes a processor and a memory. The processor and the memory communicate with each other. The processor executes instructions stored in the memory to cause the computing device to perform a graph data processing method as described in the first aspect or any implementation thereof. It should be noted that the memory can be integrated into the processor or can be independent of the processor. The computing device may also include a bus. The processor is connected to the memory via the bus. The memory may include readable storage and random access memory.

[0021] Fourthly, this application provides a computer-readable storage medium storing instructions that, when executed on a computing device, cause the computing device to perform the operational steps of the graph data processing method described in the first aspect or any implementation thereof.

[0022] Fifthly, this application provides a computer program product containing instructions that, when run on a computing device, causes the computing device to perform the operational steps of the graph data processing method described in the first aspect or any implementation thereof.

[0023] Based on the implementation methods provided in the above aspects, this application can be further combined to provide more implementation methods. Attached Figure Description

[0024] Figure 1 A schematic diagram of the structure of an exemplary data processing system provided in this application;

[0025] Figure 2 A flowchart illustrating a graph data processing method provided in this application;

[0026] Figure 3 This diagram illustrates how different zero-point coordinates can cause nodes to belong to different partitions.

[0027] Figure 4 A flowchart illustrating another graph data processing method provided in this application;

[0028] Figure 5 This application provides a schematic diagram of the structure of a graph data processing device;

[0029] Figure 6 This is a schematic diagram of the hardware structure of a computing device provided in this application. Detailed Implementation

[0030] The terms "first," "second," etc., used in the specification, claims, and accompanying drawings of this application are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that such terms can be used interchangeably where appropriate; this is merely a method of distinction used in describing objects with the same attributes in the embodiments of this application.

[0031] The technical solutions in this application will now be described with reference to the accompanying drawings.

[0032] See Figure 1 This is a schematic diagram of the structure of an exemplary data processing system 10. Figure 1 As shown, the data processing system 10 includes at least one processor and at least one memory. For ease of explanation, Figure 1 The following is an example of a data processing system including a processor 100 and a memory 200.

[0033] The processor 100 can be any type of processor with graph computing capabilities, such as a central processing unit (CPU), or other types of processors, such as accelerators, application-specific integrated circuits (ASICs), programmable logic devices (PLDs), complex programmable logical devices (CPLDs), field-programmable gate arrays (FPGAs), generic array logic (GALs), systems-on-chips (SoCs), software-defined infrastructure (SDI) chips, or any combination thereof. The accelerator can be a neural network processing unit (NPU), a graphics processing unit (GPU), a tensor processing unit (TPU), a deep-learning processing unit (DPU), or a data processing unit (DPU), etc.

[0034] Furthermore, the processor 100 may also include a cache, such as... Figure 1 The cache 101 is used to cache data that the processor 100 needs to read. For example, some data in the memory 200 can be prefetched into the cache 101, so that when the processor 100 needs to read the data, it can directly read the data from the cache 101, thereby improving the efficiency of the processor 100 in obtaining the data.

[0035] The memory 200 can be any type of memory with data storage capability, such as solid-state disk (SSD), hard disk drive (HDD), or other types of memory, without limitation.

[0036] For example, the processor 100 can communicate with the memory via a bus, which may be, for example, a peripheral component interconnect express (PCIe) bus, a non-volatile memory express (NVMe) bus, a compute expresslink (CXL) bus, a Huawei Collective Communication Server (HCCS) bus, a unified bus (UB or Ubus), or other types of buses.

[0037] In practical applications, processor 100 can be used to perform graph computation or neural network computation. During this process, processor 100 (or other devices) can predict the node data required for performing graph computation or neural network computation and prefetch this node data from memory 200 into cache 101. Thus, when processor 100 needs to read the node data later, it can directly read the node data from cache 101, which can effectively improve the efficiency of processor 100 in reading node data, thereby improving the efficiency of processor 100 in performing graph computation or neural network computation.

[0038] However, the node data that processor 100 needs to read is typically data from multiple nodes connected by edges. This data is usually stored based on node identifiers or creation order, which means the data from these nodes may be stored in geographically dispersed locations, such as different blocks or even different storage media within memory 200. Therefore, the data prefetched into cache 101 is often not the data from these interconnected nodes. In this case, processor 100 needs to read the actual data it needs from memory 200. This not only reduces the cache hit rate but also results in a longer time frame for reading data from memory 200. Consequently, the overall time spent reading data from these multiple nodes is longer, leading to lower overall efficiency for processor 100 in performing graph or neural network computations.

[0039] Based on this, Figure 1 The data processing system 10 shown may also include a graph data processing device 300, which improves the hit rate of the cache 101 by storing the data of multiple nodes that are connected by edges in close proximity as much as possible.

[0040] In practice, the graph data processing device 300 acquires graph data, which includes multiple nodes. In real-world applications, some or all of these nodes may be connected by edges. Then, based on the feature values ​​of each node, the graph data processing device 300 determines multiple partitions of the space to which the multiple nodes belong, where each node belongs to one of these partitions. Next, the graph data processing device 300 determines the arrangement order of the multiple partitions according to their positions on the space-filling curve, and sorts the multiple nodes according to this arrangement order. The graph data processing device then sequentially stores the multiple nodes according to this arrangement order.

[0041] Thus, in the process of storing graph data including multiple nodes, the graph data processing device 300 uses the space-fill curve to sort the partitions to which the multiple nodes belong, and stores the data of the multiple nodes according to the arrangement order of the multiple nodes. Since the space-fill curve has spatial clustering, determining the arrangement order of multiple nodes based on the arrangement order of the partitions determined by the space-fill curve, and further storing the data of multiple nodes sequentially, allows spatially close nodes (usually nodes connected by edges) to be stored in adjacent positions as much as possible. In this way, when reading the data of multiple nodes connected by edges in the graph data, the data of these multiple nodes can be prefetched together into the cache 101, thereby improving the hit rate of the cache 101, that is, improving the efficiency of the processor 100 in reading the data of these multiple nodes, thereby improving the overall efficiency of graph computation.

[0042] For example, the graph data processing device 300 can be implemented by software or hardware.

[0043] When the graph data processing device 300 is implemented in software, it can be an application running on hardware, such as a process or software running on the processor 100 or other hardware. When implemented in hardware, the graph data processing device 300 can be a processor or a computing device including a processor, such as a server; or, it can be a device implemented using an application-specific integrated circuit (ASIC) or a programmable logic device (PLD). The PLD can be a complex programmable logical device (CPLD), a field-programmable gate array (FPGA), generic array logic (GAL), or any combination thereof. In this case, as... Figure 1 As shown, the graph data processing device 300 can be connected to the bus memory 200 to sequentially store data from multiple nodes into the memory 200. Optionally, the graph data processing device 300 can also be connected to the processor 100 via a bus.

[0044] It is worth noting that the above Figure 1 The data processing system 10 shown is merely illustrative and not intended to be limiting. For example, in other possible embodiments, the data processing system may include a greater number or more types of processors and memory, and the types of processors and memory included in the data processing system may be the same or different. Furthermore, in other possible embodiments, the data processing system may also include more types of devices, such as devices for managing the processors or memory. Moreover, in other possible embodiments, the graph data processing device 300 may not be connected to the processor 100, or the graph data processing device 300 may be deployed as software to the processor 100 or the memory 200.

[0045] For ease of understanding, embodiments of the graph data processing method provided in this application will be described below with reference to the accompanying drawings.

[0046] See Figure 2 , Figure 2 This is a flowchart illustrating a graph data processing method provided in an embodiment of this application. This method can be applied to... Figure 1 The data processing system 10 described above can also be applied to other applicable data processing systems. For ease of explanation, this embodiment uses an application... Figure 1The data processing system 10 shown is used as an example for illustration.

[0047] in, Figure 2 The graph data processing method shown may specifically include the following steps.

[0048] S201: The graph data processing device 300 acquires graph data, which includes multiple nodes.

[0049] The acquired graph data includes not only information about multiple nodes (such as node coordinates) but also edge information, which indicates the connection relationships between multiple nodes. That is, the graph data processing device 300 can determine, based on this edge information, which nodes in the graph data are connected by edges and which are not.

[0050] Alternatively, the graph data may not include edge information. In this case, the graph data processing device 300 can determine the connection relationship between the multiple nodes based on the cutoff radius (or distance cutoff radius). For example, for any two nodes in the graph data, the graph data processing device 300 can calculate the distance between the two nodes and compare the distance with the cutoff radius. When the distance is less than or equal to the cutoff radius, the graph data processing device 300 can determine that there is an edge connecting the two nodes; while when the distance is greater than the cutoff radius, the graph data processing device 300 can determine that there is no edge connecting the two nodes.

[0051] Furthermore, each node in graph data can have a feature value, which can be the numerical value of the node across multiple dimensions, or a value in other types of representation. For example, when a node in graph data represents a product, the feature value of the node can specifically be the coordinate value of the product represented by the node across multiple dimensions.

[0052] S202: The graph data processing device 300 determines multiple partitions of the space to which the space-filling curve belongs based on the feature value of each of the multiple nodes, and each of the multiple nodes belongs to one of the multiple partitions.

[0053] In this context, a partition refers to a gridded unit in the multidimensional space corresponding to the space-filling curve. When the coordinates of a point in the multidimensional space fall within the coordinate range corresponding to one of the partitions, the graph data processing device 300 can determine that the point belongs to that partition.

[0054] Based on this, after determining the feature values ​​of each node, the graph data processing device 300 can determine the partition of the space to which the node belongs in the multidimensional space based on the feature values ​​of each node. The size of each partition in the multidimensional space can be preset or determined through optimization. For ease of understanding, this embodiment uses the example of a known partition size in the multidimensional space.

[0055] In practical applications, each node in graph data contains a large number of numerical dimensions, such as a single node having 128 feature values. Therefore, in one possible implementation, the graph data processing device 300 can reduce the dimensionality of the node features and determine the partition of the space to which the space-filling curve belongs based on the reduced node features. For ease of distinction, in this embodiment, the node features before dimensionality reduction are referred to as initial features (with a relatively large number of feature dimensions), and the node features after dimensionality reduction are referred to as target features (with a relatively small number of feature dimensions).

[0056] In practice, when determining the partition of the space to which each node belongs in the space-filling curve, the graph data processing device 300 can first reduce the dimensionality of the initial features of each node to obtain the target features of that node. The initial features have a higher dimensionality than the target features. For example, the graph data processing device 300 can transform the 128-dimensional initial features of each node into 3-dimensional target features. Then, based on the values ​​of the target features of each node, the graph data processing device 300 can determine the partition of the space to which each node belongs in the space-filling curve.

[0057] This embodiment provides the following two implementation examples for reducing the dimensionality of node features.

[0058] As a first implementation example, the graph data processing device 300 can be configured with a trained neural network, such as a graph neural network (GNN) or a multilayer perceptron (MLP). The graph data processing device 300 can then input the initial features of each node into the neural network to obtain the target features of that node output by the neural network, thereby reducing the dimensionality of the node features. The neural network can be pre-trained using at least one set of training samples; the training process will not be elaborated further. Furthermore, when the graph data includes edge information, the graph data processing device 300 can input the connection relationships between different nodes into the neural network based on this edge information; when the graph data does not include edge information, the graph data processing device 300 can determine whether there are edge connections between different nodes based on the truncation radius, thereby obtaining the connection relationships between different nodes and inputting these connection relationships into the neural network.

[0059] As a second implementation example, the graph data processing device 300 can use a graph embedding method to learn the target features of each node.

[0060] In a specific implementation, the graph data processing device 300 may be pre-configured with a neural network, and the neural network may be untrained. Then, the graph data processing device 300 may input the initial features of each node into the neural network, and the neural network may optimize the target features of each node based on the connection relationship between different nodes and the following formula (1).

[0061]

[0062] Here, Loss is the optimization objective, and when the value of Loss is minimized, the graph data processing device 300 can determine the features of each current node in the transformed d dimensions as the target features. i x j x k x l All are nodes in the graph data. D(x) i ,x j ) is x i Node and x k The square root distance between nodes in the new d-dimensional feature space. i,j∈N1(i) refers to all pairs of nodes connected by an edge. D(x i ,x k ) is x i Node and x k The square root distance between nodes in the new d-dimensional feature space. i,k∈N2(i) refers to all pairs of nodes connected by two jump edges, i.e., x i Node and x k There are no direct edges connecting the nodes; instead, they are indirectly connected through an intermediate node. Accordingly, these two jump edges are two edges that connect end-to-end. D(x) i ,x l ) is x i Node and x l The square root distance of a node in a new d-dimensional feature space. It refers to a pair of nodes with a shortest connection of more than 3 edges, i.e., x i Node and x l There are no direct edges connecting the nodes; instead, they are indirectly connected through three or more intermediate nodes. λ1 and λ2 are pre-defined coefficients.

[0063] In practical applications, the graph data processing device 300 can use a neural network to calculate multiple sets of features for the multiple nodes in the new d dimensions, and calculate the loss value corresponding to each set of features. Thus, the graph data processing device 300 can use the set of features corresponding to the minimum loss value as the target features for the multiple nodes.

[0064] Alternatively, the graph data processing device 300 can first use at least one set of training samples to train the neural network based on the above formula (1), that is, update the parameter values ​​in the neural network based on the Loss calculated by the above formula (1). Then, the graph data processing device 300 uses the trained neural network to infer the target features of each node based on the initial features of each node, thereby achieving dimensionality reduction of the features of multiple nodes.

[0065] It is understood that the above-described methods for reducing the dimensionality of node features are merely illustrative examples. In other embodiments, the graph data processing device 300 may also use other methods to reduce the dimensionality of node features, or the graph data processing device 300 may not need to perform dimensionality reduction processing on node features. This is not a limitation.

[0066] For example, after converting the initial features of a node into target features, the graph data processing device 300 can determine the partition of the space to which the space-filling curve belongs based on the value of the target features of each node and the following formula (2).

[0067] X in =Floor((x in -x 0n Formula (2)

[0068] Among them, X in This refers to the partition label of the i-th node in the n-th dimension among multiple nodes, where n is an integer less than or equal to d and greater than 0, and d is the number of dimensions of the (transformed) node features. 0n Let be the zero-point coordinates in the nth dimension; where the zero-point coordinates are the positions of the partition's zero point in the d-dimensional coordinate space, used to calculate the distances between the node's coordinates in each dimension and the zero-point coordinates. Different zero-point coordinates will affect the partition of the space filled by the node's space-filling curve; for example, when the node's target features include values ​​in two dimensions, such as... Figure 3As shown, different zero-point coordinates will cause the node to belong to different partitions; in this embodiment, the zero-point coordinates can be a pre-set fixed value. r is the size of the partition; in this embodiment, the partition size can be a pre-set fixed value. Floor means round down. Thus, based on the above formula (2), the graph data processing device 300 can determine the spatial partition number of the i-th node in each dimension, thereby determining the partition of the space filling curve to which the i-th node belongs in the multi-dimensional space (each partition is jointly located by the labels in d dimensions).

[0069] S203: The graph data processing device 300 determines the arrangement order of multiple partitions according to their positions in the space-filling curve.

[0070] Among them, the space-filling curve is a mathematical curve, for example, it could be... Figure 3 The curves shown, when infinitely approximating, can continuously cover every point in a multidimensional space, thus achieving the mapping from multidimensional spatial data to a one-dimensional continuous space. Typically, the multidimensional space can be divided into numerous grids through a finite number of recursive operations, and then a continuous curve can be drawn through all the grids, thereby solving the mapping problem between continuous and discrete spaces. Furthermore, the space-filling curve exhibits the property of spatial clustering, meaning that points that are spatially adjacent are also adjacent on the curve. Additionally, the space-filling curve possesses convergence; that is, as the order (i.e., the number of recursive operations corresponding to the space-filling curve) increases, points in the high-dimensional space gradually converge to stable values ​​between 0 and 1.

[0071] After determining the partition of the space to which each node belongs on the space-fill curve, the graph data processing device 300 can determine the positions of multiple partitions belonging to multiple nodes on the space-fill curve and determine the arrangement order of the multiple partitions. For example, the graph data processing device 300 can use a curve (i.e., the space-fill curve) to connect the various partitions, where the order of connection is the arrangement order of the multiple partitions. For example, the space-fill curve can be as follows: Figure 3 The Hilbert curve shown can also be a Z-curve or a Peano curve, or other types of curves; there is no limitation on this.

[0072] In one possible implementation, the graph data processing apparatus 300 can calculate the order of the space-fill curve based on the size of the partition. For example, the graph data processing apparatus 300 can determine the order of the space-fill curve based on the following formula (3).

[0073] L=2 l ,2 l-1 <L0<2 l Formula (3)

[0074] Where l is the order of the space-filling curve. L is the smallest integer greater than L0 raised to the power of 2, which is the number of partitions in each dimension. L0 is the quotient of m and the partition size. m is the maximum value among the coordinate differences of multiple nodes in each dimension.

[0075] Then, the graph data processing device 300 generates a space-filling curve based on the calculated order, thereby obtaining the arrangement order of multiple partitions.

[0076] S204: The graph data processing device 300 sorts multiple nodes according to the arrangement order of multiple partitions to obtain the arrangement order of multiple nodes.

[0077] S205: The graph data processing device 300 sequentially stores multiple nodes in the graph data into the memory 200 according to the arrangement order of the multiple nodes.

[0078] In this embodiment, after determining the arrangement order of the multiple partitions to which multiple nodes belong, the storage order of multiple nodes in the graph data can be determined according to this arrangement order, and the multiple nodes in the graph data (specifically, the related data of these multiple nodes) can be stored in the memory 200 according to this order. In practical application scenarios, the data processing system 10 may also include memory ( Figure 1 (Not shown in the figure), then, in the process of storing multiple nodes to the memory 200, the graph data processing device 300 can first store multiple nodes to memory, and then persistently store multiple nodes from memory to the memory 200, etc.

[0079] As an example, the graph data processing device 300 can generate a sequence list based on the space-filling curve, which records the arrangement order of multiple partitions. Then, the graph data processing device 300 can sequentially traverse the multiple partitions according to their arrangement order. For the currently traversed partition, if the partition contains one or more nodes, the identifiers of those nodes are added to the sequence list at the positions corresponding to the currently traversed partition. When a partition contains multiple nodes, the order in which the identifiers of these nodes are added to the list can be random, or they can be added sequentially according to the size of the node identifiers or other rules; there is no limitation on this.

[0080] After adding the identifiers of all nodes belonging to a partition to their corresponding positions in the sequence list, the graph data processing device 300 continues to traverse the next partition according to the arrangement order of the multiple partitions until all partitions have been traversed or the identifiers of all nodes have been added to the sequence list. In this way, the graph data processing device 300 can use the arrangement order of the identifiers of multiple nodes in the sequence list as the arrangement order of the multiple nodes, that is, as the order in which the data of the multiple nodes is stored.

[0081] Finally, the graph data processing device 300 can sequentially store the data of the multiple nodes into the memory 200 according to the order of arrangement in the sequential list, thereby realizing the sequential storage of the data of multiple nodes. In practical application scenarios, after adding the identifiers of multiple nodes to the sequential list, the graph data processing device 300 can first traverse the sequential list and delete the list elements corresponding to the partitions that do not contain nodes, and then store the nodes indicated by the identifiers in the sequential list in sequence. Thus, although when partitioning based on the above formula (3), in order to make the number of partitions a power of 2, the space covered by the partitions may be expanded, the partitions that do not contain nodes will be deleted from the sequential list during actual sorting, so no additional storage space will be occupied.

[0082] Because the space-fill curve exhibits spatial clustering, the graph data processing apparatus 300 stores nodes in the graph data based on the arrangement order of the partitions determined by the space-fill curve. This allows spatially close nodes to be stored in adjacent locations as much as possible. Consequently, when the processor 100 reads data from multiple nodes (which may be some nodes in the graph data) connected by edges, the data of these multiple nodes can be prefetched from the memory 200 into the cache 101 together. This improves the cache hit rate, thereby increasing the efficiency of the processor 100 in reading the data of these multiple nodes, and ultimately improving the overall efficiency of graph computation.

[0083] The above Figure 2 The illustrated embodiment uses a fixed partition size and zero-point coordinates as an example. In practical applications, the partition size and zero-point coordinates affect the partition of the space filled by each node, thus influencing the final storage order of multiple nodes. Therefore, the graph data processing device 300 can also determine a better partition size and zero-point coordinates for the graph data to be stored through optimization. This helps to further achieve the effect of storing spatially similar nodes in adjacent locations, thereby improving the overall efficiency of graph computation.

[0084] The following is combined Figure 4 This paper introduces another graph data processing method provided by an embodiment of this application. See also... Figure 4 This illustrates a flowchart of another graph data processing method. For ease of explanation, this embodiment uses an application... Figure 1 The data processing system 10 shown is used as an example for illustration.

[0085] in, Figure 4 The graph data processing method shown may specifically include the following steps.

[0086] S401: The graph data processing device 300 acquires graph data, which includes multiple nodes and edge information, the edge information being used to indicate the connection relationship between different nodes.

[0087] S402: The graph data processing device 300 converts the higher-dimensional initial features of each node in the multiple nodes into lower-dimensional target features based on the edge information.

[0088] The number of dimensions of the target feature is d. The implementation methods for steps S401 and S402 can be found above. Figure 2 The descriptions of reducing node feature dimensions in the illustrated embodiments are not repeated here.

[0089] Furthermore, in this embodiment, the graph data includes edge information as an example for explanation. When the graph data does not include edge information, the graph data processing device 300 can also determine the connection relationship between different nodes based on the truncation radius.

[0090] Furthermore, this embodiment uses dimensionality reduction of node features in graph data as an example for illustration. In practical applications, the graph data processing device 300 may not need to perform dimensionality reduction on node features. For example, in some application scenarios, the number of dimensions of node features in graph data is small, so the graph data processing device 300 may not need to perform dimensionality reduction on node features.

[0091] S403: The graph data processing device 300 determines multiple sets of candidate parameters based on the value of the target feature of each node. Each set of candidate parameters includes the partition size and the zero point coordinates.

[0092] In this context, each set of candidate parameters is used to determine a partitioning result, which indicates the multiple partitions to which multiple nodes belong. That is, different sets of candidate parameters will produce different partitioning results, thus different sets of candidate parameters will cause multiple nodes to be assigned to different partitions.

[0093] For ease of understanding, an exemplary implementation method for determining multiple sets of candidate parameters is given below.

[0094] In one possible implementation, the graph data processing apparatus 300 can calculate the distances between multiple pairs of nodes that are connected by edges. The distance between each pair of nodes can be, for example, the square root distance between the two nodes in a d-dimensional feature space. Then, the graph data processing apparatus 300 determines the range of values ​​for the partition size based on the distances between the multiple pairs of nodes.

[0095] For example, the graph data processing device 300 can calculate the maximum value r_max and the median r_med (or average, etc.) of the distance between multiple pairs of nodes connected by an edge, and determine the range of the partition size as [r_med, r_max].

[0096] It's understandable that if the partition size is set too large, dividing multiple nodes into multiple partitions can easily result in node pairs without connecting edges within and between adjacent partitions. When the data of multiple nodes within the same partition is randomly sorted, storing these nodes sequentially according to the partition's arrangement order reduces the effectiveness of sequential storage, meaning it diminishes the ability to store spatially close nodes in adjacent locations. Conversely, when the partition size is set too small, the number of partitions increases, increasing the size of the generated sequential list and thus increasing the processing time (e.g., increasing the time required to traverse the sequential list). Furthermore, based on the convergence characteristics of the space-fill curve, continuously increasing the resolution (corresponding to decreasing the partition size) does not significantly change the final arrangement order of multiple nodes, making it difficult to continuously improve the sequential storage effect. In practical applications, when the partition size is close to the average distance between multiple pairs of nodes' edges, connected pairs of nodes can be assigned to adjacent partitions. This not only ensures the sequential storage effect of multiple nodes but also improves the efficiency of sequential storage for multiple nodes.

[0097] In other embodiments, when the graph data does not include edge information, the graph data processing device 300 can determine the range of values ​​for the partition size based on the truncation radius. For example, the range of values ​​for the partition size can be [0.3*r_cut, r_cut]. Where r_cut is the truncation radius.

[0098] Then, the graph data processing device 300 can generate M sets of parameters (r′, x′). 01 ,x′ 02 ,…,x′ 0d M is a positive integer. The values ​​of each parameter in each group are in the range [0, 1]. For example, the value of each parameter can be randomly selected within the range [0, 1]. r′ is the parameter used to generate the partition size. x′ 01 The parameters are used to generate the zero coordinates of the first dimension out of d dimensions; x′ 02 The parameters are used to generate the zero coordinates of the second dimension out of d dimensions; ...; x′ 0d These are the parameters used to generate the zero coordinates of the d-th dimension out of d dimensions.

[0099] For each set of parameters, the graph data processing device 300 can calculate the value of the partition size based on the following formula (4).

[0100] r=r′*(R max -R min )+R min Formula (4)

[0101] Where is the value of the partition size, R max R is the maximum value within the range of possible partition sizes. min This is the minimum value within the range of possible partition sizes.

[0102] Thus, for different values ​​of r in the M groups of parameters, M different values ​​of partition size can be determined within the range of partition size values.

[0103] Furthermore, the graph data processing device 300 can also calculate M sets of zero-point coordinates based on M possible values ​​for the partition size and the target feature values ​​of each node. Each set of zero-point coordinates is calculated based on one possible value for the partition size, and each set includes zero-point coordinate values ​​for each of the d dimensions.

[0104] For example, the graph data processing device 300 can determine the maximum and minimum values ​​of the node coordinates in each of the d dimensions, obtaining max1, min1, max2, min2, ..., max d ,min d Here, max1 refers to the maximum value of the node coordinates in the first dimension out of d dimensions, min1 refers to the minimum value of the node coordinates in the first dimension out of d dimensions, and so on. Then, for the maximum and minimum values ​​of the node coordinates in each dimension, the graph data processing device 300 can calculate the maximum coordinate difference corresponding to that dimension. Here, the maximum coordinate difference corresponding to the first dimension is m1 = max1 - min1. And so on, the graph data processing device 300 can calculate the maximum coordinate differences corresponding to each of the d dimensions, which are m1, m2, ..., m d Next, the graph data processing device 300 compares the d maximum coordinate differences to obtain m1 to m... d The maximum value in the interval is denoted by m below. The graph data processing device 300 can then calculate the ratio L0 (i.e., m / r) between this maximum value m and the partition size r, and further determine the smallest integer L = 2 among powers of 2 greater than L0. l ,2 l- 1 <L0<2 l Where L is the number of partitions in each dimension; l is the order of the space-filling curve.

[0105] Then, the graph data processing device 300 can transform the zero-point coordinates of each dimension to [max] based on the maximum and minimum values ​​of the node coordinates in each of the d dimensions. n -L*r,min n Within the range of values, the zero coordinates of each dimension can be calculated based on the following formula (5).

[0106] x 0n =x′ 0n *(L*r-max n +min n )+max n -L*r, n=1,2,…,d Formula (5)

[0107] Where, x 0n This refers to the value of the zero point coordinate in the nth dimension.

[0108] Thus, the graph data processing device 300 can calculate a set of zero-point coordinates in d dimensions based on a value r for the partition size. Therefore, for multiple values ​​of the partition size, the graph data processing device 300 can calculate multiple sets of corresponding zero-point coordinates using the above method.

[0109] Accordingly, for one value of the partition size and multiple sets of zero-point coordinates calculated based on that value, a set of candidate parameters can be generated. Thus, based on M values ​​of the partition size and M sets of zero-point coordinates, M sets of candidate parameters can be generated.

[0110] After identifying multiple sets of candidate parameters, the graph data processing device 300 can use an optimization method to determine the optimal candidate parameters from these sets that improve the effect of storing spatially close nodes in mutually adjacent locations. For this purpose, such as... Figure 4 As shown, the data processing device 300 can continue to perform the following steps.

[0111] S404: The graph data processing device 300 optimizes multiple sets of candidate parameters based on the value of the target feature of each node to obtain the target candidate parameters from the multiple sets of candidate parameters.

[0112] In specific implementation, for each set of candidate parameters, the graph data processing device 300 can divide multiple nodes based on the set of candidate parameters to obtain a set of partitioning results. Based on the partitioning results corresponding to the multiple sets of candidate parameters, the target candidate parameter is determined from the multiple sets of candidate parameters. The partitioning result corresponding to the target candidate parameter enables the best effect of storing spatially close nodes in mutually adjacent positions. That is, the target candidate parameter is the optimal candidate parameter among the multiple sets of candidate parameters. Thus, the graph data processing device 300 can use the multiple partitions indicated by the partitioning result corresponding to the target candidate parameter as multiple partitions of the space where the space-filling curve of the multiple nodes finally belongs.

[0113] As an implementation example, for each set of candidate parameters, the graph data processing device 300 can use the above formula (2) to determine the partition of the space where the space-filling curve of each of the multiple nodes belongs, and obtain the partition result corresponding to the set of candidate parameters. Then, the graph data processing device 300 can use the order l of the space-filling curve calculated based on the set of candidate parameters to generate the space-filling curve corresponding to the set of candidate parameters. Then, the graph data processing device 300 can use the generated space-filling curve to connect the multiple partitions indicated by the partition result to obtain the arrangement order of the multiple partitions in the space where the set of candidate parameters is located; or, the space-filling curve can be generated in advance, so that the graph data processing device 300 can use the generated space-filling curve to connect the multiple partitions indicated by the partition result to obtain the arrangement order of the multiple partitions in the space where the set of candidate parameters is located. Thus, the graph data processing device 300 can further determine the storage order of multiple nodes based on the arrangement order of multiple partitions. The specific implementation method of the graph data processing device 300 determining the storage order of multiple nodes based on the arrangement order of multiple partitions can be found in the above. Figure 2 The details regarding determining the storage order of multiple nodes in the illustrated embodiment are described below and will not be repeated here. Referring to the above method, the graph data processing device 300 can determine the storage order of multiple nodes corresponding to each set of candidate parameters.

[0114] Next, the graph data processing device 300 can calculate the linear distance corresponding to each set of candidate parameters. The linear distance includes the distance between multiple pairs of nodes with connected edges. Thus, the graph data processing device 300 can determine the target candidate parameter from multiple sets of candidate parameters based on the linear distance corresponding to each set of candidate parameters, wherein the linear distance corresponding to the target candidate parameter is the smallest.

[0115] For example, the graph data processing device 300 can determine the optimal target candidate parameter from multiple sets of candidate parameters based on the following formula (6).

[0116]

[0117] Where S is the target parameter, and when the value of S corresponding to the candidate parameter is minimized, the graph data processing device 300 can determine the candidate parameter as the optimal target candidate parameter. i x j x k x l All are nodes in the graph data. T(x) i ,x j ) is node x i With node x j The linear distance between nodes after sorting. α1 and α2 are pre-defined coefficients. i,j∈N1(i) refers to all pairs of nodes with connected edges. i,k∈N2(i) refers to all pairs of nodes connected by two jump edges. It refers to a pair of nodes with a shortest connection edge count greater than 3.

[0118] It can be understood that the smaller the value of S, the smaller the linear distance between multiple nodes connected by edges after sorting, and the larger the linear distance between nodes that require many edges to connect after sorting. Therefore, when storing multiple nodes based on the storage order corresponding to this S value, the effect of storing spatially close nodes in mutually adjacent positions is better. In this way, the graph data processing device 300 can calculate the S value corresponding to each set of candidate parameters, determine the candidate parameter corresponding to the smallest S value as the target candidate parameter, and determine the partitioning result corresponding to the target candidate parameter, thereby determining the multiple partitions to which multiple nodes belong.

[0119] Furthermore, the graph data processing device 300 can also determine the order of the space filling curve corresponding to the target candidate parameter. The order of the space filling curve can be determined based on the partition size in the target candidate parameter, such as based on the above formula (3).

[0120] S405: The graph data processing device 300 determines the arrangement order of multiple partitions in the space where the target candidate parameters are located, and sequentially stores multiple nodes in the graph data into the memory 200.

[0121] Since the multiple partitions of the space to which the space-filling curves of multiple nodes belong, determined based on the target candidate parameters, and the arrangement order of these partitions, allow spatially close nodes to be stored in adjacent locations as much as possible, determining the arrangement order of multiple nodes in the graph data based on the arrangement order of the multiple partitions of the space to which the target candidate parameters belong, and storing the multiple nodes in the graph data sequentially according to this arrangement order, allows the processor 100 to prefetch the data of multiple nodes (which may be some nodes in the graph data) from the memory 200 into the cache 101 together during the subsequent reading of data of multiple nodes connected by edges in the graph data. This improves the hit rate of the cache 101, thereby improving the efficiency of the processor 100 in reading the data of these multiple nodes, and thus improving the overall efficiency of graph computation. The implementation method of the graph data processing device 300 determining the storage order of multiple nodes based on the target candidate parameters can be found in the above description. Figure 2 The relevant details of steps S203 and S204 in the illustrated embodiment are described, and will not be repeated here.

[0122] It should be noted that the above Figure 4 The embodiments shown are merely illustrative and are not intended to be limiting. Figure 4 Based on the illustrated embodiment, other embodiments for improving the hit rate of cache 101 can be derived by replacing or adding some steps. These will be illustrated below.

[0123] Example 1: In this embodiment, the graph data processing device 300 determines the target candidate parameter based on the M S values ​​corresponding to the M groups of candidate parameters. In other embodiments, the graph data processing device 300 may also use other optimization algorithms such as genetic algorithms or Bayesian optimization algorithms to determine the target candidate parameter. Taking the genetic algorithm as an example, after calculating the S values ​​corresponding to the M groups of candidate parameters using the above method, the graph data processing device 300 can select h groups of candidate parameters corresponding to the smaller S values, perform numerical exchange or random mutation on these h groups of candidate parameters to generate new M groups of candidate parameters, and re-execute the above process based on the new M groups of candidate parameters. When the minimum S value corresponding to the M groups of candidate parameters is less than the threshold Y, or the number of times the M groups of candidate parameters are generated exceeds the threshold Z, the candidate parameter corresponding to the minimum S value in the latest generated M groups of candidate parameters is selected as the target candidate parameter.

[0124] Example 2: In this embodiment, the candidate parameters include both partition size and zero point coordinates. In other embodiments, each set of candidate parameters may only include partition size or only include zero point coordinates (the other parameter is a fixed value). Thus, the graph data processing device 300 may only optimize the partition size or only optimize the zero point coordinates. There is no limitation on this.

[0125] Example 3: In this embodiment, the graph data processing device 300 determines the partition size and zero-point coordinate value based on the value of the node features after dimensionality reduction (i.e., the target features of each node) as an example for illustration. In other embodiments, the graph data processing device 300 can also directly determine the partition size and zero-point coordinate value based on the value of the initial features of each node, thereby realizing the sequential storage of multiple nodes.

[0126] It is worth noting that other reasonable combinations of steps that can be conceived by those skilled in the art based on the above description also fall within the scope of protection of this application. Secondly, those skilled in the art should also be aware that the embodiments described in the specification are preferred embodiments, and the actions involved are not necessarily essential to this application.

[0127] The above combination Figures 1 to 4 The graph data processing method provided in the embodiments of this application will be introduced. Next, the structure of the graph data processing apparatus and computing device provided in the embodiments of this application will be described with reference to the accompanying drawings.

[0128] See Figure 5 The diagram shows a schematic representation of a graph data processing device. Figure 5 As shown, the figure data processing device 500 includes:

[0129] The acquisition module 501 is used to acquire graph data, which includes multiple nodes.

[0130] The determination module 502 is used to determine multiple partitions of the space to which the space-filling curve of the multiple nodes belongs, based on the feature value of each node among the multiple nodes, where each node among the multiple nodes belongs to one of the multiple partitions.

[0131] The sorting module 503 determines the arrangement order of multiple partitions according to their positions in the space-filling curve; it is used to sort multiple nodes according to the arrangement order of multiple partitions to obtain the arrangement order of multiple nodes.

[0132] Storage module 504 is used to sequentially store multiple nodes according to their arrangement order.

[0133] In one possible implementation, when determining multiple partitions of the space to which the space-filling curve belongs, based on the feature value of each of the multiple nodes, the determining module 502 is used to:

[0134] Based on the feature value of each node, multiple sets of candidate parameters are determined. Each set of candidate parameters is used to determine a partitioning result. The partitioning result is used to indicate multiple partitions in the space where the space-filling curve of multiple nodes is located. Each set of candidate parameters includes partition size or zero coordinates. The zero coordinates are used to calculate the distance between the coordinates of each node in each dimension and the zero coordinates.

[0135] Based on the feature value of each node, multiple sets of candidate parameters are optimized to obtain the target candidate parameter from the multiple sets of candidate parameters;

[0136] Determine the partitioning results corresponding to the target candidate parameters.

[0137] In one possible implementation, each set of candidate parameters includes the partition size and the zero-point coordinates;

[0138] When determining multiple sets of candidate parameters based on the feature values ​​of each node, the determination module 502 is used for:

[0139] The range of values ​​for the partition size is determined based on the distance between multiple pairs of nodes that are connected by edges.

[0140] Within the range of possible partition sizes, determine the various possible values ​​for the partition size;

[0141] Based on the various possible values ​​for the partition size and the characteristic values ​​of each node, multiple sets of zero-point coordinates are calculated.

[0142] In one possible implementation, the determining module 502 is further configured to:

[0143] Before determining the range of values ​​for the partition size, based on the edge information included in the graph data, determine multiple pairs of nodes with connected edges among multiple nodes. The edge information is used to indicate the connection relationship between multiple nodes.

[0144] Alternatively, before determining the range of possible partition sizes, determine that multiple pairs of nodes with a distance less than the cutoff radius are connected by edges.

[0145] In one possible implementation, when determining multiple sets of candidate parameters based on the feature values ​​of each node, the determining module 502 is used to:

[0146] Based on the feature value of each node, the node distance corresponding to each set of candidate parameters is calculated. The linear distance includes the distance between multiple pairs of nodes with connected edges after sorting.

[0147] Based on the linear distances corresponding to each set of candidate parameters, target candidate parameters are determined from multiple sets of candidate parameters, with the target candidate parameter having the smallest linear distance.

[0148] In one possible implementation, the determining module 502 is further configured to:

[0149] The order of the space-filling curve is determined based on the partition size in the target candidate parameters.

[0150] In one possible implementation, when determining multiple partitions of the space to which the space-filling curve belongs, based on the feature value of each of the multiple nodes, the determining module 502 is used to:

[0151] The initial features of each of the multiple nodes are reduced in dimensionality to obtain the target features of each node. The number of dimensions of the initial features is higher than the number of dimensions of the target features.

[0152] Based on the value of the target feature of each node, determine multiple partitions of the space where the space-filling curve of multiple nodes is located.

[0153] because Figure 5 The image data processing device 500 shown corresponds to the above. Figure 2 as well as Figure 4 The graph data processing device 300 in the illustrated embodiment, therefore Figure 5 For a detailed description of the implementation of the graph data processing device 500 and its technical effects, please refer to the above. Figure 2 as well as Figure 4 The relevant details in the illustrated embodiments are described in detail here, and will not be repeated here.

[0154] Figure 6 This application provides a schematic diagram of the hardware structure of a computing device 600, which, for example, can implement the above-described... Figure 3 The illustrated embodiment includes a graph data processing device 300, etc.

[0155] like Figure 6As shown, the computing device 600 includes a processor 601, a memory 602, and a communication interface 603. The processor 601, memory 602, and communication interface 603 communicate via a bus 604, or via other means such as wireless transmission. The memory 602 stores instructions, and the processor 601 executes the instructions stored in the memory 602. Further, the computing device 600 may also include a memory unit 605, which is connected to the processor 601, the storage medium 602, and the communication interface 603 via the bus 604. The memory 602 stores program code, and the processor 601 can read the program code stored in the memory 602 into the memory unit 605 and execute the program code in the memory unit 605 to perform the following operations:

[0156] Acquire graph data, which includes multiple nodes;

[0157] Based on the feature value of each of the plurality of nodes, multiple partitions of the space to which the space-filling curve of the plurality of nodes belongs are determined, and each of the plurality of nodes belongs to one of the partitions of the plurality of partitions;

[0158] The arrangement order of the multiple partitions is determined according to their positions in the space-filling curve;

[0159] The nodes are sorted according to the arrangement order of the multiple partitions to obtain the arrangement order of the multiple nodes;

[0160] The multiple nodes are stored sequentially according to their arrangement order.

[0161] It should be understood that in this embodiment, the processor 601 can be a CPU, but it can also be other general-purpose processors, digital signal processors (DSPs), application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), or other programmable logic devices, discrete gate or transistor logic devices, discrete device assemblies, etc. A general-purpose processor can be a microprocessor or any conventional processor.

[0162] The memory 602 may include read-only memory and random access memory, and provides instructions and data to the processor 601. The memory 602 may also include non-volatile random access memory.

[0163] The memory 602 can be volatile memory or non-volatile memory, or it can include both. The non-volatile memory can be read-only memory (ROM), programmable read-only memory (PROM), erasable programmable read-only memory (EPROM), electrically erasable programmable read-only memory (EEPROM), or flash memory. The volatile memory can be random access memory (RAM), which is used as an external cache. By way of example, but not limitation, many forms of RAM are available, such as static random access memory (SRAM), dynamic random access memory (DRAM), synchronous dynamic random access memory (SDRAM), double data rate synchronous dynamic random access memory (DDR SDRAM), enhanced synchronous dynamic random access memory (ESDRAM), synchronous linked dynamic random access memory (SLDRAM), and direct rambus RAM (DR RAM).

[0164] The communication interface 603 is used to communicate with other devices connected to the computing device 600. The bus 604 may include a data bus, a power bus, a control bus, and a status signal bus, etc. However, for clarity, all buses are labeled as bus 604 in the figure.

[0165] It should be understood that the computing device 600 according to the embodiments of this application may correspond to the graph data processing device 300 in the embodiments of this application, and may correspond to the device executing the embodiments of this application. Figure 2 as well as Figure 4 The method executed by the graph data processing device 300 in the illustrated embodiment, and the above-mentioned and other operations and / or functions implemented by the computing device 600, are respectively for the purpose of implementing... Figure 2 as well as Figure 4 The process of the corresponding methods in [the document] will not be elaborated here for the sake of brevity.

[0166] This application also provides a computer-readable storage medium. The computer-readable storage medium can be any available medium that a computing device can store, or a data storage device such as a data center containing one or more available media. The available medium can be a magnetic medium (e.g., floppy disk, hard disk, magnetic tape), an optical medium (e.g., DVD), or a semiconductor medium (e.g., solid-state drive). The computer-readable storage medium includes instructions that instruct the computing device to perform the above-described graph data processing method.

[0167] This application also provides a computer program product. The computer program product includes one or more computer instructions. When the computer instructions are loaded and executed on a computing device, all or part of the processes or functions described in this application are generated.

[0168] The computer instructions may be stored in a computer-readable storage medium or transmitted from one computer-readable storage medium to another. For example, the computer instructions may be transmitted from one website, computer, or data center to another website, computer, or data center via wired (e.g., coaxial cable, fiber optic, digital subscriber line (DSL)) or wireless (e.g., infrared, wireless, microwave, etc.) means.

[0169] The computer program product can be a software installation package. When any of the aforementioned graph data processing methods is required, the computer program product can be downloaded and executed on a computing device.

[0170] The above embodiments can be implemented, in whole or in part, by software, hardware, firmware, or any other combination thereof. When implemented using software, the above embodiments can be implemented, in whole or in part, as a computer program product. The computer program product includes one or more computer instructions. When the computer program instructions are loaded or executed on a computer, all or part of the processes or functions described in the embodiments of this application are generated. The computer can be a general-purpose computer, a special-purpose computer, a computer network, or other programmable device. The computer instructions can be stored in a computer-readable storage medium or transmitted from one computer-readable storage medium to another. For example, the computer instructions can be transmitted from one website, computer, server, or data center to another website, computer, server, or data center via wired (e.g., coaxial cable, fiber optic, digital subscriber line (DSL)) or wireless (e.g., infrared, wireless, microwave, etc.) means. The computer-readable storage medium can be any available medium that a computer can access or a data storage device such as a server or data center that includes one or more sets of available media. The available medium can be a magnetic medium (e.g., floppy disk, hard disk, magnetic tape), an optical medium (e.g., DVD), or a semiconductor medium. A semiconductor medium can be a solid-state drive.

[0171] The terminology used in the above embodiments is for the purpose of describing specific embodiments only and is not intended to be limiting of this application. As used in the specification and appended claims of this application, the singular expressions “a,” “an,” “the,” “the,” “the,” and “this” are intended to also include expressions such as “one or more,” unless the context clearly indicates otherwise. It should also be understood that in the embodiments of this application, “one or more” refers to one, two, or more; the character “ / ” generally indicates that the preceding and following objects are in an “or” relationship. In the embodiments of this application, “simultaneously” means within the same time period, including situations where they are at the same moment.

[0172] References to "one embodiment" or "some embodiments" as described in this specification mean that one or more embodiments of this application include a specific feature, structure, or characteristic described in connection with that embodiment. Therefore, the phrases "in one embodiment," "in some embodiments," "in other embodiments," "in still other embodiments," etc., appearing in different parts of this specification do not necessarily refer to the same embodiment, but rather mean "one or more, but not all, embodiments," unless otherwise specifically emphasized. The terms "comprising," "including," "having," and variations thereof mean "including but not limited to," unless otherwise specifically emphasized.

[0173] The above description is merely a specific embodiment of this application, but the scope of protection of this application is not limited thereto. Any person skilled in the art can easily conceive of various equivalent modifications or substitutions within the technical scope disclosed in this application, and these modifications or substitutions should all be covered within the scope of protection of this application. Therefore, the scope of protection of this application should be determined by the scope of the claims.

Claims

1. A graph data processing method, characterized in that, The method includes: Acquire graph data, which includes multiple nodes; Based on the feature value of each of the plurality of nodes, multiple partitions of the space to which the space-filling curve of the plurality of nodes belongs are determined, and each of the plurality of nodes belongs to one of the partitions of the plurality of partitions; The arrangement order of the multiple partitions is determined according to their positions in the space-filling curve; The nodes are sorted according to the arrangement order of the multiple partitions to obtain the arrangement order of the multiple nodes; The multiple nodes are stored sequentially according to their arrangement order.

2. The method according to claim 1, characterized in that, The step of determining multiple partitions of the space to which the space-filling curve belongs, based on the feature value of each of the multiple nodes, includes: Based on the feature value of each node, multiple sets of candidate parameters are determined. Each set of candidate parameters is used to determine a partitioning result. The partitioning result is used to indicate multiple partitions in the space where the space-filling curve to which the multiple nodes belong is located. Each set of candidate parameters includes partition size or zero-point coordinates. The zero-point coordinates are used to calculate the distance between the coordinates of each node in each dimension and the zero-point coordinates. Based on the feature value of each node, the multiple sets of candidate parameters are optimized to obtain the target candidate parameter among the multiple sets of candidate parameters; Determine the partitioning result corresponding to the target candidate parameters.

3. The method according to claim 2, characterized in that, Each set of candidate parameters includes the partition size and the zero-point coordinates; The step of determining multiple sets of candidate parameters based on the feature values ​​of each node includes: The range of values ​​for the partition size is determined based on the distance between multiple pairs of nodes that are connected by edges. Within the range of possible partition sizes, determine multiple possible values ​​for the partition size; Based on the various possible values ​​of the partition size and the characteristic values ​​of each node, multiple sets of values ​​for the zero-point coordinates are calculated.

4. The method according to claim 3, characterized in that, Before determining the range of values ​​for the partition size, the method further includes: Based on the edge information included in the graph data, multiple pairs of nodes with connected edges are determined among the plurality of nodes, and the edge information is used to indicate the connection relationship between the plurality of nodes; Alternatively, determine that multiple pairs of nodes whose distance is less than the cutoff radius are connected by edges.

5. The method according to any one of claims 2 to 4, characterized in that, The optimization of the multiple sets of candidate parameters based on the feature values ​​of each node includes: Based on the feature value of each node, the node distance corresponding to each set of candidate parameters is calculated. The linear distance includes the distance between multiple pairs of nodes with connected edges after sorting. Based on the linear distance corresponding to each set of candidate parameters, the target candidate parameter is determined from the multiple sets of candidate parameters, and the linear distance corresponding to the target candidate parameter is the smallest.

6. The method according to claim 5, characterized in that, The method further includes: The order of the space-filling curve is determined based on the partition size in the target candidate parameters.

7. The method according to any one of claims 1 to 6, characterized in that, The step of determining multiple partitions of the space to which the space-filling curve belongs, based on the feature value of each of the multiple nodes, includes: The initial features of each of the plurality of nodes are reduced in dimensionality to obtain the target features of each node, wherein the number of dimensions of the initial features is higher than the number of dimensions of the target features; Based on the value of the target feature of each node, multiple partitions of the space to which the space-filling curve of the multiple nodes belongs are determined.

8. A graph data processing apparatus, characterized in that, The device includes: The acquisition module is used to acquire graph data, which includes multiple nodes; The determination module is used to determine multiple partitions of the space to which the space-filling curve of the multiple nodes belongs, based on the feature value of each of the multiple nodes; each of the multiple nodes belongs to one of the multiple partitions. The sorting module determines the arrangement order of the multiple partitions according to their positions in the space-filling curve; it is used to sort the multiple nodes according to the arrangement order of the multiple partitions to obtain the arrangement order of the multiple nodes. A storage module is used to sequentially store the plurality of nodes according to their arrangement order.

9. The apparatus according to claim 8, characterized in that, When determining multiple partitions of the space to which the space-filling curve belongs, based on the feature value of each of the multiple nodes, the determining module is used to: Based on the feature value of each node, multiple sets of candidate parameters are determined. Each set of candidate parameters is used to determine a partitioning result. The partitioning result is used to indicate multiple partitions in the space where the space-filling curve to which the multiple nodes belong is located. Each set of candidate parameters includes partition size or zero-point coordinates. The zero-point coordinates are used to calculate the distance between the coordinates of each node in each dimension and the zero-point coordinates. Based on the feature value of each node, the multiple sets of candidate parameters are optimized to obtain the target candidate parameter among the multiple sets of candidate parameters; Determine the partitioning result corresponding to the target candidate parameters.

10. The apparatus according to claim 9, characterized in that, Each set of candidate parameters includes the partition size and the zero-point coordinates; When determining multiple sets of candidate parameters based on the feature values ​​of each node, the determining module is used for: The range of values ​​for the partition size is determined based on the distance between multiple pairs of nodes that are connected by edges. Within the range of possible partition sizes, determine multiple possible values ​​for the partition size; Based on the various possible values ​​of the partition size and the characteristic values ​​of each node, multiple sets of values ​​for the zero-point coordinates are calculated.

11. The apparatus according to claim 10, characterized in that, The determining module is further configured to: Before determining the range of values ​​for the partition size, based on the edge information included in the graph data, multiple pairs of nodes with connected edges are determined among the plurality of nodes, and the edge information is used to indicate the connection relationship between the plurality of nodes; Alternatively, before determining the range of values ​​for the partition size, determine that multiple pairs of nodes whose distance is less than the cutoff radius have connecting edges.

12. The apparatus according to any one of claims 9 to 11, characterized in that, When the determining module optimizes the multiple sets of candidate parameters based on the feature values ​​of each node, it is used to: Based on the feature value of each node, the node distance corresponding to each set of candidate parameters is calculated. The linear distance includes the distance between multiple pairs of nodes with connected edges after sorting. Based on the linear distance corresponding to each set of candidate parameters, the target candidate parameter is determined from the multiple sets of candidate parameters, and the linear distance corresponding to the target candidate parameter is the smallest.

13. The apparatus according to claim 12, characterized in that, The determining module is further configured to: The order of the space-filling curve is determined based on the partition size in the target candidate parameters.

14. The apparatus according to any one of claims 8 to 13, characterized in that, When determining multiple partitions of the space to which the space-filling curve belongs, based on the feature value of each of the multiple nodes, the determining module is used to: The initial features of each of the plurality of nodes are reduced in dimensionality to obtain the target features of each node, wherein the number of dimensions of the initial features is higher than the number of dimensions of the target features; Based on the value of the target feature of each node, multiple partitions of the space to which the space-filling curve of the multiple nodes belongs are determined.

15. A computer-readable storage medium, characterized in that, Includes instructions that, when executed on at least one computing device, cause the at least one computing device to perform the steps of the method as described in any one of claims 1 to 7.

16. A computer program product containing instructions, characterized in that, When it is run on at least one computing device, it causes the at least one computing device to perform the method as described in any one of claims 1 to 7.

Images