AI chip and adjacency table sampling method

Abstract

An artificial intelligence (AI) chip (200) and an adjacency list sampling method. The AI ​​chip (200) includes a random number generator (201) and an NPU (202); the random number generator (201) is used to generate K random numbers; the NPU (202) is used to perform a row-column transposition on a first adjacency list input to obtain a second adjacency list, the size of the first adjacency list is M*N, and the size of the second adjacency list is N*M; the second adjacency list is randomly rearranged according to the K random numbers to obtain a third adjacency list, the size of the third adjacency list is K*M; a target adjacency list is obtained according to the third adjacency list, the size of the target adjacency list is M*S, and S is an integer less than N. Based on the characteristics of the AI ​​chip (200) structure, the adjacency list sampling process is redesigned to reduce the computational time and memory overhead.

Description

Technical Field

[0001] The present application relates to the field of graph neural networks, and specifically to an artificial intelligence (AI) chip and an adjacency list sampling method. Background Art

[0002] In scenarios such as recommendation search, smart cities, and smart transportation, it is usually necessary to extract features from non-Euclidean data. For example, in social networks, individuals and events form a graph structure; in e-commerce, individuals and commodities form a graph structure. There are hundreds of millions of nodes in these graph structures, and there are a large number of relationships between nodes that constitute edges. Graph neural networks mainly learn how to aggregate neighbor nodes and adjacent edge features. Because there is no upper bound on the adjacent edges of nodes in theory, it may take a lot of time and memory overhead to overflow during training or inference. Therefore, the industry usually samples the adjacency list to limit the number of adjacent edges of nodes, thereby limiting the time and memory overhead during training or inference to a range.

[0003] The current algorithm flow for sampling the adjacency list is as follows Figure 1a As shown, step 1: transpose the adjacency list; step 2: generate random numbers; step 3: then shuffle the adjacency list after transposing the rows and columns according to the generated random numbers; step 4: transpose the adjacency list obtained by shuffling the rows and columns; step 5: matrix segment the transposed adjacency list to obtain the sampled adjacency list. Among them, step 2 and step 3 together are a random shuffle operation. Based on different hardware, the deployment plan is different. When the above sampling plan is implemented based on the CPU, such as Figure 1b As shown, steps 1 to 5 are all executed on the CPU, but the performance of using the CPU to perform row-column transposition and matrix splitting operations is poor; Figure 1c As shown in the figure, when implemented based on the GPU+CPU hardware architecture, steps 1, 4 and 5 are executed on the GPU, and steps 2 and 3 are implemented on the CPU. However, since it involves data transfer from GPU->CPU->GPU, this solution will bring additional time overhead. Summary of the invention

[0004] The embodiments of the present application provide an AI chip and an adjacency list sampling method. Based on the characteristics of the AI ​​chip structure, the adjacency list sampling process is redesigned to reduce the calculation time and memory overhead.

[0005] In a first aspect, an embodiment of the present application provides an AI chip, the AI ​​chip comprising: a random number generator and a neural network processor NPU, wherein a CPU is connected to the NPU;

[0006] A random number generator, used to generate K random numbers;

[0007] NPU is used to perform row-column transposition on the input first adjacency list to obtain a second adjacency list, where the size of the first adjacency list is M*N, where M and N are both integers greater than 0; the size of the second adjacency list is N*M; the second adjacency list is shuffled according to K random numbers to obtain a third adjacency list, where the size of the third adjacency list is K*M; a target adjacency list is obtained according to the third adjacency list, where the size of the target adjacency list is M*S, where S is an integer less than N.

[0008] The adjacency list sampling process has been redesigned: random numbers are generated through the CPU or DSA, and the rows and columns of the adjacency list are transposed and reordered through the NPU. This is equivalent to having the NPU perform the sampling of the adjacency list, realizing the fusion of operators and reducing computing time and memory overhead.

[0009] In a feasible embodiment, the random number generator is a CPU or a domain specific accelerator (DSA).

[0010] In a feasible embodiment, the value range of the K random numbers is [0, N-1]. In terms of reordering the second adjacency list according to the K random numbers to obtain the third adjacency list, the NPU is specifically used to:

[0011] According to the K random numbers, K first vectors are obtained from the second adjacency list, the K first vectors correspond to the K random numbers one by one, the elements of the j-th first vector among the K first vectors include the elements of the i-th row in the second adjacency list, and the value of i is the value of the random number corresponding to the j-th first vector among the K random numbers; the K first vectors are arranged in the order in which the K random numbers are generated to obtain a third adjacency list.

[0012] In a feasible embodiment, the random number generator generates K random numbers before the NPU performs a row-column transposition on the first adjacency list, so that when sampling the adjacency list multiple times, the random number generation process can be hidden in the calculation process of the NPU, further reducing the sampling time.

[0013] In a feasible embodiment, in terms of obtaining the target adjacency list according to the third adjacency list, the NPU is specifically configured to:

[0014] The third adjacency list is transposed to obtain a fourth adjacency list, and the size of the fourth adjacency list is M*K; when K is equal to N, the fourth adjacency list is matrix-sliced ​​to obtain a target adjacency list; when K is less than N, the fourth adjacency list is the target adjacency list, and S is equal to K.

[0015] As for the number of random numbers, the random number generator can generate according to user instructions, and the number of random numbers is S; it can also generate according to the default method, and the number of random numbers is N.

[0016] In a second aspect, an embodiment of the present application provides an adjacency table sampling method, which is applied to an AI chip, wherein the AI ​​chip includes a random number generator and an NPU, including:

[0017] The random number generator generates K random numbers; the NPU obtains a first adjacency list, the size of the first adjacency list is M*N, and M and N are both integers greater than 0; the NPU transposes the first adjacency list to obtain a second adjacency list, and the size of the second adjacency list is N*M; the NPU shuffles the second adjacency list according to the K random numbers to obtain a third adjacency list, and the size of the third adjacency list is K*M; the NPU obtains a target adjacency list according to the third adjacency list, and the size of the target adjacency list is M*S, and S is an integer less than N.

[0018] In a feasible embodiment, the value range of the K random numbers is [0, N-1], and the NPU reorders the second adjacency list according to the K random numbers to obtain a third adjacency list, including:

[0019] The NPU obtains K first vectors from the second adjacency list based on the K random numbers. The K first vectors correspond one-to-one to the K random numbers. The elements in the j-th first vector among the K first vectors include the elements in the i-th row of the second adjacency list, and the value of i is the value of the random number corresponding to the j-th first vector among the K random numbers. The NPU arranges the K first vectors in the order in which the K random numbers are generated to obtain a third adjacency list.

[0020] In a feasible embodiment, the random number generator generates K random numbers before the NPU transposes the rows and columns of the first adjacency list.

[0021] In a feasible embodiment, the NPU obtains the target adjacency list according to the third adjacency list, including:

[0022] The third adjacency list is transposed to obtain a fourth adjacency list, and the size of the fourth adjacency list is M*K; when K is equal to N, the fourth adjacency list is matrix-sliced ​​to obtain a target adjacency list; when K is less than N, the fourth adjacency list is the target adjacency list, and S is equal to K.

[0023] In a third aspect, an embodiment of the present application provides an adjacency table sampling device, which has the function of implementing the second aspect, and the function can be implemented by hardware or by hardware executing corresponding software. The hardware or software includes one or more modules corresponding to the above functions.

[0024] In a fourth aspect, an embodiment of the present application provides a computer program product, comprising computer instructions, which, when executed on an electronic device, enables the electronic device to execute part or all of the method described in the second aspect.

[0025] In a fifth aspect, an embodiment of the present application provides a computer storage medium for storing computer software instructions used for the AI ​​chip described in the first aspect or the adjacency list sampling device described in the third aspect, which includes a program designed for executing the above aspects. BRIEF DESCRIPTION OF THE DRAWINGS

[0026] In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art are briefly introduced below.

[0027] Figure 1a It is a schematic diagram of an adjacency list sampling process;

[0028] Figure 1b A schematic diagram of an adjacency list sampling algorithm running on a CPU;

[0029] Figure 1c A schematic diagram of an adjacency list sampling algorithm running on a CPU+GPU architecture;

[0030] Figure 2a A schematic diagram of the architecture of an AI chip provided in an embodiment of the present application;

[0031] Figure 2b A schematic diagram of an adjacency table sampling process based on an AI chip provided in an embodiment of the present application;

[0032] Figure 2c A schematic diagram of another adjacency table sampling process based on an AI chip provided in an embodiment of the present application;

[0033] Figure 2d A schematic diagram of the execution order of each operation for performing two adjacency list samplings;

[0034] Figure 2e A schematic diagram of the architecture of another AI chip provided in an embodiment of the present application;

[0035] Figure 3a A schematic diagram of another adjacency table sampling process based on an AI chip provided in an embodiment of the present application;

[0036] Figure 3b A schematic diagram of another adjacency table sampling process based on an AI chip provided in an embodiment of the present application;

[0037] Figure 4 A flowchart of an adjacency list sampling method provided in an embodiment of the present application. DETAILED DESCRIPTION

[0038] In order to enable those skilled in the art to better understand the solution of the present application, the technical solution in the embodiments of the present application will be clearly and completely described below in conjunction with the drawings in the embodiments of the present application.

[0039] First, some terms of this application are explained.

[0040] An adjacency list is a storage structure of a graph. Each vertex of a graph has an adjacency list. An adjacency list is a linear list. The adjacency list of a vertex v in a graph contains all the vertices adjacent to the vertex v.

[0041] The size of the adjacency list specifically refers to the size of the matrix when the adjacency list is stored in the form of a matrix, so the adjacency list can also be regarded as a matrix.

[0042] See first Figure 2a , Figure 2a A schematic diagram of an AI chip architecture provided in an embodiment of the present application. Figure 2a As shown, the AI ​​chip includes a random number generator 201 and a neural network processor 202, wherein the random number generator 201 is connected to the neural network processor 202;

[0043] A random number generator 201 is used to generate K random numbers, where K is an integer greater than 0 and the value range of the K random numbers is [0, N];

[0044] The neural network processor 202 is used to perform row-column transposition on the input first adjacency list to obtain a second adjacency list, the size of the first adjacency list is M*N, and the size of the second adjacency list is N*M; the second adjacency list is shuffled according to K random numbers to obtain a third adjacency list, the size of the third adjacency list is K*M; and a target adjacency list is obtained according to the third adjacency list, the size of the target adjacency list is M*S, and S is an integer less than N.

[0045] Optionally, the random number generator 201 may be a CPU or a DSA.

[0046] In a feasible embodiment, in the aspect of reordering the second adjacency list according to K random numbers to obtain the third adjacency list, the neural network processor 220 is specifically configured to:

[0047] According to the K random numbers, K first vectors are obtained from the second adjacency list, the K first vectors correspond to the K random numbers one by one, the elements of the j-th first vector among the K first vectors include the elements of the i-th row in the second adjacency list, and the value of i is the value of the random number corresponding to the j-th first vector among the K random numbers; the K first vectors are arranged in the order in which the K random numbers are generated to obtain a third adjacency list.

[0048] Specifically, the neural network processor 220 obtains K first vectors from the second adjacency list according to K random numbers, specifically including: the neural network processor 220 maps each of the K random numbers to a random number address, which is the storage address of the element in the first column of the second adjacency list in the cache; the neural network processor 220 obtains the first vector corresponding to the random number from the cache according to the random address and the number of columns of the second adjacency list, and then obtains K first vectors.

[0049] It should be noted here that the values ​​of the K random numbers may be partially the same or may be different from each other.

[0050] For example, assume that the second adjacency list is: The random number generator 201 generates three random numbers, which are 2, 0 and 1 respectively. The neural network processor 202 obtains three first vectors from the second adjacency list according to the three random numbers. The three first vectors are: (C1, C2, C3, C4), (A1, A2, A3, A4) and (B1, B2, B3, B4). The three first vectors are arranged in the order in which the three random numbers are generated to obtain a third adjacency list. The third adjacency list can be expressed as:

[0051] In a feasible embodiment, in terms of obtaining the target adjacency list according to the third adjacency list, the neural network processor 220 is specifically configured to:

[0052] The third adjacency list is transposed to obtain a fourth adjacency list, the size of which is M*K; when K is equal to N, the fourth adjacency list is matrix-sliced ​​to obtain a target adjacency list; when K is less than N, the fourth adjacency list is the target adjacency list, and S is equal to K.

[0053] The random number generator 201 may generate K random numbers in a default manner, where K is equal to N. The size of the fourth adjacency list obtained by the neural network processor 202 in the above manner is M*N. Since it is necessary to segment the fourth adjacency list information matrix to implement adjacency list sampling, the above target adjacency list is obtained, such as Figure 2bAs shown; when the random number generator 201 can also generate K random numbers based on the user's instructions, K is the number of columns of the adjacency list obtained by sampling, so the fourth adjacency list obtained in the above manner is the above target adjacency list, such as Figure 2c shown.

[0054] In other words, when the user does not indicate the number of random numbers generated by the random number generator 201, the random number generator 201 will generate a default number of random numbers in a default manner. Assuming that the size of the first adjacency list is M*N, the default number here is generally N; when the user determines that the size of the adjacency list after sampling is M*S, the user will inform the random number generator 201 of S, and then the random number generator 201 generates S random numbers. It can be seen from the above description that the size of the third adjacency list obtained by shuffling is determined by the number of random numbers, so it can be determined whether to perform matrix segmentation based on the number of random numbers generated by the random number generator 201.

[0055] In a feasible embodiment, the random number generator 201 generates K random numbers before the neural network processor 202 performs a row-column transposition on the first adjacency list. For the random number generator and the NPU, the sampling of the adjacency list is repeated. Figure 2d The following diagram shows the execution order of the sampling operations for two adjacency lists, such as Figure 2d As shown, after the NPU executes the second row-column transposition operation, the NPU executes the third row-column transposition operation to enter the next adjacency list sampling. Since the generation of random numbers is relatively time-consuming for the random number generator, and the execution of the row-column transposition operation is relatively time-consuming for the NPU; in order to reduce the time consumption, before the NPU executes the third row-column transposition operation, the random number generator executes the second random number generation operation, that is, while the NPU is executing the first shuffle operation and the second row-column transposition operation, or while the NPU is executing the second row-column transposition operation, the random number generator executes the second random number generation operation, which is equivalent to hiding the entire random number generation process in the NPU calculation process, thereby reducing the sampling time of the adjacency list.

[0056] In a feasible embodiment, the neural network processor 202 can implement the above process through a graph neural network, wherein the number K of random numbers is a hyperparameter of the graph neural network, the first adjacency list and K random numbers are inputs of the graph neural network, and the neural network processor 202 calls the graph neural network to execute the actions performed by the neural network processor 202, thereby obtaining the above target adjacency list.

[0057] In an optional embodiment, if Figure 2eAs shown, the AI ​​chip 200 includes a neural network processor 202, a central processing unit 203 and a domain-specific accelerator 204; wherein the central processing unit 203 and the domain-specific accelerator 204 are both connected to the neural network processor 202; the central processing unit 203 or the domain-specific accelerator 204 is used to generate the above-mentioned K random numbers; the specific implementation process of the neural network processor 202 can be found in Figure 2a The relevant description will not be given here.

[0058] In a specific example, Figure 3a and Figure 3b As shown, the CPU or DSA in the AI ​​chip generates 25 random numbers and saves the 25 random numbers to a high bandwidth memory. memory, HBM) / DDR; NPU obtains 25 random numbers from HBM and maps the 25 random numbers to 25 random addresses; NPU obtains the first adjacency list from HBM / DDR, the scale of the first adjacency list is 5120*100, and the first adjacency list is transposed to obtain the second adjacency list, and the above 25 random addresses are the storage addresses of the elements in the first column of the second adjacency list in the cache; NPU reads 25 first vectors from the cache according to the 25 random addresses and 5120; then, based on the generation order of the 25 random numbers, the 25 first vectors are arranged to obtain the third adjacency list, which process can also be called random transport, the scale of the third adjacency list is 25*5120, and then the third adjacency list is transposed to obtain the fourth adjacency list, the scale of the fourth adjacency list is 5120*25, and the fourth adjacency list is the target adjacency list, that is, the adjacency list obtained after sampling.

[0059] It can be seen that in the scheme of the embodiment of the present application, based on the characteristics of the AI ​​chip structure, the adjacency list sampling process is redesigned: random numbers are generated by the CPU or DSA, and the rows and columns of the adjacency list are transposed and reordered by the NPU, which is equivalent to the NPU performing the sampling of the adjacency list, realizing the fusion of operators, reducing the calculation time and memory overhead, and hiding the generation of random numbers in the calculation process of the NPU, thereby further reducing the sampling time.

[0060] See also Figure 4 , Figure 4 A flow chart of an adjacency table sampling method provided in an embodiment of the present application. The method is applied to an AI chip, and the AI ​​chip includes a random number generator and an NPU. Figure 4 As shown, the method includes:

[0061] S401. A random number generator generates K random numbers.

[0062] S402: The NPU obtains a first adjacency list, and performs row-column transposition on the first adjacency list to obtain a second adjacency list.

[0063] In a feasible embodiment, the value range of the K random numbers is [0, N-1], and the NPU reorders the second adjacency list according to the K random numbers to obtain a third adjacency list, including:

[0064] The NPU obtains K first vectors from the second adjacency list based on the K random numbers. The K first vectors correspond one-to-one to the K random numbers. The elements in the j-th first vector among the K first vectors include the elements in the i-th row of the second adjacency list, and the value of i is the value of the random number corresponding to the j-th first vector among the K random numbers. The NPU arranges the K first vectors in the order in which the K random numbers are generated to obtain a third adjacency list.

[0065] In a feasible embodiment, the random number generator generates K random numbers before the NPU transposes the rows and columns of the first adjacency list.

[0066] In a feasible embodiment, the NPU obtains the target adjacency list according to the third adjacency list, including:

[0067] The third adjacency list is transposed to obtain a fourth adjacency list, and the size of the fourth adjacency list is M*K; when K is equal to N, the fourth adjacency list is matrix-sliced ​​to obtain a target adjacency list; when K is less than N, the fourth adjacency list is the target adjacency list, and S is equal to K.

[0068] S403 . The NPU reorders the second adjacency list according to the K random numbers to obtain a third adjacency list; and obtains a target adjacency list according to the third adjacency list.

[0069] It should be noted that the specific implementation process of steps S401-S403 can be found in the above Figure 2a , Figure 2b , Figure 2c , Figure 2d , Figure 3a and Figure 3b The relevant description of the illustrated embodiment will not be repeated here.

[0070] The present application also provides a computer storage medium, wherein the computer storage medium may store a program, and when the program is executed, some or all of the steps of any adjacency table sampling method recorded in the above method embodiment may be implemented. The aforementioned storage medium includes: a USB flash drive, a read-only memory (English: read-only memory), a random access memory (English: random access memory, RAM), a mobile hard disk, a magnetic disk or an optical disk, and other media that can store program codes.

[0071] It should be noted that, for the aforementioned method embodiments, for the sake of simplicity, they are all expressed as a series of action combinations, but those skilled in the art should be aware that the present application is not limited by the described order of actions, because according to the present application, certain steps can be performed in other orders or simultaneously. Secondly, those skilled in the art should also be aware that the embodiments described in the specification are all preferred embodiments, and the actions and modules involved are not necessarily required by the present application.

[0072] In the above embodiments, the description of each embodiment has its own emphasis. For parts that are not described in detail in a certain embodiment, reference can be made to the relevant descriptions of other embodiments.

[0073] In the several embodiments provided in the present application, it should be understood that the disclosed device can be implemented in other ways. For example, the device embodiments described above are only schematic, such as the division of the units, which is only a logical function division. There may be other division methods in actual implementation, such as multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some interfaces, and the indirect coupling or communication connection of the device or unit can be electrical or other forms.

[0074] The units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed on multiple network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0075] As described above, the above embodiments are only used to illustrate the technical solutions of the present application, rather than to limit them. Although the present application has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or make equivalent replacements for some of the technical features therein. However, these modifications or replacements do not cause the essence of the corresponding technical solutions to deviate from the scope of the technical solutions of the embodiments of the present application.

Claims

1. An artificial intelligence AI chip, characterized in that: The AI ​​chip includes a random number generator and a neural network processor NPU, wherein the CPU is connected to the NPU; The random number generator is used to generate K random numbers; The NPU is used to perform row-column transposition on the input first adjacency list to obtain a second adjacency list, the scale of the first adjacency list is M*N, and both M and N are integers greater than 0; the scale of the second adjacency list is N*M; the second adjacency list is randomly rearranged according to the K random numbers to obtain a third adjacency list, and the scale of the third adjacency list is K*M; a target adjacency list is obtained according to the third adjacency list, and the scale of the target adjacency list is M*S, and S is an integer less than N.

2. The chip according to claim 1, characterized in that: The random number generator is a central processing unit CPU or a domain specific accelerator DSA.

3. The chip according to claim 1 or 2, characterized in that: The random number generator generates the K random numbers before the NPU transposes the first adjacency list.

4. The chip according to claim 3, characterized in that: The value range of the K random numbers is [0, N-1]. In the aspect of reordering the second adjacency list according to the K random numbers to obtain a third adjacency list, the NPU is specifically used to: According to the K random numbers, K first vectors are obtained from the second adjacency list, wherein the K first vectors correspond to the K random numbers one-to-one, the elements in the j-th first vector among the K first vectors include the elements in the i-th row of the second adjacency list, and the value of i is the value of the random number corresponding to the j-th first vector among the K random numbers; The K first vectors are arranged in the order in which the K random numbers are generated to obtain the third adjacency list.

5. The chip according to claim 4, characterized in that: In the aspect of obtaining the target adjacency list according to the third adjacency list, the NPU is specifically used for: Transpose the third adjacency list in rows and columns to obtain a fourth adjacency list, where the size of the fourth adjacency list is M*K; When K is equal to N, performing matrix segmentation on the fourth adjacency list to obtain the target adjacency list; When K is less than N, the fourth adjacency list is the target adjacency list, and S is equal to K.

6. A method for sampling an adjacency list, characterized in that: The method is applied to an artificial intelligence (AI) chip, wherein the AI ​​chip includes a random number generator and a neural network chip, and the method includes: The random number generator generates K random numbers; The NPU obtains a first adjacency list, where the size of the first adjacency list is M*N, where both M and N are integers greater than 0; The NPU performs row-column transposition on the first adjacency list to obtain a second adjacency list, where the size of the second adjacency list is N*M; The NPU rearranges the second adjacency list in random order according to the K random numbers to obtain a third adjacency list, where the size of the third adjacency list is K*M; The NPU obtains a target adjacency list according to the third adjacency list, the size of the target adjacency list is M*S, and S is an integer less than N.

7. The method according to claim 6, characterized in that The random number generator generates the K random numbers before the NPU transposes the first adjacency list.

8. The method according to claim 6 or 7, characterized in that: The value range of the K random numbers is [0, N-1], and the NPU rearranges the second adjacency list according to the K random numbers to obtain a third adjacency list, including: The NPU obtains K first vectors from the second adjacency list according to the K random numbers, the K first vectors corresponding to the K random numbers one-to-one, the elements in the j-th first vector among the K first vectors include the elements in the i-th row of the second adjacency list, and the value of i is the value of the random number corresponding to the j-th first vector among the K random numbers; The NPU arranges the K first vectors according to the order in which the K random numbers are generated to obtain the third adjacency list.

9. The method according to claim 8, characterized in that The NPU obtains a target adjacency list according to the third adjacency list, including: Transpose the third adjacency list in rows and columns to obtain a fourth adjacency list, where the size of the fourth adjacency list is M*K; When K is equal to N, performing matrix segmentation on the fourth adjacency list to obtain the target adjacency list; When K is less than N, the fourth adjacency list is the target adjacency list, and S is equal to K.

Images