Large model calculation device and method based on weight topological connection

Abstract

The invention discloses a large model calculation device based on weight topological connection, and the calculation device comprises a plurality of hardware neuron HN modules based on weight topological connection, and the HN modules complete weight value matrix multiplication by adopting a mode based on weight topological connection for a multi-dimensional vector generated by a user input statement; wherein based on the weight topological connection, the weight value of the matrix is represented in a connection mode of the HN module circuit. The invention provides a high-efficiency large model calculation device HN (Hard Neural) based on weight topological connection, which does not need a storage unit and finishes weight numerical value operation by topological connection of combinational logic.

Description

Technical Field

[0001] The present application relates to the field of large model hardware acceleration, and in particular to a large model computing device and method based on weight topological connection. Background Art

[0002] At present, with the rapid development of artificial intelligence technology, the importance of large models in various fields has become increasingly prominent. However, the operation of large models is often accompanied by huge energy consumption, especially when performing inference calculations. The high energy consumption problem has become a bottleneck restricting its further development.

[0003] Although large language models have made significant progress in the field of natural language processing, they still face many challenges in practical applications, especially in terms of computing resources and energy consumption. The main challenges include:

[0004] Energy consumption issue: Large language models usually require a lot of computing resources for training and inference, which leads to high energy and electricity consumption and has an adverse impact on the environment.

[0005] Inference Computing Optimization: To improve throughput and reduce power consumption, the industry is exploring technologies such as in-memory computing and near-memory computing. These methods aim to reduce the energy consumption of data transmission and improve the inference efficiency of models. However, as the size of models continues to increase, the on-chip space of near-memory / in-memory may be insufficient, limiting the application potential of these technologies.

[0006] In summary, the above defects not only lead to the increasing size of large models, but also insufficient on-chip space in near memory / internal memory; the existing software technology has limited improvement in energy efficiency and requires changes in the underlying design of the hardware.

[0007] Therefore, there is an urgent need to propose an efficient large-model computing device based on weight topological connection, which does not require storage units and completes weight numerical calculations through topological connections of combinatorial logic to achieve more efficient energy efficiency pursuit. Summary of the invention

[0008] In order to solve the problems of the above-mentioned existing technologies, such as the increasing size of large models, insufficient on-chip space in near memory / internal memory, and limited improvement in energy efficiency with existing software technologies, the present invention proposes an efficient large model computing device and method based on weight topological connection.

[0009] In a first aspect, an embodiment of the present application provides a large model computing device based on weight topological connection, which is applied to large model operation, and the computing device includes: a plurality of hardware neuron HN modules based on weight topological connection, and for a multidimensional vector generated by a user input sentence, the HN module uses a weight topological connection method to complete a weight value matrix multiplication operation;

[0010] Among them, the weight-based topological connection is to represent the weight value of the matrix through the connection method of the HN module circuit.

[0011] In a specific embodiment of the present invention, the above-mentioned HN module further includes: a plurality of hardware matrix multiplication units H-MXU, which are used to perform weight value matrix multiplication operations;

[0012] The hardware matrix multiplication unit H-MXU further comprises: a bit serial module BSU and a plurality of hardware vector multiplication modules H-VXU, and the weights in the matrix are stored in the topology module of the H-VXU in a topological connection manner.

[0013] In a specific embodiment of the present invention, the HN module is configured to perform the following steps:

[0014] The matrix multiplication for each multidimensional vector x is expressed as: x*Q k ; where x is a column vector of length n and each element is q bits wide, Q1...Q k is an m*n dimensional matrix of k layers, where k is the number of layers in the large model;

[0015] For the input multidimensional vector x, select any one bit in the column vector x through the selector, denoted as x i ;

[0016] The x i In order, connected to the matrix Q k The corresponding position of each row vector, if the matrix Q k There is a weight value a in the i-th row and j-th column. The HN module is connected through a circuit, and the j-th pin of the i-th H-MXU module is connected to the Wallace tree module marked as a for matrix multiplication, wherein i and j represent the row number and column number of the matrix respectively, and a is the matrix weight value.

[0017] In a specific embodiment of the present invention, the BSU includes a plurality of bit multiplexers BM, and the H-VXU includes a plurality of merging units MGU;

[0018] The input multidimensional vector x in BM is split bit by bit, and x is broadcast to multiple H-VXUs;

[0019] The topology module in the H-VXU receives the bit-split input vector x and outputs the calculation result to the merging unit MGU;

[0020] Each MGU contains multiple layer multiplexers LM and Wallace tree modules. The calculation results are merged in each MGU and multiplied with the BFV value;

[0021] The output results of multiple MGUs are summed through the addition tree to calculate the multiplication result of the weight vector of the input vector x.

[0022] In a second aspect, an embodiment of the present application provides a large model calculation method based on weight topological connection, using a large model calculation device based on weight topological connection as described above, applied to large model calculation, the method comprising:

[0023] Topological connection calculation step: for the multidimensional vector generated by the user input sentence, multiple hardware neuron HN modules based on weight topological connection use the weight topological connection method to complete the weight value matrix multiplication operation;

[0024] Among them, the weight-based topological connection is to represent the weight value of the matrix through the connection method of the HN module circuit.

[0025] In a specific embodiment of the present invention, the above-mentioned HN module further includes: a plurality of hardware matrix multiplication units H-MXU, which are used to perform weight value matrix multiplication operations;

[0026] The hardware matrix multiplication unit H-MXU further comprises: a bit serial module BSU and a plurality of hardware vector multiplication modules H-VXU, and the weights in the matrix are stored in the topology module of the H-VXU in a topological connection manner.

[0027] In a specific embodiment of the present invention, the above topological connection calculation step includes:

[0028] The matrix multiplication for each multidimensional vector x is expressed as: x*Q k ; where x is a column vector of length n and each element is q bits wide, Q1...Q k is an m*n dimensional matrix of k layers, where k is the number of layers in the large model;

[0029] For the input multidimensional vector x, select any one bit in the column vector x through the selector, denoted as x i ;

[0030] The x i In order, connected to the matrix Q k The corresponding position of each row vector, if the matrix Q k There is a weight value a in the i-th row and j-th column. The HN module is connected through a circuit, and the j-th pin of the i-th H-MXU module is connected to the Wallace tree module marked as a for matrix multiplication, wherein i and j represent the row number and column number of the matrix respectively, and a is the matrix weight value.

[0031] In a specific embodiment of the present invention, the BSU includes a plurality of bit multiplexers BM, and the H-VXU includes a plurality of merging units MGU;

[0032] The input multidimensional vector x in BM is split bit by bit, and x is broadcast to multiple H-VXUs;

[0033] The topology module in the H-VXU receives the bit-split input vector x and outputs the calculation result to the merging unit MGU;

[0034] Each MGU includes multiple layer multiplexers LM and Wallace tree modules. The calculation results are combined in each MGU and multiplied with the BFV value, where the value range of the BFV multiplicand is between 0 and 13;

[0035] The output results of multiple MGUs are summed through the addition tree to calculate the multiplication result of the weight vector of the input vector x.

[0036] In a third aspect, an embodiment of the present application provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the steps of the large model calculation method based on weight topological connection.

[0037] In a fourth aspect, an electronic device includes a memory, a processor, and a large model computing device based on weight topological connection, wherein the large model computing device based on weight topological connection includes: a plurality of hardware neuron HN modules based on weight topological connection.

[0038] Compared with the related prior art, it has the following outstanding beneficial effects:

[0039] 1) The method of the present invention proposes an efficient large model computing device HN (HardNeural) based on weight topological connection, which does not require a storage unit and completes weight numerical calculations with a combinatorial logic topological connection;

[0040] 2) The method of the present invention proposes that the HN module is connected based on weight topology, which transforms the vector matrix multiplication at the software level into the topological connection of the combinatorial logic at the hardware level, thereby reducing the computational complexity and effectively improving the energy consumption ratio;

[0041] 3) The method of the present invention proposes that the HN module is connected based on weight topology, without the need to store matrix parameters, which significantly reduces the demand for storage. BRIEF DESCRIPTION OF THE DRAWINGS

[0042] The drawings described herein are used to provide a further understanding of the present application and constitute a part of the present application. The illustrative embodiments of the present application and their descriptions are used to explain the present application and do not constitute an improper limitation on the present application. In the drawings:

[0043] Figure 1 It is a schematic diagram of the reasoning process of the large model in the prior art;

[0044] Figure 2This is a schematic diagram of HN connection of the present invention;

[0045] Figure 3 It is a schematic diagram of the H-MXU structure of the present invention;

[0046] Figure 4 This is a schematic diagram of the weight topology connection process of the present invention;

[0047] Figure 5 It is a schematic diagram of the H-VXU structure of the present invention;

[0048] Figure 6 It is a hardware schematic diagram of the electronic device of the present invention. DETAILED DESCRIPTION

[0049] It should be noted that the processor described in the present invention is the control center of the electronic device, which may be a processor or a general term for multiple processing elements. For example, it may be one or more central processing units (CPUs), or application specific integrated circuits (ASICs), or one or more integrated circuits configured to implement the embodiments of the present invention, such as one or more digital signal processors (DSPs), or one or more field programmable gate arrays (FPGAs).

[0050] Optionally, the processor can perform various functions of the electronic device by running or executing a software program stored in the memory, and calling data stored in the memory.

[0051] In a specific implementation, as an embodiment, the processor may include one or more CPUs. Each of these processors may be a single core processor or a multi-core processor. The processor here may refer to one or more devices, circuits, and / or processing cores for processing data (e.g., computer program instructions). Electronic devices may include: servers, desktop computers, laptops, smart phones, tablet computers, embedded computers, etc., wherein the embedded computers include vehicles and robots, etc.

[0052] The memory is used to store the software program for executing the solution of the present invention, and the execution is controlled by the processor. The specific implementation method can refer to the above method embodiment and will not be repeated here.

[0053] It should be noted that the structure of the electronic device shown in the drawings of the present invention does not constitute a limitation thereto, and the actual knowledge structure recognition device may include more or fewer components than shown in the drawings, or a combination of certain components, or a different arrangement of components.

[0054] The above embodiments can be implemented in whole or in part by software, hardware (such as circuits), firmware or any other combination. When implemented by software, the above embodiments can be implemented in whole or in part in the form of a computer program product. The computer program product includes one or more computer instructions or computer programs. When the computer instructions or computer programs are loaded or executed on a computer, the process or function described in the embodiment of the present invention is generated in whole or in part. The computer can be a general-purpose computer, a special-purpose computer, a computer network, or other programmable device. The computer instructions can be stored in a computer-readable storage medium, or transmitted from one computer-readable storage medium to another computer-readable storage medium. For example, the computer instructions can be transmitted from one website, computer, server or data center to another website, computer, server or data center by wired (such as infrared, wireless, microwave, etc.). The computer-readable storage medium can be any available medium that can be accessed by a computer or a data storage device such as a server or data center that contains one or more available media sets. The available medium can be a magnetic medium (for example, a floppy disk, a hard disk, a tape), an optical medium (for example, a DVD), or a semiconductor medium. The semiconductor medium can be a solid-state hard disk.

[0055] It should also be understood that the term "and / or" in this article is only a description of the association relationship of associated objects, indicating that there can be three relationships. For example, A and / or B can represent: A exists alone, A and B exist at the same time, and B exists alone. A and B can be singular or plural. In addition, the character " / " in this article generally indicates that the associated objects before and after are in an "or" relationship, but it may also indicate an "and / or" relationship. Please refer to the context for specific understanding.

[0056] In the present invention, "at least one" means one or more, and "more than one" means two or more. "At least one of the following" or similar expressions refers to any combination of these items, including any combination of single or plural items. For example, at least one of a, b, or c can be represented by: a, b, c, ab, ac, bc, or abc, where a, b, c can be single or multiple.

[0057] It should also be understood that in various embodiments of the present invention, the size of the serial numbers of the above-mentioned processes does not mean the order of execution. The execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiments of the present invention.

[0058] In the several embodiments provided by the present invention, it should be understood that the disclosed devices, apparatuses and methods can be implemented in other ways. For example, the device embodiments described above are only schematic, for example, the division of the units is only a logical function division, and there may be other division methods in actual implementation, such as multiple units or components can be combined or integrated into another device, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some interfaces, indirect coupling or communication connection of devices or units, which can be electrical, mechanical or other forms.

[0059] The units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed on multiple network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0060] In addition, each functional unit in each embodiment of the present invention may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit.

[0061] If the functions are implemented in the form of software functional units and sold or used as independent products, they can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention can be essentially or partly embodied in the form of a software product that contributes to the prior art. The computer software product is stored in a storage medium and includes several instructions for enabling a computer device (which can be a personal computer, a server, or a network device, etc.) to perform all or part of the steps of the methods described in various embodiments of the present invention. The aforementioned storage medium includes: various media that can store program codes, such as a USB flash drive, a mobile hard disk, a read-only memory (ROM), a random access memory (RAM), a magnetic disk or an optical disk.

[0062] In order to make the above features and effects of the present invention more clearly and understandably described, embodiments are given below and described in detail with reference to the accompanying drawings. This specification discloses one or more embodiments that include the features of the present invention. The disclosed embodiments are only for illustration. The scope of protection of the present invention is not limited to the disclosed embodiments, and the present invention is defined by the attached claims.

[0063] The following is a system embodiment corresponding to the above method embodiment. This embodiment can be implemented in conjunction with the above embodiment. The relevant technical details mentioned in the above embodiment are still valid in this embodiment. In order to reduce repetition, they are not repeated here. Accordingly, the relevant technical details mentioned in this embodiment can also be applied in the above embodiment.

[0064] The present invention aims to propose an efficient large model computing device HN (Hard Neural) based on weight topological connection, which does not require a storage unit and completes weight numerical calculations with a topological connection of combinatorial logic.

[0065] In a large language model, the sentence input by the user needs to be embedded into an n-dimensional vector space first, and then the n-dimensional vector is subjected to k layers of matrix multiplication and other operations to obtain the final output vector, and then the output vector is mapped back to natural language. The existing large language model calculation process is as follows: Figure 1 The present invention accelerates the operation of matrix multiplication in the process and reduces power consumption by introducing an HN module based on weight topological connection.

[0066] The following is a detailed description with reference to specific embodiments:

[0067] Embodiment 1

[0068] like Figure 2 As shown, the embodiment of the present application provides a large model computing device based on weight topological connection, which is applied to large model operation, and the computing device includes: a plurality of hardware neurons HN based on weight topological connection 1， HN 2...， HN n Module 100, for the multidimensional vector generated by the user input sentence, the HN module 100 uses a weight value topological connection method to complete the weight value matrix multiplication operation; wherein, the weight value of the matrix is ​​represented by the connection method of the HN module 100 circuit.

[0069] The HN module transforms the vector-matrix multiplication at the software level into a topological connection of combinatorial logic at the hardware level, and represents the weight of the matrix through circuit wiring. For example, if there is a weight a in the i-th row and j-th column of the matrix, then the j-th pin of the i-th vector module (H-MXU) is connected to the Wallace tree module (a module that can efficiently perform addition operations) marked with a through circuit wiring.

[0070] like Figure 3 As shown, in a specific embodiment of the present invention, the above-mentioned HN module 100 further includes: a plurality of hardware matrix multiplication units H-MXU 101, which are used to complete the weight value matrix multiplication operation; the hardware matrix multiplication unit H-MXU 101 further includes: a one-bit serial module BSU 1011 and a plurality of hardware vector multiplication modules H-VXU 1012, and the weights in the matrix are stored in the topology module of the H-VXU 1012 in a topological connection manner.

[0071] like Figure 4 As shown, in a specific embodiment of the present invention, the HN module 100 is configured to perform the following steps:

[0072] The matrix multiplication for each multidimensional vector x is expressed as: x*Q k ; where x is a column vector of length n and each element is q bits wide, Q1...Q k is an m*n dimensional matrix of k layers, where k is the number of layers in the large model;

[0073] For the input multidimensional vector x, select any one bit in the column vector x through the selector, denoted as x i ;

[0074] The x i In order, connected to the matrix Q k The corresponding position of each row vector, if the matrix Q k There is a weight value a in the i-th row and j-th column. The HN module is connected through a circuit, and the j-th pin of the i-th H-MXU module is connected to the Wallace tree module marked as a for matrix multiplication, wherein i and j represent the row number and column number of the matrix respectively, and a is the matrix weight value.

[0075] like Figure 5 As shown, in a specific embodiment of the present invention, the above-mentioned BSU includes a plurality of bit multiplexers BM, and the H-VXU includes a plurality of merging units MGU;

[0076] The input multidimensional vector x in BM is split bit by bit, and x is broadcast to multiple H-VXUs;

[0077] The topology module in the H-VXU receives the bit-split input vector x and outputs the calculation result to the merging unit MGU;

[0078] Each MGU contains multiple layer multiplexers LM and Wallace tree modules. The calculation results are merged in each MGU and multiplied with the BFV value;

[0079] The output results of multiple MGUs are summed through the addition tree to calculate the multiplication result of the weight vector of the input vector x.

[0080] like Figure 3 As shown, for a column vector x with a length of n and a bit width of q bits per element, and an m*n dimensional matrix Q1...Q k , HN will use the overall hardware matrix multiplication unit H-MXU for calculation. The overall hardware matrix multiplication unit (H-MXU) consists of a bit serial module (BSU) and m hardware vector multiplication modules H-VXU. For the weights in the matrix, the topological connection mentioned above will be used to store them in the Topology Module of the H-VXU.

[0081] For example, Figure 3 As shown, for a vector x with a bit width of 8 bits and a length of 1536, and a 512*1536-dimensional matrix Q1...Q with 52 layers k , HN will use the overall hardware matrix multiplication unit H-MXU, first select the component of vector x with a bit width of 1 bit and a length of 1536 through BSU, and then input the component to 512 H-VXUs for multiplication calculation. Each H-VXU will eventually output a 32-bit result.

[0082] like Figure 4 As shown, for a single x*Q k , as shown in the overall process of weight topological connection, for the input x, a single bit in the column vector x is selected by the selector and recorded as x i Then, x i In order, connect to the corresponding position of each row vector of the matrix. And for each row vector in the matrix (called vector), connect to the corresponding numbered Wallace tree according to its weight.

[0083] like Figure 4 As shown, for example, for the above-mentioned vector x with a bit width of 8 bits and a length of 1536, firstly, a component x of the vector x with a length of 1536 such as 101…10 is selected through BSU. i, and then fully connected to each row of vector representing the matrix weight, and for a certain element in the vector, through the weight topology connection, if the weight corresponding to the position is 4, it is connected to the Wallace tree labeled 4; if the weight is 13, it is connected to the Wallace tree labeled 13.

[0084] like Figure 5 As shown in the figure, the BSU contains n q-to-1 Bit Mux (BM), and the input vector is split by bit. Then, the input vector is broadcast to m H-VXUs. In the H-VXU, the topology module receives the input vector after n bit splitting and outputs Sum(sigma(x i ))(xi represents each bit selected by BSU, sigma is the sign function, positive number is 1, 0 is 0, negative number is -1) ports are connected to the merging unit (MGU) #0-#13, which is used to merge the results of calculation. Each MGU contains X n-select-1 Layer Mux (LM) and 1 X-input Bit Serial Wallace Tree (BS-WT) (note that the value of X is different for each MGU). In each MGU, vector elements with the same multiplier are merged and added, and finally multiplied with the possible values ​​of BF data type (BrainFloating, a floating point data type) BFV#0-BFV#13 (excluding the case where BF is 0). 2 q The outputs of -1 MGUs are summed through the addition tree to calculate the multiplication result of the input vector weight vector.

[0085] Embodiment 2

[0086] The embodiment of the present application provides a large model calculation method based on weight topological connection, using the large model calculation device based on weight topological connection as described above, applied to large model calculation, the method includes:

[0087] Topological connection calculation step 201: for the multidimensional vector generated by the user input sentence, multiple hardware neuron HN modules based on weight topological connection use the weight topological connection method to complete the weight value matrix multiplication operation;

[0088] Among them, the weight-based topological connection is to represent the weight value of the matrix through the connection method of the HN module circuit.

[0089] In a specific embodiment of the present invention, the above-mentioned HN module further includes: a plurality of hardware matrix multiplication units H-MXU, which are used to perform weight value matrix multiplication operations;

[0090] The hardware matrix multiplication unit H-MXU further comprises: a bit serial module BSU and a plurality of hardware vector multiplication modules H-VXU, and the weights in the matrix are stored in the topology module of the H-VXU in a topological connection manner.

[0091] In a specific embodiment of the present invention, the topology connection calculation step 201 includes:

[0092] The matrix multiplication for each multidimensional vector x is expressed as: x*Q k ; where x is a column vector of length n and each element is q bits wide, Q1...Q k is an m*n dimensional matrix of k layers, where k is the number of layers of the large model;

[0093] For the input multidimensional vector x, select any one bit in the column vector x through the selector, denoted as x i ;

[0094] The x i In order, connected to the matrix Q k The corresponding position of each row vector, if the matrix Q k There is a weight value a in the i-th row and j-th column. The HN module is connected through a circuit, and the j-th pin of the i-th H-MXU module is connected to the Wallace tree module marked as a for matrix multiplication, wherein i and j represent the row number and column number of the matrix respectively, and a is the matrix weight value.

[0095] In a specific embodiment of the present invention, the BSU includes a plurality of bit multiplexers BM, and the H-VXU includes a plurality of merging units MGU;

[0096] The input multidimensional vector x in BM is split bit by bit, and x is broadcast to multiple H-VXUs;

[0097] The topology module in the H-VXU receives the bit-split input vector x and outputs the calculation result to the merging unit MGU;

[0098] Each MGU contains multiple layer multiplexers LM and Wallace tree modules. The calculation results are merged in each MGU and multiplied with the BFV value;

[0099] The output results of multiple MGUs are summed through the addition tree to calculate the multiplication result of the weight vector of the input vector x.

[0100] Embodiment 3

[0101] An embodiment of the present application provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the steps of the large model calculation method based on weight topological connection.

[0102] Embodiment 4

[0103] An embodiment of the present application provides an electronic device, including a memory, a processor, and a large model computing device based on weight topological connection. The large model computing device based on weight topological connection includes: multiple hardware neuron HN modules based on weight topological connection.

[0104] In addition, combined Figure 1 The large model calculation method based on weighted topological connection described in the embodiment of the present application can be implemented by an electronic device, such as a large model calculation device based on weighted topological connection. Figure 6 Schematic diagram of the hardware structure of a computing device according to an embodiment of the present application.

[0105] In some of these embodiments, the computing device may further include a communication interface 83 and a bus 80. Figure 6 As shown, a large model computing device 81, a memory 82, and a communication interface 83 connected based on weight topology are connected via a bus 80 and communicate with each other.

[0106] Specifically, the weight topologically connected large model computing device 81 may include a plurality of hardware neuron HN modules based on weight topological connections;

[0107] The memory 82 may be used to store or cache various data files that need to be processed and / or used for communication, as well as possible computer program instructions executed by the large model calculation device 81 of weight topological connection.

[0108] The large model calculation device 81 of weighted topological connection reads and executes the computer program instructions stored in the memory 82 to implement any one of the large model calculation methods based on weighted topological connection in the above embodiments.

[0109] The technical features of the above-described embodiments may be arbitrarily combined. To make the description concise, not all possible combinations of the technical features in the above-described embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0110] The above-mentioned embodiments only express several implementation methods of the present application, and the descriptions thereof are relatively specific and detailed, but they cannot be understood as limiting the scope of the invention patent. It should be pointed out that, for a person of ordinary skill in the art, several variations and improvements can be made without departing from the concept of the present application, and these all belong to the protection scope of the present application. Therefore, the protection scope of the patent of the present application shall be subject to the attached claims.

Claims

1. A large model computing device based on weight topological connection, applied to large model computing, characterized in that: The computing device comprises: a plurality of hardware neuron HN modules connected based on weight topology, for the multidimensional vector generated by the user input sentence, the HN modules use the weight topology connection method to complete the weight value matrix multiplication operation; The weight-based topological connection is to represent the weight value of the matrix through the connection of the HN module circuit.

2. The large model computing device based on weight topological connection according to claim 1, characterized in that: The HN module further includes: a plurality of hardware matrix multiplication units H-MXU, for completing weight value matrix multiplication operations; The hardware matrix multiplication unit H-MXU further comprises: a bit serial module BSU and a plurality of hardware vector multiplication modules H-VXU, and the weights in the matrix are stored in the topology modules of the H-VXU in a topological connection manner.

3. The large model computing device based on weight topological connection according to claim 1, characterized in that: The HN module is configured to perform the following steps: The matrix multiplication for each multidimensional vector x is expressed as: x*Q k ; where x is a column vector of length n and each element is qbit wide, Q1...Q k is an m*n dimensional matrix of k layers, where k is the number of layers of the large model; For the input multidimensional vector x, select any one bit in the column vector x through the selector, denoted as x i ; The x i In order, connected to the matrix Q k The corresponding position of each row vector, if the matrix Q k There is a weight value a in the i-th row and j-th column. The HN module is connected through a circuit, and the j-th pin of the i-th H-MXU module is connected to the Wallace tree module marked as a for matrix multiplication, wherein i and j represent the row number and column number of the matrix respectively, and a is the matrix weight value.

4. The large model computing device based on weight topological connection according to claim 2 is characterized in that: The BSU includes a plurality of bit multiplexers BM, and the H-VXU includes a plurality of merging units MGU; The multidimensional vector x input into the BM is split bit by bit, and x is broadcast to a plurality of the H-VXUs; The topology module in the H-VXU receives the bit-split input vector x and outputs the calculation result to the merging unit MGU; Each of the MGUs comprises a plurality of layer multiplexers LM and the Wallace tree module, and the calculation results are combined in each MGU and multiplied with the BFV value; The output results of multiple MGUs are summed through the addition tree to calculate the multiplication result of the weight vector of the input vector x.

5. A large model calculation method based on weighted topological connection, using the large model calculation device based on weighted topological connection as described in claims 1-4, applied to large model calculation, characterized in that: The method comprises: Topological connection calculation step: for the multidimensional vector generated by the user input sentence, the plurality of hardware neuron HN modules based on weight topological connection use a weight topological connection method to complete the weight value matrix multiplication operation; The weight-based topological connection is to represent the weight value of the matrix through the connection of the HN module circuit.

6. The large model calculation method based on weight topological connection according to claim 5 is characterized in that: The HN module further comprises: a plurality of hardware matrix multiplication units H-MXU, for completing weight value matrix multiplication operations; The hardware matrix multiplication unit H-MXU further comprises: a bit serial module BSU and a plurality of hardware vector multiplication modules H-VXU, and the weights in the matrix are stored in the topology module of the H-VXU in a topological connection manner.

7. The large model calculation method based on weight topological connection according to claim 1 is characterized in that: The topology connection calculation step comprises: The matrix multiplication for each multidimensional vector x is expressed as: x*Q k ; where x is a column vector of length n and each element is qbit wide, Q1...Q k is an m*n dimensional matrix of k layers, where k is the number of layers of the large model; For the input multidimensional vector x, select any one bit in the column vector x through the selector, denoted as x i ; The x i In order, connected to the matrix Q k The corresponding position of each row vector, if the matrix Q k There is a weight value a in the i-th row and j-th column. The HN module is connected through a circuit, and the j-th pin of the i-th H-MXU module is connected to the Wallace tree module marked as a for matrix multiplication, wherein i and j represent the row number and column number of the matrix respectively, and a is the matrix weight value.

8. The large model calculation method based on weight topological connection according to claim 6 is characterized in that: The BSU includes a plurality of bit multiplexers BM, and the H-VXU includes a plurality of merging units MGU; The multidimensional vector x input into the BM is split bit by bit, and x is broadcast to a plurality of the H-VXUs; The topology module in the H-VXU receives the bit-split input vector x and outputs the calculation result to the merging unit MGU; Each of the MGUs comprises a plurality of layer multiplexers LM and the Wallace tree module, and the calculation results are combined in each MGU and multiplied with the BFV value; The output results of multiple MGUs are summed through the addition tree to calculate the multiplication result of the weight vector of the input vector x.

9. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the program is executed by a processor, the steps of the large model calculation method based on weighted topological connection described in any one of claims 5 to 8 are implemented.

10. An electronic device comprising a memory, a processor and a large model computing device based on weight topological connection as claimed in any one of claims 1 to 4, characterized in that: The large model computing device based on weight topological connection includes: a plurality of hardware neuron HN modules based on weight topological connection.

Images