Large model reasoning calculation architecture, method and device based on topological calculation

Abstract

The invention provides a large model reasoning calculation architecture based on topological calculation, which comprises a CPU (Central Processing Unit), an SDRAM (Synchronous Dynamic Random Access Memory), a code memory, a cue word memory and a large model reasoning accelerator, and the CPU, the SDRAM, the code memory, the cue word memory and the large model reasoning accelerator are in routing connection through a bus; when large model reasoning calculation is carried out, firstly, codes in the code storage and cue words and word lists in the cue word storage are read into the SDRAM, then the word lists in the SDRAM are inquired according to mark IDs of the cue words to obtain embedded vectors, and then the embedded vectors are sent to the large model reasoning accelerator for large model reasoning calculation. And sending an output result obtained by the large model reasoning accelerator to an upper computer. The invention further provides a large model reasoning calculation method and device based on topological calculation. According to the method, the problems that in the large model reasoning process, the overhead of weight loading is too large, and the overhead of the SRAM implementation area of the KV cache is too large are solved.

Description

Technical Field

[0001] The present invention belongs to the field of deep learning technology, and in particular relates to a large-model inference computing architecture and an inference computing method based on topological computing. Background Art

[0002] In recent years, with the rapid development of deep learning technology, large-scale language models (LLMs) have attracted widespread attention due to their outstanding performance in natural language processing tasks. These models are usually based on the Transformer architecture and can obtain rich semantic understanding and generation capabilities by pre-training on large amounts of text data. However, when trying to apply these models to edge devices, existing technical solutions face a series of challenges.

[0003] At present, the mainstream large-model edge model deployment solutions mainly include technologies such as model pruning, quantization, and distillation. Model pruning reduces the number of model parameters by removing redundant weights and connections; quantization converts floating-point numbers to low-precision integers to reduce storage and computing costs; distillation trains a small model to imitate the behavior of the large model, thereby reducing the model size while maintaining certain performance. These methods have alleviated the problem of limited edge device resources to a certain extent, but there are still obvious difficulties in performing efficient reasoning on large models at the edge.

[0004] The huge scale of parameters in large models leads to huge overhead in weight loading, especially in the context of single-tenant use at the edge, where weight loading becomes the main bottleneck for reasoning. At the same time, the SRAM implementation of KV cache will bring unacceptable area overhead at the edge. Summary of the invention

[0005] In view of the above problems, the present invention proposes a large model reasoning computing architecture based on topological computing, including: a CPU, an SDRAM, a code memory, a prompt word memory, and a large model reasoning accelerator, and the CPU, the SDRAM, the code memory, the prompt word memory and the large model reasoning accelerator are routed and connected through a bus; when performing large model reasoning computing, first, the code in the code memory and the prompt word and the vocabulary in the prompt word memory are read to the SDRAM, and then the vocabulary in the SDRAM is queried according to the tag ID of the prompt word to obtain an embedded vector, and then the embedded vector is sent to the large model reasoning accelerator for large model reasoning computing, and the final output result obtained by the large model reasoning accelerator is sent to the host computer.

[0006] The large model reasoning computing architecture described in the present invention, wherein the large model reasoning accelerator includes: an Emb module, used to obtain the embedded vector and send it to the Reg register; an Attn module, used to obtain the embedded vector from the Reg register to perform a large model reasoning calculation based on a self-attention mechanism; an HN module, used to obtain the embedded vector from the Reg register to perform the FFN layer calculation of the large model reasoning calculation; a VEX module, used to obtain the embedded vector from the Reg register to perform vector calculation of the large model reasoning calculation; a Sample module, used to sample and generate a tag ID for the output result; a Reg register, used to store the embedded vector; an output tag buffer, used to cache and output the output The tag ID of the output result; wherein the Emb module is electrically connected to the bus, and receives and sends data, the Emb module is electrically connected to the Reg register, and sends data to the Reg register; the Reg register is electrically connected to the HN module, the Attn module and the VEX module, and receives and sends data to the HN module, the Attn module and the VEX module, the HN module is electrically connected to the Sample module, and sends data to the Sample module; the Sample module is electrically connected to the Emb module and the output tag buffer, and sends data to the Emb module and the output tag buffer; the output tag buffer is electrically connected to the bus, and sends data to the bus.

[0007] The large model inference computing architecture described in the present invention, wherein the code memory is SPI Flash, the prompt word memory is SD Card, and the bus is an AXI4 bus.

[0008] The present invention also proposes a large model reasoning calculation method based on topological calculation, which adopts the large model reasoning calculation architecture as described above to perform large model reasoning calculation, including: reading code from the code memory and sending it to SDRAM; reading the prompt words and vocabulary in the prompt word memory to the SDRAM; querying the vocabulary in the SDRAM according to the tag ID of the prompt word to obtain an embedded vector, and sending the embedded vector to the large model reasoning accelerator; performing large model reasoning calculation and obtaining the final output result, and sending the final output result to the host computer.

[0009] The large model reasoning calculation method of the present invention, wherein the step of performing large model reasoning calculation on the large model reasoning accelerator includes: a pre-filling operation step and a decoding operation step; the pre-filling operation step includes: the Emb module of the large model reasoning accelerator obtains the embedded vector from the SDRAM via a bus, and the Emb module transfers the embedded vector to the Reg register of the large model reasoning accelerator; the VEX module of the large model reasoning accelerator performs a root mean square normalization operation on the embedded vector to obtain a normalized result; the Attn module of the large model reasoning accelerator calculates the attention value according to the normalized result, and updates the key value cache of the large model reasoning accelerator; the VEX module performs a row root mean square normalization operation and a residual addition operation on the attention value; the HN module of the large model reasoning accelerator and the VEX module jointly perform FFN layer calculation to obtain an intermediate output result; iterate according to the depth of the large model Decoder layer until the pre-filled output result is obtained, and the HN module performs a logit calculation on the pre-filled output result to obtain an output probability vector;

[0010] The decoding operation steps include: performing a sampling operation on the final output result and generating a tag ID of the final output result, and sending the tag ID of the final output result to the output tag buffer of the large model inference accelerator and the Emb module.

[0011] The present invention also proposes a large model reasoning calculation device based on topological calculation, including: a data pre-reading module, used to read the code from the code memory and send it to the SDRAM; read the prompt words and vocabulary in the prompt word memory to the SDRAM; a data loading module, used to query the vocabulary in the SDRAM according to the tag ID of the prompt word to obtain an embedded vector, and send the embedded vector to a large model reasoning accelerator; a result acquisition module, used to perform large model reasoning calculation, and send the output result obtained by the large model reasoning accelerator to a host computer.

[0012] The large model reasoning computing device described in the present invention, wherein the result acquisition module includes a pre-filling module and a decoding module: the pre-filling module is used to perform a pre-filling operation, including: the Emb module of the large model reasoning accelerator obtains the embedded vector from the SDRAM via a bus, and the Emb module transfers the embedded vector to the Reg register of the large model reasoning accelerator; the VEX module of the large model reasoning accelerator performs a root mean square normalization operation on the embedded vector to obtain a normalized result; the Attn module of the large model reasoning accelerator calculates the attention value according to the normalized result, and updates the key value cache of the large model reasoning accelerator; the VEX module The attention value is subjected to a row root mean square normalization operation and a residual addition operation; the HN module and the VEX module of the large model inference accelerator jointly perform FFN layer calculations to obtain an intermediate output result; it is iterated according to the depth of the large model Decoder layer until a pre-filled output result is obtained, and the HN module performs a logit calculation on the pre-filled output result to obtain an output probability vector; the decoding module is used to perform a decoding operation, including: sampling the final output result and generating a tag ID of the final output result, and sending the tag ID of the final output result to the output tag buffer of the large model inference accelerator and the Emb module.

[0013] The present invention also proposes an electronic device, comprising the large model reasoning and computing device based on topological computing as described above.

[0014] The present invention also proposes a computer-readable storage medium storing computer-executable instructions, characterized in that when the computer-executable instructions are executed, the large model reasoning calculation method based on topological calculation as described above is implemented.

[0015] The present invention reduces the demand for FFN layer computing resources and weight storage resources by performing binary quantization on model weights; performs int4 quantization on keys and values, reducing the capacity demand for KV cache. A large model reasoning computing architecture based on topological calculation is also designed, which completely eliminates the large-scale weight loading process, avoids the weight loading from memory to on-chip cache, and alleviates memory access overhead. BRIEF DESCRIPTION OF THE DRAWINGS

[0016] Figure 1 It is a schematic diagram of topological calculation of a large model reasoning calculation architecture based on topological calculation of the present invention.

[0017] Figure 2 It is a schematic diagram of the large-model reasoning computing architecture based on topological computing of the present invention.

[0018] Figure 3 It is a schematic diagram of the large model hardware accelerator architecture of the present invention.

[0019] Figure 4 It is a schematic diagram of the large model reasoning calculation data flow of the present invention.

[0020] Figure 5 It is a schematic diagram of the large model reasoning and computing device of the present invention.

[0021] Figure 6 It is a schematic diagram of an electronic device of the present invention. DETAILED DESCRIPTION

[0022] In order to make the purpose, technical solution and advantages of the present invention more clearly understood, the present invention is further described in detail below in conjunction with the accompanying drawings. It should be understood that the specific implementation method described herein is only used to explain the present invention and is not used to limit the present invention.

[0023] It should be noted that, in this application, relational terms such as first and second, etc. are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Moreover, the terms "include", "comprises" or any other variants thereof are intended to cover non-exclusive inclusion, so that a process, method, article or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or device.

[0024] Without more constraints, an element defined by the phrase "comprising a..." does not exclude the existence of other identical elements in the process, method, article or apparatus comprising the element.

[0025] The present invention aims to solve the technical problems of excessive overhead of weight loading and excessive area overhead of SRAM implementation of KV cache during large model inference.

[0026] In terms of algorithm, the present invention performs binary quantization on model weights, reducing the storage requirements of weights. Performs int4 quantization on keys and values, reducing the capacity requirements of KV cache. In terms of hardware, a large model inference computing architecture based on topological calculation is designed to avoid loading weights from memory to on-chip cache. Through the above technical means, the beneficial effects obtained by the present invention include: binarization of weights alleviates the demand for FFN layer computing resources and weight storage resources; int4 quantization of keys and values ​​greatly reduces the SRAM implementation area overhead of KV cache; the large model inference computing architecture based on topological calculation completely eliminates the large-scale weight loading process, greatly alleviating the memory access overhead.

[0027] The following is a further explanation of the large model reasoning calculation architecture and large model reasoning calculation method based on topological calculation of the present invention in conjunction with the accompanying drawings.

[0028] The first embodiment of the present invention comprises:

[0029] 1. Topological calculation principle:

[0030] Topological calculation is based on the topological relationship of interconnected wires at the physical level. Specifically, in neural network calculations, the presence or absence of a connection represents the presence or absence of a weight, and the topological relationship of the connection defines the data flow of the calculation. Figure 1 As shown, compared with the traditional computing paradigm, topological computing is based on topological connection calculation, without the need to load and transmit weights.

[0031] 2. SoC architecture design:

[0032] SoC architecture design Figure 2 As shown in the figure, it includes CPU, SDRAM, SPI Flash for storing code, SD Card for storing prompts and vocabulary, large model reasoning accelerator H-LLM core and other necessary peripherals and controllers. The CPU execution process is divided into the following three steps:

[0033] Step S11, reading code from SPI Flash to SDRAM.

[0034] Step S12: Configure peripherals. Move the prompt word and word list in the SD card to the SDRAM.

[0035] Step S13, configure the H-LLM core. According to the token ID (token_id) of the prompt word prompt, query the word table in SDRAM to obtain the embedding vector (Embedding Vector) and send it to the H-LLM core. Get the output from the output buffer of the H-LLM core and send it to the host computer through uart. Specifically include:

[0036] 3. Large model hardware accelerator architecture design:

[0037] Hardware accelerator architecture design Figure 3As shown in the figure, the accelerator is connected to the outside world through the AXI4 bus, where the Embedding module is responsible for obtaining the Embedding Vector from the vocabulary to the Reg; the Attn module is responsible for the calculation of the self-attention mechanism; the HN module is responsible for the calculation of the FFN layer; the VEX module is responsible for other vector calculations; and the Sample module is responsible for sampling to generate the final output token id. In addition, there is a key-value cache KV cache, a configuration register Configure Register, and a system controller Controller.

[0038] The HN module hardens the weights of all FFN layers into the module through topological calculation, so the HN module does not need to load weights from off-chip memory during inference. After int4 quantization, the KV cache can be fully implemented using on-chip SRAM, so the Attn module does not need to read key-value pairs from off-chip to local cache during inference.

[0039] 4. Large model reasoning and calculation data execution steps;

[0040] The inference process of a large language model can be divided into a pre-population phase and a decoding phase:

[0041] Step S21, H-LLM core executes the prompt pre-filling stage. The pre-filling stage is the initial step of reasoning. At this time, the model takes the prompt word prompt as input. The CPU controls DMA to query the vocabulary in SDRAM according to the token id of the prompt word prompt to obtain the Embedding Vector and send it to the Embedding module. The Embedding module transfers it to Reg. The VEX module performs RMS normalization on the Embedding Vector and then writes it back to Reg. The Attn module calculates the attention value based on the normalized result and stores it in Reg, while updating the KV cache. The VEX module performs row RMS normalization and residual addition on the attention value. The HN module and the VEX module jointly perform the calculation of the FFN layer to obtain the output result x of the i-th layer. i Repeat the above process according to the depth of the Decoder layer until the pre-filled output result x is obtained. layer_num , HN module to x layer_num Perform logit calculation to get the output probability vector l. l can start the next round of calculation without sampling until all prompt tokens are executed.

[0042] Step S22, H-LLM core performs decoding phase. In the decoding phase, based on the pre-filling phase, l is sampled to generate a token id and sent to the output buffer and Emb module. The Emb module queries the vocabulary in SDRAM to obtain the Embedding Vector to start a new round of calculation, and autoregressively executes the complete calculation process until a token_id indicating the end is encountered: <eos>.

[0043] It should be noted that, in various embodiments of the present invention, the sequence numbers of the above steps do not mean the order of execution. The order of execution of the steps should be determined by their functions and internal logic, and should not constitute any limitation on the implementation process of the embodiments of the present invention.

[0044] The following is a system embodiment corresponding to the above method embodiment. This embodiment can be implemented in conjunction with the above embodiment. The relevant technical details mentioned in the above embodiment are still valid in this embodiment. In order to reduce repetition, they are not repeated here. Accordingly, the relevant technical details mentioned in this embodiment can also be applied in the above embodiment.

[0045] In the second embodiment of the present invention, a large model reasoning computing device 10 based on topological computing is proposed, such as Figure 5 As shown, including:

[0046] The data pre-reading module 11 is used to read the code from the code memory and send it to the SDRAM; read the prompt words and the word list from the prompt word memory to the SDRAM;

[0047] A data loading module 12 is used to query the word table in the SDRAM according to the tag ID of the prompt word to obtain an embedding vector, and send the embedding vector to the large model reasoning accelerator;

[0048] The result acquisition module 13 is used to perform large model reasoning calculations and send the output results obtained by the large model reasoning accelerator to the host computer.

[0049] The result acquisition module 13 further includes a pre-filling module 131 and a decoding module 132, wherein the pre-filling module 131 is used to perform a pre-filling operation; the decoding module 132 is used to perform a decoding operation;

[0050] The pre-filling stage is the initial step of reasoning. At this time, the model takes the prompt word as input. The CPU controls DMA to query the word list in SDRAM according to the token id of the prompt word to obtain EmbeddingVector and send it to the Embedding module. The EmbeddingVector is transferred to Reg. The VEX module performs RMS normalization on the EmbeddingVector and then writes it back to Reg. The Attn module calculates the attention value based on the normalization result and stores it in Reg, while updating KVcache. The VEX module performs RMS normalization and residual addition on the attention value. The HN module and the VEX module jointly perform the calculation of the FFN layer to obtain the output result of the i-th layer. i Repeat the above process according to the depth of the Decoder layer until x is obtained. layer_num , HN module to x layer_num Perform logit calculation to get the output probability vector l. l can start the next round of calculation without sampling until all prompttokens are executed.

[0051] The decoding phase samples l based on the pre-filling phase to generate a tokenid and sends it to the output buffer and Emb module. The Emb module queries the vocabulary in SDRAM to obtain the Embedding Vector to start a new round of calculations, and autoregressively executes the complete calculation process until it encounters a token_id indicating the end: <eos>.

[0052] Specifically, the model takes prompt as input, and the CPU controls DMA to query the vocabulary in SDRAM according to the token id of prompt to obtain the Embedding Vector x i and sends it to Emb, which stores it in Reg. VEX reads x from Reg i Perform RMS normalization to get and write back to Reg. HN according to And weight calculation q i , k i 、v i And write back to Reg. Attn reads the cached K from KV cache i-1 、V i-1 and Reg's q i , k i 、v i Perform attention calculation together and convert the intermediate result a i Write back to Reg and update KV cache. The HN module will a i Multiply by the weight matrix to get the final self-attention mechanism output o i and write back Reg. VEX to o i Perform RMS normalization again to get HN Multiply by the weight to get u i , g i VEX will i Activate and multiply by g i get HN will u i Multiply by the weight d i , VEX vs. i Perform RMS normalization to get x i+1 Repeat the above process according to the depth of the Decoder layer until x is obtained. layer_num , HN module to x layer_num Logit calculation is performed to obtain l. In the pre-filling stage, l can start the next round of calculation without sampling until all prompttokens are executed. The decoding module 132 samples l based on the data obtained by the pre-filling module 131 to generate a tokenid and send it to the output Buffer and Emb. Embedding queries the vocabulary in SDRAM to obtain EmbeddingVector to start a new round of calculation, and autoregressively executes the complete calculation process until a token_id indicating the end is encountered.

[0053] In the third embodiment of the present invention, a computer-readable storage medium is proposed. The large model reasoning and computing device based on topological calculation of the present invention, if its function is implemented in the form of a software functional unit and sold or used as an independent product, can be stored in a computer-readable storage medium. Based on such an understanding, the technical solution of the present invention, in essence, or the part that contributes to the prior art or the part of the technical solution, can be embodied in the form of a software product, and the computer software product is stored in a computer-readable storage medium, including a number of instructions for a computer device (which can be a personal computer, a server, or a network device, etc.) to perform all or part of the steps of the method described in each embodiment of the present invention. Therefore, in the third embodiment of the present invention, a computer-readable storage medium is provided for storing a computer program of a large model reasoning and computing method based on topological calculation. It should be understood that the computer-readable storage medium in the embodiment of the present invention can be a volatile memory and / or a non-volatile memory. Among them, the non-volatile memory can be a read-only memory (ROM), a programmable ROM (PROM), an erasable PROM (EPROM), an electrically erasable programmable read-only memory (EEPROM) or a flash memory. The volatile memory can be a random access memory (RAM), which is used as an external cache. By way of example but not limitation, many forms of random access memory (RAM) are available, such as static random access memory (SRAM), dynamic random access memory (DRAM), synchronous dynamic random access memory (SDRAM), double data rate synchronous dynamic random access memory (DDR SDRAM), enhanced synchronous dynamic random access memory (ESDRAM), synchronous link dynamic random access memory (SLDRAM) and direct rambus RAM (DRRAM).

[0054] Figure 6 Schematic diagram of an electronic device of the present invention. Figure 6 As shown, in the fourth embodiment of the present invention, an electronic device 100 is proposed, including a large model reasoning and computing device based on topological calculation as described above. A person of ordinary skill in the art can understand that all or part of the steps in the above method can be completed by instructing related hardware (such as a processor, FPGA, ASIC, etc.) through a program. All or part of the steps of the above embodiment can also be implemented using one or more integrated circuits. Accordingly, each module in the above embodiment can be implemented in the form of hardware, such as implementing its corresponding functions through an integrated circuit, or in the form of a software function module, such as implementing its corresponding functions through a processor executing a program / instruction stored in a memory. The embodiments of the present invention are not limited to any specific form of combination of hardware and software.

[0055] It should be noted that the structure of the electronic device shown in the drawings of the present invention does not constitute a limitation thereto, and the actual knowledge structure recognition device may include more or fewer components than shown in the drawings, or a combination of certain components, or a different arrangement of components.

[0056] When implemented using software, the above embodiments may be implemented in whole or in part in the form of a computer program product. The computer program product includes one or more computer instructions or computer programs. When the computer instructions or computer programs are loaded or executed on a computer, the process or function described in the embodiment of the present invention is generated in whole or in part. The computer may be a general-purpose computer, a special-purpose computer, a computer network, or other programmable device. The computer instructions may be stored in a computer-readable storage medium, or transmitted from one computer-readable storage medium to another computer-readable storage medium. For example, the computer instructions may be transmitted from one website site, computer, server or data center to another website site, computer, server or data center by wired (e.g., infrared, wireless, microwave, etc.) means. The computer-readable storage medium may be any available medium that a computer can access or a data storage device such as a server or data center that includes one or more available media sets. The available medium may be a magnetic medium (e.g., a floppy disk, a hard disk, a tape), an optical medium (e.g., a DVD), or a semiconductor medium. The semiconductor medium may be a solid-state hard disk.

[0057] The above implementation modes are only used to illustrate the present invention, but not to limit the present invention. Ordinary technicians in the relevant technical field can make various changes and modifications without departing from the spirit and scope of the present invention. Therefore, all equivalent technical solutions also belong to the scope of the present invention. The patent protection scope of the present invention should be defined by the claims.< / eos> < / eos>

Claims

1. A large model reasoning computing architecture based on topological computing, characterized in that: include: A CPU, an SDRAM, a code memory, a prompt word memory, and a large model reasoning accelerator, and the CPU, the SDRAM, the code memory, the prompt word memory and the large model reasoning accelerator are routed and connected via a bus; When performing large model inference calculations, first read the code in the code memory and the prompt words and vocabulary in the prompt word memory into the SDRAM, then query the vocabulary in the SDRAM according to the tag ID of the prompt word to obtain the embedding vector, and then send the embedding vector to the large model inference accelerator for large model inference calculations, and send the final output result obtained by the large model inference accelerator to the host computer.

2. The large model inference computing architecture as described in claim 1, characterized in that: The large model inference accelerator includes: The Emb module is used to obtain the embedding vector and send it to the Reg register; The Attn module is used to obtain the embedding vector from the Reg register to perform large model inference calculation based on the self-attention mechanism; The HN module is used to obtain the embedding vector from the Reg register to perform calculation of the FFN layer of the large model inference calculation; A VEX module is used to obtain the embedded vector from the Reg register to perform vector calculation for large model inference calculation; Sample module, used to sample and generate the tag ID of the output result; Reg register, used to store the embedded vector; An output tag buffer is used to cache and output the tag ID of the output result; Among them, the Emb module is electrically connected to the bus, and receives and sends data; the Emb module is electrically connected to the Reg register, and sends data to the Reg register; the Reg register is electrically connected to the HN module, the Attn module and the VEX module, and receives and sends data to the HN module, the Attn module and the VEX module; the HN module is electrically connected to the Sample module, and sends data to the Sample module; the Sample module is electrically connected to the Emb module and the output tag buffer, and sends data to the Emb module and the output tag buffer; the output tag buffer is electrically connected to the bus, and sends data to the bus.

3. The large model inference computing architecture as described in claim 1, characterized in that: The code memory is SPI Flash, and the prompt word memory is SD Card.

4. The large model inference computing architecture as claimed in claim 1, characterized in that: The bus is an AXI4 bus.

5. A large model reasoning calculation method based on topological calculation, using the large model reasoning calculation architecture as described in any one of claims 1 to 4 to perform large model reasoning calculation, characterized in that: The large model reasoning calculation method includes: Read the code from the code memory and send it to the SDRAM; read the prompt word and the word list from the prompt word memory to the SDRAM; According to the tag ID of the prompt word, the word table in the SDRAM is queried to obtain an embedding vector, and the embedding vector is sent to the large model inference accelerator; Perform large model inference calculations and obtain the final output results, which are then sent to the host computer.

6. The large model reasoning calculation method according to claim 5, characterized in that: The steps of performing large model inference calculation on the large model inference accelerator include: a pre-filling operation step and a decoding operation step, The pre-filling operation steps include: The Emb module of the large model reasoning accelerator obtains the embedded vector from the SDRAM via a bus, and the Emb module transfers the embedded vector to the Reg register of the large model reasoning accelerator; The VEX module of the large model reasoning accelerator performs a root mean square normalization operation on the embedded vector to obtain a normalized result; the Attn module of the large model reasoning accelerator calculates the attention value according to the normalized result, and updates the key value cache of the large model reasoning accelerator; the VEX module performs a row root mean square normalization operation and a residual addition operation on the attention value; the HN module of the large model reasoning accelerator and the VEX module jointly perform FFN layer calculations to obtain an intermediate output result; Iterate according to the depth of the decoder layer of the large model until the pre-filled output result is obtained. The HN module performs logit calculation on the pre-filled output result to obtain the output probability vector; The decoding operation steps include: A sampling operation is performed on the final output result and a tag ID of the final output result is generated, and the tag ID of the final output result is sent to the output tag buffer of the large model reasoning accelerator and the Emb module.

7. A large model reasoning computing device based on topological computing, characterized in that: include: A data pre-reading module, used for reading the code from the code memory and sending it to the SDRAM; Read the cue words and word list from the cue word memory to the SDRAM; A data loading module, used for querying the word table in the SDRAM according to the tag ID of the prompt word to obtain an embedding vector, and sending the embedding vector to the large model inference accelerator; The result acquisition module is used to perform large model reasoning calculations and send the output results obtained by the large model reasoning accelerator to the host computer.

8. The large model inference computing device according to claim 7, characterized in that: The result acquisition module includes a pre-filling module and a decoding module: The pre-filling module is used to perform a pre-filling operation, including: the Emb module of the large model reasoning accelerator obtains the embedded vector from the SDRAM through a bus, and the Emb module transfers the embedded vector to the Reg register of the large model reasoning accelerator; the VEX module of the large model reasoning accelerator performs a root mean square normalization operation on the embedded vector to obtain a normalized result; the Attn module of the large model reasoning accelerator calculates the attention value according to the normalized result, and updates the key value cache of the large model reasoning accelerator; the VEX module performs a row root mean square normalization operation and a residual addition operation on the attention value; the HN module of the large model reasoning accelerator and the VEX module jointly perform FFN layer calculations to obtain an intermediate output result; iterate according to the depth of the large model Decoder layer until a pre-filling output result is obtained, and the HN module performs a logit calculation on the pre-filling output result to obtain an output probability vector; The decoding module is used to perform decoding operations, including: sampling the final output result and generating a tag ID of the final output result, and sending the tag ID of the final output result to the output tag buffer of the large model inference accelerator and the Emb module.

9. An electronic device comprising the large model reasoning and computing device based on topological computing as claimed in claim 7 or 8.

10. A computer-readable storage medium storing computer-executable instructions, characterized in that: When the computer executable instructions are executed, the large model reasoning calculation method based on topological calculation as described in claim 5 or 6 is implemented.

Images